stc8: implement option handling

Implement option handling for STC8 series, based on STC8A8K64S4A12 reverse engineering. This mostly wraps up all important parts of the STC8 implementation. Interoperability was tested with STC-ISP V6.86O. v2: update documentation
stc8: finish up frequency calibration
2018-08-21 02:01:39 +02:00 · 2018-08-20 23:47:42 +02:00 · 2018-06-27 01:10:46 +02:00 · 2018-06-27 01:05:06 +02:00 · 2018-06-27 01:03:22 +02:00 · 2018-06-27 00:10:41 +02:00
23 changed files with 1080 additions and 295 deletions
--- a/.gitignore
+++ b/.gitignore
@ -1,8 +1,12 @@
 *~
 *.pyc
 *.egg-info
-__pycache__
+*.eggs/
+*.pybuild/
+__pycache__/
 /build
 /dist
 /deb_dist
+/debian/stcgal*
+/debian/files
 /.vscode
--- a/.travis.yml
+++ b/.travis.yml
@ -11,7 +11,7 @@ python:
 before_install:
  - sudo apt install rpm dpkg-dev debhelper dh-python python3-setuptools fakeroot python3-serial python3-yaml
 install:
-  - pip install pyserial pyusb
+  - pip install pyserial pyusb tqdm
 script:
  - python setup.py build
  - python setup.py test
--- a/README.md
+++ b/README.md
@ -23,7 +23,7 @@ suitable for automation.
 Supported MCU models
 --------------------

-stcgal should fully support STC 89/90/10/11/12/15 series MCUs.
+stcgal should fully support STC 89/90/10/11/12/15 series MCUs. Support for STC8 series MCUs is work in progress.

 So far, stcgal was tested with the following MCU models:

@ -47,6 +47,7 @@ So far, stcgal was tested with the following MCU models:
 * STC15L2K16S2 (BSL version: 7.2.4S)
 * STC15W408AS (BSL version: 7.2.4T)
 * STC15W4K56S4 (BSL version: 7.3.4T, UART and USB mode)
+* STC8A8K64S4A12 (BSL version: 7.3.9U)

 Compatibility reports, both negative and positive, are welcome.

@ -59,8 +60,8 @@ Features
 * Program flash memory
 * Program IAP/EEPROM
 * Set device options
-* Read unique device ID (STC 10/11/12/15)
-* Trim RC oscillator frequency (STC 15)
+* Read unique device ID (STC 10/11/12/15/8)
+* Trim RC oscillator frequency (STC 15/8)
 * Automatic power-cycling with DTR toggle or a custom shell command
 * Automatic UART protocol detection

@ -126,6 +127,7 @@ and MCU series is as follows:
 * ```stc12``` Most STC10/11/12 series
 * ```stc15a``` STC15x104E and STC15x204E(A) series
 * ```stc15``` Most STC15 series
+* ```stc8``` STC8 series
 * ```usb15``` USB support on STC15W4 series
 * ```auto``` Automatic detection of UART based protocols (default)

@ -257,17 +259,20 @@ Option key                    | Possible values   | Protocols/Models    | Descri
 ```por_reset_delay```         | short/long        | STC12+              | Power-on reset (POR) delay
 ```low_voltage_threshold```   | 0...7             | STC15A+             | Low-voltage detection threshold. Model specific.
 ```eeprom_lvd_inhibit```      | true/false        | STC15A+             | Ignore EEPROM writes in low-voltage situations
-```rstout_por_state```        | low/high          | STC15+              | RSTOUT pin state after power-on reset
+```rstout_por_state```        | low/high          | STC15+              | RSTOUT/RSTSV pin state after power-on reset
+```uart1_remap```             | true/false        | STC8                | Remap UART1 pins (P3.0/P3.1) to UART2 pins (P3.6/P3.7)
 ```uart2_passthrough```       | true/false        | STC15+              | Pass-through UART1 to UART2 pins (for single-wire UART mode)
 ```uart2_pin_mode```          | push-pull/normal  | STC15+              | Output mode of UART2 TX pin
 ```cpu_core_voltage```        | low/mid/high      | STC15W+             | CPU core voltage (low: ~2.7V, mid: ~3.3V, high: ~3.6V)
+```epwm_open_drain```         | true/false        | STC8                | Use open-drain pin mode for EPWM pins after power-on reset
+```program_eeprom_split```    | 512 - 65024       | STC8A8 w/ 64 KB     | Select split between code flash and EEPROM flash (in 512 byte blocks)

 ### Frequency trimming

 If the internal RC oscillator is used (```clock_source=internal```),
 stcgal can execute a trim procedure to adjust it to a given value. This
-is only supported by STC15 series. The trim values are stored with
-device options. Use the ```-t``` flag to request trimming to a certain
+is only supported by STC15 series and newer. The trim values are stored
+with device options. Use the ```-t``` flag to request trimming to a certain
 value. Generally, frequencies between 4 and 35 MHz can be achieved. If
 trimming fails, stcgal will abort.

--- a/debian/control
+++ b/debian/control
@ -2,14 +2,14 @@ Source: stcgal
 Section: electronics
 Priority: optional
 Maintainer: Andrew Andrianov <andrew@ncrmnt.org>
-Build-Depends: debhelper (>= 9), python3, python3-setuptools, dh-python
+Build-Depends: debhelper (>= 9), python3, python3-setuptools, dh-python, python3-serial, python3-tqdm, python3-yaml
 Standards-Version: 3.9.5
 Homepage: https://github.com/grigorig/stcgal
 X-Python3-Version: >= 3.2

 Package: stcgal
 Architecture: all
-Depends: ${misc:Depends}, python3, python3-serial
+Depends: ${misc:Depends}, python3, python3-serial, python3-tqdm
 Recommends: python3-usb (>= 1.0.0~b2)
 Description: STC MCU ISP flash tool
 stcgal is a command line flash programming tool for STC MCU Ltd.
--- a/doc/stc8-options.txt
+++ b/doc/stc8-options.txt
@ -0,0 +1,71 @@
+MCS bytes
+=========
+
+            46 b9 6a 00 33 04 00 00 5a a5 ff ff ff 00 ff ff
+            00 ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff
+            00 ff 01 31 20 80 34 00 01 ff ff ff ff ff 8b bf
+                  ^^^^^^^^^^^       ^^    ^^          ^^ ^^
+                  frequency         clkdiv 5)         1) 3)
+                              ^^^^^
+                              trim?
+            f7 fe 1f cc 16
+            ^^ ^^
+            4) 2)
+
+1) not stricty related to some register
+aka MCS1
+bit 0: ? always 1
+bit 1: oscillator high gain
+bit 2: EPWM push-pull enabled
+bit 3: p2.0 state after boot
+bit 4: TXD signal source from RXD
+bit 5: p3.7 push-pull enabled
+bit 6: UART1 remap enabled
+bit 7: long power-on reset delay
+
+2) not strictly related to some register
+aka MCS4
+eeprom size / code space upper limit (in pages)
+only seems to apply to devices with max. flash size
+e.g. fe -> 63.5K, e0 -> 56K
+
+3) like RSTCFG? inverted?
+aka MCS2
+bit 0: LVD0
+bit 1: LVD1
+bit 2: ? always 1
+bit 3: ? always 1
+bit 4: ~reset pin enabled
+bit 5: ? always 1
+bit 6: ~enable lvd reset
+bit 7: ? always 1
+
+LVD:
+2.20V -> 0xbf
+2.40V -> 0xbe
+2.70V -> 0xbd
+3.00V -> 0xbc
+
+4) like WDT_CONTR
+aka MCS3
+bit 0: WDPS0
+bit 1: WDPS1
+bit 2: WDPS2
+bit 3: ~stop wdt in idle
+bit 4: ? always 1
+bit 5: ~enable wdt on por
+bit 6: ? always 1
+bit 7: ? always 1
+
+WDPS like in datasheet
+
+5)
+aka MCS0
+bit 0: ? ~BSLD / bootloader enabled
+bit 1: erase eeprom enabled
+bit 2: ?
+bit 3: ?
+bit 4: ?
+bit 5: ?
+bit 6: ?
+bit 7: ?
--- a/doc/stc8-protocol.txt
+++ b/doc/stc8-protocol.txt
@ -0,0 +1,137 @@
+Overview of changes
+-------------------
+
+The following changes have been observed compared to STC15:
+
+- Many differences in the status packet
+- At least some differences in MCS
+- Different challenge
+  - no separate program speed
+  - clock division was introduced; calibration always in the ~20-30 MHz range, lower clocks
+    use division
+  - the meaning of the calibration ranges and trim has changed
+
+The good:
+
+- Erase, Program, etc. operations are apparently unchanged. :)
+
+
+Status packet
+-------------
+
+46 B9 68 00 30   50 00 54 62 58 5D 00 04 FF FD 8B BF FF 27 4A F7 FE 73 55 00 F6 28 09 85 E3 5F 80 07 20 20 20 01 00 00 FE 05 3A 17 05 25 91 FF   10 AE 16
+                                                                                      ^^^^^ wakeup clock
+
+Clock set to 20 MHz by STC-ISP (encoding is different compared to STC15):
+
+46 B9 68 00 30   50 01 31 2E 90 38 01 01 FF FD 8B BF FF 27 35 F7 FE 73 55 00 F6 28 09 85 E3 5F 80 07 20 20 20 01 00 00 FE 05 3A 17 05 25 91 FF   10 54 16
+46 B9 68 00 30   50 01 31 2E 90 38 01 01 FF FD 8B BF FF 27 3B F7 FE 73 55 00 F6 28 09 85 E3 5F 80 07 20 20 20 01 00 00 FE 05 3A 17 05 25 91 FF   10 5A 16
+                                                        ^^^^^ some 24 MHz reference or other clk measurement?
+                                ^^^^^ trim/adjust?
+                                      ^^ clkdiv
+                    ^^^^^^^^^^^ clk
+
+MCS bytes
+
+46 B9 68 00 30   50 01 31 2E 90 38 01 01 FF FD 8B BF FF 27 35 F7 FE 73 55 00 F6 28 09 85 E3 5F 80 07 20 20 20 01 00 00 FE 05 3A 17 05 25 91 FF   10 54 16
+                                            ^^^^^^^^          ^^^^^
+
+Disconnect
+----------
+
+Uses FF command byte.
+
+
+Basic challenge operation
+-------------------------
+
+Host sends a challenge of some kind, followed by 0xfe pulsing
+
+46 B9 6A 00 0C   00 02 00 00 80 00   00 F8 16
+
+Much simpler than in STC15
+
+MCU sends back some response:
+
+46 B9 68 00 0C   00 02 36 AD 4E 83   02 2A 16
+
+Host now sends some longer challenge, followed by more pulses:
+
+46 B9 6A 00 20   00 0C 7C 00 7C 01 7C 02 7C 03 7D 00 7D 01 7D 02 7D 03 7E 00 7E 01 7E 02 7E 03   06 84 16
+
+MCU sends back some response:
+
+46 B9 68 00 20   00 0C 4D C6 4D DB 4D E7 4D F3 4D F6 4E 0E 4E 11 4E 26 4E 26 4E 32 4E 41 4E 56   09 DC 16
+
+Host now seems to initiate a baud switch or something like that
+
+46 B9 6A 00 0E   01 00 00 FF CC 01 7C 80   03 41 16
+
+MCU acknowlegdes it:
+
+46 B9 68 00 07   01   00 70 16
+
+Now the MCU switches to the new baud rate.
+
+
+Challenges observed
+-------------------
+
+6 MHz:
+
+46B96A0020000C 1400 1401 1402 1403 1500 1501 1502 1503 1600 1601 1602 1603 01A416
+
+5.5 MHz:
+
+46B96A0020000C 5C00 5C01 5C02 5C03 5D00 5D01 5D02 5D03 5E00 5E01 5E02 5E03 050416
+
+
+11 MHz:
+
+46B96A0020000C 5B00 5B01 5B02 5B03 5C00 5C01 5C02 5C03 5D00 5D01 5D02 5D03 04F816
+
+20 MHz:
+
+46B96A0020000C 3600 3601 3602 3603 3700 3701 3702 3703 3800 3801 3802 3803 033C16
+
+24 MHz:
+
+46B96A0020000C 7C00 7C01 7C02 7C03 7D00 7D01 7D02 7D03 7E00 7E01 7E02 7E03 068416
+
+27 MHz:
+
+46B96A0020000C B000 B001 B002 B003 B100 B101 B102 B103 B200 B201 B202 B203 08F416
+
+
+Ranges vs trim value
+--------------------
+
+46 B9 6A 00 20 00 0C   00 00  80 00  FF 00  00 01  80 01  FF 01  00 02  80 02  FF 02  00 03  80 03  FF 03   06 A4 16
+46 B9 68 00 20 00 0C   36 9B  4E 92  65 E4  36 CB  4E 7D  66 29  36 D1  4E 83  66 05  36 CB  4E C2  66 47   0A EA 16
+
+first byte determines general trim value... range of ~16 to ~30 MHz, the second byte (00..03) is a fine adjustment.
+
+
+Clock division?
+---------------
+
+5.5 MHz vs 11 Mhz: challenge is about the same. it's likely some kind of clock divider is used!
+
+5.5 Mhz switch: 01 00 00 FF CC 01 5C 80 clkdiv = 4?
+11  MHz switch: 01 00 00 FF CC 01 5B 80 clkdiv = 2?
+22  MHz switch: 01 00 00 FF CC 01 5C 80 clkdiv = 1?
+
+22 Mhz  option packet: 0400005AA5FFFFFF00FFFF00FFFFFFFFFFFFFFFFFFFFFFFFFFFFFF00FF01516D405D0201FFFDFFFFFF8BBFF7FE
+11 MHz  option packet: 0400005AA5FFFFFF00FFFF00FFFFFFFFFFFFFFFFFFFFFFFFFFFFFF00FF00A8AF985D0102FFFDFFFFFF8BBFF7FE
+5.5 MHz option packet: 0400005AA5FFFFFF00FFFF00FFFFFFFFFFFFFFFFFFFFFFFFFFFFFF00FF005462585D0004FFFDFFFFFF8BBFF7FE
+                                                                                             ^^ clkdiv?	
+                                                                                 ^^^^^^^^ clkspeed
+
+Always 24 MHz for programming
+-----------------------------
+
+Calibration for anything but 24 Mhz (and around that) fails when switching baud. Another observation is that there is no
+programming speed being calibrated anymore. This may suggest that a fixed speed is used for programming.
+
+Adjusting BRT calculation to 24 MHz in the switch packet seems to work. So it is really using 24 MHz by default;
+probably some pre-calibrated value.
--- a/setup.py
+++ b/setup.py
@ -24,10 +24,13 @@
 import stcgal
 from setuptools import setup, find_packages

+with open("README.md", "r") as fh:
+    long_description = fh.read()
+
 setup(
    name = "stcgal",
    version = stcgal.__version__,
-    packages = find_packages(exclude=["doc", "test"]),
+    packages = find_packages(exclude=["doc", "tests"]),
    install_requires = ["pyserial"],
    extras_require = {
        "usb": ["pyusb>=1.0.0"]
@ -38,6 +41,9 @@ setup(
        ],
    },
    description = "STC MCU ISP flash tool",
+    long_description = long_description,
+    long_description_content_type = "text/markdown",
+    keywords = "stc mcu microcontroller 8051 mcs-51",
    url = "https://github.com/grigorig/stcgal",
    author = "Grigori Goronzy",
    author_email = "greg@kinoho.net",
@ -52,9 +58,12 @@ setup(
        "Operating System :: Microsoft :: Windows",
        "Operating System :: MacOS",
        "Programming Language :: Python :: 3",
+        "Programming Language :: Python :: 3.4",
+        "Programming Language :: Python :: 3.5",
+        "Programming Language :: Python :: 3.6",
        "Topic :: Software Development :: Embedded Systems",
        "Topic :: Software Development",
    ],
-    test_suite = "test",
+    test_suite = "tests",
    tests_require = ["PyYAML"],
 )
--- a/stcgal/frontend.py
+++ b/stcgal/frontend.py
@ -23,8 +23,19 @@
 import sys
 import argparse
 import stcgal
+import serial
 from stcgal.utils import BaudType
-from stcgal.protocols import *
+from stcgal.protocols import Stc89Protocol
+from stcgal.protocols import Stc12AProtocol
+from stcgal.protocols import Stc12BProtocol
+from stcgal.protocols import Stc12Protocol
+from stcgal.protocols import Stc15Protocol
+from stcgal.protocols import Stc15AProtocol
+from stcgal.protocols import StcUsb15Protocol
+from stcgal.protocols import Stc8Protocol
+from stcgal.protocols import StcAutoProtocol
+from stcgal.protocols import StcProtocolException
+from stcgal.protocols import StcFramingException
 from stcgal.ihex import IHex

 class StcGal:
@ -32,7 +43,10 @@ class StcGal:

    def __init__(self, opts):
        self.opts = opts
+        self.initialize_protocol(opts)

+    def initialize_protocol(self, opts):
+        """Initialize protocol backend"""
        if opts.protocol == "stc89":
            self.protocol = Stc89Protocol(opts.port, opts.handshake, opts.baud)
        elif opts.protocol == "stc12a":
@ -47,11 +61,13 @@ class StcGal:
        elif opts.protocol == "stc15":
            self.protocol = Stc15Protocol(opts.port, opts.handshake, opts.baud,
                                          round(opts.trim * 1000))
+        elif opts.protocol == "stc8":
+            self.protocol = Stc8Protocol(opts.port, opts.handshake, opts.baud,
+                                         round(opts.trim * 1000))
        elif opts.protocol == "usb15":
            self.protocol = StcUsb15Protocol()
        else:
-            self.protocol = StcBaseProtocol(opts.port, opts.handshake, opts.baud)
-
+            self.protocol = StcAutoProtocol(opts.port, opts.handshake, opts.baud)
        self.protocol.debug = opts.debug

    def emit_options(self, options):
@ -133,14 +149,14 @@ class StcGal:

        try:
            self.protocol.connect(autoreset=self.opts.autoreset, resetcmd=self.opts.resetcmd)
-            if self.opts.protocol == "auto":
+            if isinstance(self.protocol, StcAutoProtocol):
                if not self.protocol.protocol_name:
                    raise StcProtocolException("cannot detect protocol")
                base_protocol = self.protocol
                self.opts.protocol = self.protocol.protocol_name
                print("Protocol detected: %s" % self.opts.protocol)
                # recreate self.protocol with proper protocol class
-                self.__init__(self.opts)
+                self.initialize_protocol(self.opts)
            else:
                base_protocol = None

@ -167,9 +183,8 @@ class StcGal:
            if self.opts.code_image:
                self.program_mcu()
                return 0
-            else:
-                self.protocol.disconnect()
-                return 0
+            self.protocol.disconnect()
+            return 0
        except NameError as ex:
            sys.stdout.flush()
            print("Option error: %s" % ex, file=sys.stderr)
@ -198,12 +213,14 @@ class StcGal:
 def cli():
    # check arguments
    parser = argparse.ArgumentParser(formatter_class=argparse.RawDescriptionHelpFormatter,
-                                     description="stcgal %s - an STC MCU ISP flash tool\n(C) 2014-2017 Grigori Goronzy\nhttps://github.com/grigorig/stcgal" %stcgal.__version__)
+                                     description="stcgal {} - an STC MCU ISP flash tool\n".format(stcgal.__version__) +
+                                                 "(C) 2014-2017 Grigori Goronzy\nhttps://github.com/grigorig/stcgal")
    parser.add_argument("code_image", help="code segment file to flash (BIN/HEX)", type=argparse.FileType("rb"), nargs='?')
    parser.add_argument("eeprom_image", help="eeprom segment file to flash (BIN/HEX)", type=argparse.FileType("rb"), nargs='?')
    parser.add_argument("-a", "--autoreset", help="cycle power automatically by asserting DTR", action="store_true")
    parser.add_argument("-r", "--resetcmd",  help="Use this shell command for board power-cycling (instead of DTR assertion)", action="store")
-    parser.add_argument("-P", "--protocol", help="protocol version (default: auto)", choices=["stc89", "stc12a", "stc12b", "stc12", "stc15a", "stc15", "usb15", "auto"], default="auto")
+    parser.add_argument("-P", "--protocol", help="protocol version (default: auto)",
+                        choices=["stc89", "stc12a", "stc12b", "stc12", "stc15a", "stc15", "stc8", "usb15", "auto"], default="auto")
    parser.add_argument("-p", "--port", help="serial port device", default="/dev/ttyUSB0")
    parser.add_argument("-b", "--baud", help="transfer baud rate (default: 19200)", type=BaudType(), default=19200)
    parser.add_argument("-l", "--handshake", help="handshake baud rate (default: 2400)", type=BaudType(), default=2400)
--- a/stcgal/ihex.py
+++ b/stcgal/ihex.py
@ -5,201 +5,213 @@
 import struct
 import codecs

-class IHex(object):
-  @classmethod
-  def read(cls, lines):
-    ihex = cls()

-    segbase = 0
-    for line in lines:
-      line = line.strip()
-      if not line: continue
+class IHex:
+    """Intel HEX parser and writer"""

-      t, a, d = ihex.parse_line(line)
-      if t == 0x00:
-        ihex.insert_data(segbase + a, d)
+    @classmethod
+    def read(cls, lines):
+        """Read Intel HEX data from string or lines"""
+        ihex = cls()

-      elif t == 0x01:
-        break # Should we check for garbage after this?
+        segbase = 0
+        for line in lines:
+            line = line.strip()
+            if not line:
+                continue

-      elif t == 0x02:
-        ihex.set_mode(16)
-        segbase = struct.unpack(">H", d[0:2])[0] << 4
+            t, a, d = ihex.parse_line(line)
+            if t == 0x00:
+                ihex.insert_data(segbase + a, d)

-      elif t == 0x03:
-        ihex.set_mode(16)
+            elif t == 0x01:
+                break  # Should we check for garbage after this?

-        cs, ip = struct.unpack(">2H", d[0:2])
-        ihex.set_start((cs, ip))
+            elif t == 0x02:
+                ihex.set_mode(16)
+                segbase = struct.unpack(">H", d[0:2])[0] << 4

-      elif t == 0x04:
-        ihex.set_mode(32)
-        segbase = struct.unpack(">H", d[0:2])[0] << 16
+            elif t == 0x03:
+                ihex.set_mode(16)

-      elif t == 0x05:
-        ihex.set_mode(32)
-        ihex.set_start(struct.unpack(">I", d[0:4])[0])
+                cs, ip = struct.unpack(">2H", d[0:2])
+                ihex.set_start((cs, ip))

-      else:
-        raise ValueError("Invalid type byte")
+            elif t == 0x04:
+                ihex.set_mode(32)
+                segbase = struct.unpack(">H", d[0:2])[0] << 16

-    return ihex
+            elif t == 0x05:
+                ihex.set_mode(32)
+                ihex.set_start(struct.unpack(">I", d[0:4])[0])

-  @classmethod
-  def read_file(cls, fname):
-    f = open(fname, "rb")
-    ihex = cls.read(f)
-    f.close()
-    return ihex
+            else:
+                raise ValueError("Invalid type byte")

-  def __init__(self):
-    self.areas = {}
-    self.start = None
-    self.mode = 8
-    self.row_bytes = 16
+        return ihex

-  def set_row_bytes(self, row_bytes):
-      """Set output hex file row width (bytes represented per row)."""
-      if row_bytes < 1 or row_bytes > 0xff:
-          raise ValueError("Value out of range: (%r)" % row_bytes)
-      self.row_bytes = row_bytes
+    @classmethod
+    def read_file(cls, fname):
+        """Read Intel HEX data from file"""
+        f = open(fname, "rb")
+        ihex = cls.read(f)
+        f.close()
+        return ihex

-  def extract_data(self, start=None, end=None):
-    if start is None:
-      start = 0
+    def __init__(self):
+        self.areas = {}
+        self.start = None
+        self.mode = 8
+        self.row_bytes = 16

-    if end is None:
-      result = bytearray()
+    def set_row_bytes(self, row_bytes):
+        """Set output hex file row width (bytes represented per row)."""
+        if row_bytes < 1 or row_bytes > 0xff:
+            raise ValueError("Value out of range: (%r)" % row_bytes)
+        self.row_bytes = row_bytes

-      for addr, data in self.areas.items():
-        if addr >= start:
-          if len(result) < (addr - start):
-            result[len(result):addr-start] = bytes(addr-start-len(result))
-          result[addr-start:addr-start+len(data)] = data
+    def extract_data(self, start=None, end=None):
+        """Extract binary data"""
+        if start is None:
+            start = 0

-      return bytes(result)
+        if end is None:
+            result = bytearray()

-    else:
-      result = bytearray()
+            for addr, data in self.areas.items():
+                if addr >= start:
+                    if len(result) < (addr - start):
+                        result[len(result):addr - start] = bytes(
+                            addr - start - len(result))
+                    result[addr - start:addr - start + len(data)] = data

-      for addr, data in self.areas.items():
-        if addr >= start and addr < end:
-          data = data[:end-addr]
-          if len(result) < (addr - start):
-            result[len(result):addr-start] = bytes(addr-start-len(result))
-          result[addr-start:addr-start+len(data)] = data
+            return bytes(result)

-      return bytes(result)
+        result = bytearray()

-  def set_start(self, start=None):
-    self.start = start
+        for addr, data in self.areas.items():
+            if addr >= start and addr < end:
+                data = data[:end - addr]
+                if len(result) < (addr - start):
+                    result[len(result):addr - start] = bytes(
+                        addr - start - len(result))
+                result[addr - start:addr - start + len(data)] = data

-  def set_mode(self, mode):
-    self.mode = mode
+        return bytes(result)

-  def get_area(self, addr):
-    for start, data in self.areas.items():
-      end = start + len(data)
-      if addr >= start and addr <= end:
-        return start
+    def set_start(self, start=None):
+        self.start = start

-    return None
+    def set_mode(self, mode):
+        self.mode = mode

-  def insert_data(self, istart, idata):
-    iend = istart + len(idata)
+    def get_area(self, addr):
+        for start, data in self.areas.items():
+            end = start + len(data)
+            if addr >= start and addr <= end:
+                return start

-    area = self.get_area(istart)
-    if area is None:
-      self.areas[istart] = idata
+        return None

-    else:
-      data = self.areas[area]
-      # istart - iend + len(idata) + len(data)
-      self.areas[area] = data[:istart-area] + idata + data[iend-area:]
+    def insert_data(self, istart, idata):
+        iend = istart + len(idata)

-  def calc_checksum(self, bytes):
-    total = sum(bytes)
-    return (-total) & 0xFF
+        area = self.get_area(istart)
+        if area is None:
+            self.areas[istart] = idata

-  def parse_line(self, rawline):
-    if rawline[0:1] != b":":
-      raise ValueError("Invalid line start character (%r)" % rawline[0])
+        else:
+            data = self.areas[area]
+            # istart - iend + len(idata) + len(data)
+            self.areas[area] = data[
+                :istart - area] + idata + data[iend - area:]

-    try:
-      #line = rawline[1:].decode("hex")
-      line = codecs.decode(rawline[1:], "hex_codec")
-    except:
-      raise ValueError("Invalid hex data")
+    def calc_checksum(self, data):
+        total = sum(data)
+        return (-total) & 0xFF

-    length, addr, type = struct.unpack(">BHB", line[:4])
+    def parse_line(self, rawline):
+        if rawline[0:1] != b":":
+            raise ValueError("Invalid line start character (%r)" % rawline[0])

-    dataend = length + 4
-    data = line[4:dataend]
+        try:
+            line = codecs.decode(rawline[1:], "hex_codec")
+        except ValueError:
+            raise ValueError("Invalid hex data")

-    #~ print line[dataend:dataend + 2], repr(line)
-    cs1 = line[dataend]
-    cs2 = self.calc_checksum(line[:dataend])
+        length, addr, line_type = struct.unpack(">BHB", line[:4])

-    if cs1 != cs2:
-      raise ValueError("Checksums do not match")
+        dataend = length + 4
+        data = line[4:dataend]

-    return (type, addr, data)
+        cs1 = line[dataend]
+        cs2 = self.calc_checksum(line[:dataend])

-  def make_line(self, type, addr, data):
-    line = struct.pack(">BHB", len(data), addr, type)
-    line += data
-    line += chr(self.calc_checksum(line))
-    #~ return ":" + line.encode("hex")
-    return ":" + line.encode("hex").upper() + "\r\n"
+        if cs1 != cs2:
+            raise ValueError("Checksums do not match")

-  def write(self):
-    output = ""
+        return (line_type, addr, data)

-    for start, data in sorted(self.areas.items()):
-      i = 0
-      segbase = 0
+    def make_line(self, line_type, addr, data):
+        line = struct.pack(">BHB", len(data), addr, line_type)
+        line += data
+        line += chr(self.calc_checksum(line))
+        return ":" + line.encode("hex").upper() + "\r\n"

-      while i < len(data):
-        chunk = data[i:i + self.row_bytes]
+    def write(self):
+        """Write Intel HEX data to string"""
+        output = ""

-        addr = start
-        newsegbase = segbase
+        for start, data in sorted(self.areas.items()):
+            i = 0
+            segbase = 0

-        if self.mode == 8:
-          addr = addr & 0xFFFF
+            while i < len(data):
+                chunk = data[i:i + self.row_bytes]

-        elif self.mode == 16:
-          t = addr & 0xFFFF
-          newsegbase = (addr - t) >> 4
-          addr = t
+                addr = start
+                newsegbase = segbase

-          if newsegbase != segbase:
-            output += self.make_line(0x02, 0, struct.pack(">H", newsegbase))
-            segbase = newsegbase
+                if self.mode == 8:
+                    addr = addr & 0xFFFF

-        elif self.mode == 32:
-          newsegbase = addr >> 16
-          addr = addr & 0xFFFF
+                elif self.mode == 16:
+                    t = addr & 0xFFFF
+                    newsegbase = (addr - t) >> 4
+                    addr = t

-          if newsegbase != segbase:
-            output += self.make_line(0x04, 0, struct.pack(">H", newsegbase))
-            segbase = newsegbase
+                    if newsegbase != segbase:
+                        output += self.make_line(
+                            0x02, 0, struct.pack(">H", newsegbase))
+                        segbase = newsegbase

-        output += self.make_line(0x00, addr, chunk)
+                elif self.mode == 32:
+                    newsegbase = addr >> 16
+                    addr = addr & 0xFFFF

-        i += self.row_bytes
-        start += self.row_bytes
+                    if newsegbase != segbase:
+                        output += self.make_line(
+                            0x04, 0, struct.pack(">H", newsegbase))
+                        segbase = newsegbase

-    if self.start is not None:
-      if self.mode == 16:
-        output += self.make_line(0x03, 0, struct.pack(">2H", self.start[0], self.start[1]))
-      elif self.mode == 32:
-        output += self.make_line(0x05, 0, struct.pack(">I", self.start))
+                output += self.make_line(0x00, addr, chunk)

-    output += self.make_line(0x01, 0, "")
-    return output
+                i += self.row_bytes
+                start += self.row_bytes

-  def write_file(self, fname):
-    f = open(fname, "w")
-    f.write(self.write())
-    f.close()
+        if self.start is not None:
+            if self.mode == 16:
+                output += self.make_line(
+                    0x03, 0, struct.pack(">2H", self.start[0], self.start[1]))
+            elif self.mode == 32:
+                output += self.make_line(
+                    0x05, 0, struct.pack(">I", self.start))
+
+        output += self.make_line(0x01, 0, "")
+        return output
+
+    def write_file(self, fname):
+        """Write Intel HEX data to file"""
+        f = open(fname, "w")
+        f.write(self.write())
+        f.close()
--- a/stcgal/options.py
+++ b/stcgal/options.py
@ -21,15 +21,24 @@
 #

 import struct
+from abc import ABC
 from stcgal.utils import Utils

-class BaseOption:
+class BaseOption(ABC):
+    """Base class for options"""
+
+    def __init__(self):
+        self.options = ()
+        self.msr = None
+
    def print(self):
+        """Print current configuration to standard output"""
        print("Target options:")
        for name, get_func, _ in self.options:
            print("  %s=%s" % (name, get_func()))

    def set_option(self, name, value):
+        """Set value of a specific option"""
        for opt, _, set_func in self.options:
            if opt == name:
                print("Option %s=%s" % (name, value))
@ -38,12 +47,14 @@ class BaseOption:
        raise ValueError("unknown")

    def get_option(self, name):
+        """Get option value for a specific option"""
        for opt, get_func, _ in self.options:
            if opt == name:
                return get_func(name)
        raise ValueError("unknown")

    def get_msr(self):
+        """Get array of model-specific configuration registers"""
        return bytes(self.msr)


@ -51,6 +62,7 @@ class Stc89Option(BaseOption):
    """Manipulation STC89 series option byte"""

    def __init__(self, msr):
+        super().__init__()
        self.msr = msr
        self.options = (
            ("cpu_6t_enabled", self.get_t6, self.set_t6),
@ -129,6 +141,7 @@ class Stc12AOption(BaseOption):
    """Manipulate STC12A series option bytes"""

    def __init__(self, msr):
+        super().__init__()
        assert len(msr) == 4
        self.msr = bytearray(msr)

@ -213,6 +226,7 @@ class Stc12Option(BaseOption):
    """Manipulate STC10/11/12 series option bytes"""

    def __init__(self, msr):
+        super().__init__()
        assert len(msr) == 4
        self.msr = bytearray(msr)

@ -337,6 +351,7 @@ class Stc12Option(BaseOption):

 class Stc15AOption(BaseOption):
    def __init__(self, msr):
+        super().__init__()
        assert len(msr) == 13
        self.msr = bytearray(msr)

@ -435,6 +450,7 @@ class Stc15AOption(BaseOption):

 class Stc15Option(BaseOption):
    def __init__(self, msr):
+        super().__init__()
        assert len(msr) >= 4
        self.msr = bytearray(msr)

@ -457,7 +473,7 @@ class Stc15Option(BaseOption):
        )

        if len(msr) > 4:
-            self.options += ("cpu_core_voltage", self.get_core_voltage, self.set_core_voltage),
+            self.options += (("cpu_core_voltage", self.get_core_voltage, self.set_core_voltage),)

    def get_reset_pin_enabled(self):
        return not bool(self.msr[2] & 16)
@ -599,10 +615,177 @@ class Stc15Option(BaseOption):
        if self.msr[4] == 0xea: return "low"
        elif self.msr[4] == 0xf7: return "mid"
        elif self.msr[4] == 0xfd: return "high"
-        else: return "unknown"
+        return "unknown"

    def set_core_voltage(self, val):
        volt_vals = {"low": 0xea, "mid": 0xf7, "high": 0xfd}
        if val not in volt_vals.keys():
            raise ValueError("must be one of %s" % list(volt_vals.keys()))
        self.msr[4] = volt_vals[val]
+
+class Stc8Option(BaseOption):
+    def __init__(self, msr):
+        super().__init__()
+        assert len(msr) >= 5
+        self.msr = bytearray(msr)
+
+        self.options = (
+            ("reset_pin_enabled", self.get_reset_pin_enabled, self.set_reset_pin_enabled),
+            ("clock_gain", self.get_clock_gain, self.set_clock_gain),
+            ("watchdog_por_enabled", self.get_watchdog, self.set_watchdog),
+            ("watchdog_stop_idle", self.get_watchdog_idle, self.set_watchdog_idle),
+            ("watchdog_prescale", self.get_watchdog_prescale, self.set_watchdog_prescale),
+            ("low_voltage_reset", self.get_lvrs, self.set_lvrs),
+            ("low_voltage_threshold", self.get_low_voltage, self.set_low_voltage),
+            ("eeprom_erase_enabled", self.get_ee_erase, self.set_ee_erase),
+            ("bsl_pindetect_enabled", self.get_pindetect, self.set_pindetect),
+            ("por_reset_delay", self.get_por_delay, self.set_por_delay),
+            ("rstout_por_state", self.get_p20_state, self.set_p20_state),
+            ("uart1_remap", self.get_uart1_remap, self.set_uart1_remap),
+            ("uart2_passthrough", self.get_uart_passthrough, self.set_uart_passthrough),
+            ("uart2_pin_mode", self.get_uart_pin_mode, self.set_uart_pin_mode),
+            ("epwm_open_drain", self.get_epwm_pp, self.set_epwm_pp),
+            ("program_eeprom_split", self.get_flash_split, self.set_flash_split),
+        )
+
+    def get_reset_pin_enabled(self):
+        return not bool(self.msr[2] & 16)
+
+    def set_reset_pin_enabled(self, val):
+        val = Utils.to_bool(val)
+        self.msr[2] &= 0xef
+        self.msr[2] |= 0x10 if not bool(val) else 0x00
+
+    def get_clock_gain(self):
+        gain = bool(self.msr[1] & 0x02)
+        return "high" if gain else "low"
+
+    def set_clock_gain(self, val):
+        gains = {"low": 0, "high": 1}
+        if val not in gains.keys():
+            raise ValueError("must be one of %s" % list(gains.keys()))
+        self.msr[1] &= 0xfd
+        self.msr[1] |= gains[val] << 1
+
+    def get_watchdog(self):
+        return not bool(self.msr[3] & 32)
+
+    def set_watchdog(self, val):
+        val = Utils.to_bool(val)
+        self.msr[3] &= 0xdf
+        self.msr[3] |= 0x20 if not val else 0x00
+
+    def get_watchdog_idle(self):
+        return not bool(self.msr[3] & 8)
+
+    def set_watchdog_idle(self, val):
+        val = Utils.to_bool(val)
+        self.msr[3] &= 0xf7
+        self.msr[3] |= 0x08 if not val else 0x00
+
+    def get_watchdog_prescale(self):
+        return 2 ** (((self.msr[3]) & 0x07) + 1)
+
+    def set_watchdog_prescale(self, val):
+        val = Utils.to_int(val)
+        wd_vals = {2: 0, 4: 1, 8: 2, 16: 3, 32: 4, 64: 5, 128: 6, 256: 7}
+        if val not in wd_vals.keys():
+            raise ValueError("must be one of %s" % list(wd_vals.keys()))
+        self.msr[3] &= 0xf8
+        self.msr[3] |= wd_vals[val]
+
+    def get_lvrs(self):
+        return not bool(self.msr[2] & 64)
+
+    def set_lvrs(self, val):
+        val = Utils.to_bool(val)
+        self.msr[2] &= 0xbf
+        self.msr[2] |= 0x40 if not val else 0x00
+
+    def get_low_voltage(self):
+        return 3 - self.msr[2] & 0x03
+
+    def set_low_voltage(self, val):
+        val = Utils.to_int(val)
+        if val not in range(0, 4):
+            raise ValueError("must be one of %s" % list(range(0, 4)))
+        self.msr[2] &= 0xfc
+        self.msr[2] |= 3 - val
+
+    def get_ee_erase(self):
+        return bool(self.msr[0] & 2)
+
+    def set_ee_erase(self, val):
+        val = Utils.to_bool(val)
+        self.msr[0] &= 0xfd
+        self.msr[0] |= 0x02 if val else 0x00
+
+    def get_pindetect(self):
+        return not bool(self.msr[0] & 1)
+
+    def set_pindetect(self, val):
+        val = Utils.to_bool(val)
+        self.msr[0] &= 0xfe
+        self.msr[0] |= 0x01 if not val else 0x00
+
+    def get_por_delay(self):
+        delay = bool(self.msr[1] & 128)
+        return "long" if delay else "short"
+
+    def set_por_delay(self, val):
+        delays = {"short": 0, "long": 1}
+        if val not in delays.keys():
+            raise ValueError("must be one of %s" % list(delays.keys()))
+        self.msr[1] &= 0x7f
+        self.msr[1] |= delays[val] << 7
+
+    def get_p20_state(self):
+        return "high" if self.msr[1] & 0x08 else "low"
+
+    def set_p20_state(self, val):
+        val = Utils.to_bool(val)
+        self.msr[1] &= 0xf7
+        self.msr[1] |= 0x08 if val else 0x00
+
+    def get_uart_passthrough(self):
+        return bool(self.msr[1] & 0x10)
+
+    def set_uart_passthrough(self, val):
+        val = Utils.to_bool(val)
+        self.msr[1] &= 0xef
+        self.msr[1] |= 0x10 if val else 0x00
+
+    def get_uart_pin_mode(self):
+        return "push-pull" if bool(self.msr[1] & 0x20) else "normal"
+
+    def set_uart_pin_mode(self, val):
+        modes = {"normal": 0, "push-pull": 1}
+        if val not in modes.keys():
+            raise ValueError("must be one of %s" % list(modes.keys()))
+        self.msr[1] &= 0xdf
+        self.msr[1] |= 0x20 if modes[val] else 0x00
+
+    def get_epwm_pp(self):
+        return bool(self.msr[1] & 0x04)
+
+    def set_epwm_pp(self, val):
+        val = Utils.to_bool(val)
+        self.msr[1] &= 0xfb
+        self.msr[1] |= 0x04 if val else 0x00
+
+    def get_uart1_remap(self):
+        return bool(self.msr[1] & 0x40)
+
+    def set_uart1_remap(self, val):
+        val = Utils.to_bool(val)
+        self.msr[1] &= 0xbf
+        self.msr[1] |= 0x40 if val else 0x00
+
+    def get_flash_split(self):
+        return self.msr[4] * 256
+
+    def set_flash_split(self, val):
+        num_val = Utils.to_int(val)
+        if num_val < 512 or num_val > 65024 or (num_val % 512) != 0:
+            raise ValueError("must be between 512 and 65024 bytes and a multiple of 512 bytes")
+        self.msr[4] = num_val // 256
--- a/stcgal/protocols.py
+++ b/stcgal/protocols.py
@ -21,16 +21,28 @@
 #

 import serial
-import sys, os, time, struct, re, errno
-import argparse
-import collections
+import sys
+import os
+import time
+import struct
+import re
+import errno
 from stcgal.models import MCUModelDatabase
 from stcgal.utils import Utils
-from stcgal.options import *
+from stcgal.options import Stc89Option
+from stcgal.options import Stc12Option
+from stcgal.options import Stc12AOption
+from stcgal.options import Stc15Option
+from stcgal.options import Stc15AOption
+from stcgal.options import Stc8Option
+from abc import ABC
+from abc import abstractmethod
 import functools
+import tqdm

 try:
-    import usb.core, usb.util
+    import usb.core
+    import usb.util
    _usb_available = True
 except ImportError:
    _usb_available = False
@ -46,22 +58,23 @@ class StcProtocolException(Exception):
    pass


-class StcBaseProtocol:
+class StcBaseProtocol(ABC):
    """Basic functionality for STC BSL protocols"""

-    """magic word that starts a packet"""
    PACKET_START = bytes([0x46, 0xb9])
+    """magic word that starts a packet"""

-    """magic byte that ends a packet"""
    PACKET_END = bytes([0x16])
+    """magic byte that ends a packet"""

-    """magic byte for packets received from MCU"""
    PACKET_MCU = bytes([0x68])
+    """magic byte for packets received from MCU"""

-    """magic byte for packets sent by host"""
    PACKET_HOST = bytes([0x6a])
+    """magic byte for packets sent by host"""

    PARITY = serial.PARITY_NONE
+    """parity configuration for serial communication"""

    def __init__(self, port, baud_handshake, baud_transfer):
        self.port = port
@ -77,6 +90,22 @@ class StcBaseProtocol:
        self.debug = False
        self.status_packet = None
        self.protocol_name = None
+        self.progress = None
+        self.progress_cb = self.progress_bar_cb
+
+    def progress_text_cb(self, current, written, maximum):
+        print(current, written, maximum)
+
+    def progress_bar_cb(self, current, written, maximum):
+        if not self.progress:
+            self.progress = tqdm.tqdm(
+                total = maximum,
+                unit = " Bytes",
+                desc = "Writing flash"
+                )
+        self.progress.update(written)
+        if current == maximum:
+            self.progress.close()

    def dump_packet(self, data, receive=True):
        if self.debug:
@ -103,6 +132,10 @@ class StcBaseProtocol:

        return packet[5:-1]

+    @abstractmethod
+    def write_packet(self, packet_data):
+        pass
+
    def read_packet(self):
        """Read and check packet from MCU.

@ -192,20 +225,6 @@ class StcBaseProtocol:
            mcu_name += "E" if self.status_packet[17] < 0x70 else "W"
            self.model = self.model._replace(name = mcu_name)

-        protocol_database = [("stc89", r"STC(89|90)(C|LE)\d"),
-                             ("stc12a", r"STC12(C|LE)\d052"),
-                             ("stc12b", r"STC12(C|LE)(52|56)"),
-                             ("stc12", r"(STC|IAP)(10|11|12)\D"),
-                             ("stc15a", r"(STC|IAP)15[FL][01]0\d(E|EA|)$"),
-                             ("stc15", r"(STC|IAP|IRC)15\D")]
-
-        for protocol_name, pattern in protocol_database:
-            if re.match(pattern, self.model.name):
-                self.protocol_name = protocol_name
-                break
-        else:
-            self.protocol_name = None
-
    def get_status_packet(self):
        """Read and decode status packet"""

@ -287,6 +306,8 @@ class StcBaseProtocol:
            try:
                self.pulse()
                self.status_packet = self.get_status_packet()
+                if len(self.status_packet) < 23:
+                    raise StcProtocolException("status packet too short")
            except (StcFramingException, serial.SerialTimeoutException): pass
        print("done")

@ -296,7 +317,21 @@ class StcBaseProtocol:

        self.initialize_model()

-    def initialize(self, base_protocol = None):
+    @abstractmethod
+    def initialize_status(self, status_packet):
+        """Initialize internal state from status packet"""
+        pass
+
+    @abstractmethod
+    def initialize_options(self, status_packet):
+        """Initialize options from status packet"""
+        pass
+
+    def initialize(self, base_protocol=None):
+        """
+        Initialize from another instance. This is an alternative for calling
+        connect() and is used by protocol autodetection.
+        """
        if base_protocol:
            self.ser = base_protocol.ser
            self.ser.parity = self.PARITY
@ -324,14 +359,48 @@ class StcBaseProtocol:
        print("Disconnected!")


+class StcAutoProtocol(StcBaseProtocol):
+    """
+    Protocol handler for autodetection of protocols. Does not implement full
+    functionality for any device class.
+    """
+
+    def initialize_model(self):
+        super().initialize_model()
+
+        protocol_database = [("stc89", r"STC(89|90)(C|LE)\d"),
+                             ("stc12a", r"STC12(C|LE)\d052"),
+                             ("stc12b", r"STC12(C|LE)(52|56)"),
+                             ("stc12", r"(STC|IAP)(10|11|12)\D"),
+                             ("stc15a", r"(STC|IAP)15[FL][012]0\d(E|EA|)$"),
+                             ("stc15", r"(STC|IAP|IRC)15\D"),
+                             ("stc8", r"(STC|IAP|IRC)8")]
+
+        for protocol_name, pattern in protocol_database:
+            if re.match(pattern, self.model.name):
+                self.protocol_name = protocol_name
+                break
+        else:
+            self.protocol_name = None
+
+    def initialize_options(self, status_packet):
+        raise NotImplementedError
+
+    def initialize_status(self, status_packet):
+        raise NotImplementedError
+
+    def write_packet(self, packet_data):
+        raise NotImplementedError
+
+
 class Stc89Protocol(StcBaseProtocol):
    """Protocol handler for STC 89/90 series"""

-    """These don't use any parity"""
    PARITY = serial.PARITY_NONE
+    """Parity configuration - these don't use any parity"""

-    """block size for programming flash"""
    PROGRAM_BLOCKSIZE = 128
+    """block size for programming flash"""

    def __init__(self, port, baud_handshake, baud_transfer):
        StcBaseProtocol.__init__(self, port, baud_handshake, baud_transfer)
@ -350,7 +419,7 @@ class Stc89Protocol(StcBaseProtocol):
        payload = StcBaseProtocol.extract_payload(self, packet)
        return payload[:-1]

-    def write_packet(self, data):
+    def write_packet(self, packet_data):
        """Send packet to MCU.

        Constructs a packet with supplied payload and sends it to the MCU.
@ -362,8 +431,8 @@ class Stc89Protocol(StcBaseProtocol):
        packet += self.PACKET_HOST

        # packet length and payload
-        packet += struct.pack(">H", len(data) + 5)
-        packet += data
+        packet += struct.pack(">H", len(packet_data) + 5)
+        packet += packet_data

        # checksum and end code
        packet += bytes([sum(packet[2:]) & 0xff])
@ -384,6 +453,9 @@ class Stc89Protocol(StcBaseProtocol):
    def initialize_options(self, status_packet):
        """Initialize options"""

+        if len(status_packet) < 20:
+            raise StcProtocolException("invalid options in status packet")
+
        self.options = Stc89Option(status_packet[19])
        self.options.print()

@ -416,19 +488,19 @@ class Stc89Protocol(StcBaseProtocol):

        return brt, brt_csum, iap_wait, delay

-    def initialize_status(self, packet):
+    def initialize_status(self, status_packet):
        """Decode status packet and store basic MCU info"""

-        self.cpu_6t = not bool(packet[19] & 1)
+        self.cpu_6t = not bool(status_packet[19] & 1)

        cpu_t = 6.0 if self.cpu_6t else 12.0
        freq_counter = 0
        for i in range(8):
-            freq_counter += struct.unpack(">H", packet[1+2*i:3+2*i])[0]
+            freq_counter += struct.unpack(">H", status_packet[1+2*i:3+2*i])[0]
        freq_counter /= 8.0
        self.mcu_clock_hz = (self.baud_handshake * freq_counter * cpu_t) / 7.0

-        bl_version, bl_stepping = struct.unpack("BB", packet[17:19])
+        bl_version, bl_stepping = struct.unpack("BB", status_packet[17:19])
        self.mcu_bsl_version = "%d.%d%s" % (bl_version >> 4, bl_version & 0x0f,
                                            chr(bl_stepping))

@ -473,7 +545,7 @@ class Stc89Protocol(StcBaseProtocol):
        sys.stdout.flush()
        packet = bytes([0x80, 0x00, 0x00, 0x36, 0x01])
        packet += struct.pack(">H", self.mcu_magic)
-        for i in range(4):
+        for _ in range(4):
            self.write_packet(packet)
            response = self.read_packet()
            if response[0] != 0x80:
@ -481,7 +553,7 @@ class Stc89Protocol(StcBaseProtocol):

        print("done")

-    def erase_flash(self, erase_size, flash_size):
+    def erase_flash(self, erase_size, _):
        """Erase the MCU's flash memory.

        Erase the flash memory with a block-erase command.
@ -505,8 +577,6 @@ class Stc89Protocol(StcBaseProtocol):
        as the block size (depends on MCU's RAM size).
        """

-        print("Writing %d bytes: " % len(data), end="")
-        sys.stdout.flush()
        for i in range(0, len(data), self.PROGRAM_BLOCKSIZE):
            packet = bytes(3)
            packet += struct.pack(">H", i)
@ -516,13 +586,12 @@ class Stc89Protocol(StcBaseProtocol):
            csum = sum(packet[7:]) & 0xff
            self.write_packet(packet)
            response = self.read_packet()
-            if response[0] != 0x80:
+            if len(response) < 1 or response[0] != 0x80:
                raise StcProtocolException("incorrect magic in write packet")
-            elif response[1] != csum:
+            elif len(response) < 2 or response[1] != csum:
                raise StcProtocolException("verification checksum mismatch")
-            print(".", end="")
-            sys.stdout.flush()
-        print(" done")
+            self.progress_cb(i, self.PROGRAM_BLOCKSIZE, len(data))
+        self.progress_cb(len(data), self.PROGRAM_BLOCKSIZE, len(data))

    def program_options(self):
        """Program option byte into flash"""
@ -576,16 +645,16 @@ class Stc12AProtocol(Stc12AOptionsMixIn, Stc89Protocol):
    def __init__(self, port, baud_handshake, baud_transfer):
        Stc89Protocol.__init__(self, port, baud_handshake, baud_transfer)

-    def initialize_status(self, packet):
+    def initialize_status(self, status_packet):
        """Decode status packet and store basic MCU info"""

        freq_counter = 0
        for i in range(8):
-            freq_counter += struct.unpack(">H", packet[1+2*i:3+2*i])[0]
+            freq_counter += struct.unpack(">H", status_packet[1+2*i:3+2*i])[0]
        freq_counter /= 8.0
        self.mcu_clock_hz = (self.baud_handshake * freq_counter * 12.0) / 7.0

-        bl_version, bl_stepping = struct.unpack("BB", packet[17:19])
+        bl_version, bl_stepping = struct.unpack("BB", status_packet[17:19])
        self.mcu_bsl_version = "%d.%d%s" % (bl_version >> 4, bl_version & 0x0f,
                                            chr(bl_stepping))

@ -620,6 +689,9 @@ class Stc12AProtocol(Stc12AOptionsMixIn, Stc89Protocol):
    def initialize_options(self, status_packet):
        """Initialize options"""

+        if len(status_packet) < 31:
+            raise StcProtocolException("invalid options in status packet")
+
        # create option state
        self.options = Stc12AOption(status_packet[23:26] + status_packet[29:30])
        self.options.print()
@ -661,7 +733,7 @@ class Stc12AProtocol(Stc12AOptionsMixIn, Stc89Protocol):
        sys.stdout.flush()
        packet = bytes([0x80, 0x00, 0x00, 0x36, 0x01])
        packet += struct.pack(">H", self.mcu_magic)
-        for i in range(4):
+        for _ in range(4):
            self.write_packet(packet)
            response = self.read_packet()
            if response[0] != 0x80:
@ -718,14 +790,14 @@ class Stc12OptionsMixIn:
 class Stc12BaseProtocol(StcBaseProtocol):
    """Base class for STC 10/11/12 series protocol handlers"""

-    """block size for programming flash"""
    PROGRAM_BLOCKSIZE = 128
+    """block size for programming flash"""

-    """countdown value for flash erase"""
    ERASE_COUNTDOWN = 0x0d
+    """countdown value for flash erase"""

-    """Parity for error correction was introduced with STC12"""
    PARITY = serial.PARITY_EVEN
+    """Parity for error correction was introduced with STC12"""

    def __init__(self, port, baud_handshake, baud_transfer):
        StcBaseProtocol.__init__(self, port, baud_handshake, baud_transfer)
@ -742,7 +814,7 @@ class Stc12BaseProtocol(StcBaseProtocol):
        payload = StcBaseProtocol.extract_payload(self, packet)
        return payload[:-2]

-    def write_packet(self, data):
+    def write_packet(self, packet_data):
        """Send packet to MCU.

        Constructs a packet with supplied payload and sends it to the MCU.
@ -754,8 +826,8 @@ class Stc12BaseProtocol(StcBaseProtocol):
        packet += self.PACKET_HOST

        # packet length and payload
-        packet += struct.pack(">H", len(data) + 6)
-        packet += data
+        packet += struct.pack(">H", len(packet_data) + 6)
+        packet += packet_data

        # checksum and end code
        packet += struct.pack(">H", sum(packet[2:]) & 0xffff)
@ -765,16 +837,16 @@ class Stc12BaseProtocol(StcBaseProtocol):
        self.ser.write(packet)
        self.ser.flush()

-    def initialize_status(self, packet):
+    def initialize_status(self, status_packet):
        """Decode status packet and store basic MCU info"""

        freq_counter = 0
        for i in range(8):
-            freq_counter += struct.unpack(">H", packet[1+2*i:3+2*i])[0]
+            freq_counter += struct.unpack(">H", status_packet[1+2*i:3+2*i])[0]
        freq_counter /= 8.0
        self.mcu_clock_hz = (self.baud_handshake * freq_counter * 12.0) / 7.0

-        bl_version, bl_stepping = struct.unpack("BB", packet[17:19])
+        bl_version, bl_stepping = struct.unpack("BB", status_packet[17:19])
        self.mcu_bsl_version = "%d.%d%s" % (bl_version >> 4, bl_version & 0x0f,
                                            chr(bl_stepping))

@ -809,6 +881,9 @@ class Stc12BaseProtocol(StcBaseProtocol):
    def initialize_options(self, status_packet):
        """Initialize options"""

+        if len(status_packet) < 29:
+            raise StcProtocolException("invalid options in status packet")
+
        # create option state
        self.options = Stc12Option(status_packet[23:26] + status_packet[27:28])
        self.options.print()
@ -886,8 +961,6 @@ class Stc12BaseProtocol(StcBaseProtocol):
        as the block size (depends on MCU's RAM size).
        """

-        print("Writing %d bytes: " % len(data), end="")
-        sys.stdout.flush()
        for i in range(0, len(data), self.PROGRAM_BLOCKSIZE):
            packet = bytes(3)
            packet += struct.pack(">H", i)
@ -898,9 +971,8 @@ class Stc12BaseProtocol(StcBaseProtocol):
            response = self.read_packet()
            if response[0] != 0x00:
                raise StcProtocolException("incorrect magic in write packet")
-            print(".", end="")
-            sys.stdout.flush()
-        print(" done")
+            self.progress_cb(i, self.PROGRAM_BLOCKSIZE, len(data))
+        self.progress_cb(len(data), self.PROGRAM_BLOCKSIZE, len(data))

        print("Finishing write: ", end="")
        sys.stdout.flush()
@ -943,6 +1015,9 @@ class Stc15AProtocol(Stc12Protocol):
    def initialize_options(self, status_packet):
        """Initialize options"""

+        if len(status_packet) < 37:
+            raise StcProtocolException("invalid options in status packet")
+
        # create option state
        self.options = Stc15AOption(status_packet[23:36])
        self.options.print()
@ -961,20 +1036,20 @@ class Stc15AProtocol(Stc12Protocol):
            raise StcProtocolException("incorrect magic in status packet")
        return status_packet

-    def initialize_status(self, packet):
+    def initialize_status(self, status_packet):
        """Decode status packet and store basic MCU info"""

        freq_counter = 0
        for i in range(4):
-            freq_counter += struct.unpack(">H", packet[1+2*i:3+2*i])[0]
+            freq_counter += struct.unpack(">H", status_packet[1+2*i:3+2*i])[0]
        freq_counter /= 4.0
        self.mcu_clock_hz = (self.baud_handshake * freq_counter * 12.0) / 7.0

-        bl_version, bl_stepping = struct.unpack("BB", packet[17:19])
+        bl_version, bl_stepping = struct.unpack("BB", status_packet[17:19])
        self.mcu_bsl_version = "%d.%d%s" % (bl_version >> 4, bl_version & 0x0f,
                                            chr(bl_stepping))

-        self.trim_data = packet[51:58]
+        self.trim_data = status_packet[51:58]
        self.freq_counter = freq_counter

    def get_trim_sequence(self, frequency):
@ -1055,15 +1130,19 @@ class Stc15AProtocol(Stc12Protocol):
        self.write_packet(packet)
        self.pulse(timeout=1.0)
        response = self.read_packet()
-        if response[0] != 0x65:
+        if len(response) < 36 or response[0] != 0x65:
            raise StcProtocolException("incorrect magic in handshake packet")

        # determine programming speed trim value
        target_trim_a, target_count_a = struct.unpack(">HH", response[28:32])
        target_trim_b, target_count_b = struct.unpack(">HH", response[32:36])
+        if target_count_a == target_count_b:
+            raise StcProtocolException("frequency trimming failed")
        m = (target_trim_b - target_trim_a) / (target_count_b - target_count_a)
        n = target_trim_a - m * target_count_a
        program_trim = round(m * program_count + n)
+        if program_trim > 65535 or program_trim < 0:
+            raise StcProtocolException("frequency trimming failed")

        # determine trim trials for second round
        trim_a, count_a = struct.unpack(">HH", response[12:16])
@ -1082,10 +1161,14 @@ class Stc15AProtocol(Stc12Protocol):
            target_count_a = count_a
            target_count_b = count_b
        # linear interpolate to find range to try next
+        if target_count_a == target_count_b:
+            raise StcProtocolException("frequency trimming failed")
        m = (target_trim_b - target_trim_a) / (target_count_b - target_count_a)
        n = target_trim_a - m * target_count_a
        target_trim = round(m * user_count + n)
        target_trim_start = min(max(target_trim - 5, target_trim_a), target_trim_b)
+        if target_trim_start + 11 > 65535 or target_trim_start < 0:
+            raise StcProtocolException("frequency trimming failed")

        # trim challenge-response, second round
        packet = bytes([0x65])
@ -1097,7 +1180,7 @@ class Stc15AProtocol(Stc12Protocol):
        self.write_packet(packet)
        self.pulse(timeout=1.0)
        response = self.read_packet()
-        if response[0] != 0x65:
+        if len(response) < 56 or response[0] != 0x65:
            raise StcProtocolException("incorrect magic in handshake packet")

        # determine best trim value
@ -1156,34 +1239,38 @@ class Stc15Protocol(Stc15AProtocol):
    def initialize_options(self, status_packet):
        """Initialize options"""

+        if len(status_packet) < 14:
+            raise StcProtocolException("invalid options in status packet")
+
        # create option state
+        # XXX: check how option bytes are concatenated here
        self.options = Stc15Option(status_packet[5:8] + status_packet[12:13] + status_packet[37:38])
        self.options.print()

-    def initialize_status(self, packet):
+    def initialize_status(self, status_packet):
        """Decode status packet and store basic MCU info"""

        # check bit that control internal vs. external clock source
        # get frequency either stored from calibration or from
        # frequency counter
-        self.external_clock = (packet[7] & 0x01) == 0
+        self.external_clock = (status_packet[7] & 0x01) == 0
        if self.external_clock:
-            count, = struct.unpack(">H", packet[13:15])
+            count, = struct.unpack(">H", status_packet[13:15])
            self.mcu_clock_hz = self.baud_handshake * count
        else:
-            self.mcu_clock_hz, = struct.unpack(">I", packet[8:12])
+            self.mcu_clock_hz, = struct.unpack(">I", status_packet[8:12])
            # all ones means no calibration
            # new chips are shipped without any calibration
            if self.mcu_clock_hz == 0xffffffff: self.mcu_clock_hz = 0

        # pre-calibrated trim adjust for 24 MHz, range 0x40
-        self.freq_count_24 = packet[4]
+        self.freq_count_24 = status_packet[4]

        # wakeup timer factory value
-        self.wakeup_freq, = struct.unpack(">H", packet[1:3])
+        self.wakeup_freq, = struct.unpack(">H", status_packet[1:3])

-        bl_version, bl_stepping = struct.unpack("BB", packet[17:19])
-        bl_minor = packet[22] & 0x0f
+        bl_version, bl_stepping = struct.unpack("BB", status_packet[17:19])
+        bl_minor = status_packet[22] & 0x0f
        self.mcu_bsl_version = "%d.%d.%d%s" % (bl_version >> 4, bl_version & 0x0f,
                                               bl_minor, chr(bl_stepping))
        self.bsl_version = bl_version
@ -1201,6 +1288,8 @@ class Stc15Protocol(Stc15AProtocol):
        calib_data = response[2:]
        challenge_data = packet[2:]
        calib_len = response[1]
+        if len(calib_data) < 2 * calib_len:
+            raise StcProtocolException("range calibration data missing")

        for i in range(calib_len - 1):
            count_a, count_b = struct.unpack(">HH", calib_data[2*i:2*i+4])
@ -1210,6 +1299,8 @@ class Stc15Protocol(Stc15AProtocol):
                m = (trim_b - trim_a) / (count_b - count_a)
                n = trim_a - m * count_a
                target_trim = round(m * target_count + n)
+                if target_trim > 65536 or target_trim < 0:
+                    raise StcProtocolException("frequency trimming failed")
                return (target_trim, trim_range)

        return None
@ -1221,6 +1312,8 @@ class Stc15Protocol(Stc15AProtocol):
        calib_data = response[2:]
        challenge_data = packet[2:]
        calib_len = response[1]
+        if len(calib_data) < 2 * calib_len:
+            raise StcProtocolException("trim calibration data missing")

        best = None
        best_count = sys.maxsize
@ -1231,6 +1324,9 @@ class Stc15Protocol(Stc15AProtocol):
                best_count = abs(count - target_count)
                best = (trim_adj, trim_range), count

+        if not best:
+            raise StcProtocolException("frequency trimming failed")
+
        return best

    def calibrate(self):
@ -1260,13 +1356,13 @@ class Stc15Protocol(Stc15AProtocol):
        self.write_packet(packet)
        self.pulse(b"\xfe", timeout=1.0)
        response = self.read_packet()
-        if response[0] != 0x00:
+        if len(response) < 2 or response[0] != 0x00:
            raise StcProtocolException("incorrect magic in handshake packet")

        # select ranges and trim values
        user_trim = self.choose_range(packet, response, target_user_count)
        prog_trim = self.choose_range(packet, response, target_prog_count)
-        if user_trim == None or prog_trim == None:
+        if user_trim is None or prog_trim is None:
            raise StcProtocolException("frequency trimming unsuccessful")

        # calibration, round 2
@ -1279,12 +1375,12 @@ class Stc15Protocol(Stc15AProtocol):
        self.write_packet(packet)
        self.pulse(b"\xfe", timeout=1.0)
        response = self.read_packet()
-        if response[0] != 0x00:
+        if len(response) < 2 or response[0] != 0x00:
            raise StcProtocolException("incorrect magic in handshake packet")

        # select final values
        user_trim, user_count = self.choose_trim(packet, response, target_user_count)
-        prog_trim, prog_count = self.choose_trim(packet, response, target_prog_count)
+        prog_trim, _ = self.choose_trim(packet, response, target_prog_count)
        self.trim_value = user_trim
        self.trim_frequency = round(user_count * (self.baud_handshake / 2))
        print("%.03f MHz" % (self.trim_frequency / 1E6))
@ -1305,7 +1401,7 @@ class Stc15Protocol(Stc15AProtocol):
        packet += bytes([iap_wait])
        self.write_packet(packet)
        response = self.read_packet()
-        if response[0] != 0x01:
+        if len(response) < 1 or response[0] != 0x01:
            raise StcProtocolException("incorrect magic in handshake packet")
        time.sleep(0.2)
        self.ser.baudrate = self.baud_transfer
@ -1322,7 +1418,7 @@ class Stc15Protocol(Stc15AProtocol):
        packet += bytes([0x00, 0x00, iap_wait])
        self.write_packet(packet)
        response = self.read_packet()
-        if response[0] != 0x01:
+        if len(response) < 1 or response[0] != 0x01:
            raise StcProtocolException("incorrect magic in handshake packet")
        time.sleep(0.2)
        self.ser.baudrate = self.baud_transfer
@ -1348,9 +1444,9 @@ class Stc15Protocol(Stc15AProtocol):
            packet += bytes([0x00, 0x00, 0x5a, 0xa5])
        self.write_packet(packet)
        response = self.read_packet()
-        if response[0] == 0x0f:
+        if len(response) == 1 and response[0] == 0x0f:
            raise StcProtocolException("MCU is locked")
-        if response[0] != 0x05:
+        if len(response) < 1 or response[0] != 0x05:
            raise StcProtocolException("incorrect magic in handshake packet")

        print("done")
@ -1371,18 +1467,20 @@ class Stc15Protocol(Stc15AProtocol):
            packet += bytes([0x00, 0x5a, 0xa5])
        self.write_packet(packet)
        response = self.read_packet()
-        if response[0] != 0x03:
+        if len(response) < 1 or response[0] != 0x03:
            raise StcProtocolException("incorrect magic in handshake packet")
        print("done")

        if len(response) >= 8:
            self.uid = response[1:8]

+        # we should have a UID at this point
+        if not self.uid:
+            raise StcProtocolException("UID is missing")
+
    def program_flash(self, data):
        """Program the MCU's flash memory."""

-        print("Writing %d bytes: " % len(data), end="")
-        sys.stdout.flush()
        for i in range(0, len(data), self.PROGRAM_BLOCKSIZE):
            packet = bytes([0x22]) if i == 0 else bytes([0x02])
            packet += struct.pack(">H", i)
@ -1392,11 +1490,10 @@ class Stc15Protocol(Stc15AProtocol):
            while len(packet) < self.PROGRAM_BLOCKSIZE + 3: packet += b"\x00"
            self.write_packet(packet)
            response = self.read_packet()
-            if response[0] != 0x02 or response[1] != 0x54:
+            if len(response) < 2 or response[0] != 0x02 or response[1] != 0x54:
                raise StcProtocolException("incorrect magic in write packet")
-            print(".", end="")
-            sys.stdout.flush()
-        print(" done")
+            self.progress_cb(i, self.PROGRAM_BLOCKSIZE, len(data))
+        self.progress_cb(len(data), self.PROGRAM_BLOCKSIZE, len(data))

        # BSL 7.2+ needs a write finish packet according to dumps
        if self.bsl_version >= 0x72:
@ -1405,7 +1502,7 @@ class Stc15Protocol(Stc15AProtocol):
            packet = bytes([0x07, 0x00, 0x00, 0x5a, 0xa5])
            self.write_packet(packet)
            response = self.read_packet()
-            if response[0] != 0x07 or response[1] != 0x54:
+            if len(response) < 2 or response[0] != 0x07 or response[1] != 0x54:
                raise StcProtocolException("incorrect magic in finish packet")
            print("done")

@ -1444,13 +1541,152 @@ class Stc15Protocol(Stc15AProtocol):
        packet += self.build_options()
        self.write_packet(packet)
        response = self.read_packet()
-        if response[0] != 0x04 or response[1] != 0x54:
+        if len(response) < 2 or response[0] != 0x04 or response[1] != 0x54:
            raise StcProtocolException("incorrect magic in option packet")
        print("done")

        print("Target UID: %s" % Utils.hexstr(self.uid))


+class Stc8Protocol(Stc15Protocol):
+    """Protocol handler for STC8 series"""
+
+    def __init__(self, port, handshake, baud, trim):
+        Stc15Protocol.__init__(self, port, handshake, baud, trim)
+        self.trim_divider = None
+
+    def initialize_options(self, status_packet):
+        """Initialize options"""
+        if len(status_packet) < 17:
+            raise StcProtocolException("invalid options in status packet")
+
+        # create option state
+        self.options = Stc8Option(status_packet[9:12] + status_packet[15:17])
+        self.options.print()
+
+    def initialize_status(self, packet):
+        """Decode status packet and store basic MCU info"""
+
+        self.mcu_clock_hz, = struct.unpack(">I", packet[1:5])
+        # XXX: external clock not supported nor tested
+        self.external_clock = False
+        # all ones means no calibration
+        # new chips are shipped without any calibration
+        # XXX: somehow check if that still holds
+        if self.mcu_clock_hz == 0xffffffff: self.mcu_clock_hz = 0
+
+        # pre-calibrated trim adjust for 24 MHz, range 0x40
+        self.freq_count_24 = packet[4]
+
+        # wakeup timer factory value
+        self.wakeup_freq, = struct.unpack(">H", packet[23:25])
+
+        bl_version, bl_stepping = struct.unpack("BB", packet[17:19])
+        bl_minor = packet[22] & 0x0f
+        self.mcu_bsl_version = "%d.%d.%d%s" % (bl_version >> 4, bl_version & 0x0f,
+                                               bl_minor, chr(bl_stepping))
+        self.bsl_version = bl_version
+
+    def calibrate(self):
+        """Calibrate selected user frequency frequency and switch to selected baudrate."""
+
+        # handle uncalibrated chips
+        if self.mcu_clock_hz == 0 and self.trim_frequency <= 0:
+            raise StcProtocolException("uncalibrated, please provide a trim value")
+
+        # determine target counter
+        user_speed = self.trim_frequency
+        if user_speed <= 0: user_speed = self.mcu_clock_hz
+        target_user_count = round(user_speed / (self.baud_handshake/2))
+
+        # calibration, round 1
+        # XXX: challenges need work. the ranges and how they related to clock frequency
+        # is different. A clock divider is used for lower frequencies.
+        print("Trimming frequency: ", end="")
+        sys.stdout.flush()
+        packet = bytes([0x00])
+        packet += struct.pack(">B", 12)
+        packet += bytes([0x00, 0x00, 23*1, 0x00, 23*2, 0x00])
+        packet += bytes([23*3, 0x00, 23*4, 0x00, 23*5, 0x00])
+        packet += bytes([23*6, 0x00, 23*7, 0x00, 23*8, 0x00])
+        packet += bytes([23*9, 0x00, 23*10, 0x00, 255, 0x00])
+        self.write_packet(packet)
+        self.pulse(b"\xfe", timeout=1.0)
+        response = self.read_packet()
+        if len(response) < 2 or response[0] != 0x00:
+            raise StcProtocolException("incorrect magic in handshake packet")
+
+        # select ranges and trim values
+        for divider in (1, 2, 3, 4, 5):
+            user_trim = self.choose_range(packet, response, target_user_count * divider)
+            if user_trim is not None:
+                self.trim_divider = divider
+                break
+        if user_trim is None:
+            raise StcProtocolException("frequency trimming unsuccessful")
+
+        # calibration, round 2
+        packet = bytes([0x00])
+        packet += struct.pack(">B", 12)
+        for i in range(user_trim[0] - 1, user_trim[0] + 2):
+            packet += bytes([i & 0xff, 0x00])
+        for i in range(user_trim[0] - 1, user_trim[0] + 2):
+            packet += bytes([i & 0xff, 0x01])
+        for i in range(user_trim[0] - 1, user_trim[0] + 2):
+            packet += bytes([i & 0xff, 0x02])
+        for i in range(user_trim[0] - 1, user_trim[0] + 2):
+            packet += bytes([i & 0xff, 0x03])
+        self.write_packet(packet)
+        self.pulse(b"\xfe", timeout=1.0)
+        response = self.read_packet()
+        if len(response) < 2 or response[0] != 0x00:
+            raise StcProtocolException("incorrect magic in handshake packet")
+
+        # select final values
+        user_trim, user_count = self.choose_trim(packet, response, target_user_count)
+        self.trim_value = user_trim
+        self.trim_frequency = round(user_count * (self.baud_handshake / 2) / self.trim_divider)
+        print("%.03f MHz" % (self.trim_frequency / 1E6))
+
+        # switch to programming frequency
+        print("Switching to %d baud: " % self.baud_transfer, end="")
+        sys.stdout.flush()
+        packet = bytes([0x01, 0x00, 0x00])
+        bauds = self.baud_transfer * 4
+        packet += struct.pack(">H", int(65535 - 24E6 / bauds))
+        packet += bytes([user_trim[1], user_trim[0]])
+        iap_wait = self.get_iap_delay(24E6)
+        packet += bytes([iap_wait])
+        self.write_packet(packet)
+        response = self.read_packet()
+        if len(response) < 1 or response[0] != 0x01:
+            raise StcProtocolException("incorrect magic in handshake packet")
+        self.ser.baudrate = self.baud_transfer
+
+    def build_options(self):
+        """Build a packet of option data from the current configuration."""
+
+        msr = self.options.get_msr()
+        packet = 40 * bytearray([0xff])
+        packet[3] = 0
+        packet[6] = 0
+        packet[22] = 0
+        packet[24:28] = struct.pack(">I", self.trim_frequency)
+        packet[28:30] = self.trim_value
+        packet[30] = self.trim_divider
+        packet[32] = msr[0]
+        packet[36:40] = msr[1:5]
+        return bytes(packet)
+
+    def disconnect(self):
+        """Disconnect from MCU"""
+
+        # reset mcu
+        packet = bytes([0xff])
+        self.write_packet(packet)
+        self.ser.close()
+        print("Disconnected!")
+
 class StcUsb15Protocol(Stc15Protocol):
    """USB should use large blocks"""
    PROGRAM_BLOCKSIZE = 128
@ -1579,8 +1815,6 @@ class StcUsb15Protocol(Stc15Protocol):
    def program_flash(self, data):
        """Program the MCU's flash memory."""

-        print("Writing %d bytes: " % len(data), end="")
-        sys.stdout.flush()
        for i in range(0, len(data), self.PROGRAM_BLOCKSIZE):
            packet = data[i:i+self.PROGRAM_BLOCKSIZE]
            while len(packet) < self.PROGRAM_BLOCKSIZE: packet += b"\x00"
@ -1590,9 +1824,8 @@ class StcUsb15Protocol(Stc15Protocol):
            response = self.read_packet()
            if response[0] != 0x02 or response[1] != 0x54:
                raise StcProtocolException("incorrect magic in write packet")
-            print(".", end="")
-            sys.stdout.flush()
-        print(" done")
+            self.progress_cb(i, self.PROGRAM_BLOCKSIZE, len(data))
+        self.progress_cb(len(data), self.PROGRAM_BLOCKSIZE, len(data))

    def program_options(self):
        print("Setting options: ", end="")
--- a/stcgal/utils.py
+++ b/stcgal/utils.py
@ -29,16 +29,13 @@ class Utils:
    def to_bool(cls, val):
        """make sensible boolean from string or other type value"""

-        if val is None:
+        if not val:
            return False
        if isinstance(val, bool):
            return val
        elif isinstance(val, int):
            return bool(val)
-        elif len(val) == 0:
-            return False
-        else:
-            return True if val[0].lower() == "t" or val[0] == "1" else False
+        return True if val[0].lower() == "t" or val[0] == "1" else False

    @classmethod
    def to_int(cls, val):
@ -46,7 +43,7 @@ class Utils:

        try:
            return int(val, 0)
-        except:
+        except (TypeError, ValueError):
            raise ValueError("invalid integer")

    @classmethod
--- a/tests/init.py
+++ b/tests/init.py
--- a/tests/iap15f2k61s2.yml
+++ b/tests/iap15f2k61s2.yml
--- a/tests/stc12c2052ad.yml
+++ b/tests/stc12c2052ad.yml
--- a/tests/stc12c5a60s2.yml
+++ b/tests/stc12c5a60s2.yml
--- a/tests/stc15f104e.yml
+++ b/tests/stc15f104e.yml
--- a/tests/stc15l104w.yml
+++ b/tests/stc15l104w.yml
--- a/tests/stc15w4k56s4.yml
+++ b/tests/stc15w4k56s4.yml
--- a/tests/stc89c52rc.yml
+++ b/tests/stc89c52rc.yml
--- a/tests/test_fuzzing.py
+++ b/tests/test_fuzzing.py
@ -0,0 +1,119 @@
+#
+# Copyright (c) 2017 Grigori Goronzy <greg@chown.ath.cx>
+#
+# Permission is hereby granted, free of charge, to any person obtaining a copy
+# of this software and associated documentation files (the "Software"), to deal
+# in the Software without restriction, including without limitation the rights
+# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+# copies of the Software, and to permit persons to whom the Software is
+# furnished to do so, subject to the following conditions:
+#
+# The above copyright notice and this permission notice shall be included in all
+# copies or substantial portions of the Software.
+#
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+# SOFTWARE.
+#
+
+"""Tests with fuzzing of input data"""
+
+import random
+import sys
+import unittest
+from unittest.mock import patch
+import yaml
+import stcgal.frontend
+import stcgal.protocols
+from tests.test_program import get_default_opts, convert_to_bytes
+
+class ByteArrayFuzzer:
+    """Fuzzer for byte arrays"""
+
+    def __init__(self):
+        self.rng = random.Random()
+        self.cut_propability = 0.01 # probability for cutting off an array early
+        self.cut_min = 0 # minimum cut amount
+        self.cut_max = sys.maxsize # maximum cut amount
+        self.bitflip_probability = 0.0001 # probability for flipping a bit
+        self.randomize_probability = 0.001 # probability for randomizing a char
+
+    def fuzz(self, inp):
+        """Fuzz an array of bytes according to predefined settings"""
+        arr = bytearray(inp)
+        arr = self.cut_off(arr)
+        self.randomize(arr)
+        return bytes(arr)
+
+    def randomize(self, arr):
+        """Randomize array contents with bitflips and random bytes"""
+        for i, _ in enumerate(arr):
+            for j in range(8):
+                if self.rng.random() < self.bitflip_probability:
+                    arr[i] ^= (1 << j)
+            if self.rng.random() < self.randomize_probability:
+                arr[i] = self.rng.getrandbits(8)
+
+    def cut_off(self, arr):
+        """Cut off data from end of array"""
+        if self.rng.random() < self.cut_propability:
+            cut_limit = min(len(arr), self.cut_max)
+            cut_len = self.rng.randrange(self.cut_min, cut_limit)
+            arr = arr[0:len(arr) - cut_len]
+        return arr
+
+class TestProgramFuzzed(unittest.TestCase):
+    """Special programming cycle tests that use a fuzzing approach"""
+
+    @patch("stcgal.protocols.StcBaseProtocol.read_packet")
+    @patch("stcgal.protocols.Stc89Protocol.write_packet")
+    @patch("stcgal.protocols.serial.Serial", autospec=True)
+    @patch("stcgal.protocols.time.sleep")
+    @patch("sys.stdout")
+    @patch("sys.stderr")
+    def test_program_fuzz(self, err, out, sleep_mock, serial_mock, write_mock, read_mock):
+        """Test programming cycles with fuzzing enabled"""
+        yml = [
+            "./tests/iap15f2k61s2.yml",
+            "./tests/stc12c2052ad.yml",
+            "./tests/stc15w4k56s4.yml",
+            "./tests/stc12c5a60s2.yml",
+            "./tests/stc89c52rc.yml",
+            "./tests/stc15l104w.yml",
+            "./tests/stc15f104e.yml",
+        ]
+        fuzzer = ByteArrayFuzzer()
+        fuzzer.cut_propability = 0.01
+        fuzzer.bitflip_probability = 0.005
+        fuzzer.rng = random.Random(1)
+        for y in yml:
+            with self.subTest(msg="trace {}".format(y)):
+                self.single_fuzz(y, serial_mock, fuzzer, read_mock, err, out,
+                                 sleep_mock, write_mock)
+
+    def single_fuzz(self, yml, serial_mock, fuzzer, read_mock, err, out, sleep_mock, write_mock):
+        """Test a single programming cycle with fuzzing"""
+        with open(yml) as test_file:
+            test_data = yaml.load(test_file.read())
+            for _ in range(1000):
+                with self.subTest():
+                    opts = get_default_opts()
+                    opts.protocol = test_data["protocol"]
+                    opts.code_image.read.return_value = bytes(test_data["code_data"])
+                    serial_mock.return_value.inWaiting.return_value = 1
+                    fuzzed_responses = []
+                    for arr in convert_to_bytes(test_data["responses"]):
+                        fuzzed_responses.append(fuzzer.fuzz(arr))
+                    read_mock.side_effect = fuzzed_responses
+                    gal = stcgal.frontend.StcGal(opts)
+                    self.assertGreaterEqual(gal.run(), 0)
+                    err.reset_mock()
+                    out.reset_mock()
+                    sleep_mock.reset_mock()
+                    serial_mock.reset_mock()
+                    write_mock.reset_mock()
+                    read_mock.reset_mock()
--- a/tests/test_program.py
+++ b/tests/test_program.py
@ -57,7 +57,7 @@ class ProgramTests(unittest.TestCase):
    @patch("sys.stdout")
    def test_program_stc89(self, out, sleep_mock, serial_mock, write_mock, read_mock):
        """Test a programming cycle with STC89 protocol"""
-        self._program_yml("./test/stc89c52rc.yml", serial_mock, read_mock)
+        self._program_yml("./tests/stc89c52rc.yml", serial_mock, read_mock)

    @patch("stcgal.protocols.StcBaseProtocol.read_packet")
    @patch("stcgal.protocols.Stc89Protocol.write_packet")
@ -66,7 +66,7 @@ class ProgramTests(unittest.TestCase):
    @patch("sys.stdout")
    def test_program_stc12(self, out, sleep_mock, serial_mock, write_mock, read_mock):
        """Test a programming cycle with STC12 protocol"""
-        self._program_yml("./test/stc12c5a60s2.yml", serial_mock, read_mock)
+        self._program_yml("./tests/stc12c5a60s2.yml", serial_mock, read_mock)

    @patch("stcgal.protocols.StcBaseProtocol.read_packet")
    @patch("stcgal.protocols.Stc89Protocol.write_packet")
@ -75,7 +75,7 @@ class ProgramTests(unittest.TestCase):
    @patch("sys.stdout")
    def test_program_stc12a(self, out, sleep_mock, serial_mock, write_mock, read_mock):
        """Test a programming cycle with STC12A protocol"""
-        self._program_yml("./test/stc12c2052ad.yml", serial_mock, read_mock)
+        self._program_yml("./tests/stc12c2052ad.yml", serial_mock, read_mock)

    def test_program_stc12b(self):
        """Test a programming cycle with STC12B protocol"""
@ -88,7 +88,7 @@ class ProgramTests(unittest.TestCase):
    @patch("sys.stdout")
    def test_program_stc15f2(self, out, sleep_mock, serial_mock, write_mock, read_mock):
        """Test a programming cycle with STC15 protocol, F2 series"""
-        self._program_yml("./test/iap15f2k61s2.yml", serial_mock, read_mock)
+        self._program_yml("./tests/iap15f2k61s2.yml", serial_mock, read_mock)

    @patch("stcgal.protocols.StcBaseProtocol.read_packet")
    @patch("stcgal.protocols.Stc89Protocol.write_packet")
@ -97,7 +97,7 @@ class ProgramTests(unittest.TestCase):
    @patch("sys.stdout")
    def test_program_stc15w4(self, out, sleep_mock, serial_mock, write_mock, read_mock):
        """Test a programming cycle with STC15 protocol, W4 series"""
-        self._program_yml("./test/stc15w4k56s4.yml", serial_mock, read_mock)
+        self._program_yml("./tests/stc15w4k56s4.yml", serial_mock, read_mock)

    @unittest.skip("trace is broken")
    @patch("stcgal.protocols.StcBaseProtocol.read_packet")
@ -107,7 +107,7 @@ class ProgramTests(unittest.TestCase):
    @patch("sys.stdout")
    def test_program_stc15a(self, out, sleep_mock, serial_mock, write_mock, read_mock):
        """Test a programming cycle with STC15A protocol"""
-        self._program_yml("./test/stc15f104e.yml", serial_mock, read_mock)
+        self._program_yml("./tests/stc15f104e.yml", serial_mock, read_mock)

    @patch("stcgal.protocols.StcBaseProtocol.read_packet")
    @patch("stcgal.protocols.Stc89Protocol.write_packet")
@ -116,7 +116,7 @@ class ProgramTests(unittest.TestCase):
    @patch("sys.stdout")
    def test_program_stc15l1(self, out, sleep_mock, serial_mock, write_mock, read_mock):
        """Test a programming cycle with STC15 protocol, L1 series"""
-        self._program_yml("./test/stc15l104w.yml", serial_mock, read_mock)
+        self._program_yml("./tests/stc15l104w.yml", serial_mock, read_mock)

    def test_program_stc15w4_usb(self):
        """Test a programming cycle with STC15W4 USB protocol"""
@ -133,4 +133,3 @@ class ProgramTests(unittest.TestCase):
            read_mock.side_effect = convert_to_bytes(test_data["responses"])
            gal = stcgal.frontend.StcGal(opts)
            self.assertEqual(gal.run(), 0)
-    
--- a/tests/test_utils.py
+++ b/tests/test_utils.py
@ -24,7 +24,6 @@

 import argparse
 import unittest
-from unittest.mock import patch
 from stcgal.utils import Utils, BaudType

 class TestUtils(unittest.TestCase):
Author	SHA1	Message	Date
Grigori Goronzy	fe60e647bf	stc8: implement option handling Implement option handling for STC8 series, based on STC8A8K64S4A12 reverse engineering. This mostly wraps up all important parts of the STC8 implementation. Interoperability was tested with STC-ISP V6.86O. v2: update documentation	2018-08-21 02:01:39 +02:00
Grigori Goronzy	0ffcbd197b	stc8: finish up frequency calibration This is a collection of various changes: * Also try divider of 5 (it's used by STC-ISP) * Use the correct IAP delay for the 24 MHz programming frequency * Also try a trim adjust value of 0x03 and adjust trim ranges * Fix display of calibrated frequency	2018-08-20 23:47:42 +02:00
Grigori Goronzy	85e815366c	stc8: try dividers 2 and 4 for frequency calibration	2018-06-27 01:10:46 +02:00
Grigori Goronzy	8bc9d89257	Add preliminary STC8 support What works: * Frequency calibration of internal RC oscillator * Flash/EEPROM programming What doesn't work yet: * Everything else	2018-06-27 01:05:06 +02:00
Grigori Goronzy	b47092093e	Fix frontend argument parser Refactoring sneaked in an issue with the string formatting.	2018-06-27 01:03:22 +02:00
Grigori Goronzy	bc5e8ce6cb	setup: load long description from README.md	2018-06-27 00:10:41 +02:00
Grigori Goronzy	d9e71a8694	Misc style and consistency fixes (NFC) Various style, naming and other changes as recommended by pep8, pyflakes and pylint.	2018-06-27 00:05:16 +02:00
Grigori Goronzy	4dcde5cc49	Clean up imports (NFC)	2018-06-27 00:05:16 +02:00
Grigori Goronzy	3ec6f5b6bd	setup: improve metadata	2018-06-26 23:06:30 +02:00
Andrew Andrianov	eb6df3b645	debian: Update Build-Depends and Depends This closes #32 Signed-off-by: Andrew Andrianov <andrew@ncrmnt.org>	2018-06-26 22:14:41 +02:00
Grigori Goronzy	eb8eecbc9b	Merge pull request #38 from zerog2k/master stc15f204ea should use protocol stc15a	2018-02-04 18:13:22 +01:00
Jens J	c71e455f16	stc15f204ea should use protocol stc15a	2018-02-04 09:46:10 -06:00
Grigori Goronzy	a5e1cc26ee	Merge pull request #31 from nekromant/progressbar Implement progress callback and tqdm progressbar	2017-10-22 15:55:45 +02:00
Andrew Andrianov	b77157bc40	.travis.yml: Install tqdm to make ci happy Signed-off-by: Andrew Andrianov <andrew@ncrmnt.org>	2017-10-19 11:26:42 +03:00
Andrew Andrianov	092fbdc842	protocols.py: Implement progress callback and tqdm progressbar Signed-off-by: Andrew Andrianov <andrew@ncrmnt.org>	2017-10-19 11:26:27 +03:00
Grigori Goronzy	e0bda73fed	Merge pull request #29 from grigorig/advanced-tests Advanced tests	2017-10-18 23:22:40 +02:00
Grigori Goronzy	57100062af	Rename test/ to tests/	2017-10-12 23:02:02 +02:00
Grigori Goronzy	030497beb0	Extract StcAutoProtocol class, fix autodetection With the introduction of real abstract classes, it is not possible anymore to instantiate StcBaseProtocol. Instead, extract some of the code for autodetection into the new class StcAutoProtocol and use that for autodetection.	2017-10-12 23:02:02 +02:00
Grigori Goronzy	fd923f3a92	Cleanup utils Just a tiny simplification, found by pylint.	2017-10-12 23:02:02 +02:00
Grigori Goronzy	b145fb364a	Remove unneeded include	2017-10-12 23:02:02 +02:00
Grigori Goronzy	a29c9bf42e	Add fuzzing programming cycle tests	2017-10-12 23:02:02 +02:00
Grigori Goronzy	1cde6da007	stc15: check that a UID has been received Found by fuzzing. In some cases it's possible that we end up without a valid UID. Detect and workaround.	2017-10-12 23:02:02 +02:00
Grigori Goronzy	ca30a508aa	Fix various issues in frequency trimming Found by fuzzing. The frequency trimming functions did a bad job of checking for possible out of bounds accesses and didn't handle various failure cases correctly. Add suitable checks to fix the issues found. v2: fix one check, add several new ones	2017-10-12 23:01:50 +02:00
Grigori Goronzy	b9208c4772	Add length checks for status packets Fuzzing found a number of issues when status packets are cut short. Introduce checks on the length of status packets to fix these issues.	2017-10-11 23:20:20 +02:00
Grigori Goronzy	ad5a89297f	Check length of responses Fuzzing found lots of issues when packets are cut short. This should rarely happen, but stcgal should be able to handle it without crashing. This adds length checks when checking the magic of packets or when checking checksums.	2017-10-11 23:20:20 +02:00
Grigori Goronzy	0cb56f4919	Use abc for StcBaseProtocol Use the abc module to declare StcBaseProtocol as an abstract base class and clean up imports while at it.	2017-10-11 23:20:20 +02:00
Grigori Goronzy	f195258eb5	Clean up options utilities Use abc to declare an abstract base class and add some documentation.	2017-10-11 23:20:20 +02:00
Grigori Goronzy	ff9530833d	Update gitignore	2017-10-11 23:19:11 +02:00
Grigori Goronzy	8b0fdcb42a	Clean up Intel HEX utilities No functional change intended.	2017-10-11 23:18:59 +02:00