Rename test/ to tests/

With the introduction of real abstract classes, it is not possible
anymore to instantiate StcBaseProtocol. Instead, extract some of the
code for autodetection into the new class StcAutoProtocol and use
that for autodetection.

.gitignore

@@ -1,7 +1,12 @@
 *~
 *.pyc
 *.egg-info
-__pycache__
+*.eggs/
+*.pybuild/
+__pycache__/
 /build
 /dist
 /deb_dist
+/debian/stcgal*
+/debian/files
+/.vscode

.travis.yml

@@ -0,0 +1,33 @@
+sudo: required
+dist: trusty
+language: python
+cache:
+  - pip
+python:
+  - "3.4"
+  - "3.5"
+  - "3.6"
+  - "pypy3"
+before_install:
+  - sudo apt install rpm dpkg-dev debhelper dh-python python3-setuptools fakeroot python3-serial python3-yaml
+install:
+  - pip install pyserial pyusb
+script:
+  - python setup.py build
+  - python setup.py test
+before_deploy:
+  - deactivate
+  - python3 setup.py bdist_rpm
+  - dpkg-buildpackage -uc -us
+  - cp ../*.deb dist/
+deploy:
+  provider: releases
+  api_key: $GH_TOKEN
+  file_glob: true
+  file:
+  - dist/stcgal*_all.deb
+  - dist/stcgal*.noarch.rpm
+  skip_cleanup: true
+  on:
+    tags: true
+    python: "3.4"

README.md

@@ -1,3 +1,5 @@
+[![Build Status](https://travis-ci.org/grigorig/stcgal.svg)](https://travis-ci.org/grigorig/stcgal)
+
 stcgal - STC MCU ISP flash tool
 ===============================
 
@@ -25,14 +27,14 @@ stcgal should fully support STC 89/90/10/11/12/15 series MCUs.
 
 So far, stcgal was tested with the following MCU models:
 
-* STC89C52RC (BSL version: 4.3C)
+* STC89C52RC (BSL version: 4.3C/6.6C)
 * STC90C52RC (BSL version: 4.3C)
 * STC89C54RD+ (BSL version: 4.3C)
 * STC12C2052 (BSL version: 5.8D)
 * STC12C2052AD (BSL version: 5.8D)
 * STC12C5608AD (BSL version: 6.0G)
 * STC12C5A16S2 (BSL version: 6.2I)
-* STC12C5A60S2 (BSL version: 6.2I)
+* STC12C5A60S2 (BSL version: 6.2I/7.1I)
 * STC11F02E (BSL version: 6.5K)
 * STC10F04XE (BSL version: 6.5J)
 * STC11F08XE (BSL version: 6.5M)
@@ -59,7 +61,7 @@ Features
 * Set device options
 * Read unique device ID (STC 10/11/12/15)
 * Trim RC oscillator frequency (STC 15)
-* Automatic power-cycling with DTR toggle
+* Automatic power-cycling with DTR toggle or a custom shell command
 * Automatic UART protocol detection
 
 Installation
@@ -94,6 +96,9 @@ positional arguments:
 optional arguments:
   -h, --help            show this help message and exit
   -a, --autoreset       cycle power automatically by asserting DTR
+  -r RESETCMD, --resetcmd RESETCMD
+                        Use this shell command for board power-cycling
+                        (instead of DTR assertion)
   -P {stc89,stc12a,stc12,stc15a,stc15,auto}, --protocol {stc89,stc12a,stc12,stc15a,stc15,auto}
                         protocol version
   -p PORT, --port PORT  serial port device
@@ -274,7 +279,22 @@ serial interface to automate this. The DTR signal is asserted for
 approximately 500 ms when the autoreset feature is enabled with the
 ```-a``` flag. This requires external circuitry to actually switch the
 power. In some cases, when the microcontroller draws only little power,
-it is possible to directly supply power from the DTR signal, however.
+it is possible to directly supply power from the DTR signal.
+
+As an alternative to DTR, you can use a custom shell command or an external
+script (via -r option) to reset the  device. You should specify the command
+along with -a option. Do not forget the quotes!
+
+Example:
+
+```
+  $ ./stcgal.py -P stc15 -a -r "echo 1 > /sys/class/gpio/gpio666/value"
+```
+or
+
+```
+  $ ./stcgal.py -P stc15 -a -r "./powercycle.sh"
+```
 
 ### Exit status
 

debian/changelog

@@ -1,3 +1,15 @@
+stcgal (1.4) unstable; urgency=low
+
+  * Update to 1.4
+
+ -- Grigori <greg@chown.ath.cx>  Tue, 19 Sep 2017 17:57:11 +0200
+
+stcgal (1.3) unstable; urgency=low
+
+  * Update to 1.3
+
+ -- Grigori Goronzy <greg@chown.ath.cx>  Sat, 10 Jun 2017 10:01:07 +0200
+
 stcgal (1.2) unstable; urgency=low
 
   * Update to 1.2

doc/stc15-usb-protocol.txt

@@ -0,0 +1,46 @@
+STC15 series USB ISP protocol
+=============================
+
+General principle
+-----------------
+
+- host does OUT and IN control transfers for write and read
+- IN transfer with wLength = 132, wValue = 0, wIndex = 0, bRequest = 0 are used for all reads
+- OUT transfers with with specific bRequest, wValue, wIndex are used for writes
+
+
+Packet coding
+-------------
+
+- packets from MCU
+  always start with 0x46 0xb9, similar to serial protocols
+  third byte is packet length, followed by data bytes
+  checksum at the end: 8 bit modular sum
+
+- packets from host
+  no header bytes
+  bRequest sets packet type
+  wValue, wIndex interpretation according to packet type
+  8 bit modular checksum for every 7 bytes, interleaved
+  
+- packet types derived from the serial protocol
+
+Specific packet information
+---------------------------
+
+- flash data
+  wIndex specifies write address
+  wValue is 0xa55a
+  bRequest is 0x22 for first packet, 0x02 for the following ones
+  unusually encoded: a total of 128 bytes per packet,
+  with every 7 byte checksummed in some way,
+  for a total of 18x7 byte segments and a final 2 byte segment
+  checksum: 8 bit modular sum
+
+- option packet
+  generally same as with serial protocol, some header stuff omitted
+  wIndex is 0
+  wValue is 0xa55a
+  bRequest is 4
+  seems to use the same checksumming scheme as flash writes
+

setup.py

@@ -27,7 +27,7 @@ from setuptools import setup, find_packages
 setup(
     name = "stcgal",
     version = stcgal.__version__,
-    packages = find_packages(exclude=["doc"]),
+    packages = find_packages(exclude=["doc", "tests"]),
     install_requires = ["pyserial"],
     extras_require = {
         "usb": ["pyusb>=1.0.0"]
@@ -55,4 +55,6 @@ setup(
         "Topic :: Software Development :: Embedded Systems",
         "Topic :: Software Development",
     ],
+    test_suite = "tests",
+    tests_require = ["PyYAML"],
 )

stcgal/__init__.py

@@ -1 +1 @@
-__version__ = "1.2"
+__version__ = "1.4"

stcgal/__main__.py

@@ -0,0 +1,27 @@
+#
+# Copyright (c) 2013-2015 Grigori Goronzy <greg@chown.ath.cx>
+#
+# Permission is hereby granted, free of charge, to any person obtaining a copy
+# of this software and associated documentation files (the "Software"), to deal
+# in the Software without restriction, including without limitation the rights
+# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+# copies of the Software, and to permit persons to whom the Software is
+# furnished to do so, subject to the following conditions:
+#
+# The above copyright notice and this permission notice shall be included in all
+# copies or substantial portions of the Software.
+#
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+# SOFTWARE.
+#
+
+import sys
+import stcgal.frontend
+
+if __name__ == "__main__":
+    sys.exit(stcgal.frontend.cli())

stcgal/frontend.py

@@ -20,10 +20,10 @@
 # SOFTWARE.
 #
 
-import sys, os, time, struct
+import sys
 import argparse
 import stcgal
-from stcgal.utils import Utils, BaudType
+from stcgal.utils import BaudType
 from stcgal.protocols import *
 from stcgal.ihex import IHex
 
@@ -32,7 +32,10 @@ class StcGal:
 
     def __init__(self, opts):
         self.opts = opts
+        self.initialize_protocol(opts)
 
+    def initialize_protocol(self, opts):
+        """Initialize protocol backend"""
         if opts.protocol == "stc89":
             self.protocol = Stc89Protocol(opts.port, opts.handshake, opts.baud)
         elif opts.protocol == "stc12a":
@@ -50,18 +53,20 @@ class StcGal:
         elif opts.protocol == "usb15":
             self.protocol = StcUsb15Protocol()
         else:
-            self.protocol = StcBaseProtocol(opts.port, opts.handshake, opts.baud)
-
+            self.protocol = StcAutoProtocol(opts.port, opts.handshake, opts.baud)
         self.protocol.debug = opts.debug
 
     def emit_options(self, options):
-        for o in options:
+        """Set options from command line to protocol handler."""
+
+        for opt in options:
             try:
-                kv = o.split("=", 1)
-                if len(kv) < 2: raise ValueError("incorrect format")
+                kv = opt.split("=", 1)
+                if len(kv) < 2:
+                    raise ValueError("incorrect format")
                 self.protocol.set_option(kv[0], kv[1])
-            except ValueError as e:
-                raise NameError("invalid option '%s' (%s)" % (kv[0], e))
+            except ValueError as ex:
+                raise NameError("invalid option '%s' (%s)" % (kv[0], ex))
 
     def load_file_auto(self, fileobj):
         """Load file with Intel Hex autodetection."""
@@ -74,14 +79,16 @@ class StcGal:
                 binary = hexfile.extract_data()
                 print("%d bytes (Intel HEX)" %len(binary))
                 return binary
-            except ValueError as e:
-                raise IOError("invalid Intel HEX file (%s)" %e)
+            except ValueError as ex:
+                raise IOError("invalid Intel HEX file (%s)" %ex)
         else:
             binary = fileobj.read()
             print("%d bytes (Binary)" %len(binary))
             return binary
 
     def program_mcu(self):
+        """Execute the standard programming flow."""
+
         code_size = self.protocol.model.code
         ee_size = self.protocol.model.eeprom
 
@@ -124,37 +131,38 @@ class StcGal:
         self.protocol.disconnect()
 
     def run(self):
-        try:
-            self.protocol.connect(autoreset=self.opts.autoreset)
+        """Run programmer, main entry point."""
 
-            if self.opts.protocol == "auto":
+        try:
+            self.protocol.connect(autoreset=self.opts.autoreset, resetcmd=self.opts.resetcmd)
+            if isinstance(self.protocol, StcAutoProtocol):
                 if not self.protocol.protocol_name:
                     raise StcProtocolException("cannot detect protocol")
                 base_protocol = self.protocol
                 self.opts.protocol = self.protocol.protocol_name
                 print("Protocol detected: %s" % self.opts.protocol)
                 # recreate self.protocol with proper protocol class
-                self.__init__(self.opts)
+                self.initialize_protocol(self.opts)
             else:
                 base_protocol = None
 
             self.protocol.initialize(base_protocol)
         except KeyboardInterrupt:
-            sys.stdout.flush();
+            sys.stdout.flush()
             print("interrupted")
             return 2
-        except (StcFramingException, StcProtocolException) as e:
-            sys.stdout.flush();
-            print("Protocol error: %s" % e, file=sys.stderr)
+        except (StcFramingException, StcProtocolException) as ex:
+            sys.stdout.flush()
+            print("Protocol error: %s" % ex, file=sys.stderr)
             self.protocol.disconnect()
             return 1
-        except serial.SerialException as e:
-            sys.stdout.flush();
-            print("Serial port error: %s" % e, file=sys.stderr)
+        except serial.SerialException as ex:
+            sys.stdout.flush()
+            print("Serial port error: %s" % ex, file=sys.stderr)
             return 1
-        except IOError as e:
-            sys.stdout.flush();
-            print("I/O error: %s" % e, file=sys.stderr)
+        except IOError as ex:
+            sys.stdout.flush()
+            print("I/O error: %s" % ex, file=sys.stderr)
             return 1
 
         try:
@@ -164,27 +172,27 @@ class StcGal:
             else:
                 self.protocol.disconnect()
                 return 0
-        except NameError as e:
-            sys.stdout.flush();
-            print("Option error: %s" % e, file=sys.stderr)
+        except NameError as ex:
+            sys.stdout.flush()
+            print("Option error: %s" % ex, file=sys.stderr)
             self.protocol.disconnect()
             return 1
-        except (StcFramingException, StcProtocolException) as e:
-            sys.stdout.flush();
-            print("Protocol error: %s" % e, file=sys.stderr)
+        except (StcFramingException, StcProtocolException) as ex:
+            sys.stdout.flush()
+            print("Protocol error: %s" % ex, file=sys.stderr)
             self.protocol.disconnect()
             return 1
         except KeyboardInterrupt:
-            sys.stdout.flush();
+            sys.stdout.flush()
             print("interrupted", file=sys.stderr)
             self.protocol.disconnect()
             return 2
-        except serial.SerialException as e:
-            print("Serial port error: %s" % e, file=sys.stderr)
+        except serial.SerialException as ex:
+            print("Serial port error: %s" % ex, file=sys.stderr)
             return 1
-        except IOError as e:
-            sys.stdout.flush();
-            print("I/O error: %s" % e, file=sys.stderr)
+        except IOError as ex:
+            sys.stdout.flush()
+            print("I/O error: %s" % ex, file=sys.stderr)
             self.protocol.disconnect()
             return 1
 
@@ -192,10 +200,11 @@ class StcGal:
 def cli():
     # check arguments
     parser = argparse.ArgumentParser(formatter_class=argparse.RawDescriptionHelpFormatter,
-                                     description="stcgal %s - an STC MCU ISP flash tool\n(C) 2014-2015 Grigori Goronzy\nhttps://github.com/grigorig/stcgal" %stcgal.__version__)
+                                     description="stcgal %s - an STC MCU ISP flash tool\n(C) 2014-2017 Grigori Goronzy\nhttps://github.com/grigorig/stcgal" %stcgal.__version__)
     parser.add_argument("code_image", help="code segment file to flash (BIN/HEX)", type=argparse.FileType("rb"), nargs='?')
     parser.add_argument("eeprom_image", help="eeprom segment file to flash (BIN/HEX)", type=argparse.FileType("rb"), nargs='?')
     parser.add_argument("-a", "--autoreset", help="cycle power automatically by asserting DTR", action="store_true")
+    parser.add_argument("-r", "--resetcmd",  help="Use this shell command for board power-cycling (instead of DTR assertion)", action="store")
     parser.add_argument("-P", "--protocol", help="protocol version (default: auto)", choices=["stc89", "stc12a", "stc12b", "stc12", "stc15a", "stc15", "usb15", "auto"], default="auto")
     parser.add_argument("-p", "--port", help="serial port device", default="/dev/ttyUSB0")
     parser.add_argument("-b", "--baud", help="transfer baud rate (default: 19200)", type=BaudType(), default=19200)

stcgal/ihex.py

@@ -5,201 +5,214 @@
 import struct
 import codecs
 
-class IHex(object):
-  @classmethod
-  def read(cls, lines):
-    ihex = cls()
 
-    segbase = 0
-    for line in lines:
-      line = line.strip()
-      if not line: continue
+class IHex:
+    """Intel HEX parser and writer"""
 
-      t, a, d = ihex.parse_line(line)
-      if t == 0x00:
-        ihex.insert_data(segbase + a, d)
+    @classmethod
+    def read(cls, lines):
+        """Read Intel HEX data from string or lines"""
+        ihex = cls()
 
-      elif t == 0x01:
-        break # Should we check for garbage after this?
+        segbase = 0
+        for line in lines:
+            line = line.strip()
+            if not line:
+                continue
 
-      elif t == 0x02:
-        ihex.set_mode(16)
-        segbase = struct.unpack(">H", d[0:2])[0] << 4
+            t, a, d = ihex.parse_line(line)
+            if t == 0x00:
+                ihex.insert_data(segbase + a, d)
 
-      elif t == 0x03:
-        ihex.set_mode(16)
+            elif t == 0x01:
+                break  # Should we check for garbage after this?
 
-        cs, ip = struct.unpack(">2H", d[0:2])
-        ihex.set_start((cs, ip))
+            elif t == 0x02:
+                ihex.set_mode(16)
+                segbase = struct.unpack(">H", d[0:2])[0] << 4
 
-      elif t == 0x04:
-        ihex.set_mode(32)
-        segbase = struct.unpack(">H", d[0:2])[0] << 16
+            elif t == 0x03:
+                ihex.set_mode(16)
 
-      elif t == 0x05:
-        ihex.set_mode(32)
-        ihex.set_start(struct.unpack(">I", d[0:4])[0])
+                cs, ip = struct.unpack(">2H", d[0:2])
+                ihex.set_start((cs, ip))
 
-      else:
-        raise ValueError("Invalid type byte")
+            elif t == 0x04:
+                ihex.set_mode(32)
+                segbase = struct.unpack(">H", d[0:2])[0] << 16
 
-    return ihex
+            elif t == 0x05:
+                ihex.set_mode(32)
+                ihex.set_start(struct.unpack(">I", d[0:4])[0])
 
-  @classmethod
-  def read_file(cls, fname):
-    f = open(fname, "rb")
-    ihex = cls.read(f)
-    f.close()
-    return ihex
+            else:
+                raise ValueError("Invalid type byte")
 
-  def __init__(self):
-    self.areas = {}
-    self.start = None
-    self.mode = 8
-    self.row_bytes = 16
+        return ihex
 
-  def set_row_bytes(self, row_bytes):
-      """Set output hex file row width (bytes represented per row)."""
-      if row_bytes < 1 or row_bytes > 0xff:
-          raise ValueError("Value out of range: (%r)" % row_bytes)
-      self.row_bytes = row_bytes
-  
-  def extract_data(self, start=None, end=None):
-    if start is None:
-      start = 0
-    
-    if end is None:
-      result = bytearray()
-      
-      for addr, data in self.areas.items():
-        if addr >= start:
-          if len(result) < (addr - start):
-            result[len(result):addr-start] = bytes(addr-start-len(result))
-          result[addr-start:addr-start+len(data)] = data
-      
-      return bytes(result)
-    
-    else:
-      result = bytearray()
-      
-      for addr, data in self.areas.items():
-        if addr >= start and addr < end:
-          data = data[:end-addr]
-          if len(result) < (addr - start):
-            result[len(result):addr-start] = bytes(addr-start-len(result))
-          result[addr-start:addr-start+len(data)] = data
-      
-      return bytes(result)
-  
-  def set_start(self, start=None):
-    self.start = start
+    @classmethod
+    def read_file(cls, fname):
+        """Read Intel HEX data from file"""
+        f = open(fname, "rb")
+        ihex = cls.read(f)
+        f.close()
+        return ihex
 
-  def set_mode(self, mode):
-    self.mode = mode
+    def __init__(self):
+        self.areas = {}
+        self.start = None
+        self.mode = 8
+        self.row_bytes = 16
 
-  def get_area(self, addr):
-    for start, data in self.areas.items():
-      end = start + len(data)
-      if addr >= start and addr <= end:
-        return start
+    def set_row_bytes(self, row_bytes):
+        """Set output hex file row width (bytes represented per row)."""
+        if row_bytes < 1 or row_bytes > 0xff:
+            raise ValueError("Value out of range: (%r)" % row_bytes)
+        self.row_bytes = row_bytes
 
-    return None
+    def extract_data(self, start=None, end=None):
+        """Extract binary data"""
+        if start is None:
+            start = 0
 
-  def insert_data(self, istart, idata):
-    iend = istart + len(idata)
+        if end is None:
+            result = bytearray()
 
-    area = self.get_area(istart)
-    if area is None:
-      self.areas[istart] = idata
+            for addr, data in self.areas.items():
+                if addr >= start:
+                    if len(result) < (addr - start):
+                        result[len(result):addr - start] = bytes(
+                            addr - start - len(result))
+                    result[addr - start:addr - start + len(data)] = data
 
-    else:
-      data = self.areas[area]
-      # istart - iend + len(idata) + len(data)
-      self.areas[area] = data[:istart-area] + idata + data[iend-area:]
+            return bytes(result)
 
-  def calc_checksum(self, bytes):
-    total = sum(bytes)
-    return (-total) & 0xFF
+        else:
+            result = bytearray()
 
-  def parse_line(self, rawline):
-    if rawline[0:1] != b":":
-      raise ValueError("Invalid line start character (%r)" % rawline[0])
+            for addr, data in self.areas.items():
+                if addr >= start and addr < end:
+                    data = data[:end - addr]
+                    if len(result) < (addr - start):
+                        result[len(result):addr - start] = bytes(
+                            addr - start - len(result))
+                    result[addr - start:addr - start + len(data)] = data
 
-    try:
-      #line = rawline[1:].decode("hex")
-      line = codecs.decode(rawline[1:], "hex_codec")
-    except:
-      raise ValueError("Invalid hex data")
+            return bytes(result)
 
-    length, addr, type = struct.unpack(">BHB", line[:4])
+    def set_start(self, start=None):
+        self.start = start
 
-    dataend = length + 4
-    data = line[4:dataend]
+    def set_mode(self, mode):
+        self.mode = mode
 
-    #~ print line[dataend:dataend + 2], repr(line)
-    cs1 = line[dataend]
-    cs2 = self.calc_checksum(line[:dataend])
+    def get_area(self, addr):
+        for start, data in self.areas.items():
+            end = start + len(data)
+            if addr >= start and addr <= end:
+                return start
 
-    if cs1 != cs2:
-      raise ValueError("Checksums do not match")
+        return None
 
-    return (type, addr, data)
+    def insert_data(self, istart, idata):
+        iend = istart + len(idata)
 
-  def make_line(self, type, addr, data):
-    line = struct.pack(">BHB", len(data), addr, type)
-    line += data
-    line += chr(self.calc_checksum(line))
-    #~ return ":" + line.encode("hex")
-    return ":" + line.encode("hex").upper() + "\r\n"
+        area = self.get_area(istart)
+        if area is None:
+            self.areas[istart] = idata
 
-  def write(self):
-    output = ""
-    
-    for start, data in sorted(self.areas.items()):
-      i = 0
-      segbase = 0
+        else:
+            data = self.areas[area]
+            # istart - iend + len(idata) + len(data)
+            self.areas[area] = data[
+                :istart - area] + idata + data[iend - area:]
 
-      while i < len(data):
-        chunk = data[i:i + self.row_bytes]
+    def calc_checksum(self, data):
+        total = sum(data)
+        return (-total) & 0xFF
 
-        addr = start
-        newsegbase = segbase
+    def parse_line(self, rawline):
+        if rawline[0:1] != b":":
+            raise ValueError("Invalid line start character (%r)" % rawline[0])
 
-        if self.mode == 8:
-          addr = addr & 0xFFFF
+        try:
+            line = codecs.decode(rawline[1:], "hex_codec")
+        except:
+            raise ValueError("Invalid hex data")
 
-        elif self.mode == 16:
-          t = addr & 0xFFFF
-          newsegbase = (addr - t) >> 4
-          addr = t
+        length, addr, line_type = struct.unpack(">BHB", line[:4])
 
-          if newsegbase != segbase:
-            output += self.make_line(0x02, 0, struct.pack(">H", newsegbase))
-            segbase = newsegbase
+        dataend = length + 4
+        data = line[4:dataend]
 
-        elif self.mode == 32:
-          newsegbase = addr >> 16
-          addr = addr & 0xFFFF
+        cs1 = line[dataend]
+        cs2 = self.calc_checksum(line[:dataend])
 
-          if newsegbase != segbase:
-            output += self.make_line(0x04, 0, struct.pack(">H", newsegbase))
-            segbase = newsegbase
+        if cs1 != cs2:
+            raise ValueError("Checksums do not match")
 
-        output += self.make_line(0x00, addr, chunk)
+        return (line_type, addr, data)
 
-        i += self.row_bytes
-        start += self.row_bytes
+    def make_line(self, line_type, addr, data):
+        line = struct.pack(">BHB", len(data), addr, line_type)
+        line += data
+        line += chr(self.calc_checksum(line))
+        return ":" + line.encode("hex").upper() + "\r\n"
 
-    if self.start is not None:
-      if self.mode == 16:
-        output += self.make_line(0x03, 0, struct.pack(">2H", self.start[0], self.start[1]))
-      elif self.mode == 32:
-        output += self.make_line(0x05, 0, struct.pack(">I", self.start))
+    def write(self):
+        """Write Intel HEX data to string"""
+        output = ""
 
-    output += self.make_line(0x01, 0, "")
-    return output
+        for start, data in sorted(self.areas.items()):
+            i = 0
+            segbase = 0
 
-  def write_file(self, fname):
-    f = open(fname, "w")
-    f.write(self.write())
-    f.close()
+            while i < len(data):
+                chunk = data[i:i + self.row_bytes]
+
+                addr = start
+                newsegbase = segbase
+
+                if self.mode == 8:
+                    addr = addr & 0xFFFF
+
+                elif self.mode == 16:
+                    t = addr & 0xFFFF
+                    newsegbase = (addr - t) >> 4
+                    addr = t
+
+                    if newsegbase != segbase:
+                        output += self.make_line(
+                            0x02, 0, struct.pack(">H", newsegbase))
+                        segbase = newsegbase
+
+                elif self.mode == 32:
+                    newsegbase = addr >> 16
+                    addr = addr & 0xFFFF
+
+                    if newsegbase != segbase:
+                        output += self.make_line(
+                            0x04, 0, struct.pack(">H", newsegbase))
+                        segbase = newsegbase
+
+                output += self.make_line(0x00, addr, chunk)
+
+                i += self.row_bytes
+                start += self.row_bytes
+
+        if self.start is not None:
+            if self.mode == 16:
+                output += self.make_line(
+                    0x03, 0, struct.pack(">2H", self.start[0], self.start[1]))
+            elif self.mode == 32:
+                output += self.make_line(
+                    0x05, 0, struct.pack(">I", self.start))
+
+        output += self.make_line(0x01, 0, "")
+        return output
+
+    def write_file(self, fname):
+        """Write Intel HEX data to file"""
+        f = open(fname, "w")
+        f.write(self.write())
+        f.close()

stcgal/models.py

@@ -973,6 +973,46 @@ class MCUModelDatabase:
         MCUModel(name='STC90LE513AD', magic=0xf18d, total=65536, code=53248, eeprom=10240),
         MCUModel(name='STC90LE514AD', magic=0xf18e, total=65536, code=57344, eeprom=6144),
         MCUModel(name='STC90LE516AD', magic=0xf190, total=65536, code=63488, eeprom=0),
+
+        # Warning, these definitions lack a valid eeprom size.
+        MCUModel(name='STC15F04AD', magic=0xd444, total=4096, code=4096, eeprom=0),
+        MCUModel(name='STC15F06AD', magic=0xd446, total=6144, code=6144, eeprom=0),
+        MCUModel(name='STC15F08AD', magic=0xd448, total=8192, code=8192, eeprom=0),
+        MCUModel(name='STC15F10AD', magic=0xd44a, total=10240, code=10240, eeprom=0),
+        MCUModel(name='STC15F12AD', magic=0xd44c, total=12288, code=12288, eeprom=0),
+        MCUModel(name='STC15F04CCP', magic=0xd434, total=4096, code=4096, eeprom=0),
+        MCUModel(name='STC15F06CCP', magic=0xd436, total=6144, code=6144, eeprom=0),
+        MCUModel(name='STC15F08CCP', magic=0xd438, total=8192, code=8192, eeprom=0),
+        MCUModel(name='STC15F10CCP', magic=0xd43a, total=10240, code=10240, eeprom=0),
+        MCUModel(name='STC15F12CCP', magic=0xd43c, total=12288, code=12288, eeprom=0),
+        MCUModel(name='STC15F04', magic=0xd404, total=4096, code=4096, eeprom=0),
+        MCUModel(name='STC15F06', magic=0xd406, total=6144, code=6144, eeprom=0),
+        MCUModel(name='STC15F08', magic=0xd408, total=8192, code=8192, eeprom=0),
+        MCUModel(name='STC15F10', magic=0xd40a, total=10240, code=10240, eeprom=0),
+        MCUModel(name='STC15F12', magic=0xd40c, total=12288, code=12288, eeprom=0),
+        MCUModel(name='IAP15F08AD', magic=0xd458, total=8192, code=8192, eeprom=0),
+        MCUModel(name='IAP15F10AD', magic=0xd45a, total=10240, code=10240, eeprom=0),
+        MCUModel(name='IAP15F12AD', magic=0xd45c, total=12288, code=12288, eeprom=0),
+        MCUModel(name='IAP15F14AD', magic=0xd45e, total=14336, code=14336, eeprom=0),
+        MCUModel(name='STC15L04AD', magic=0xd4c4, total=4096, code=4096, eeprom=0),
+        MCUModel(name='STC15L06AD', magic=0xd4c6, total=6144, code=6144, eeprom=0),
+        MCUModel(name='STC15L08AD', magic=0xd4c8, total=8192, code=8192, eeprom=0),
+        MCUModel(name='STC15L10AD', magic=0xd4ca, total=10240, code=10240, eeprom=0),
+        MCUModel(name='STC15L12AD', magic=0xd4cc, total=12288, code=12288, eeprom=0),
+        MCUModel(name='STC15L04CCP', magic=0xd4b4, total=4096, code=4096, eeprom=0),
+        MCUModel(name='STC15L06CCP', magic=0xd4b6, total=6144, code=6144, eeprom=0),
+        MCUModel(name='STC15L08CCP', magic=0xd4b8, total=8192, code=8192, eeprom=0),
+        MCUModel(name='STC15L10CCP', magic=0xd4ba, total=10240, code=10240, eeprom=0),
+        MCUModel(name='STC15L12CCP', magic=0xd4bc, total=12288, code=12288, eeprom=0),
+        MCUModel(name='STC15L04', magic=0xd484, total=4096, code=4096, eeprom=0),
+        MCUModel(name='STC15L06', magic=0xd486, total=6144, code=6144, eeprom=0),
+        MCUModel(name='STC15L08', magic=0xd488, total=8192, code=8192, eeprom=0),
+        MCUModel(name='STC15L10', magic=0xd48a, total=10240, code=10240, eeprom=0),
+        MCUModel(name='STC15L12', magic=0xd48c, total=12288, code=12288, eeprom=0),
+        MCUModel(name='IAP15L08AD', magic=0xd4d8, total=8192, code=8192, eeprom=0),
+        MCUModel(name='IAP15L10AD', magic=0xd4da, total=10240, code=10240, eeprom=0),
+        MCUModel(name='IAP15L12AD', magic=0xd4dc, total=12288, code=12288, eeprom=0),
+        MCUModel(name='IAP15L14AD', magic=0xd4de, total=14336, code=14336, eeprom=0),
     )
 
     @classmethod
@@ -988,6 +1028,3 @@ class MCUModelDatabase:
         print("  Magic: %02X%02X" % (model.magic >> 8, model.magic & 0xff))
         print("  Code flash: %.1f KB" % (model.code / 1024.0))
         print("  EEPROM flash: %.1f KB" % (model.eeprom / 1024.0))
-
-
-

stcgal/options.py

@@ -21,15 +21,24 @@
 #
 
 import struct
+from abc import ABC
 from stcgal.utils import Utils
 
-class BaseOption:
+class BaseOption(ABC):
+    """Base class for options"""
+
+    def __init__(self):
+        self.options = ()
+        self.msr = None
+
     def print(self):
+        """Print current configuration to standard output"""
         print("Target options:")
         for name, get_func, _ in self.options:
             print("  %s=%s" % (name, get_func()))
 
     def set_option(self, name, value):
+        """Set value of a specific option"""
         for opt, _, set_func in self.options:
             if opt == name:
                 print("Option %s=%s" % (name, value))
@@ -38,12 +47,14 @@ class BaseOption:
         raise ValueError("unknown")
 
     def get_option(self, name):
+        """Get option value for a specific option"""
         for opt, get_func, _ in self.options:
             if opt == name:
                 return get_func(name)
         raise ValueError("unknown")
 
     def get_msr(self):
+        """Get array of model-specific configuration registers"""
         return bytes(self.msr)
 
 
@@ -51,6 +62,7 @@ class Stc89Option(BaseOption):
     """Manipulation STC89 series option byte"""
 
     def __init__(self, msr):
+        super().__init__()
         self.msr = msr
         self.options = (
             ("cpu_6t_enabled", self.get_t6, self.set_t6),
@@ -69,7 +81,7 @@ class Stc89Option(BaseOption):
         return not bool(self.msr & 1)
 
     def set_t6(self, val):
-        val = Utils.to_bool(val);
+        val = Utils.to_bool(val)
         self.msr &= 0xfe
         self.msr |= 0x01 if not bool(val) else 0x00
 
@@ -77,7 +89,7 @@ class Stc89Option(BaseOption):
         return not bool(self.msr & 4)
 
     def set_pindetect(self, val):
-        val = Utils.to_bool(val);
+        val = Utils.to_bool(val)
         self.msr &= 0xfb
         self.msr |= 0x04 if not bool(val) else 0x00
 
@@ -85,7 +97,7 @@ class Stc89Option(BaseOption):
         return not bool(self.msr & 8)
 
     def set_ee_erase(self, val):
-        val = Utils.to_bool(val);
+        val = Utils.to_bool(val)
         self.msr &= 0xf7
         self.msr |= 0x08 if not bool(val) else 0x00
 
@@ -104,7 +116,7 @@ class Stc89Option(BaseOption):
         return bool(self.msr & 32)
 
     def set_ale(self, val):
-        val = Utils.to_bool(val);
+        val = Utils.to_bool(val)
         self.msr &= 0xdf
         self.msr |= 0x20 if bool(val) else 0x00
 
@@ -112,7 +124,7 @@ class Stc89Option(BaseOption):
         return bool(self.msr & 64)
 
     def set_xram(self, val):
-        val = Utils.to_bool(val);
+        val = Utils.to_bool(val)
         self.msr &= 0xbf
         self.msr |= 0x40 if bool(val) else 0x00
 
@@ -120,7 +132,7 @@ class Stc89Option(BaseOption):
         return not bool(self.msr & 128)
 
     def set_watchdog(self, val):
-        val = Utils.to_bool(val);
+        val = Utils.to_bool(val)
         self.msr &= 0x7f
         self.msr |= 0x80 if not bool(val) else 0x00
 
@@ -129,6 +141,7 @@ class Stc12AOption(BaseOption):
     """Manipulate STC12A series option bytes"""
 
     def __init__(self, msr):
+        super().__init__()
         assert len(msr) == 4
         self.msr = bytearray(msr)
 
@@ -150,7 +163,7 @@ class Stc12AOption(BaseOption):
     def set_low_voltage_detect(self, val):
         lvds = {"low": 1, "high": 0}
         if val not in lvds.keys():
-            raise ValueError("must be one of %s" % list(sources.keys()))
+            raise ValueError("must be one of %s" % list(lvds.keys()))
         self.msr[3] &= 0xbf
         self.msr[3] |= lvds[val] << 6
 
@@ -169,7 +182,7 @@ class Stc12AOption(BaseOption):
         return not bool(self.msr[1] & 32)
 
     def set_watchdog(self, val):
-        val = Utils.to_bool(val);
+        val = Utils.to_bool(val)
         self.msr[1] &= 0xdf
         self.msr[1] |= 0x20 if not val else 0x00
 
@@ -177,7 +190,7 @@ class Stc12AOption(BaseOption):
         return not bool(self.msr[1] & 8)
 
     def set_watchdog_idle(self, val):
-        val = Utils.to_bool(val);
+        val = Utils.to_bool(val)
         self.msr[1] &= 0xf7
         self.msr[1] |= 0x08 if not val else 0x00
 
@@ -196,7 +209,7 @@ class Stc12AOption(BaseOption):
         return not bool(self.msr[2] & 2)
 
     def set_ee_erase(self, val):
-        val = Utils.to_bool(val);
+        val = Utils.to_bool(val)
         self.msr[2] &= 0xfd
         self.msr[2] |= 0x02 if not val else 0x00
 
@@ -204,7 +217,7 @@ class Stc12AOption(BaseOption):
         return not bool(self.msr[2] & 1)
 
     def set_pindetect(self, val):
-        val = Utils.to_bool(val);
+        val = Utils.to_bool(val)
         self.msr[2] &= 0xfe
         self.msr[2] |= 0x01 if not val else 0x00
 
@@ -213,6 +226,7 @@ class Stc12Option(BaseOption):
     """Manipulate STC10/11/12 series option bytes"""
 
     def __init__(self, msr):
+        super().__init__()
         assert len(msr) == 4
         self.msr = bytearray(msr)
 
@@ -235,7 +249,7 @@ class Stc12Option(BaseOption):
         return bool(self.msr[0] & 1)
 
     def set_reset_pin_enabled(self, val):
-        val = Utils.to_bool(val);
+        val = Utils.to_bool(val)
         self.msr[0] &= 0xfe
         self.msr[0] |= 0x01 if bool(val) else 0x00
 
@@ -243,7 +257,7 @@ class Stc12Option(BaseOption):
         return not bool(self.msr[0] & 64)
 
     def set_low_voltage_detect(self, val):
-        val = Utils.to_bool(val);
+        val = Utils.to_bool(val)
         self.msr[0] &= 0xbf
         self.msr[0] |= 0x40 if not val else 0x00
 
@@ -295,7 +309,7 @@ class Stc12Option(BaseOption):
         return not bool(self.msr[2] & 32)
 
     def set_watchdog(self, val):
-        val = Utils.to_bool(val);
+        val = Utils.to_bool(val)
         self.msr[2] &= 0xdf
         self.msr[2] |= 0x20 if not val else 0x00
 
@@ -303,7 +317,7 @@ class Stc12Option(BaseOption):
         return not bool(self.msr[2] & 8)
 
     def set_watchdog_idle(self, val):
-        val = Utils.to_bool(val);
+        val = Utils.to_bool(val)
         self.msr[2] &= 0xf7
         self.msr[2] |= 0x08 if not val else 0x00
 
@@ -322,7 +336,7 @@ class Stc12Option(BaseOption):
         return not bool(self.msr[3] & 2)
 
     def set_ee_erase(self, val):
-        val = Utils.to_bool(val);
+        val = Utils.to_bool(val)
         self.msr[3] &= 0xfd
         self.msr[3] |= 0x02 if not val else 0x00
 
@@ -330,13 +344,14 @@ class Stc12Option(BaseOption):
         return not bool(self.msr[3] & 1)
 
     def set_pindetect(self, val):
-        val = Utils.to_bool(val);
+        val = Utils.to_bool(val)
         self.msr[3] &= 0xfe
         self.msr[3] |= 0x01 if not val else 0x00
 
 
 class Stc15AOption(BaseOption):
     def __init__(self, msr):
+        super().__init__()
         assert len(msr) == 13
         self.msr = bytearray(msr)
 
@@ -359,7 +374,7 @@ class Stc15AOption(BaseOption):
         return bool(self.msr[0] & 16)
 
     def set_reset_pin_enabled(self, val):
-        val = Utils.to_bool(val);
+        val = Utils.to_bool(val)
         self.msr[0] &= 0xef
         self.msr[0] |= 0x10 if bool(val) else 0x00
 
@@ -367,7 +382,7 @@ class Stc15AOption(BaseOption):
         return not bool(self.msr[2] & 32)
 
     def set_watchdog(self, val):
-        val = Utils.to_bool(val);
+        val = Utils.to_bool(val)
         self.msr[2] &= 0xdf
         self.msr[2] |= 0x20 if not val else 0x00
 
@@ -375,7 +390,7 @@ class Stc15AOption(BaseOption):
         return not bool(self.msr[2] & 8)
 
     def set_watchdog_idle(self, val):
-        val = Utils.to_bool(val);
+        val = Utils.to_bool(val)
         self.msr[2] &= 0xf7
         self.msr[2] |= 0x08 if not val else 0x00
 
@@ -394,7 +409,7 @@ class Stc15AOption(BaseOption):
         return bool(self.msr[1] & 64)
 
     def set_lvrs(self, val):
-        val = Utils.to_bool(val);
+        val = Utils.to_bool(val)
         self.msr[1] &= 0xbf
         self.msr[1] |= 0x40 if val else 0x00
 
@@ -402,7 +417,7 @@ class Stc15AOption(BaseOption):
         return bool(self.msr[1] & 128)
 
     def set_eeprom_lvd(self, val):
-        val = Utils.to_bool(val);
+        val = Utils.to_bool(val)
         self.msr[1] &= 0x7f
         self.msr[1] |= 0x80 if val else 0x00
 
@@ -420,7 +435,7 @@ class Stc15AOption(BaseOption):
         return not bool(self.msr[12] & 2)
 
     def set_ee_erase(self, val):
-        val = Utils.to_bool(val);
+        val = Utils.to_bool(val)
         self.msr[12] &= 0xfd
         self.msr[12] |= 0x02 if not val else 0x00
 
@@ -428,13 +443,14 @@ class Stc15AOption(BaseOption):
         return not bool(self.msr[12] & 1)
 
     def set_pindetect(self, val):
-        val = Utils.to_bool(val);
+        val = Utils.to_bool(val)
         self.msr[12] &= 0xfe
         self.msr[12] |= 0x01 if not val else 0x00
 
 
 class Stc15Option(BaseOption):
     def __init__(self, msr):
+        super().__init__()
         assert len(msr) >= 4
         self.msr = bytearray(msr)
 
@@ -463,7 +479,7 @@ class Stc15Option(BaseOption):
         return not bool(self.msr[2] & 16)
 
     def set_reset_pin_enabled(self, val):
-        val = Utils.to_bool(val);
+        val = Utils.to_bool(val)
         self.msr[2] &= 0xef
         self.msr[2] |= 0x10 if not bool(val) else 0x00
 
@@ -493,7 +509,7 @@ class Stc15Option(BaseOption):
         return not bool(self.msr[0] & 32)
 
     def set_watchdog(self, val):
-        val = Utils.to_bool(val);
+        val = Utils.to_bool(val)
         self.msr[0] &= 0xdf
         self.msr[0] |= 0x20 if not val else 0x00
 
@@ -501,7 +517,7 @@ class Stc15Option(BaseOption):
         return not bool(self.msr[0] & 8)
 
     def set_watchdog_idle(self, val):
-        val = Utils.to_bool(val);
+        val = Utils.to_bool(val)
         self.msr[0] &= 0xf7
         self.msr[0] |= 0x08 if not val else 0x00
 
@@ -520,7 +536,7 @@ class Stc15Option(BaseOption):
         return not bool(self.msr[1] & 64)
 
     def set_lvrs(self, val):
-        val = Utils.to_bool(val);
+        val = Utils.to_bool(val)
         self.msr[1] &= 0xbf
         self.msr[1] |= 0x40 if not val else 0x00
 
@@ -528,7 +544,7 @@ class Stc15Option(BaseOption):
         return bool(self.msr[1] & 128)
 
     def set_eeprom_lvd(self, val):
-        val = Utils.to_bool(val);
+        val = Utils.to_bool(val)
         self.msr[1] &= 0x7f
         self.msr[1] |= 0x80 if val else 0x00
 
@@ -546,7 +562,7 @@ class Stc15Option(BaseOption):
         return bool(self.msr[3] & 2)
 
     def set_ee_erase(self, val):
-        val = Utils.to_bool(val);
+        val = Utils.to_bool(val)
         self.msr[3] &= 0xfd
         self.msr[3] |= 0x02 if val else 0x00
 
@@ -554,7 +570,7 @@ class Stc15Option(BaseOption):
         return not bool(self.msr[3] & 1)
 
     def set_pindetect(self, val):
-        val = Utils.to_bool(val);
+        val = Utils.to_bool(val)
         self.msr[3] &= 0xfe
         self.msr[3] |= 0x01 if not val else 0x00
 

stcgal/protocols.py

@@ -21,12 +21,18 @@
 #
 
 import serial
-import sys, os, time, struct, re, errno
+import sys
+import os
+import time
+import struct
+import re
+import errno
 import argparse
 import collections
 from stcgal.models import MCUModelDatabase
 from stcgal.utils import Utils
-from stcgal.options import *
+from stcgal.options import Stc89Option, Stc12Option, Stc12AOption, Stc15Option, Stc15AOption
+from abc import ABC, abstractmethod
 import functools
 
 try:
@@ -46,7 +52,7 @@ class StcProtocolException(Exception):
     pass
 
 
-class StcBaseProtocol:
+class StcBaseProtocol(ABC):
     """Basic functionality for STC BSL protocols"""
 
     """magic word that starts a packet"""
@@ -103,6 +109,10 @@ class StcBaseProtocol:
 
         return packet[5:-1]
 
+    @abstractmethod
+    def write_packet(self, packet_data):
+        pass
+
     def read_packet(self):
         """Read and check packet from MCU.
 
@@ -142,7 +152,7 @@ class StcBaseProtocol:
         packet += self.read_bytes_safe(packet_len - 3)
 
         # verify checksum and extract payload
-        payload = self.extract_payload(packet);
+        payload = self.extract_payload(packet)
 
         self.dump_packet(packet, receive=True)
 
@@ -192,20 +202,6 @@ class StcBaseProtocol:
             mcu_name += "E" if self.status_packet[17] < 0x70 else "W"
             self.model = self.model._replace(name = mcu_name)
 
-        protocol_database = [("stc89", "STC(89|90)(C|LE)\d"),
-                             ("stc12a", "STC12(C|LE)\d052"),
-                             ("stc12b", "STC12(C|LE)(52|56)"),
-                             ("stc12", "(STC|IAP)(10|11|12)\D"),
-                             ("stc15a", "(STC|IAP)15[FL][01]0\d(E|EA|)$"),
-                             ("stc15", "(STC|IAP|IRC)15\D")]
-
-        for protocol_name, pattern in protocol_database:
-            if re.match(pattern, self.model.name):
-                self.protocol_name = protocol_name
-                break
-        else:
-            self.protocol_name = None
-
     def get_status_packet(self):
         """Read and decode status packet"""
 
@@ -241,7 +237,22 @@ class StcBaseProtocol:
     def set_option(self, name, value):
         self.options.set_option(name, value)
 
-    def connect(self, autoreset=False):
+    def reset_device(self, resetcmd=False):
+        if not resetcmd:
+            print("Cycling power: ", end="")
+            sys.stdout.flush()
+            self.ser.setDTR(True)
+            time.sleep(0.5)
+            self.ser.setDTR(False)
+            print("done")
+        else:
+            print("Cycling power via shell cmd: " + resetcmd)
+            os.system(resetcmd)
+
+        print("Waiting for MCU: ", end="")
+        sys.stdout.flush()
+
+    def connect(self, autoreset=False, resetcmd=False):
         """Connect to MCU and initialize communication.
 
         Set up serial port, send sync sequence and get part info.
@@ -260,14 +271,7 @@ class StcBaseProtocol:
         self.ser.flushInput()
 
         if autoreset:
-            print("Cycling power: ", end="")
-            sys.stdout.flush()
-            self.ser.setDTR(True)
-            time.sleep(0.5)
-            self.ser.setDTR(False)
-            print("done")
-            print("Waiting for MCU: ", end="")
-            sys.stdout.flush()
+            self.reset_device(resetcmd)
         else:
             print("Waiting for MCU, please cycle power: ", end="")
             sys.stdout.flush()
@@ -279,6 +283,8 @@ class StcBaseProtocol:
             try:
                 self.pulse()
                 self.status_packet = self.get_status_packet()
+                if len(self.status_packet) < 23:
+                    raise StcProtocolException("status packet too short")
             except (StcFramingException, serial.SerialTimeoutException): pass
         print("done")
 
@@ -288,7 +294,21 @@ class StcBaseProtocol:
 
         self.initialize_model()
 
-    def initialize(self, base_protocol = None):
+    @abstractmethod
+    def initialize_status(self, status_packet):
+        """Initialize internal state from status packet"""
+        pass
+
+    @abstractmethod
+    def initialize_options(self, status_packet):
+        """Initialize options from status packet"""
+        pass
+
+    def initialize(self, base_protocol=None):
+        """
+        Initialize from another instance. This is an alternative for calling
+        connect() and is used by protocol autodetection.
+        """
         if base_protocol:
             self.ser = base_protocol.ser
             self.ser.parity = self.PARITY
@@ -316,6 +336,39 @@ class StcBaseProtocol:
         print("Disconnected!")
 
 
+class StcAutoProtocol(StcBaseProtocol):
+    """
+    Protocol handler for autodetection of protocols. Does not implement full
+    functionality for any device class.
+    """
+
+    def initialize_model(self):
+        super().initialize_model()
+
+        protocol_database = [("stc89", r"STC(89|90)(C|LE)\d"),
+                             ("stc12a", r"STC12(C|LE)\d052"),
+                             ("stc12b", r"STC12(C|LE)(52|56)"),
+                             ("stc12", r"(STC|IAP)(10|11|12)\D"),
+                             ("stc15a", r"(STC|IAP)15[FL][01]0\d(E|EA|)$"),
+                             ("stc15", r"(STC|IAP|IRC)15\D")]
+
+        for protocol_name, pattern in protocol_database:
+            if re.match(pattern, self.model.name):
+                self.protocol_name = protocol_name
+                break
+        else:
+            self.protocol_name = None
+
+    def initialize_options(self, status_packet):
+        raise NotImplementedError
+
+    def initialize_status(self, status_packet):
+        raise NotImplementedError
+
+    def write_packet(self, packet_data):
+        raise NotImplementedError
+
+
 class Stc89Protocol(StcBaseProtocol):
     """Protocol handler for STC 89/90 series"""
 
@@ -376,6 +429,9 @@ class Stc89Protocol(StcBaseProtocol):
     def initialize_options(self, status_packet):
         """Initialize options"""
 
+        if len(status_packet) < 20:
+            raise StcProtocolException("invalid options in status packet")
+
         self.options = Stc89Option(status_packet[19])
         self.options.print()
 
@@ -422,7 +478,7 @@ class Stc89Protocol(StcBaseProtocol):
 
         bl_version, bl_stepping = struct.unpack("BB", packet[17:19])
         self.mcu_bsl_version = "%d.%d%s" % (bl_version >> 4, bl_version & 0x0f,
-                                           chr(bl_stepping))
+                                            chr(bl_stepping))
 
     def handshake(self):
         """Switch to transfer baudrate
@@ -508,9 +564,9 @@ class Stc89Protocol(StcBaseProtocol):
             csum = sum(packet[7:]) & 0xff
             self.write_packet(packet)
             response = self.read_packet()
-            if response[0] != 0x80:
+            if len(response) < 1 or response[0] != 0x80:
                 raise StcProtocolException("incorrect magic in write packet")
-            elif response[1] != csum:
+            elif len(response) < 2 or response[1] != csum:
                 raise StcProtocolException("verification checksum mismatch")
             print(".", end="")
             sys.stdout.flush()
@@ -579,7 +635,7 @@ class Stc12AProtocol(Stc12AOptionsMixIn, Stc89Protocol):
 
         bl_version, bl_stepping = struct.unpack("BB", packet[17:19])
         self.mcu_bsl_version = "%d.%d%s" % (bl_version >> 4, bl_version & 0x0f,
-                                           chr(bl_stepping))
+                                            chr(bl_stepping))
 
         self.bsl_version = bl_version
 
@@ -612,6 +668,9 @@ class Stc12AProtocol(Stc12AOptionsMixIn, Stc89Protocol):
     def initialize_options(self, status_packet):
         """Initialize options"""
 
+        if len(status_packet) < 31:
+            raise StcProtocolException("invalid options in status packet")
+
         # create option state
         self.options = Stc12AOption(status_packet[23:26] + status_packet[29:30])
         self.options.print()
@@ -768,7 +827,7 @@ class Stc12BaseProtocol(StcBaseProtocol):
 
         bl_version, bl_stepping = struct.unpack("BB", packet[17:19])
         self.mcu_bsl_version = "%d.%d%s" % (bl_version >> 4, bl_version & 0x0f,
-                                           chr(bl_stepping))
+                                            chr(bl_stepping))
 
         self.bsl_version = bl_version
 
@@ -801,6 +860,9 @@ class Stc12BaseProtocol(StcBaseProtocol):
     def initialize_options(self, status_packet):
         """Initialize options"""
 
+        if len(status_packet) < 29:
+            raise StcProtocolException("invalid options in status packet")
+
         # create option state
         self.options = Stc12Option(status_packet[23:26] + status_packet[27:28])
         self.options.print()
@@ -886,7 +948,6 @@ class Stc12BaseProtocol(StcBaseProtocol):
             packet += struct.pack(">H", self.PROGRAM_BLOCKSIZE)
             packet += data[i:i+self.PROGRAM_BLOCKSIZE]
             while len(packet) < self.PROGRAM_BLOCKSIZE + 7: packet += b"\x00"
-            csum = sum(packet[7:]) & 0xff
             self.write_packet(packet)
             response = self.read_packet()
             if response[0] != 0x00:
@@ -936,6 +997,9 @@ class Stc15AProtocol(Stc12Protocol):
     def initialize_options(self, status_packet):
         """Initialize options"""
 
+        if len(status_packet) < 37:
+            raise StcProtocolException("invalid options in status packet")
+
         # create option state
         self.options = Stc15AOption(status_packet[23:36])
         self.options.print()
@@ -965,7 +1029,7 @@ class Stc15AProtocol(Stc12Protocol):
 
         bl_version, bl_stepping = struct.unpack("BB", packet[17:19])
         self.mcu_bsl_version = "%d.%d%s" % (bl_version >> 4, bl_version & 0x0f,
-                                           chr(bl_stepping))
+                                            chr(bl_stepping))
 
         self.trim_data = packet[51:58]
         self.freq_counter = freq_counter
@@ -1019,7 +1083,8 @@ class Stc15AProtocol(Stc12Protocol):
         """
 
         user_speed = self.trim_frequency
-        if user_speed <= 0: user_speed = self.mcu_clock_hz
+        if user_speed <= 0:
+            user_speed = self.mcu_clock_hz
         program_speed = 22118400
 
         user_count = int(self.freq_counter * (user_speed / self.mcu_clock_hz))
@@ -1047,15 +1112,19 @@ class Stc15AProtocol(Stc12Protocol):
         self.write_packet(packet)
         self.pulse(timeout=1.0)
         response = self.read_packet()
-        if response[0] != 0x65:
+        if len(response) < 36 or response[0] != 0x65:
             raise StcProtocolException("incorrect magic in handshake packet")
 
         # determine programming speed trim value
         target_trim_a, target_count_a = struct.unpack(">HH", response[28:32])
         target_trim_b, target_count_b = struct.unpack(">HH", response[32:36])
+        if target_count_a == target_count_b:
+            raise StcProtocolException("frequency trimming failed")
         m = (target_trim_b - target_trim_a) / (target_count_b - target_count_a)
         n = target_trim_a - m * target_count_a
         program_trim = round(m * program_count + n)
+        if program_trim > 65535 or program_trim < 0:
+            raise StcProtocolException("frequency trimming failed")
 
         # determine trim trials for second round
         trim_a, count_a = struct.unpack(">HH", response[12:16])
@@ -1074,10 +1143,14 @@ class Stc15AProtocol(Stc12Protocol):
             target_count_a = count_a
             target_count_b = count_b
         # linear interpolate to find range to try next
+        if target_count_a == target_count_b:
+            raise StcProtocolException("frequency trimming failed")
         m = (target_trim_b - target_trim_a) / (target_count_b - target_count_a)
         n = target_trim_a - m * target_count_a
         target_trim = round(m * user_count + n)
         target_trim_start = min(max(target_trim - 5, target_trim_a), target_trim_b)
+        if target_trim_start + 11 > 65535 or target_trim_start < 0:
+            raise StcProtocolException("frequency trimming failed")
 
         # trim challenge-response, second round
         packet = bytes([0x65])
@@ -1089,7 +1162,7 @@ class Stc15AProtocol(Stc12Protocol):
         self.write_packet(packet)
         self.pulse(timeout=1.0)
         response = self.read_packet()
-        if response[0] != 0x65:
+        if len(response) < 56 or response[0] != 0x65:
             raise StcProtocolException("incorrect magic in handshake packet")
 
         # determine best trim value
@@ -1148,7 +1221,11 @@ class Stc15Protocol(Stc15AProtocol):
     def initialize_options(self, status_packet):
         """Initialize options"""
 
+        if len(status_packet) < 14:
+            raise StcProtocolException("invalid options in status packet")
+
         # create option state
+        # XXX: check how option bytes are concatenated here
         self.options = Stc15Option(status_packet[5:8] + status_packet[12:13] + status_packet[37:38])
         self.options.print()
 
@@ -1177,8 +1254,7 @@ class Stc15Protocol(Stc15AProtocol):
         bl_version, bl_stepping = struct.unpack("BB", packet[17:19])
         bl_minor = packet[22] & 0x0f
         self.mcu_bsl_version = "%d.%d.%d%s" % (bl_version >> 4, bl_version & 0x0f,
-                                           bl_minor,
-                                           chr(bl_stepping))
+                                               bl_minor, chr(bl_stepping))
         self.bsl_version = bl_version
 
     def print_mcu_info(self):
@@ -1194,6 +1270,8 @@ class Stc15Protocol(Stc15AProtocol):
         calib_data = response[2:]
         challenge_data = packet[2:]
         calib_len = response[1]
+        if len(calib_data) < 2 * calib_len:
+            raise StcProtocolException("range calibration data missing")
 
         for i in range(calib_len - 1):
             count_a, count_b = struct.unpack(">HH", calib_data[2*i:2*i+4])
@@ -1203,6 +1281,8 @@ class Stc15Protocol(Stc15AProtocol):
                 m = (trim_b - trim_a) / (count_b - count_a)
                 n = trim_a - m * count_a
                 target_trim = round(m * target_count + n)
+                if target_trim > 65536 or target_trim < 0:
+                    raise StcProtocolException("frequency trimming failed")
                 return (target_trim, trim_range)
 
         return None
@@ -1214,6 +1294,8 @@ class Stc15Protocol(Stc15AProtocol):
         calib_data = response[2:]
         challenge_data = packet[2:]
         calib_len = response[1]
+        if len(calib_data) < 2 * calib_len:
+            raise StcProtocolException("trim calibration data missing")
 
         best = None
         best_count = sys.maxsize
@@ -1224,6 +1306,9 @@ class Stc15Protocol(Stc15AProtocol):
                 best_count = abs(count - target_count)
                 best = (trim_adj, trim_range), count
 
+        if not best:
+            raise StcProtocolException("frequency trimming failed")
+
         return best
 
     def calibrate(self):
@@ -1253,7 +1338,7 @@ class Stc15Protocol(Stc15AProtocol):
         self.write_packet(packet)
         self.pulse(b"\xfe", timeout=1.0)
         response = self.read_packet()
-        if response[0] != 0x00:
+        if len(response) < 2 or response[0] != 0x00:
             raise StcProtocolException("incorrect magic in handshake packet")
 
         # select ranges and trim values
@@ -1272,7 +1357,7 @@ class Stc15Protocol(Stc15AProtocol):
         self.write_packet(packet)
         self.pulse(b"\xfe", timeout=1.0)
         response = self.read_packet()
-        if response[0] != 0x00:
+        if len(response) < 2 or response[0] != 0x00:
             raise StcProtocolException("incorrect magic in handshake packet")
 
         # select final values
@@ -1293,12 +1378,12 @@ class Stc15Protocol(Stc15AProtocol):
         # This is a bit of a hack, but it works.
         bauds = self.baud_transfer if (self.mcu_magic >> 8) == 0xf2 else self.baud_transfer * 4
         packet += struct.pack(">H", int(65535 - program_speed / bauds))
-        packet += struct.pack(">H", int(65535 - (program_speed / bauds) * 1.5))
+        packet += bytes(user_trim)
         iap_wait = self.get_iap_delay(program_speed)
         packet += bytes([iap_wait])
         self.write_packet(packet)
         response = self.read_packet()
-        if response[0] != 0x01:
+        if len(response) < 1 or response[0] != 0x01:
             raise StcProtocolException("incorrect magic in handshake packet")
         time.sleep(0.2)
         self.ser.baudrate = self.baud_transfer
@@ -1315,7 +1400,7 @@ class Stc15Protocol(Stc15AProtocol):
         packet += bytes([0x00, 0x00, iap_wait])
         self.write_packet(packet)
         response = self.read_packet()
-        if response[0] != 0x01:
+        if len(response) < 1 or response[0] != 0x01:
             raise StcProtocolException("incorrect magic in handshake packet")
         time.sleep(0.2)
         self.ser.baudrate = self.baud_transfer
@@ -1341,9 +1426,9 @@ class Stc15Protocol(Stc15AProtocol):
             packet += bytes([0x00, 0x00, 0x5a, 0xa5])
         self.write_packet(packet)
         response = self.read_packet()
-        if response[0] == 0x0f:
+        if len(response) == 1 and response[0] == 0x0f:
             raise StcProtocolException("MCU is locked")
-        if response[0] != 0x05:
+        if len(response) < 1 or response[0] != 0x05:
             raise StcProtocolException("incorrect magic in handshake packet")
 
         print("done")
@@ -1364,13 +1449,17 @@ class Stc15Protocol(Stc15AProtocol):
             packet += bytes([0x00, 0x5a, 0xa5])
         self.write_packet(packet)
         response = self.read_packet()
-        if response[0] != 0x03:
+        if len(response) < 1 or response[0] != 0x03:
             raise StcProtocolException("incorrect magic in handshake packet")
         print("done")
 
         if len(response) >= 8:
             self.uid = response[1:8]
 
+        # we should have a UID at this point
+        if not self.uid:
+            raise StcProtocolException("UID is missing")
+
     def program_flash(self, data):
         """Program the MCU's flash memory."""
 
@@ -1385,7 +1474,7 @@ class Stc15Protocol(Stc15AProtocol):
             while len(packet) < self.PROGRAM_BLOCKSIZE + 3: packet += b"\x00"
             self.write_packet(packet)
             response = self.read_packet()
-            if response[0] != 0x02 or response[1] != 0x54:
+            if len(response) < 2 or response[0] != 0x02 or response[1] != 0x54:
                 raise StcProtocolException("incorrect magic in write packet")
             print(".", end="")
             sys.stdout.flush()
@@ -1398,7 +1487,7 @@ class Stc15Protocol(Stc15AProtocol):
             packet = bytes([0x07, 0x00, 0x00, 0x5a, 0xa5])
             self.write_packet(packet)
             response = self.read_packet()
-            if response[0] != 0x07 or response[1] != 0x54:
+            if len(response) < 2 or response[0] != 0x07 or response[1] != 0x54:
                 raise StcProtocolException("incorrect magic in finish packet")
             print("done")
 
@@ -1407,7 +1496,7 @@ class Stc15Protocol(Stc15AProtocol):
         configuration."""
 
         msr = self.options.get_msr()
-        packet  = bytes([0xff] * 23)
+        packet = bytes([0xff] * 23)
         packet += bytes([(self.trim_frequency >> 24) & 0xff,
                          0xff,
                          (self.trim_frequency >> 16) & 0xff,
@@ -1437,7 +1526,7 @@ class Stc15Protocol(Stc15AProtocol):
         packet += self.build_options()
         self.write_packet(packet)
         response = self.read_packet()
-        if response[0] != 0x04 or response[1] != 0x54:
+        if len(response) < 2 or response[0] != 0x04 or response[1] != 0x54:
             raise StcProtocolException("incorrect magic in option packet")
         print("done")
 
@@ -1461,8 +1550,9 @@ class StcUsb15Protocol(Stc15Protocol):
 
     def dump_packet(self, data, request=0, value=0, index=0, receive=True):
         if self.debug:
-            print("%s bRequest=%02X wValue=%04X wIndex=%04X data: %s" % (("<-" if receive else "->"),
-                  request, value, index, Utils.hexstr(data, " ")), file=sys.stderr)
+            print("%s bRequest=%02X wValue=%04X wIndex=%04X data: %s" %
+                  (("<-" if receive else "->"), request, value, index,
+                   Utils.hexstr(data, " ")), file=sys.stderr)
 
     def read_packet(self):
         """Read a packet from the MCU"""
@@ -1502,15 +1592,16 @@ class StcUsb15Protocol(Stc15Protocol):
 
         self.dump_packet(chunks, request, value, index, receive=False)
         host2dev = usb.util.CTRL_TYPE_VENDOR | usb.util.CTRL_RECIPIENT_DEVICE | usb.util.CTRL_OUT
-        self.dev.ctrl_transfer(host2dev, request, value, index, chunks);
+        self.dev.ctrl_transfer(host2dev, request, value, index, chunks)
 
-    def connect(self, autoreset=False):
+    def connect(self, autoreset=False, resetcmd=False):
         """Connect to USB device and read info packet"""
 
         # USB support is optional. Provide an error if pyusb is not available.
-        if _usb_available == False:
-            raise StcProtocolException("USB support not available. "
-                + "pyusb is not installed or not working correctly.")
+        if not _usb_available:
+            raise StcProtocolException(
+                "USB support not available. " +
+                "pyusb is not installed or not working correctly.")
 
         print("Waiting for MCU, please cycle power: ", end="")
         sys.stdout.flush()

stcgal/utils.py

@@ -19,31 +19,39 @@
 # SOFTWARE.
 #
 
-import serial
 import argparse
+import serial
 
 class Utils:
+    """Common utility functions"""
+
     @classmethod
-    def to_bool(self, val):
+    def to_bool(cls, val):
         """make sensible boolean from string or other type value"""
 
-        if isinstance(val, bool): return val
-        if isinstance(val, int): return bool(val)
-        if len(val) == 0: return False
-        return True if val[0].lower() == "t" or val[0] == "1" else False
+        if not val:
+            return False
+        if isinstance(val, bool):
+            return val
+        elif isinstance(val, int):
+            return bool(val)
+        else:
+            return True if val[0].lower() == "t" or val[0] == "1" else False
 
     @classmethod
-    def to_int(self, val):
+    def to_int(cls, val):
         """make int from any value, nice error message if not possible"""
 
-        try: return int(val, 0)
-        except: raise ValueError("invalid integer")
+        try:
+            return int(val, 0)
+        except:
+            raise ValueError("invalid integer")
 
     @classmethod
-    def hexstr(self, bytestr, sep=""):
+    def hexstr(cls, bytestr, sep=""):
         """make formatted hex string output from byte sequence"""
 
-        return sep.join(["%02X" % x for x in bytestr])
+        return sep.join(["%02X" % x for x in bytes(bytestr)])
 
 
 class BaudType:
@@ -55,5 +63,5 @@ class BaudType:
             raise argparse.ArgumentTypeError("illegal baudrate")
         return baud
 
-    def __repr__(self): return "baudrate"
-
+    def __repr__(self):
+        return "baudrate"

tests/iap15f2k61s2.yml

@@ -0,0 +1,19 @@
+name: IAP15F2K61S2 programming test
+protocol: stc15
+code_data: [49, 50, 51, 52, 53, 54, 55, 56, 57]
+responses:
+    - [0x50, 0x87, 0xD3, 0x75, 0x9C, 0xF5, 0x3B, 0x17, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0x09, 0x81, 0x00, 0x00, 0x71, 0x53, 0x00, 0xF4, 0x49, 0x04, 0x06, 0x58, 0x9C, 0x02, 0x0E, 0x14, 0x17, 0x19, 0x19, 0x00, 0xF4, 0xF4, 0x04, 0xD2]
+    - [0x00, 0x0B, 0x03, 0x37, 0x04, 0x9A, 0x06, 0x02, 0x06, 0x6B, 0x09, 0x27, 0x0B, 0xE8, 0x0D, 0x0A, 0x12, 0x5A, 0x17, 0x9B, 0x14, 0x8F, 0x1C, 0x96, 0x00, 0x00]
+    - [0x00, 0x0C, 0x09, 0x04, 0x09, 0x09, 0x09, 0x0E, 0x09, 0x0E, 0x09, 0x18, 0x09, 0x1D, 0x12, 0x00, 0x12, 0x0F, 0x12, 0x19, 0x12, 0x23, 0x12, 0x2D, 0x12, 0x37]
+    - [0x01]
+    - [0x05]
+    - [0x03, 0x0D, 0x00, 0x00, 0x21, 0x02, 0x26, 0x32]
+    - [0x02, 0x54]
+    - [0x02, 0x54]
+    - [0x02, 0x54]
+    - [0x02, 0x54]
+    - [0x02, 0x54]
+    - [0x02, 0x54]
+    - [0x02, 0x54]
+    - [0x02, 0x54]
+    - [0x04, 0x54]

tests/stc12c2052ad.yml

@@ -0,0 +1,19 @@
+name: STC12C2052AD programming test
+protocol: stc12a
+code_data: [49, 50, 51, 52, 53, 54, 55, 56, 57]
+responses:
+    - [0x00, 0x04, 0xEC, 0x04, 0xEC, 0x04, 0xEC, 0x04, 0xEC, 0x04, 0xEC, 0x04, 0xEC, 0x04, 0xEB, 0x04, 0xEB, 0x58, 0x44, 0x00, 0xF2, 0x12, 0x83, 0xFD, 0xF7, 0xF7, 0xFF, 0xFF, 0xFF, 0xBF, 0xFF, 0xFD, 0xF7, 0xF7, 0xFF]
+    - [0x8F, 0xC0, 0x79, 0x3F, 0xFE, 0x28, 0x85]
+    - [0x8E, 0xC0, 0x79, 0x3F, 0xFE, 0x28]
+    - [0x80]
+    - [0x80]
+    - [0x80]
+    - [0x80]
+    - [0x80]
+    - [0x80, 0x66]
+    - [0x80, 0x80]
+    - [0x80, 0x80]
+    - [0x80, 0x80]
+    - [0x80, 0xEE]
+    - [0x10, 0xC0, 0x16, 0xF7, 0xFF, 0xBF, 0x03, 0xFF, 0x58, 0x44, 0xFD, 0xF7, 0xF7, 0xFF, 0xFF, 0xFF, 0xBF, 0xFF, 0xFD, 0xF7, 0xF7, 0xFF]
+    - [0x80]

tests/stc12c5a60s2.yml

@@ -0,0 +1,15 @@
+name: STC12C5A60S2 programming test
+protocol: stc12
+code_data: [49, 50, 51, 52, 53, 54, 55, 56, 57]
+responses:
+    - [0x50, 0x04, 0xBD, 0x04, 0xBC, 0x04, 0xBC, 0x04, 0xBD, 0x04, 0xBC, 0x04, 0xBC, 0x04, 0xBC, 0x04, 0xBC, 0x62, 0x49, 0x00, 0xD1, 0x7E, 0x8C, 0xFF, 0x7F, 0xF7, 0xFF, 0xFF, 0xFF, 0x00, 0x00, 0x00, 0x03, 0x00, 0xB0, 0x02, 0x2E, 0x6B, 0x00, 0xCD, 0x80, 0x00, 0x00]
+    - [0x8F]
+    - [0x8F, 0xC0, 0x7E, 0x3F, 0xFE, 0xA0, 0x83, 0x04]
+    - [0x84, 0xC0, 0x7E, 0x3F, 0xFE, 0xA0, 0x04]
+    - [0x00]
+    - [0x00, 0x03]
+    - [0x00, 0x00]
+    - [0x00, 0x00]
+    - [0x00, 0x00]
+    - [0x8D]
+    - [0x50, 0xFF, 0x7F, 0xF7, 0xFF, 0xFF, 0x03, 0xFF, 0x62, 0x49, 0xFF, 0x7F, 0xF7, 0xFF, 0xFF, 0xFF, 0xFF, 0x01, 0x00, 0x03, 0x00, 0xB0, 0x02, 0x2E, 0x6B, 0x00, 0xCD, 0x80, 0x00, 0x00]

tests/stc15f104e.yml

@@ -0,0 +1,20 @@
+name: STC15F104E programming test
+protocol: stc15a
+code_data: [49, 50, 51, 52, 53, 54, 55, 56, 57]
+responses:
+    - [0x50, 0x02, 0xB0, 0x02, 0xB0, 0x02, 0xAF, 0x02, 0xB0, 0x02, 0xE6, 0x02, 0xE7, 0x00, 0x00, 0x00, 0x00, 0x67, 0x51, 0xFF, 0xF2, 0x94, 0x8C, 0xEF, 0x3B, 0xF5, 0x58, 0x34, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0x00, 0x00, 0x00, 0x00, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0x58, 0x50, 0x0C, 0x94, 0x21, 0xFF, 0x29]
+    - [0x8f]
+    - [0x65, 0x58, 0x50, 0x0C, 0x95, 0x21, 0xFF, 0x2B, 0xFF, 0xFF, 0x06, 0x06, 0x58, 0x00, 0x02, 0x00, 0x58, 0x80, 0x02, 0x00, 0x58, 0x80, 0x02, 0x00, 0x58, 0xFF, 0x02, 0x00, 0x58, 0x00, 0x02, 0x00, 0x58, 0x80, 0x02, 0x00]
+    - [0x65, 0x58, 0x50, 0x0C, 0x95, 0x21, 0xFF, 0x2B, 0xFF, 0xFF, 0x06, 0x0B, 0x58, 0x24, 0x02, 0x00, 0x58, 0x25, 0x02, 0x00, 0x58, 0x26, 0x02, 0x00, 0x58, 0x27, 0x02, 0x00, 0x58, 0x28, 0x02, 0x00, 0x58, 0x29, 0x02, 0x00, 0x58, 0x2A, 0x02, 0x00, 0x58, 0x2B, 0x02, 0x00, 0x58, 0x2C, 0x02, 0x00, 0x58, 0x2D, 0x02, 0x00, 0x58, 0x2E, 0x02, 0x00]
+    - [0x01]
+    - [0x05]
+    - [0x03, 0x0C, 0x00, 0x00, 0x17, 0x01, 0xA0, 0xE0]
+    - [0x02, 0x54]
+    - [0x02, 0x54]
+    - [0x02, 0x54]
+    - [0x02, 0x54]
+    - [0x02, 0x54]
+    - [0x02, 0x54]
+    - [0x02, 0x54]
+    - [0x02, 0x54]
+    - [0x04, 0x54]

tests/stc15l104w.yml

@@ -0,0 +1,19 @@
+name: STC15L104W programming test
+protocol: stc15
+code_data: [49, 50, 51, 52, 53, 54, 55, 56, 57]
+responses:
+    - [0x50, 0x66, 0x3C, 0x93, 0xBA, 0xF7, 0xBB, 0x9F, 0x00, 0x5B, 0x68, 0x00, 0xFD, 0x00, 0x00, 0x00, 0x00, 0x71, 0x51, 0x03, 0xF2, 0xD4, 0x04, 0x06, 0x58, 0xBA, 0x02, 0x2A, 0x31, 0x32, 0x38, 0x30, 0x80, 0x14, 0x10, 0x04, 0xD9]
+    - [0x00, 0x0B, 0x03, 0x0A, 0x04, 0x4F, 0x05, 0x9E, 0x06, 0x20, 0x08, 0xB9, 0x0B, 0x5C, 0x0C, 0x6A, 0x11, 0x7E, 0x16, 0x79, 0x13, 0x77, 0x1A, 0xB1, 0x00, 0x00]
+    - [0x00, 0x0C, 0x04, 0xD6, 0x04, 0xDB, 0x04, 0xE0, 0x04, 0xE0, 0x04, 0xE0, 0x04, 0xE5, 0x11, 0xE2, 0x11, 0xF1, 0x11, 0xFB, 0x12, 0x05, 0x12, 0x0A, 0x12, 0x19]
+    - [0x01]
+    - [0x05]
+    - [0x03, 0x0C, 0x00, 0x00, 0x17, 0x01, 0xA0, 0xE0]
+    - [0x02, 0x54]
+    - [0x02, 0x54]
+    - [0x02, 0x54]
+    - [0x02, 0x54]
+    - [0x02, 0x54]
+    - [0x02, 0x54]
+    - [0x02, 0x54]
+    - [0x02, 0x54]
+    - [0x04, 0x54]

tests/stc15w4k56s4.yml

@@ -0,0 +1,20 @@
+name: STC15W4K56S4 programming test
+protocol: stc15
+code_data: [49, 50, 51, 52, 53, 54, 55, 56, 57]
+responses:
+    - [0x50, 0x8D, 0xFF, 0x73, 0x96, 0xF5, 0x7B, 0x9F, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0x27, 0xED, 0x00, 0x00, 0x73, 0x54, 0x00, 0xF5, 0x28, 0x04, 0x06, 0x70, 0x96, 0x02, 0x15, 0x19, 0x1C, 0x1E, 0x23, 0x00, 0xEC, 0xE0, 0x04, 0xD7, 0xF8, 0x73, 0xBF, 0xFF, 0xFF, 0x15, 0x09, 0x25, 0x60]
+    - [0x00, 0x0B, 0x0D, 0x21, 0x12, 0xBC, 0x18, 0x3E, 0x1A, 0x05, 0x24, 0xFA, 0x2F, 0xB3, 0x34, 0xD1, 0x4A, 0x52, 0x5E, 0xC0, 0x52, 0xDB, 0x73, 0x1A, 0x00, 0x00]
+    - [0x00, 0x0C, 0x23, 0xBF, 0x23, 0xD3, 0x23, 0xE7, 0x23, 0xF6, 0x24, 0x0F, 0x24, 0x23, 0x47, 0x73, 0x47, 0xB9, 0x47, 0xE1, 0x48, 0x09, 0x48, 0x36, 0x48, 0x59]
+    - [0x01]
+    - [0x05]
+    - [0x03, 0xF5, 0x28, 0x00, 0xA5, 0x03, 0x27, 0x49]
+    - [0x02, 0x54]
+    - [0x02, 0x54]
+    - [0x02, 0x54]
+    - [0x02, 0x54]
+    - [0x02, 0x54]
+    - [0x02, 0x54]
+    - [0x02, 0x54]
+    - [0x02, 0x54]
+    - [0x07, 0x54]
+    - [0x04, 0x54]

tests/stc89c52rc.yml

@@ -0,0 +1,19 @@
+name: STC89C52RC programming test
+protocol: stc89
+code_data: [49, 50, 51, 52, 53, 54, 55, 56, 57]
+responses:
+    - [0x00, 0x25, 0xE6, 0x25, 0xE6, 0x25, 0xE6, 0x25, 0xE6, 0x25, 0xE6, 0x25, 0xE6, 0x25, 0xE2, 0x25, 0xE6, 0x43, 0x43, 0xFC, 0xF0, 0x02, 0x82, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00]
+    - [0x8F, 0xFD, 0xF8, 0x02, 0x10, 0x28, 0x81]
+    - [0x8E, 0xFD, 0xF8, 0x02, 0x10, 0x28]
+    - [0x80]
+    - [0x80]
+    - [0x80]
+    - [0x80]
+    - [0x80]
+    - [0x80, 0x66]
+    - [0x80, 0x80]
+    - [0x80, 0x80]
+    - [0x80, 0x80]
+    - [0x8D, 0xFC, 0xFF, 0xF6, 0xFF]
+    - [0x10, 0xC0, 0x16, 0xF6, 0xFF, 0xF1, 0x03, 0xFF, 0x43, 0x43, 0xFC]
+    - [0x80]

tests/test_fuzzing.py

@@ -0,0 +1,119 @@
+#
+# Copyright (c) 2017 Grigori Goronzy <greg@chown.ath.cx>
+#
+# Permission is hereby granted, free of charge, to any person obtaining a copy
+# of this software and associated documentation files (the "Software"), to deal
+# in the Software without restriction, including without limitation the rights
+# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+# copies of the Software, and to permit persons to whom the Software is
+# furnished to do so, subject to the following conditions:
+#
+# The above copyright notice and this permission notice shall be included in all
+# copies or substantial portions of the Software.
+#
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+# SOFTWARE.
+#
+
+"""Tests with fuzzing of input data"""
+
+import random
+import sys
+import unittest
+from unittest.mock import patch
+import yaml
+import stcgal.frontend
+import stcgal.protocols
+from tests.test_program import get_default_opts, convert_to_bytes
+
+class ByteArrayFuzzer:
+    """Fuzzer for byte arrays"""
+
+    def __init__(self):
+        self.rng = random.Random()
+        self.cut_propability = 0.01 # probability for cutting off an array early
+        self.cut_min = 0 # minimum cut amount
+        self.cut_max = sys.maxsize # maximum cut amount
+        self.bitflip_probability = 0.0001 # probability for flipping a bit
+        self.randomize_probability = 0.001 # probability for randomizing a char
+
+    def fuzz(self, inp):
+        """Fuzz an array of bytes according to predefined settings"""
+        arr = bytearray(inp)
+        arr = self.cut_off(arr)
+        self.randomize(arr)
+        return bytes(arr)
+
+    def randomize(self, arr):
+        """Randomize array contents with bitflips and random bytes"""
+        for i, _ in enumerate(arr):
+            for j in range(8):
+                if self.rng.random() < self.bitflip_probability:
+                    arr[i] ^= (1 << j)
+            if self.rng.random() < self.randomize_probability:
+                arr[i] = self.rng.getrandbits(8)
+
+    def cut_off(self, arr):
+        """Cut off data from end of array"""
+        if self.rng.random() < self.cut_propability:
+            cut_limit = min(len(arr), self.cut_max)
+            cut_len = self.rng.randrange(self.cut_min, cut_limit)
+            arr = arr[0:len(arr) - cut_len]
+        return arr
+
+class TestProgramFuzzed(unittest.TestCase):
+    """Special programming cycle tests that use a fuzzing approach"""
+
+    @patch("stcgal.protocols.StcBaseProtocol.read_packet")
+    @patch("stcgal.protocols.Stc89Protocol.write_packet")
+    @patch("stcgal.protocols.serial.Serial", autospec=True)
+    @patch("stcgal.protocols.time.sleep")
+    @patch("sys.stdout")
+    @patch("sys.stderr")
+    def test_program_fuzz(self, err, out, sleep_mock, serial_mock, write_mock, read_mock):
+        """Test programming cycles with fuzzing enabled"""
+        yml = [
+            "./tests/iap15f2k61s2.yml",
+            "./tests/stc12c2052ad.yml",
+            "./tests/stc15w4k56s4.yml",
+            "./tests/stc12c5a60s2.yml",
+            "./tests/stc89c52rc.yml",
+            "./tests/stc15l104w.yml",
+            "./tests/stc15f104e.yml",
+        ]
+        fuzzer = ByteArrayFuzzer()
+        fuzzer.cut_propability = 0.01
+        fuzzer.bitflip_probability = 0.005
+        fuzzer.rng = random.Random(1)
+        for y in yml:
+            with self.subTest(msg="trace {}".format(y)):
+                self.single_fuzz(y, serial_mock, fuzzer, read_mock, err, out,
+                                 sleep_mock, write_mock)
+
+    def single_fuzz(self, yml, serial_mock, fuzzer, read_mock, err, out, sleep_mock, write_mock):
+        """Test a single programming cycle with fuzzing"""
+        with open(yml) as test_file:
+            test_data = yaml.load(test_file.read())
+            for _ in range(1000):
+                with self.subTest():
+                    opts = get_default_opts()
+                    opts.protocol = test_data["protocol"]
+                    opts.code_image.read.return_value = bytes(test_data["code_data"])
+                    serial_mock.return_value.inWaiting.return_value = 1
+                    fuzzed_responses = []
+                    for arr in convert_to_bytes(test_data["responses"]):
+                        fuzzed_responses.append(fuzzer.fuzz(arr))
+                    read_mock.side_effect = fuzzed_responses
+                    gal = stcgal.frontend.StcGal(opts)
+                    self.assertGreaterEqual(gal.run(), 0)
+                    err.reset_mock()
+                    out.reset_mock()
+                    sleep_mock.reset_mock()
+                    serial_mock.reset_mock()
+                    write_mock.reset_mock()
+                    read_mock.reset_mock()

tests/test_program.py

@@ -0,0 +1,135 @@
+#
+# Copyright (c) 2017 Grigori Goronzy <greg@chown.ath.cx>
+#
+# Permission is hereby granted, free of charge, to any person obtaining a copy
+# of this software and associated documentation files (the "Software"), to deal
+# in the Software without restriction, including without limitation the rights
+# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+# copies of the Software, and to permit persons to whom the Software is
+# furnished to do so, subject to the following conditions:
+#
+# The above copyright notice and this permission notice shall be included in all
+# copies or substantial portions of the Software.
+#
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+# SOFTWARE.
+#
+
+"""Tests that simulate a whole programming cycle"""
+
+import unittest
+from unittest.mock import patch
+import yaml
+import stcgal.frontend
+import stcgal.protocols
+
+def convert_to_bytes(list_of_lists):
+    """Convert lists of integer lists to list of byte lists"""
+    return [bytes(x) for x in list_of_lists]
+
+def get_default_opts():
+    """Get a default preconfigured option object"""
+    opts = unittest.mock.MagicMock()
+    opts.protocol = "stc89"
+    opts.autoreset = False
+    opts.port = ""
+    opts.baud = 19200
+    opts.handshake = 9600
+    opts.trim = 22118
+    opts.eeprom_image = None
+    opts.debug = False
+    opts.code_image.name = "test.bin"
+    opts.code_image.read.return_value = b"123456789"
+    return opts
+
+class ProgramTests(unittest.TestCase):
+    """Test MCU programming cycles for different families, based on traces"""
+
+    @patch("stcgal.protocols.StcBaseProtocol.read_packet")
+    @patch("stcgal.protocols.Stc89Protocol.write_packet")
+    @patch("stcgal.protocols.serial.Serial", autospec=True)
+    @patch("stcgal.protocols.time.sleep")
+    @patch("sys.stdout")
+    def test_program_stc89(self, out, sleep_mock, serial_mock, write_mock, read_mock):
+        """Test a programming cycle with STC89 protocol"""
+        self._program_yml("./tests/stc89c52rc.yml", serial_mock, read_mock)
+
+    @patch("stcgal.protocols.StcBaseProtocol.read_packet")
+    @patch("stcgal.protocols.Stc89Protocol.write_packet")
+    @patch("stcgal.protocols.serial.Serial", autospec=True)
+    @patch("stcgal.protocols.time.sleep")
+    @patch("sys.stdout")
+    def test_program_stc12(self, out, sleep_mock, serial_mock, write_mock, read_mock):
+        """Test a programming cycle with STC12 protocol"""
+        self._program_yml("./tests/stc12c5a60s2.yml", serial_mock, read_mock)
+
+    @patch("stcgal.protocols.StcBaseProtocol.read_packet")
+    @patch("stcgal.protocols.Stc89Protocol.write_packet")
+    @patch("stcgal.protocols.serial.Serial", autospec=True)
+    @patch("stcgal.protocols.time.sleep")
+    @patch("sys.stdout")
+    def test_program_stc12a(self, out, sleep_mock, serial_mock, write_mock, read_mock):
+        """Test a programming cycle with STC12A protocol"""
+        self._program_yml("./tests/stc12c2052ad.yml", serial_mock, read_mock)
+
+    def test_program_stc12b(self):
+        """Test a programming cycle with STC12B protocol"""
+        self.skipTest("trace missing")
+
+    @patch("stcgal.protocols.StcBaseProtocol.read_packet")
+    @patch("stcgal.protocols.Stc89Protocol.write_packet")
+    @patch("stcgal.protocols.serial.Serial", autospec=True)
+    @patch("stcgal.protocols.time.sleep")
+    @patch("sys.stdout")
+    def test_program_stc15f2(self, out, sleep_mock, serial_mock, write_mock, read_mock):
+        """Test a programming cycle with STC15 protocol, F2 series"""
+        self._program_yml("./tests/iap15f2k61s2.yml", serial_mock, read_mock)
+
+    @patch("stcgal.protocols.StcBaseProtocol.read_packet")
+    @patch("stcgal.protocols.Stc89Protocol.write_packet")
+    @patch("stcgal.protocols.serial.Serial", autospec=True)
+    @patch("stcgal.protocols.time.sleep")
+    @patch("sys.stdout")
+    def test_program_stc15w4(self, out, sleep_mock, serial_mock, write_mock, read_mock):
+        """Test a programming cycle with STC15 protocol, W4 series"""
+        self._program_yml("./tests/stc15w4k56s4.yml", serial_mock, read_mock)
+
+    @unittest.skip("trace is broken")
+    @patch("stcgal.protocols.StcBaseProtocol.read_packet")
+    @patch("stcgal.protocols.Stc89Protocol.write_packet")
+    @patch("stcgal.protocols.serial.Serial", autospec=True)
+    @patch("stcgal.protocols.time.sleep")
+    @patch("sys.stdout")
+    def test_program_stc15a(self, out, sleep_mock, serial_mock, write_mock, read_mock):
+        """Test a programming cycle with STC15A protocol"""
+        self._program_yml("./tests/stc15f104e.yml", serial_mock, read_mock)
+
+    @patch("stcgal.protocols.StcBaseProtocol.read_packet")
+    @patch("stcgal.protocols.Stc89Protocol.write_packet")
+    @patch("stcgal.protocols.serial.Serial", autospec=True)
+    @patch("stcgal.protocols.time.sleep")
+    @patch("sys.stdout")
+    def test_program_stc15l1(self, out, sleep_mock, serial_mock, write_mock, read_mock):
+        """Test a programming cycle with STC15 protocol, L1 series"""
+        self._program_yml("./tests/stc15l104w.yml", serial_mock, read_mock)
+
+    def test_program_stc15w4_usb(self):
+        """Test a programming cycle with STC15W4 USB protocol"""
+        self.skipTest("USB not supported yet, trace missing")
+
+    def _program_yml(self, yml, serial_mock, read_mock):
+        """Program MCU with data from YAML file"""
+        with open(yml) as test_file:
+            test_data = yaml.load(test_file.read())
+            opts = get_default_opts()
+            opts.protocol = test_data["protocol"]
+            opts.code_image.read.return_value = bytes(test_data["code_data"])
+            serial_mock.return_value.inWaiting.return_value = 1
+            read_mock.side_effect = convert_to_bytes(test_data["responses"])
+            gal = stcgal.frontend.StcGal(opts)
+            self.assertEqual(gal.run(), 0)

tests/test_utils.py

@@ -0,0 +1,76 @@
+#
+# Copyright (c) 2017 Grigori Goronzy <greg@chown.ath.cx>
+#
+# Permission is hereby granted, free of charge, to any person obtaining a copy
+# of this software and associated documentation files (the "Software"), to deal
+# in the Software without restriction, including without limitation the rights
+# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+# copies of the Software, and to permit persons to whom the Software is
+# furnished to do so, subject to the following conditions:
+#
+# The above copyright notice and this permission notice shall be included in all
+# copies or substantial portions of the Software.
+#
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+# SOFTWARE.
+#
+
+"""Tests for utility functions and other misc parts"""
+
+import argparse
+import unittest
+from stcgal.utils import Utils, BaudType
+
+class TestUtils(unittest.TestCase):
+    """Test for utility functions in the Utils class"""
+
+    def test_to_bool(self):
+        """Test special utility function for bool conversion"""
+        self.assertTrue(Utils.to_bool(True))
+        self.assertTrue(Utils.to_bool("true"))
+        self.assertTrue(Utils.to_bool("True"))
+        self.assertTrue(Utils.to_bool("t"))
+        self.assertTrue(Utils.to_bool("T"))
+        self.assertTrue(Utils.to_bool(1))
+        self.assertTrue(Utils.to_bool(-1))
+        self.assertFalse(Utils.to_bool(0))
+        self.assertFalse(Utils.to_bool(None))
+        self.assertFalse(Utils.to_bool("false"))
+        self.assertFalse(Utils.to_bool("False"))
+        self.assertFalse(Utils.to_bool("f"))
+        self.assertFalse(Utils.to_bool("F"))
+        self.assertFalse(Utils.to_bool(""))
+
+    def test_to_int(self):
+        """Test wrapped integer conversion"""
+        self.assertEqual(Utils.to_int("2"), 2)
+        self.assertEqual(Utils.to_int("0x10"), 16)
+        with self.assertRaises(ValueError):
+            Utils.to_int("a")
+        with self.assertRaises(ValueError):
+            Utils.to_int("")
+        with self.assertRaises(ValueError):
+            Utils.to_int(None)
+
+    def test_hexstr(self):
+        """Test byte array formatter"""
+        self.assertEqual(Utils.hexstr([10]), "0A")
+        self.assertEqual(Utils.hexstr([1, 2, 3]), "010203")
+        with self.assertRaises(Exception):
+            Utils.hexstr([400, 500])
+
+class TestBaudType(unittest.TestCase):
+    """Test BaudType class"""
+
+    def test_create_baud_type(self):
+        """Test creation of BaudType instances"""
+        baud_type = BaudType()
+        self.assertEqual(baud_type("2400"), 2400)
+        self.assertEqual(baud_type("115200"), 115200)
+        with self.assertRaises(argparse.ArgumentTypeError):
+            baud_type("2374882")