CI: try to fix and simplify apt setup

Run apt update to update the package cache, as this seems to be needed by
Travis now. Also reduce the package set - we don't build any deb or rpm
packages anymore. This should hopefully fix the CI build.

.travis.yml

@@ -9,25 +9,10 @@ python:
   - "3.6"
   - "pypy3"
 before_install:
-  - sudo apt install rpm dpkg-dev debhelper dh-python python3-setuptools fakeroot python3-serial python3-yaml
+  - sudo apt update
+  - sudo apt install python3-setuptools python3-serial python3-yaml
 install:
   - pip install pyusb
 script:
   - python setup.py build
   - python setup.py test
-before_deploy:
-  - deactivate
-  - python3 setup.py bdist_rpm
-  - dpkg-buildpackage -uc -us
-  - cp ../*.deb dist/
-deploy:
-  provider: releases
-  api_key: $GH_TOKEN
-  file_glob: true
-  file:
-  - dist/stcgal*_all.deb
-  - dist/stcgal*.noarch.rpm
-  skip_cleanup: true
-  on:
-    tags: true
-    python: "3.4"

README.md

@@ -32,6 +32,17 @@ Features
 * Automatic power-cycling with DTR toggle or a custom shell command
 * Automatic UART protocol detection
 
+Quickstart
+----------
+
+Install stcgal (might need root/administrator privileges):
+    
+    pip3 install stcgal
+
+Call stcgal and show usage:
+
+    stcgal -h
+
 Further information
 -------------------
 

debian/changelog

@@ -1,3 +1,9 @@
+stcgal (1.6) unstable; urgency=low
+
+  * Update to 1.6
+
+ -- Grigori Goronzy <greg@chown.ath.cx>  Mon, 24 Sep 2018 22:56:31 +0200
+
 stcgal (1.4) unstable; urgency=low
 
   * Update to 1.4

debian/docs

@@ -1,2 +1,4 @@
 README.md
-TODO.md
+doc/USAGE.md
+doc/MODELS.md
+doc/FAQ.md

doc/FAQ.md

@@ -7,7 +7,7 @@ By design, this is not possible with STC's bootloader protocols. This is conside
 
 ### Which serial interfaces have been tested with stcgal?
 
-The following USB-based UART interface chips have been successfully tested with stcgal:
+stcgal should work fine with common 16550 compatible UARTs that are traditionally available on many platforms. However, nowadays, USB-based UARTs are the typical case. The following USB-based UART interface chips have been successfully tested with stcgal:
 
 * FT232 family (OS: Linux, Windows)
 * CH340/CH341 (OS: Windows, Linux requires Kernel 4.10)
@@ -20,6 +20,10 @@ Interfaces that are known to not work:
 
 In general, stcgal requires accurate baud rate timings and parity support.
 
+### stcgal fails to start with the error `module 'serial' has no attribute 'PARITY_NONE'` or similar
+
+There is a module name conflict between the PyPI package 'serial' (a data serialization library) and the PyPI package 'pyserial' (the serial port access library needed by stcgal). You have to uninstall the 'serial' package (`pip3 uninstall serial`) and reinstall 'pyserial' (`pip3 install --force-reinstall pyserial`) to fix this. There is no other known solution at the moment.
+
 ### stcgal fails to recognize the MCU and is stuck at "Waiting for MCU"
 
 There are a number of issues that can result in this symptom:
@@ -40,7 +44,7 @@ Various remedies are possible to avoid parasitic powering.
 
 First, make sure that the frequency specified uses the correct unit. The frequency is specified in kHz and the safe range is approximately 5000 kHz - 30000 kHz. Furthermore, frequency trimming uses the UART clock as the clock reference, so UART incompatibilities or clock inaccuracies can also result in issues with frequency trimming. If possible, try another UART chip.
 
-### Baud rate switching fails
+### Baud rate switching fails or flash programming fails
 
 This can especially happen at high programming baud rates, e.g. 115200 baud. Try a lower baudrate, or stick to the default of 19200 baud. Some USB UARTs are known to cause problems due to inaccurate timing as well, which can lead to various issues.
 

doc/INSTALL.md

@@ -1,16 +1,15 @@
 Installation
 ============
 
-stcgal requires Python 3.2 (or later) and pySerial. USB support is
-optional and requires pyusb 1.0.0b2 or later. You can run stcgal
-directly with the included ```stcgal.py``` script if the dependencies
-are already installed.
+stcgal requires Python 3.2 (or later), pyserial 3.0 or later and
+TQDM 4.0.0 or later. USB support is optional and requires pyusb
+1.0.0b2 or later. You can run stcgal directly with the included
+```stcgal.py``` script if the dependencies are already installed.
 
 There are several options for permanent installation:
 
-* Use Python3 and ```pip```. Run
-```pip3 install git+https://github.com/grigorig/stcgal.git```
-to install the latest version of stcgal globally on your system.
+* Use Python3 and ```pip```. Run ```pip3 install stcgal``` to
+install the latest release of stcgal globally on your system.
 This may require administrator/root permissions for write access
 to system directories.
 

doc/MODELS.md

@@ -1,7 +1,7 @@
 Supported MCU models
 ====================
 
-stcgal should fully support STC 89/90/10/11/12/15 series MCUs. Support for STC8 series MCUs is work in progress.
+stcgal should fully support STC 89/90/10/11/12/15/8 series MCUs.
 
 So far, stcgal was tested with the following MCU models:
 

doc/USAGE.md

@@ -4,12 +4,14 @@ Usage
 Call stcgal with ```-h``` for usage information.
 
 ```
-usage: stcgal.py [-h] [-a] [-P {stc89,stc12a,stc12,stc15a,stc15,auto}]
+usage: stcgal.py [-h] [-a] [-r RESETCMD]
+                 [-P {stc89,stc12a,stc12b,stc12,stc15a,stc15,stc8,usb15,auto}]
                  [-p PORT] [-b BAUD] [-l HANDSHAKE] [-o OPTION] [-t TRIM] [-D]
+                 [-V]
                  [code_image] [eeprom_image]
 
-stcgal 1.0 - an STC MCU ISP flash tool
-(C) 2014-2015 Grigori Goronzy
+stcgal 1.5 - an STC MCU ISP flash tool
+(C) 2014-2018 Grigori Goronzy and others
 https://github.com/grigorig/stcgal
 
 positional arguments:
@@ -20,18 +22,20 @@ optional arguments:
   -h, --help            show this help message and exit
   -a, --autoreset       cycle power automatically by asserting DTR
   -r RESETCMD, --resetcmd RESETCMD
-                        Use this shell command for board power-cycling
-                        (instead of DTR assertion)
-  -P {stc89,stc12a,stc12,stc15a,stc15,auto}, --protocol {stc89,stc12a,stc12,stc15a,stc15,auto}
-                        protocol version
+                        shell command for board power-cycling (instead of DTR
+                        assertion)
+  -P {stc89,stc12a,stc12b,stc12,stc15a,stc15,stc8,usb15,auto}, --protocol {stc89,stc12a,stc12b,stc12,stc15a,stc15,stc8,usb15,auto}
+                        protocol version (default: auto)
   -p PORT, --port PORT  serial port device
   -b BAUD, --baud BAUD  transfer baud rate (default: 19200)
   -l HANDSHAKE, --handshake HANDSHAKE
                         handshake baud rate (default: 2400)
   -o OPTION, --option OPTION
-                        set option (can be used multiple times)
-  -t TRIM, --trim TRIM  RC oscillator frequency in kHz (STC15 series only)
+                        set option (can be used multiple times, see
+                        documentation)
+  -t TRIM, --trim TRIM  RC oscillator frequency in kHz (STC15+ series only)
   -D, --debug           enable debug output
+  -V, --version         print version info and exit
 ```
 
 Most importantly, ```-p``` sets the serial port to be used for programming.
@@ -43,6 +47,7 @@ BSL. The protocol can be specified with the ```-P``` flag. By default
 UART protocol autodetection is used. The mapping between protocols
 and MCU series is as follows:
 
+* ```auto``` Automatic detection of UART based protocols (default)
 * ```stc89``` STC89/90 series
 * ```stc12a``` STC12x052 series and possibly others
 * ```stc12b``` STC12x52 series, STC12x56 series and possibly others
@@ -51,11 +56,10 @@ and MCU series is as follows:
 * ```stc15``` Most STC15 series
 * ```stc8``` STC8 series
 * ```usb15``` USB support on STC15W4 series
-* ```auto``` Automatic detection of UART based protocols (default)
 
-The text files in the doc/ subdirectory provide an overview over
-the reverse engineered protocols used by the BSLs. For more details,
-please read the source code.
+The text files in the doc/reverse-engineering subdirectory provide an
+overview over the reverse engineered protocols used by the BSLs. For
+more details, please read the source code.
 
 ### Getting MCU information
 
@@ -92,6 +96,8 @@ Target options:
 Disconnected!
 ```
 
+If the identification fails, see the [FAQ](FAQ.md) for troubleshooting.
+
 ### Program the flash memory
 
 stcgal supports Intel HEX encoded files as well as binary files. Intel
@@ -195,8 +201,8 @@ If the internal RC oscillator is used (```clock_source=internal```),
 stcgal can execute a trim procedure to adjust it to a given value. This
 is only supported by STC15 series and newer. The trim values are stored
 with device options. Use the ```-t``` flag to request trimming to a certain
-value. Generally, frequencies between 4 and 35 MHz can be achieved. If
-trimming fails, stcgal will abort.
+value. Generally, frequencies between 4000 and 30000 kHz can be achieved.
+If trimming fails, stcgal will abort.
 
 ### Automatic power-cycling
 

setup.py

@@ -31,7 +31,6 @@ setup(
     name = "stcgal",
     version = stcgal.__version__,
     packages = find_packages(exclude=["doc", "tests"]),
-    data_files = [("doc", ["README.md", "doc/FAQ.md", "doc/MODELS.md", "doc/USAGE.md"])],
     install_requires = ["pyserial>=3.0", "tqdm>=4.0.0"],
     extras_require = {
         "usb": ["pyusb>=1.0.0"]

stcgal/__init__.py

@@ -1 +1 @@
-__version__ = "1.5"
+__version__ = "1.6"

stcgal/frontend.py

@@ -147,6 +147,10 @@ class StcGal:
     def run(self):
         """Run programmer, main entry point."""
 
+        if self.opts.version:
+            print("stcgal {}".format(stcgal.__version__))
+            return 0
+
         try:
             self.protocol.connect(autoreset=self.opts.autoreset, resetcmd=self.opts.resetcmd)
             if isinstance(self.protocol, StcAutoProtocol):
@@ -178,6 +182,10 @@ class StcGal:
             sys.stdout.flush()
             print("I/O error: %s" % ex, file=sys.stderr)
             return 1
+        except Exception as ex:
+            sys.stdout.flush()
+            print("Unexpected error: %s" % ex, file=sys.stderr)
+            return 1
 
         try:
             if self.opts.code_image:
@@ -218,15 +226,16 @@ def cli():
     parser.add_argument("code_image", help="code segment file to flash (BIN/HEX)", type=argparse.FileType("rb"), nargs='?')
     parser.add_argument("eeprom_image", help="eeprom segment file to flash (BIN/HEX)", type=argparse.FileType("rb"), nargs='?')
     parser.add_argument("-a", "--autoreset", help="cycle power automatically by asserting DTR", action="store_true")
-    parser.add_argument("-r", "--resetcmd",  help="Use this shell command for board power-cycling (instead of DTR assertion)", action="store")
+    parser.add_argument("-r", "--resetcmd",  help="shell command for board power-cycling (instead of DTR assertion)", action="store")
     parser.add_argument("-P", "--protocol", help="protocol version (default: auto)",
                         choices=["stc89", "stc12a", "stc12b", "stc12", "stc15a", "stc15", "stc8", "usb15", "auto"], default="auto")
     parser.add_argument("-p", "--port", help="serial port device", default="/dev/ttyUSB0")
     parser.add_argument("-b", "--baud", help="transfer baud rate (default: 19200)", type=BaudType(), default=19200)
     parser.add_argument("-l", "--handshake", help="handshake baud rate (default: 2400)", type=BaudType(), default=2400)
-    parser.add_argument("-o", "--option", help="set option (can be used multiple times)", action="append")
-    parser.add_argument("-t", "--trim", help="RC oscillator frequency in kHz (STC15 series only)", type=float, default=0.0)
+    parser.add_argument("-o", "--option", help="set option (can be used multiple times, see documentation)", action="append")
+    parser.add_argument("-t", "--trim", help="RC oscillator frequency in kHz (STC15+ series only)", type=float, default=0.0)
     parser.add_argument("-D", "--debug", help="enable debug output", action="store_true")
+    parser.add_argument("-V", "--version", help="print version info and exit", action="store_true")
     opts = parser.parse_args()
 
     # run programmer

tests/stc8f2k08s2-untrimmed.yml

@@ -0,0 +1,5 @@
+name: STC8F2K08S2 untrimmed programming test
+protocol: stc8
+code_data: [49, 50, 51, 52, 53, 54, 55, 56, 57]
+responses:
+  - [0x50, 0xFF, 0xFF, 0xFF, 0xFF, 0x8F, 0x00, 0x04, 0xFF, 0xFF, 0x8B, 0xFD, 0xFF, 0x27, 0x38, 0xF5, 0x73, 0x73, 0x55, 0x00, 0xF6, 0x41, 0x0A, 0x88, 0x86, 0x6F, 0x8F, 0x08, 0x20, 0x20, 0x20, 0x01, 0x00, 0x00, 0x20, 0x05, 0x3C, 0x18, 0x05, 0x22, 0x32, 0xFF]

tests/test_program.py

@@ -27,6 +27,7 @@ from unittest.mock import patch
 import yaml
 import stcgal.frontend
 import stcgal.protocols
+from stcgal.protocols import StcProtocolException
 
 def convert_to_bytes(list_of_lists):
     """Convert lists of integer lists to list of byte lists"""
@@ -43,6 +44,7 @@ def get_default_opts():
     opts.trim = 22118
     opts.eeprom_image = None
     opts.debug = False
+    opts.version = False
     opts.code_image.name = "test.bin"
     opts.code_image.read.return_value = b"123456789"
     return opts
@@ -127,6 +129,24 @@ class ProgramTests(unittest.TestCase):
         """Test a programming cycle with STC15 protocol, L1 series"""
         self._program_yml("./tests/stc15l104w.yml", serial_mock, read_mock)
 
+    @patch("stcgal.protocols.StcBaseProtocol.read_packet")
+    @patch("stcgal.protocols.Stc89Protocol.write_packet")
+    @patch("stcgal.protocols.serial.Serial", autospec=True)
+    @patch("stcgal.protocols.time.sleep")
+    @patch("sys.stdout")
+    def test_program_stc8_untrimmed(self, out, sleep_mock, serial_mock, write_mock, read_mock):
+        """Test error with untrimmed MCU"""
+        with open("./tests/stc8f2k08s2-untrimmed.yml") as test_file:
+            test_data = yaml.load(test_file.read())
+            opts = get_default_opts()
+            opts.trim = 0.0
+            opts.protocol = test_data["protocol"]
+            opts.code_image.read.return_value = bytes(test_data["code_data"])
+            serial_mock.return_value.inWaiting.return_value = 1
+            read_mock.side_effect = convert_to_bytes(test_data["responses"])
+            gal = stcgal.frontend.StcGal(opts)
+            self.assertEqual(gal.run(), 1)
+
     def test_program_stc15w4_usb(self):
         """Test a programming cycle with STC15W4 USB protocol"""
         self.skipTest("USB not supported yet, trace missing")