New board revison, bulk buy, discount on 2.0.4 gateway

Last week I've received new 2.0.4 gateway boards. There are a few visible changes like connector description on silk screen and cut out hole for zip tie to hold on the RFM69 pigtail cable. And a few not so visible changes in routing and such, but none of them are really a reason to call it a new revision.

One problem that I've started to realize is relatively pricier GSM modem. The 2G/3G/4G SIM7600x are starting at 40EUR for regional and 50EUR for Global band coverage. I was looking for some cheaper alternative while still keeping at least 4G, and the apparent replacement seemed to be a new SimCom A7670x. Unfortunately these use different package, and are only covering Europe and China, still I deiced to give it a go and spun a new version of gateway. Just few days before sending it to fab I received a newsletter from SimCom that they released same modem module called A7600E which is pin compatible to one that gateway already use (7600X). One difference it is still Europe only and only 2G/4G. I think it is reasonable as 3G is, at least in Europe, being replaced by 4G/5G networks. Good thing is it requires only one antenna and it is 5EUR cheaper. I've ordered 2 units to test them.

For gateway, I have consolidated all the components, and I was able to get the price down by making bulk buy for all pricier components. Like buying the STM32F437 at RS Components, were it is 50% of my local distributor price.. This should reduce the final price to some 92EUR without a modem.

Adding MQTT to gateway 2.x

MQTT was somehow last step of adding features present in gateways 1.x to gateway 2.x. Now with last version of 1.3 release, already available in GitHub releases, this come to live. For now only publish functions are implemented, but subscribe procedures and skeleton is already prepared. Beware that 1.3 release forces whole configuration to be erased! I suggest to take screenshots of each page before upgrading. I know it is not very convenient, and I will try to implement mechanisms that such future changes will be a bit elegant. That is without the need to reconfigure whole gateway again.

MQTT publish functions are now having their own thread, that is collecting queue of publish request. The thread then pushes one publish a time when system allows it.

The publish structure is a bit different, as it now more relies on given indexes and addresses rather then on names of elements. Main topis is OHS with subtopics:

 /state {On, Off} - Indicates if system is on
 /group
   /{#} - index of group
     /name
     /state {disarmed, arming, armed_home, armed_away, triggered, disarming}
 /zone
   /{#} - index of zone
     /name
     /state {OK, alarm, tamper}
 /sensor
   /{address} - node address like W:2:K:i:0
     /name
     /value

There are other small changes like Nodes tab is now showing value of authentication nodes, that is a name of person who last armed/disarmed this node. Same name is also passed to MQTT sensor value for same node type.

You can test now new 1.3 version with initial MQTT support, but there can be some rough edges.

Expansion board

Gateways 2.x have now a slightly reduced numbers of input. This is a little trade off from previous versions 1.x that have 8 analog(balanced) and 4 digital(unbalanced) ports. 2.x gateways have increased the balanced inputs to 10 ports, and also added ability to switch any of this port to be treated as unbalanced one. But there is no digital only port. Instead I started to design new extension board that will allow to add 8 extra ports. These ports will also be hybrid, meaning that they can be switched in software to unbalanced. The board is already in fabrication and should look like picture on right.

I did not yet started coding, so I hope it will be still possible with good old ATmega328P. Also not sure if this application has need for RTOS, or if it is doable only in as standard Arduino sketch.

Anyway it has possibility to expand the zones by 8, and the connection can be established via wire(RS485) or radio(RFM69). It has onboard relay and similar power detection circuit as gateway. There are only 2 free pins left that are taken out to allow some user application like additional relay board.

Gateway firmware is already able to allow 2 such boards to be connected.

Pages reorganization.

Pages menu on right are now reorganized by gateway 1.x and 2.x for better access, and some pages like compiling and firmware were grouped together. Also gateway 2.x configuration is now complete. 

Further more new static page 2.x scripts is created where I will publish summary for scripts and TCL commands along with example scripts.

Travis CI and precompiled firmware for OHS 2.x

Over last weekend I've created a Travis continuous integration for OHS 2.x gateway repository. This CI performs a test build on every commit that is pushed to it. What it does is that it verifies that the commit is buildable, and maybe more importantly, it creates new compiled firmware related to every new commit, and places it back in releases section directly in GitHub.

This means that the latest, and also historical firmware are now available for download in https://github.com/vysocan/OHS_2-gateway/releases 

Binary files .bin can be directly used for DFU - Device Firmware Upgrade.

TCL scripts

TCL scripts is a new functionality in OHS 2.x. They allow various enhancements to be added by users to the automation part of the gateway. As the base of TCL language I used the following library: https://github.com/zserge/partcl, and embed it into gateway firmware. Scripts are handled by a separate thread that has a queue that serves all incoming scripts, and passes the result back asynchronously as a call back. Scripts are using their own separated heap, provided by umm_malloc library, that performs great heap management. Embedded SPI FRAM is used as storage for scripts with 64kB of available space. I have written a simple block access library called uBS (micro block system) that allows basic read write operations.

Scripts also have their own tab in the web interface, which allows you to create and run the TCL scripts directly on the gateway. As shown on the picture, there is statistics about the heap and the storage used by scripts. as well as helper icons </> and #. The </>, when you hover over it, shows build in language helps. And the # shows all variables currently in heap with their values.

Button "Run" passes the edited script into execution queue. "Refresh" button simply refreshes the page, allowing the "Last output:" to be seen. "Save" stores the script with given name to uBS storage, and allows this scripts to be assigned to triggers and timers.

Due to performance limitations, since scripts are not primary function of the gateway, there are build in limits to maximum script execution time, and maximum script interactions.

Language syntax

Tcl script is made up of commands separated by semicolons or newline symbols. Commands in their turn are made up of words separated by whitespace. To make whitespace a part of the word one may use double quotes or braces.

An important part of the language is command substitution, when the result of a command inside square braces is returned as a part of the outer command, e.g. puts [+ 1 2] performs addition of 1 + 2 and passes the result to puts.

Currently the only data type of the language is a string. Even numbers are stored in string format and converted to numeric types when needed.

Any symbol can be part of the word, except for the following special symbols:

•     whitespace, tab - used to delimit words
•     \r, \n, semicolon or EOF - used to delimit commands
•     Braces, square brackets, dollar sign - used for substitution and grouping

 

Interpreter

Partcl interpreter is a simple structure which keeps the current environment, array of available commands and a last result value. Interpreter logic is wrapped around two functions - evaluation and substitution.

Substitution:

• If argument starts with $ - create a temporary command [set name] and evaluate it. In Tcl $foo is just a shortcut to [set foo], which returns the value of "foo" variable in the current environment.
• If argument starts with [ - evaluate what's inside the square brackets and return the result.
• If argument is a quoted string (e.g. {foo bar}) - return it as is, just without braces.
• Otherwise return the argument as is.

Evaluation:

• Iterates over each token in a list.
• Appends words into a list.
• If the command end is met (semicolor, or newline, or end-of-file - our lexer has a special token type TCMD for them) - then find a suitable command (the first word in the list) and call it.

Each command has a name, arity (how many arguments is shall take - interpreter checks it before calling the command, use zero arity for varargs) and a C function pointer that actually implements the command. 

 

Builtin commands

• "set" - assigns value to the variable (if any) and returns the current variable value.
• "subst" - does command substitution in the argument string.
• "puts" - prints argument to the tcl_stdout buffer, followed by a newline
• "proc" - creates a new command appending it to the list of current interpreter commands. That's how user-defined commands are built.
• "if" - does a simple if {cond} {then} {cond2} {then2} {else}.
• "while" - runs a while loop while {cond} {body}. One may use "break", "continue" or "return" inside the loop to contol the flow.
• Various math operations are implemented like:  !, +, -, *, /, >, >=, <, <=, ==, !=, &&, ||.
• "string" - performs simple string manipulation, like"compare" and "length".
• "clock" - allows time and date manipulation, like "seconds", "format" and "add".
  

Gateway 2.x configuration

Just started to update new page section for gateway 2.x configuration. It can be find in menu on right. It will describe basics terminology for OHS, new shell interface for OHS 2.x gateways and of course a web interface used to configure the functionality.  Currently it is work in progress, but I will try to add all sections that are missing, and finish it as primary guide for new users.