Gateway Version 1.5.6

Stability, SMS Integration, and UI Refinement

I'd like to announce the release of v1.5.6 for the OHS_2-gateway. This update brings critical under-the-hood refactoring for better performance while introducing several requested features for daily usability.

🚀 Key New Features

• HTTP Server Advanced Alerting System:  The Web UI alerts are overhauled. You’ll now see distinct visual feedback for Info, Warning, and Error levels, making it much easier to confirm when settings have been saved correctly.

• "Zones Needed to ARM" Logic: This is a major security enhancement. You can now flag specific zones as "required." If you attempt to arm a group while one of these zones is compromised, the request will be rejected. There are plans for follow-up updates for authentication nodes to provide audible feedback when an arming request is rejected for this reason.

• MQTT SMS Integration: We’ve expanded our MQTT capabilities. You can now trigger SMS messages via two new subscribe topics:

  /SMS/contact/{index}/message – Target a specific user.
  /SMS/group/{index}/message – Broadcast to an entire alarm group.

• Config Portability: You can now download and upload simple configuration files directly through the web interface.

🎨 UI & UX Improvements

• Cleaner Layout: We’ve removed the bottom slider for a more streamlined page layout.

• Cross-Platform Consistency: CSS and button styles have been updated to ensure text visibility and button alignment are consistent, specifically targeting fixes for users on macOS.

• SMS Status Checks: You can now query system status via SMS commands for better remote management.

🛠️ Under the Hood: Performance & Refactoring

A massive amount of work went into the codebase for this release to ensure the gateway remains fast and maintainable:

• HTTP Server Overhaul: The web server code was refactored, and handlers were moved into dedicated files.

• Memory Efficiency: Optimization of getPostData handling are faster and lighter on memory.

• uBS Library Redesign: Introduced common static buffers and enhanced the Free and Master maps for better resource management

---

Download & Update

The full release is now live on GitHub.

👉 Download Version 1.5.6 here

Thanks to the community for the ongoing feedback and testing!

Gateway Release 1.5.5

Release 1.5.5 brings significant updates centered around ChibiOS migration and new communication features.

ChibiOS 21.11.x Migration

The gateway has been migrated from ChibiOS 20.3.x to the latest stable version 21.11.x, which is now the primary RTOS version for OHS_2. This major update required extensive work including migration to kernel 7, backporting custom Ethernet and RS485 changes, and updating the modem component to the new UART API. A float number display bug in the web interface was identified and fixed shortly after the initial migration.

SMS Command System

A new SMS command feature enables remote control through text messages using a hierarchical command dispatcher library. The system parses incoming SMS messages and routes them to specific handler functions that can execute actions and respond back to the requester. Security is enforced through phone number verification against stored contacts. The implementation includes a built-in help system that leverages the static command structure for user guidance.

MQTT Improvements

MQTT processing received several enhancements focused on reliability and functionality. A dedicated worker thread now handles incoming subscribe events, moving processing out of the data callback to follow lwIP best practices for ISR-like callback handling. A new /OHS/set/zone/refresh and /OHS/set/group/refresh subscribe topic was added, allowing all enabled zone or group states to be republished to MQTT on demand without requiring a payload.

Home Assistant Integration

Home Assistant auto-discovery support was added for sensors, enabling automatic configuration of OHS sensors that are directly supported by both platforms.

The release is available for download with the OHS_F437.bin firmware binary.

ChibiOS 20.3.x to 21.11.x Migration Success for Gateway

The migration from ChibiOS 20.3.x to the latest stable 21.11.x branch has been completed internally for the gateway. This upgrade adopts RT kernel version 7, with custom backports for Ethernet and RS485 functionality, plus adaptations to the new UART API for modem operations. A successful build is now available for testing.

​Key Migration Steps

• Updated core OHS components to kernel 7 compatibility, addressing API changes in HAL 8.0.0 including SPIv2 drivers and improved STM32 support.
• Backported custom Ethernet and RS485 drivers, leveraging UART driver callbacks for transmission end events essential in half-duplex RS485 setups.
• Modified modem UART integration to align with the new driver model, ensuring reliable communication in IoT gateway scenarios.

Resources and Downloads

•  Forked ChibiOS repository: https://github.com/vysocan/ChibiOS_21.11.x.git – contains the adapted 21.11.x branch for OHS.
•  Test release with binaries: https://github.com/vysocan/OHS_2-gateway/releases/tag/test_ChibiOS_21_11 – ready for download and deployment on STM32F4x7 hardware.

Potential Hiccups and Tips

Fell free to test the new binary and report any bugs through the discord @ #gw-latest

Site Shop not Functional

The ECWID shop platform was transferred to Shopify and the free tear option was removed. It is an end, for now :), for the possibility to buy the hardware directly on my blogger. I'll have to look up a new way to integrate web shop into it.
 

For now you can look and buy on Tindie. It is free, but taking like 8-10% from the purchase. Or directly ping me via Discord or email. 

Old Forum Removal

The old forum has come to an end, and Nabble now gathers only dust.
All past posts have been sent out to users as backups.

From here on, new conversations will take place on Discord.

 

GW Firmware Updates

Recently, there have been a few noteworthy updates to the GW firmware, primarily from the new GW owners. As always, the latest version 1.5.4 is available on the GitHub  releases page.

Here are some key changes:

• Automatic republishing of zone states to MQTT after reconnecting to the MQTT server. This is useful when updating Home Assistant (HA).
• The lwIP network library has been updated to stable version 2.2.0.
• mDNS has been added, both as a client and resolver. The GW is now accessible at http://ohs.local, eliminating the need to search for its DHCP-assigned address.
• A test alert feature has been added to the web interface, making it easier to verify that all alert paths are properly configured directly from the browser.
• A new alert path has been introduced. Now, all alerts can be propagated to the MQTT topic /OHS/alert and subsequently consumed in HA.
• The shell, accessible via USB, now allows for resetting the web admin user to its default settings.
  

 

 

Fingerprint Reader

An interesting addition to authentication node by p2baron. Check the video of the integration. Source code is available on discord.

Discord

For those who are interested, here is a link to Discord server:
https://discord.gg/PEUAUv6XxV

Discord is replacing rather old and not anymore used OHS forum. Also, I will try to redirect all email conversation to Discord in future, just for the convenience for the newcomers.

MQTT Homeassistant Auto Discovery

A new version of the gateway firmware (v1.5.0) is now available, offering enhanced MQTT support with the addition of Home Assistant Auto Discovery. As always, the updated firmware image is available on GitHub.

Now, let's take a closer look at what MQTT and Home Assistant Auto Discovery are.

MQTT (Message Queuing Telemetry Transport) is a lightweight messaging protocol ideal for devices that need to communicate in real-time, such as sensors and smart home devices. By implementing MQTT in your burglar alarm system, you enable real-time data transfer between the system and Home Assistant. 

Home Assistant Auto Discovery is a feature that allows Home Assistant to automatically detect and configure devices that support MQTT. This eliminates the need for manual configuration, as Home Assistant will automatically generate entities such as sensors, switches, and binary sensors based on the MQTT messages from your alarm system.

If you're new to MQTT, Home Assistant offers a convenient solution with the Mosquitto Broker add-on. You can easily install it directly from Home Assistant by navigating to Settings > Add-on Store and searching for "Mosquitto Broker." Once installed, head over to the Mosquitto Broker page in the Add-ons section, and click Start to launch the broker. For added security, it's recommended to create a dedicated user (e.g., "mosquitto") with its own password, under which the broker will operate.

Now let's go back to new gateway features. As always they are configurable over the web
interface. Main option to enable Home Assistant Auto Discovery is right on the Setting page as a toggle switch as shown on the picture. Enabling this will create the main MQTT topic for system state, allowing the Home Assistant to see it along with hardware and software version information and link to gateway web configuration page.

Additionally, similar switches are added to all groups and zones. When these switches are enabled, they are automatically propagated to Home Assistant as corresponding entities. This allows you to manage and control these groups and zones seamlessly within Home Assistant, enhancing integration and automation possibilities.

Following picture shows my gateway as new device inside Devices & services for MQTT.

This then can be then added to dashboard, and interacted with as any other Home Assistant device. 

Hope you like it :)

Home Assistant MQTT Alarm Control Panel Integration

Home Assistant is becoming increasingly popular in DIY home automation projects. To help you integrate OHS groups as alarm control panels, I've put together a small guide. The steps are straightforward:

First, you need to access the main configuration file, located at /homeassistant/configuration.yaml. The easiest way to do this is by using the File editor add-on. Go to Settings, then Add-ons, and click on the ADD-ON STORE button at the bottom right. Type "File" in the search bar, and the first add-on to appear should be the File editor. Install it, start it, and set it to start on boot.

Once the File editor is installed, you can access the configuration file at /homeassistant/configuration.yaml. All local files are conveniently accessible from the Folder icon located at the top left. Edit the file and add the following section at the bottom, making sure to maintain proper YAML indentation:

# OHS Alarm
mqtt:
  - alarm_control_panel:
    name: "Alarm Group House"
    supported_features:
      - arm_home
      - arm_away
    payload_arm_away: "arm_away"
    payload_arm_home: "arm_home"
    payload_disarm: "disarm"
    code_arm_required: false
    state_topic: "OHS/group/1/state"
    command_topic: "OHS/set/group/1/state"

This section can be repeated for each group number you have defined in OHS. Simply change the group number in both the state and command topics to match the actual number in OHS, and adjust the alarm panel name as needed.

After editing, save the configuration and then restart Home Assistant. The restart option is conveniently accessible from the editor menu at the top right under the settings icon.

After restarting, the following panel will be added to the main overview. Here, you can directly arm or disarm the OHS group and see its current state.

 

Note: Each time you edit or remove the MQTT configuration, you need to restart Home Assistant and also log off and log on again to clear the web cache.

Now as the GW firmware 1.5.0 allows Home Assistant Auto Discovery, using this manual approach is not recommended. 

Alarm State Machine / Group Zones Example

Just to update of gateway configuration page,  I've created a state machine diagram showing arm/disarm cycle.Also here is an example of my own groups (Garage, House) and zones (PIR, reed) setting for ground floor. You can see here how authentication unit is attached to particular zone. Only these zone is then set up with authentication delay. That is, when someone trips these zones there is time allowed for the person to authenticate. All other zones in house group trigger the alarm state immediately. You can read more about it in zone's setting.

 

Updated Radio NodeMini with STM32

Here comes the updated version of Radio Node Mini on STM32f103. There are only a few visual changes, like places for the required pull-up resistors for the I2C bus. I add them automatically when someone orders the optional HTU21D temperature/humidity sensor. Other than that, only silkscreen labels are enhanced. But the most important fact is the change of way I make those boards. 

The chip crisis is fading out and with it all the components are getting cheaper. This brought me to an idea to use fabrication services. And since I'm getting busier and I value my free time even more, I tried to compromise and I let the new nodes be assembled and soldered at the PCB shop. I invested the effort into making a small panel of the boards and created BOM and CPL files for the machine assembly. 

Make it short, the service is great. Actually it allows me to lower the price of the Mini Nodes, simply because it takes me much less time. And not only by saving my labor time, but also the components are priced similarly, or sometimes even cheaper, to my preferred shop TME. There are only a few cosmetic problems with the micro USB ports, or the micro buttons which are not always aligned, but the boards are nicely soldered and even cleaned of the residue flux.

To celebrate this accomplishment, I made a new graphical pin-out diagram for the STM32 Radio Nodes Mini. Take a look:

STM32 Radio Nodes Mini

STM32 Radio Nodes Mini are here and available for sale in my shop and on Tindie.com

I created this board to use it as base for various battery powered sensors I use at, and around my house. It is evolution of my Radio Mini Node based on ATmega328P, but instead of ATmega, it's using STM32F103 (basically well known BluePill). It suited for RFM69HCW(CW) or various LoRa RFM9X(RFM95/96) modules. It can have an u.UL connector for pigtail, or it has a solder pin for simple wire antenna. It comes with micro USB charging for 3.7V Li-ion cells with capacity ranging from 500mAh to 2000mAh. Quite good are 102050 (1000mAh) or 102550 (1500mAh) with JST-PH2.0 connector, that are almost matching the size of this board. It also includes battery voltage measurement connected to MCU analog pin. USB port can be used for programming, or as serial port. It has also standard ST-LINK SWD port. There are RESET and BOOT0 push buttons for convenience. Then there is also external RTC quartz crystal which allows indefinite sleeps instead of ATmegas maximum 8 seconds. It has LED on PC13. Also on top, there's a place to mount various temperature/humidity sensors like HTU21D. PCB comes with mounting holes to fit in various places.

Specification is as follows, it's STM32Duino BluePill compatible board with STM32F103C8:

• Size 24 x 49 mm (0.95 x 1.92 in)
• STM32F103C8 - FLASH 64KB, SRAM 20KB, Max. 72MHz @ 3V3.
• 8MHz main crystal, RTC 32.678kHz crystal.
• USB charging designed for 3.7 Li-Ion battery (500 - 2000mAh). Great thermal dissipation.
• Included, but not soldered, JST PH2.0 battery connector.
• Charging state (Hi-Z) available on pin PA8 via NO(Normally Open) PCB jumper.
• Voltage measurement on PB1 pin via 2:1 voltage divider, consuming 2,1uA @ 4.2V.
• LED on PC13, with NC(Normally Closed) PCB jumper.
• ST-LINK SWD 4-pin connector (3V3, SWDIO, SWCLK, GND).
• RESET and BOOT 0 push buttons. BOOT 1 (PB2) pulled-up by 100k resistor.
• SPI1 routed to RFM69/RFM9X modules. PA4(CS), PA5(SCK), PA6(MISO), PA7(MOSI)
• NO PCB jumper for Radio Reset routed to PB11.
• NC PCB jumper for Radio Interrupt(DIO0) routed to PA3
• PCB footprint for HTU2XD, SHT2X, SI70XX humidity/temperature sensor routed to I2C.
• Optional HTU21D humidity and temperature sensor.
• RFM69HCW 868Mhz as soldered on option.
• Optional u.FL antenna connector.
• Optional high quality 20cm u.FL to SMA pigtail.
• Optional rubber SMA 868/915MHz antenna.

The consumption of the board alone is around 60uA while MCU is in deep sleep, when clocked at default 72MHz. With soldered on RFM69HCW radio module and temperature/humidity HTU21D, the consumption raises to around 120uA while sleeping. This is about the same as a normal Li-Ion 1500mAh self-discharge current. It is enough to keep the node sending packet every 10 minutes for many months with a 1000mAh battery.

New Radio Mini Node

There was a little happening on development side of OHS lately. The gateway line 2.0.x
works very stable. No issues found in software or hardware. Arduino based nodes are also reliable. Basically it works as intended, "install and forget". 

But the word is changing and so are the evolution of chips. ATmegas, and also other components are becoming hard to get, and the prices of ATmega tripled. While using STM32 Cortex M4 chips for gateway, I realised they are great replacements, allowing us to do stuff not imaginable on AVR. For example STM32duino, I've been watching the progress of it for long time, and also, I have some blue/black pills on hand. So I decided to give them a try also on OHS environment. I've created a STM32 Radio Node Mini, which render is shown on picture. 

It is 50*24mm board with STM32F103C6T6 Cortex M3 running up to 72MHz with 64KB of flash and 20KB SRAM. Basically well known bluepill. It's meant to be used as radio node, so on back there is place to mount RFM69 or LoRa RFM95/96 module. It has a u.UL connector or place for wire antenna. Board has also 3.7V Li-ion battery charger, and battery measurement resistor divider connected to MCU ADC pin. It is programmable directly via USB port, or it has SWD. There is also RTC quartz crystal which allows indefinite sleeps instead of ATmegas maximum 8 seconds. It has LED on PC13, and also on top a place to mount various temperature/humidity sensors like HTU21D. PCB comes with mounting holes to fit in various places.

Currently I'm having those fabricated, and as soon as I have some of them tested and I'm ready with proper software, I will report more.

Gateway 2.0.4 pinout

Just short update, using nice tool called pinion with few hiccups,  I've created quite nice diagram for pin-out of OHS gateway 2.0.4.

https://vysocan.github.io/OHS_20x.html

It offers nice visuals with highlighting of pins and grouping by type. It has to be hosted out of blogger, as blogger does not allow embedding your own CSS, JS and pages easily.

Authentication added.

New firmware version 1.3.9 has been just released. It includes few minor improvements, and as main new feature an authentication, that was added to web interface.  How it works is pictured on right side. Every new client connection is asked for user and password that must match with internal values. Default user is: admin, and default password is: pass. Both then can be changed in Settings.

At first I was thinking to use obsolete HTTP basic authentication, to make it simple. But it is very obsolete, and also not supported by vanilla HTTPD from lwip contrib package. I was thinking how to add an authentication for a longer time. There are several methods to be used. First would be creating a single page application with authentication message encrypted in URL as parameters. Witch would require a bigger re-work on the HTML page generation. Now HTTPD has several pages, on which user can set tens of options for each section on a page. Quite a difficult task.  Second, to use a cookie to store some secret after user authenticate successfully on browser side, and request the browser to send this secret with each request. But lwip's HTTPD does not support cookies. 

Well I deiced for second option. I created a simple patch to HTTPD  to handle cookie generation and reception. First is done as part of dynamic header option for pages that are not statically liked, that is not generated via makefsdata program. Then, I added an additional callback, that is preceding request for every fs_open (file system open function). This allows an application to verify received cookie and then, if needed, rewrite the page( or file) requested to something else, for example login page.

I think it is nice and neat addition to lwip's HTTPD, so I will try to push it upstream, hoping it will get accepted.

New pre-compiled version is as always available GitHub, together with all sources. Enjoy!

P.S. Finally this is new version for my "production" home gateway. Resetting it after 163 days. Quite satisfied 💪

Forum migration

 Nabble forum was migrated to new server http://ohs.356.s1.nabble.com

Thanks to Nabble for support and keeping this small forum up and free. 

Summer vacation and 1.3.8 continuity test

Summer vacation is almost over, I hope for everyone it was spent in nice and restful manner. I personally, was traveling abroad and also much in my country to visit places I've wanted to see for longer time. After long stay-at-home period of covid 19, it was very welcome and pleasant.

But as always, there is the the other side of vacation, I did not do do much development for OHS. The last one which bothered me about lwip's MQTT was solved, and it took longer time. Also as I think about it now, the current firmware has all the major functionality I wanted initially. And it is in state that it simply works. Yes! Look at the picture, I have now running the last 83 days without interruption.

So what will come next? I have few ideas:

• There is a big task ahead to integrate SSL/TLS layer.
  
• Based on above create "push" notification as new alert delivery path.
• Add authentication to web interface.
  
• Create examples for OHS integrated scripting language (modified TCL).
• ...

If you have some other idea let me know.

 

Solving lwip troubles.

Some time after the introduction of lwip core locking into ChibiOS port, I started to notice that MQTT still tends to disconnect and connect right after. Same problem as before core locking. This is not very good that all other lwip process seems to work well, but MQTT is keep on reconnecting. Well lwip is a big and complex library to jump in and decode all possible problems, and I decided to start with MQTT client part only. There are quite a few debugging facilities in lwip and I turned on many to start seeing the whole picture.

Nice informative way to see information about lwip is to enable LWIP_STATS. There are many options what to see, like to examine possible memory problems by defining:
#define LWIP_STATS_DISPLAY 1
#define MEMP_STATS 1
#define PBUF_STATS 1
#define SYS_STATS 1

And adding:

#ifdef LWIP_STATS
  stats_display(); // Show lwip statistics
#endif

into while (true) lwip thread.

I started to torture again OHS application fir frequent radio packets resulting into MQTT publish messages. All was fine even with heavy usage of web interface and MQTT subscribe. But after 5 days the statistics started to show an error on MEMP TCP PCB and the in used number of same did not cease even after no other TCP communication then MQTT was used. I increased the number of MEMP_NUM_TCP_PCB to 50 and started again. 5 days later the same problem.

Examining MQTT library or searching forums did not help much. In logs I found that the client receives an empty pbuf which is forcing the client to disconnect. Mosquitto broker then does not see the keep alive periodic packet and also closes the connection. But I started to measure the exact time when the MQTT get out of pbufs, and it a bit surprised me that it happens exactly after 4.9 days. To be precise after 2^32 / 10000 = 429496,7296 seconds. There are timers in lwip that make various time dependent tasks, like removing used pbufs or dhcp. Deep diving into it I found out that the problem is not in lwip, but in ChibiOS port. The sys_now() function does not overflow on u32_t boundary but actually much earlier, causing the timers wait forever.

A patch of the sys_now() function seems to fix the problem completely. It comes with new OHS version 1.3.8 with a few cosmetics enhancements described in particular commits on GitHub. 

It is recommended to upgrade if you use MQTT.

Fixing MQTT

There was lately a lot of changes done in gateway MQTT processing. First of all I've introduced a lwip_assert_core_locked() function which present as helper macro throughout lwip network stack, but it is defined as empty. This macro is a quite useful feature, especially if lwip is running multiple sockets. ChibiOS has very functional and well defined bindings to lwip, however lacks the lwip_assert_core_locked() function definition. To benefit from this lock I've added following code to lwipopts.h:

#if LWIP_TCPIP_CORE_LOCKING
void lwip_assert_core_locked(void);
#define LWIP_ASSERT_CORE_LOCKED() lwip_assert_core_locked();
#endif

And the actual function to /arch/sys_arch.c:

#if LWIP_TCPIP_CORE_LOCKING
void lwip_assert_core_locked(void) {
  // If the mutex hasn't been initialized yet, then give it a pass.
  if (lock_tcpip_core == SYS_SEM_NULL) return;
  chSysLock();
  // Ensure that the mutex is currently taken (locked).
  if (chSemWaitTimeoutS(lock_tcpip_core, TIME_IMMEDIATE) == MSG_OK) {
    chSysHalt("TCPIP core is not locked!");
  }
  chSysUnlock();
}
#endif

Then basically you need to wrap every lwip API call with following macros:
LOCK_TCPIP_CORE();
lwip_function();
UNLOCK_TCPIP_CORE();


This helped a lot, since without the locking the gateway after some time was not sending ACK to MQTT broker, and was forced to reconnect every 5 or so minutes.

Second lwip source was updated to latest dev branch, which has some fixes for MQTT in it as well. 

Last, but not least, there are several changes introduced to MQTT thread. The connection/re-connection function was moved there as well from service thread.

All this is is pushed to GitHub as version 1.3.6.