Saturday, January 15, 2022

Gateway 2.0.4 pinout

Just short update, using nice tool called pinion with few hiccups,  I've created quite nice diagram for pin-out of OHS gateway 2.0.4.

It offers nice visuals with highlighting of pins and grouping by type. It has to be hosted out of blogger, as blogger does not allow embedding your own CSS, JS and pages easily.

Saturday, November 13, 2021

Authentication added.

New firmware version 1.3.9 has been just released. It includes few minor improvements, and as main new feature an authentication, that was added to web interface.  How it works is pictured on right side. Every new client connection is asked for user and password that must match with internal values. Default user is: admin, and default password is: pass. Both then can be changed in Settings.

At first I was thinking to use obsolete HTTP basic authentication, to make it simple. But it is very obsolete, and also not supported by vanilla HTTPD from lwip contrib package. I was thinking how to add an authentication for a longer time. There are several methods to be used. First would be creating a single page application with authentication message encrypted in URL as parameters. Witch would require a bigger re-work on the HTML page generation. Now HTTPD has several pages, on which user can set tens of options for each section on a page. Quite a difficult task.  Second, to use a cookie to store some secret after user authenticate successfully on browser side, and request the browser to send this secret with each request. But lwip's HTTPD does not support cookies. 

Well I deiced for second option. I created a simple patch to HTTPD  to handle cookie generation and reception. First is done as part of dynamic header option for pages that are not statically liked, that is not generated via makefsdata program. Then, I added an additional callback, that is preceding request for every fs_open (file system open function). This allows an application to verify received cookie and then, if needed, rewrite the page( or file) requested to something else, for example login page.

I think it is nice and neat addition to lwip's HTTPD, so I will try to push it upstream, hoping it will get accepted.

New pre-compiled version is as always available GitHub, together with all sources. Enjoy!

P.S. Finally this is new version for my "production" home gateway. Resetting it after 163 days. Quite satisfied 💪

Thursday, September 16, 2021

Forum migration

 Nabble forum was migrated to new server

Thanks to Nabble for support and keeping this small forum up and free. 

Monday, August 30, 2021

Summer vacation and 1.3.8 continuity test

Summer vacation is almost over, I hope for everyone it was spent in nice and restful manner. I personally, was traveling abroad and also much in my country to visit places I've wanted to see for longer time. After long stay-at-home period of covid 19, it was very welcome and pleasant.

But as always, there is the the other side of vacation, I did not do do much development for OHS. The last one which bothered me about lwip's MQTT was solved, and it took longer time. Also as I think about it now, the current firmware has all the major functionality I wanted initially. And it is in state that it simply works. Yes! Look at the picture, I have now running the last 83 days without interruption.

So what will come next? I have few ideas:

  • There is a big task ahead to integrate SSL/TLS layer.
  • Based on above create "push" notification as new alert delivery path.
  • Add authentication to web interface.
  • Create examples for OHS integrated scripting language (modified TCL).
  • ...

If you have some other idea let me know.


Sunday, June 20, 2021

Solving lwip troubles.

Some time after the introduction of lwip core locking into ChibiOS port, I started to notice that MQTT still tends to disconnect and connect right after. Same problem as before core locking. This is not very good that all other lwip process seems to work well, but MQTT is keep on reconnecting. Well lwip is a big and complex library to jump in and decode all possible problems, and I decided to start with MQTT client part only. There are quite a few debugging facilities in lwip and I turned on many to start seeing the whole picture.

Nice informative way to see information about lwip is to enable LWIP_STATS. There are many options what to see, like to examine possible memory problems by defining:
#define MEMP_STATS 1
#define PBUF_STATS 1
#define SYS_STATS 1

And adding:

  stats_display(); // Show lwip statistics

into while (true) lwip thread.

I started to torture again OHS application fir frequent radio packets resulting into MQTT publish messages. All was fine even with heavy usage of web interface and MQTT subscribe. But after 5 days the statistics started to show an error on MEMP TCP PCB and the in used number of same did not cease even after no other TCP communication then MQTT was used. I increased the number of MEMP_NUM_TCP_PCB to 50 and started again. 5 days later the same problem.

Examining MQTT library or searching forums did not help much. In logs I found that the client receives an empty pbuf which is forcing the client to disconnect. Mosquitto broker then does not see the keep alive periodic packet and also closes the connection. But I started to measure the exact time when the MQTT get out of pbufs, and it a bit surprised me that it happens exactly after 4.9 days. To be precise after 2^32 / 10000 = 429496,7296 seconds. There are timers in lwip that make various time dependent tasks, like removing used pbufs or dhcp. Deep diving into it I found out that the problem is not in lwip, but in ChibiOS port. The sys_now() function does not overflow on u32_t boundary but actually much earlier, causing the timers wait forever.

A patch of the sys_now() function seems to fix the problem completely. It comes with new OHS version 1.3.8 with a few cosmetics enhancements described in particular commits on GitHub. 

It is recommended to upgrade if you use MQTT.

Wednesday, April 7, 2021

Fixing MQTT

There was lately a lot of changes done in gateway MQTT processing. First of all I've introduced a lwip_assert_core_locked() function which present as helper macro throughout lwip network stack, but it is defined as empty. This macro is a quite useful feature, especially if lwip is running multiple sockets. ChibiOS has very functional and well defined bindings to lwip, however lacks the lwip_assert_core_locked() function definition. To benefit from this lock I've added following code to lwipopts.h:

void lwip_assert_core_locked(void);
#define LWIP_ASSERT_CORE_LOCKED() lwip_assert_core_locked();

And the actual function to /arch/sys_arch.c:

void lwip_assert_core_locked(void) {
  // If the mutex hasn't been initialized yet, then give it a pass.
  if (lock_tcpip_core == SYS_SEM_NULL) return;
  // Ensure that the mutex is currently taken (locked).
  if (chSemWaitTimeoutS(lock_tcpip_core, TIME_IMMEDIATE) == MSG_OK) {
    chSysHalt("TCPIP core is not locked!");

Then basically you need to wrap every lwip API call with following macros:

This helped a lot, since without the locking the gateway after some time was not sending ACK to MQTT broker, and was forced to reconnect every 5 or so minutes.

Second lwip source was updated to latest dev branch, which has some fixes for MQTT in it as well. 

Last, but not least, there are several changes introduced to MQTT thread. The connection/re-connection function was moved there as well from service thread.

All this is is pushed to GitHub as version 1.3.6.

Saturday, March 20, 2021

Release notification

Just a short post about OHS firmware. In case you would like to be notified about new version being  available for download, you can subscribe in GitHub directly by "Watch" button on top right. Either watch all, or "custom" and then just the releases.

More about releases, I've just pushed few commits to GitHub releasing 1.3.5. It's adding more support for remote zones, adding debug message to alarm thread, and fixing few small bugs.