@ -5,17 +5,17 @@ On one side, it interacts with the hardware in a system specific way by implemen
On the other side, it provides use-case dependent functionality in interaction with general purpose hardware components.
On the other side, it provides use-case dependent functionality in interaction with general purpose hardware components.
Embedded systems are often thought as systems that never change their requirements or functionality.
Embedded systems are often thought as systems that never change their requirements or functionality.
But practical use shows that the environment in which these systems run in do, in fact, change.
But practical use shows that the environment in which these systems run do, in fact, change.
These changes include, but are not limited to, modifications of to the expected behavior or additions to it, reconfiguration of parameters related to communication with other systems and the users and correcting errors that have been reported after deployment.
These changes include, but are not limited to, modifications of to the expected behavior or additions to it, reconfiguration of parameters related to communication with other systems and the users and correcting errors that have been reported after deployment.
In almost all cases, the requirements can be accomplished by changing the firmware and does no need alternations of the hardware.
In almost all cases, the requirements can be accomplished by changing the firmware and does no need any modification of the hardware.
For updating the firmware on a system being deployed, the system must provide an interface for altering the firmware.
For updating the firmware on a system being deployed, the system must provide an interface for altering the firmware.
In addition, such an interface should provide mechanisms to check which firmware is currently installed and which configuration parameters are used.
In addition, such an interface should provide mechanisms to check which firmware is currently installed and which configuration parameters are used.
Even if systems are equipped with a interface for applying updates, the maintenance cost can still be enormous because administrators have to interact with each device physically.
Even if systems are equipped with an interface for applying updates, the maintenance cost can still be enormous because administrators have to interact with each device physically.
For systems that are located in areas where reachability is limited the cost is increased even more.
For systems that are located in areas where reachability is limited the cost is increased even more.
If a system is already able to communicate over a network interface, this can be leveraged to apply updates on these system - this is typically referred to as \textit{Over the Air (OTA)}.
If a system is already able to communicate over a network interface, this can be leveraged to apply updates on these system - this is typically referred to as \textit{Over the Air (OTA)}.
By reusing the existing communication channels, the dedicated update interface can be omitted which leads to smaller packaging and reduces production cost.
By reusing the existing communication channels, the dedicated update interface can be omitted which leads to smaller packaging and reduces production cost.
It also decreases the maintenance cost drastically because updates can be installed remotely.
It also decreases the maintenance cost drastically because updates can be installed remotely.
Updates OTA allows the administrators to apply automation methods on the update process allowing to roll out updates in a controlled fashion. I.e. updates can be don on critical deployments first and uncritical ones can be delayed to times where the device is not utilized.
\textit{OTA} updates allows the administrators to apply automation methods on the update process allowing to roll out updates in a controlled fashion. I.e. updates can be done on testing devices first, followed by security-critical deployments and subordinate ones can be delayed to times where the device is not utilized.
Information about the update status provided by the devices allows administrators to apply monitoring techniques ensuring all updates are installed and devices ar in the desired state.
Information about the update status provided by the devices allows administrators to apply monitoring techniques ensuring all updates are installed and devices ar in the desired state.
The home-automation projects developed by \textit{Magrathea Laboratories e.V.}\autocite{maglab}, the local hackerspace in Fulda, are used to provide control over the different actors and sensors in the foundations rooms to visitors and members locally and remotely.
The home-automation projects developed by \textit{Magrathea Laboratories e.V.}\autocite{maglab}, the local hackerspace in Fulda, are used to provide control over the different actors and sensors in the foundations rooms to visitors and members locally and remotely.
The different components available (like the door status, power sockets, projectors and screens, temperature, etc.) are all managed by the home-automation controller driven by the software home-assistant\autocite{HASS}.
The different components available (like the door status, power sockets, projectors and screens, temperature sensors, etc.) are all managed by the home-automation controller driven by the software \textit{home-assistant}\autocite{HASS}.
It provides direct control over all existing components using a web UI and allows to define rules and automations on how these components interact.
It provides direct control over all existing components using a web UI and allows to define rules and automations on how these components interact.
The hackerspace has developed a common software and hardware platform for its home-automation projects called \textit{ESPer}\autocite{ESPer}.
The hackerspace has developed a common software and hardware platform for its home-automation projects called \textit{ESPer}\autocite{ESPer}.
For the hardware, boards based on the ESP8266 micro-controllers, mostly ESP-01s\autocite{ESP-01s} boards, are used in combination with self-developed power supplies and use-case specific hardware components.
These boards provide a MCU fast enough for all required scenarios integrate WiFi without requiring any extra components.
The software is based on the Sming\autocite{Sming} library, which in turn is based on the open source SDK for ESP8266 and integrates a lot of other software components for easy use.
To build the software, a Makefile\autocite{make} is used and provides a simple way for reproducible builds.
For the hardware, boards based on the \textit{ESP8266} micro-controllers, mostly \textit{ESP-01s}\autocite{ESP-01s} boards, are used in combination with self-developed power supplies and use-case specific hardware components.
These boards provide a Microcontroller Unit (MCU) fast enough for all required scenarios and integrate WiFi without requiring any extra components.
The software is based on the \textit{Sming}\autocite{Sming} library, which in turn is based on the open source SDK for \textit{ESP8266} and integrates a lot of other software components for easy use.
To build the software, a \textit{Makefile}\autocite{make} is used and provides a simple way for reproducible builds.
For communication with the controller, the MQTT\autocite{MQTT} protocol is used.
For communication with the controller, the \textit{MQTT}\autocite{MQTT} protocol is used.
It provides a lightweight messaging mechanism implementing the publish-subscribe pattern which allows devices to listen for commands and publish their current state to the controller and other interested parties.
It provides a lightweight messaging mechanism implementing the publish-subscribe pattern which allows devices to listen for commands and publish their current state to the controller and other interested parties.
The controller software has out-of-the-box support for this protocol which allows easy integration of all different device types using the same patterns.
The controller software has out-of-the-box support for this protocol which allows easy integration of all different device types using the same patterns.
The components all share the same configuration in regard of the network access and the controller to communicate with.
The components all share the same configuration in regard of the network access and the controller to communicate with.
The configuration is provided during build time which eschews the need for a configuration interface and thus reducing the manage overhead and increases the security.
The configuration is provided during build time which eschews the need for a configuration interface and reducing the management overhead thus minimizing security leaks.
The following requirements are defined as global project targets and has been refined during the work on the project multiple times.
The following requirements are defined as global project goals and has been refined during the work on the project multiple times.
\begin{itemize}
\begin{itemize}
@ -12,7 +12,7 @@ All these devices should then download and install the new software version and
Even if the installation of an update fails in the middle of reprogramming the controller, the system should continue to work full functional immediately and after a reboot.
Even if the installation of an update fails in the middle of reprogramming the controller, the system should continue to work full functional immediately and after a reboot.
\item Downloading the updated firmware should be done over the WiFi interface using the same network connection as used during normal operation.
\item Downloading the updated firmware should be done over the WiFi interface using the same network connection as used during normal operation.
Fetching the firmware should be done side-by-side other traffic used during operation.
Fetching the firmware should be done side-by-side with other traffic used during operation.
\item Reducing network load and aiming for the maximum possible device uptime is critical.
\item Reducing network load and aiming for the maximum possible device uptime is critical.
Therefore, the update process should only be done if a new version is available.
Therefore, the update process should only be done if a new version is available.
@ -24,7 +24,7 @@ During the checking for available updates and while downloading such an update,
\item Devices are categorized by types.
\item Devices are categorized by types.
Each device type runs the same software and therefore provides the same functionality.
Each device type runs the same software and therefore provides the same functionality.
As the device type is hardly coupled to the hardware.
As the device type is hardly coupled to the hardware.
It is provided as a global constant while compile time and it must never be changed during operation.
It is provided through a global constant at compile time and it must never be changed during operation.
Updates must ensure that the correct firmware according to the device type is used while reprogramming.
Updates must ensure that the correct firmware according to the device type is used while reprogramming.
The implementation of the update mechanism consists of three parts which interact closely: checking for updates, reprogramming the device and reconfiguring the boot process.
The implementation of the update mechanism consists of three parts which interact closely: checking for updates, reprogramming the device and reconfiguring the boot process.
This sections describes all these parts in detail.
This sections describes all these parts in detail.
The build-time configuration was extended to include a new option called \texttt{UPDATER\_URL} which is the base URL sued to query the update server.
The build-time configuration was extended to include a new option called \texttt{UPDATER\_URL} which is the base URL used to query the update server.
Each device requires to have this option set to make the updates work.
Each device requires to have this option set to make the updates work.
If the option is skipped, the while code for update management is excluded during the build.
If the option is skipped, the code for update management is excluded during the build.
\subsubsection{Checking for update}
\subsubsection{Checking for update}
Initially, each device checks the update server regularly for the current firmware version and initializes the update process if remote and local versions differ.
Initially, each device checks the update server regularly for the current firmware version and initializes the update process if remote and local versions differ.
To do so, the update server provides a file for each device type containing the available version identifier which are stored beside the firmware binary files.
To do so, the update server provides a file for each device type containing the available version identifier which are stored beside the firmware binary files.
These version identifier files are provided by the update server using HTTP 1.1\autocite{HTTP_1.1} under the following path pattern: \texttt{\${DEVICE}.version} (whereas \texttt{\${DEVICE}} is the device type name).
These version identifier files are provided by the update server using \textit{HTTP 1.1}\autocite{HTTP_1.1} under the following path pattern: \texttt{\${DEVICE}.version} (whereas \texttt{\${DEVICE}} is the device type name).
The version identifier can be an arbitrary string as the content is not interpreted semantically but only compared to the version identifier used during build time.
The version identifier can be an arbitrary string as the content is not interpreted semantically but only compared to the version identifier used during build time.
Each device tries to fetch the version identifier file once every hour.
Each device tries to fetch the version identifier file once every hour.
@ -26,7 +26,7 @@ This allows faster roll-outs of updates and finer control for manual maintenance
\subsubsection{Reprogramming the device}
\subsubsection{Reprogramming the device}
As the binary to download and to flash maybe exceed the size of free memory heap space, the received data must be written to the flash directly.
As the binary to download and to flash possibly exceed the size of free memory heap space, the received data must be written to the flash directly.
In contrasts, executing the code from the memory mapped flash while writing the same area with the downloaded update leads to errors, as the executed code changes immediately to the updated one.
In contrasts, executing the code from the memory mapped flash while writing the same area with the downloaded update leads to errors, as the executed code changes immediately to the updated one.
To avoid that, the flash is split in half to contain two firmware ROMs with different versions, one being executed and one which is being downloaded.
To avoid that, the flash is split in half to contain two firmware ROMs with different versions, one being executed and one which is being downloaded.
This standby firmware also acts as a safety mechanism if the download fails or is interrupted as the previous version stays intact and can still be used.
This standby firmware also acts as a safety mechanism if the download fails or is interrupted as the previous version stays intact and can still be used.
Microcontroller boards based on the \textit{ESP8266} MCU are mostly following the same layout: the MCU is attached to a flash chip which contains bootloader, firmware and other application data.
Microcontroller boards based on the \textit{ESP8266} MCU are mostly following the same layout: the MCU is attached to a flash chip which contains bootloader, firmware and other application data.
The memory mapping mechanism of the MCU allows only a single page of 1 MB flash to be mapped at the same time\autocite{ESP8266_Memory_Map} and the selected range must be aligned to 1 MB blocks.
The memory mapping mechanism of the MCU allows only a single page of 1 MB flash to be mapped at the same time\autocite{ESP8266_Memory_Map} and the selected range must be aligned to 1 MB blocks.
As the ESP-01s is only equipped with 1MB of flash, this means that the whole memory is mapped to a continuous address space.
As the \textit{ESP-01s} is only equipped with 1MB of flash, this means that the whole memory is mapped to a continuous address space.
Therefore, the second ROM can not be re-mapped to have the same start address as the first ROM.
Therefore, the second ROM can not be re-mapped to have the same start address as the first ROM.
While the firmware is executed without any dynamic linking mechanism and the chip does not support position independent code, the addresses used in the ROMs are dependent to the offset at which the firmware is stored.
While the firmware is executed without any dynamic linking mechanism and the chip does not support position independent code, the addresses used in the ROMs are dependent to the offset at which the firmware is stored.
This arises the need of building two firmware binaries, one for each target location.
This arises the need of building two firmware binaries, one for each target location.
To do so, a linker script for each of the two ROM slots was created which are used to create two variations of the same firmware which only differ in ROM placement.
To do so, a linker script for each of the two ROM slots was created which are used to create two variations of the same firmware which only differ in ROM placement.
The two resulting firmware binaries files are both provided for download via HTTP 1.1 - which of these files to download is selected by the device during the download process depending on the target ROM slot.
The two resulting firmware binaries files are both provided for download via HTTP 1.1 - which of these files to download is selected by the device during the download process depending on the target ROM slot.
Snippet~\ref{lst:linker_script} shows the only difference between the two linker scripts, whereas \texttt{\${SLOT}} must be replaced with the slot number according to the current build.
Listing~\ref{lst:linker_script} shows the only difference between the two linker scripts, whereas \texttt{\${SLOT}} must be replaced with the slot number according to the current build.
In addition to the two ROMs, the flash must provide room for the bootloader and its configuration.
In addition to the two ROMs, the flash must provide room for the bootloader and its configuration.
\textit{rBoot}\autocite{rBoot} has been choosen as it is integrated within the \textit{SMING} framework and allows to boot to multiple ROMs.
\textit{rBoot}\autocite{rBoot} has been choosen as it is integrated within the \textit{Sming} framework and allows to boot to multiple ROMs.
For configuration, a \textit{rBoot} specific structure is placed in the flash at a well-known location directly after the space reserved for the bootloader code.
For configuration, a \textit{rBoot} specific structure is placed in the flash at a well-known location directly after the space reserved for the bootloader code.
This structure contains, among other things, the target offsets for all known ROMs and the number of the ROM to boot from on next reboot.
This structure contains, among other things, the target offsets for all known ROMs and the number of the ROM to boot from on next reboot.
@ -99,7 +99,7 @@ These provided firmware file are the exact same one as used to initially flash t
Using the same file for flashing and updating allows better debugging by eliminating errors related to the update process and makes development and initial installation easy.
Using the same file for flashing and updating allows better debugging by eliminating errors related to the update process and makes development and initial installation easy.
Listing~\ref{lst:choosing_rom} shows the algorithm used to determine the download address and reconfiguring the bootloader.
Listing~\ref{lst:choosing_rom} shows the algorithm used to determine the download address and reconfiguring the bootloader.
After the download of a new ROM has finished successfully, the bootloader configuration is altered to boot to the new ROM slot and the device is rebooted.
After the download of a new ROM has been finished successfully, the bootloader configuration is altered to boot to the new ROM slot and the device is rebooted.
The firmware for all ESP8266 based devices in the hackerspace are all based on the same framework.
\textit{SMING} provides the base library for this framework.
The firmware for all \textit{ESP8266} based devices in the hackerspace are all based on the same framework.
\textit{Sming} provides the base library for this framework.
In addition, components and functionality shared by all devices has been identified and are providing a framework for the existing and possible further devices.
In addition, components and functionality shared by all devices has been identified and are providing a framework for the existing and possible further devices.
This framework provides a functional base for all devices and allows to reuse code providing functionality which is common in multiple devices.
This framework provides a functional base for all devices and allows to reuse code providing functionality which is common in multiple devices.
The framework also includes a build system which allows to configure some basic parameters for all devices.
The framework also includes a build system which allows to configure some basic parameters for all devices.
@ -12,7 +12,7 @@ This eases configuration and allows to collect information about all devices at
Each device firmware exists as a separate project and includes a link to the framework.
Each device firmware exists as a separate project and includes a link to the framework.
As development on these devices happens in cycles, older projects are missing updates of the framework and there do not benefit from added features or fixed problems.
As development on these devices happens in cycles, older projects are missing updates of the framework and there do not benefit from added features or fixed problems.
Updating the framework version and rebuilding the firmware would often result in an easy gain of these benefits but requires manual interaction.
Updating the framework version and rebuilding the firmware would often result in an easy gain of these benefits but requires manual interaction.
More problems will arise if the API of the framework has changed.
More problems will arise if the application programming interface (API) of the framework has changed.
Then the device firmware must be updated to use the changed API which can be an unpleasant and complex task and leads to higher latency for firmware updates.
Then the device firmware must be updated to use the changed API which can be an unpleasant and complex task and leads to higher latency for firmware updates.
To prevent these problems the device firmware of all devices in the space is now integrated with the framework into a larger project.
To prevent these problems the device firmware of all devices in the space is now integrated with the framework into a larger project.
@ -24,7 +24,7 @@ The build system has been modified to scan for all device specific folders and c
\subsubsection{Framework integration}
\subsubsection{Framework integration}
The framework has been changed to keep control over the application life-cycle.
The framework has been changed to keep control over the application life-cycle.
It ensures that the device unspecific code is executed at the right time and provides integration points for device specific functionality.
It ensures that the device unspecific code is executed at the right time and provide an API for device specific functionality.
\begin{figure}[h]
\begin{figure}[h]
\begin{lstlisting}
\begin{lstlisting}
@ -48,15 +48,15 @@ Device* getDevice() {
The framework specifies a simple interface which must be implemented by each device.
The framework specifies a simple interface which must be implemented by each device.
A single function \texttt{Device* getDevice()} must be defined exactly once in each device specific folder.
A single function \texttt{Device* getDevice()} must be defined exactly once in each device specific folder.
To implement this interface, a static instance of \texttt{Device} is created returned.
To implement this interface, a static instance of \texttt{Device} is created and returned.
Each \texttt{Device} is populated with device specific \texttt{Feature} instances.
Each \texttt{Device} is populated with device specific \texttt{Feature} instances.
While the \texttt{Feature}-API leverages common runtime polymorphism to share functionality between features, the initial \texttt{Device} creation uses compile-time polymorphism which reduces the need of memory management and increases performance by avoiding virtual function tables.
While the \texttt{Feature}-API leverages common runtime polymorphism to share functionality between features, the initial \texttt{Device} creation uses compile-time polymorphism which reduces the need of memory management and increases performance by avoiding virtual function tables.
Snippet~\ref{lst:create_device_socket} shows the complete device specific code used for a simple power socket.
Listing~\ref{lst:create_device_socket} shows the complete device specific code used for a simple power socket.
\subsubsection{Build system}
\subsubsection{Build system}
The existing \textit{Makefile} has been refactored accept a parameter for device type identifiers called \texttt{DEVICE} and to create its whole output inside a subdirectory specific to the device type.
The existing \textit{Makefile} has been refactored to accept a parameter for device type identifiers called \texttt{DEVICE} and to create its whole output inside a subdirectory specific to the device type.
Another \textit{Makefile} has been created which scans a project subdirectory and uses each directory in there as container for device specific code.
Another \textit{Makefile} has been created which scans a project subdirectory and uses each directory in there as container for device specific code.
For each of these directories, the other \textit{Makefile} is called and the subdirectories name is used as \texttt{DEVICE} parameter.
For each of these directories, the other \textit{Makefile} is called and the subdirectories name is used as \texttt{DEVICE} parameter.
By splitting the build and recompiling the framework each time before intermixing it with the device specific code, the device type identifier can be used inside the shared framework code.
By splitting the build and recompiling the framework each time before intermixing it with the device specific code, the device type identifier can be used inside the shared framework code.
The source code of the \textit{ESPer} project is published into a GIT repository which is provided by the Hackerspace.
The source code of the \textit{ESPer} project is published into a \textit{GIT} source code repository which is provided by the hackerspace.
To avoid interferences between different build environments on developers computers and roll out new versions as early as possible, the code has been integrated into a continuous integration (CI) system.
To avoid interferences between different build environments on developers computers and roll out new versions as early as possible, the code has been integrated into a continuous integration (CI) system.
The CI based on \textit{drone}\autocite{drone}, provided as part of the hackerspace infrastructure, allows to execute commands on each version published into the GIT repository.
Therefore a \textit{drone} configuration file as shown in Snippet~\ref{lst:drone} has been added to the source code as \texttt{.drone.yml}.
The CI based on \textit{drone}\autocite{drone}, provided as part of the hackerspace infrastructure, allows to execute commands on each version published into the \textit{GIT} repository.
Therefore a \textit{drone} configuration file as shown in Listing~\ref{lst:drone} has been added to the source code as \texttt{.drone.yml}.
\begin{figure}[h]
\begin{figure}[h]
\begin{lstlisting}[language=]
\begin{lstlisting}[language=]
@ -33,10 +33,10 @@ publish:
As shown in the configuration Snippet, the build environment includes some special settings.
As shown in the configuration Snippet, the build environment includes some special settings.
First, the \texttt{CONFIG=maglab} option let the build system use \texttt{Configurion.mk.maglab} instead of the default configuration file.
First, the \texttt{CONFIG=maglab} option let the build system use \texttt{Configurion.mk.maglab} instead of the default configuration file.
This configuration file is stored inside the repository, too.
This configuration file is stored inside the repository, too.
To keep the WiFi password secret, it is not written down in the configuration but mus be specified in the environment.
To keep the WiFi password secret, it is not written down in the configuration but must be specified in the environment.
To include secrets into a build process while allowing to keep the configuration public, \textit{drone} allows to encrypt these with a repository specific key.
To include secrets into a build process while allowing to keep the configuration public, \textit{drone} allows to encrypt these with a repository specific key.
Using this method, the password is stored as \texttt{.drone.sec} file inside the repository from where it is injected into the build environment.
Using this method, the password is stored as \texttt{.drone.sec} file inside the repository from where it is injected into the build environment.
At last, the firmware version is configured to be made out of the first 8 letters of the GIT commit hash, which uniquely identifies a version of the source code.
At last, the firmware version is configured to be made out of the first 8 letters of the \textit{GIT} commit hash, which uniquely identifies a version of the source code.
For deployment, only the master branch is considered.
For deployment, only the master branch is considered.
After a successful build, all distribution files (the binary firmware files and the version files) of all devices are copied to the machine running the home automation controller software into a directory served by a HTTP server.
After a successful build, all distribution files (the binary firmware files and the version files) of all devices are copied to the machine running the home automation controller software into a directory served by a HTTP server.
\title{Implement Updates \textit{Over the Air} for ESP8266 microcontrollers}
\title{Implement Updates \textit{Over the Air} for \textit{ESP8266} microcontrollers}
\author{Dustin Frisch\\ C/C++ for Embedded Systems and Physical Computing\\ Angewandte Informatik\\ University of Applied Sciences Fulda\\\texttt{dustin.frisch@informatik.hs-fulda.de}}
\author{Dustin Frisch\\ C/C++ for Embedded Systems and Physical Computing\\ Angewandte Informatik\\ University of Applied Sciences Fulda\\\texttt{dustin.frisch@informatik.hs-fulda.de}}
\date{\today}
\date{\today}
\maketitle
\maketitle
\newpage
\newpage
\begin{abstract}
\begin{abstract}
This describes the implementation of durable and stable system for building firmware updates for embedded systems based on ESP8266\autocite{ESP8266} microcontrollers. This includes the mechanisms used to build the updates, distribute and install them on the device.
This article describes the implementation of durable and stable system for building firmware updates for embedded systems based on \textit{ESP8266}\autocite{ESP8266} microcontrollers. This includes the mechanisms used to build the updates, distribute and install them on the device.
