The firmware for all \textit{ESP8266} based devices in the hackerspace are all based on the same framework.
The firmware for all \textit{ESP8266} based devices in the hackerspace are all based on the same framework.
\textit{Sming} provides the base library for the\textit{ESPer} framework which provides components and functionality shared by all devices.
\textit{Sming} provides the base library for this\textit{ESPer} framework, which provides components and functionality shared by all devices.
This allows to reuse code providing functionality which is common in multiple devices.
This allows to reuse code providing functionality which is common in multiple devices.
To implement \textit{OTA} updates under the given requirements, we define a topology, which integrates our build infrastructure, firmware repository and MQTT-broker with the IoT wifi network the devices are connected to.
To implement \textit{OTA} updates under the given requirements, we define a topology, which integrates our build infrastructure, firmware repository and MQTT-broker with the IoT wifi network the devices are connected to.
@ -13,35 +13,43 @@ The base topology used is shown in Figure \ref{fig:topology}.
\label{fig:topology}
\label{fig:topology}
\end{figure}
\end{figure}
The continuous integration system is responsible for automatically building and publishing the firmware binary files, as soon as updated source code is available.
Both systems are described in detail in the following section.
Updates to the devices firmware is either actively triggered (manually or by the CI system) or on a regular schedule by the devices themselves.
This process is described in section \ref{flashlayout}.
For monitoring and maintenance purposes, each device publishes a set of information to a well-known \textit{MQTT} topic after connecting to the network.
Beside already existing dates like device type, chip and flash ID, the information block has been extended with details about the bootloader, SDK and firmware version as well as relevant details from the bootloader configuration, like the currently booted ROM slot and the default ROM slot to boot from.
This allows administrators to find devices with outdated bootloaders and helps to find missing and failed updates.
\subsection{Common framework and build infrastructure}
\subsection{Common framework and build infrastructure}
The framework also includes a build system, which allows to configure basic parameters for all devices.
Including, but not limited to, the Wi-Fi access parameters, the \textit{MQTT} connection settings and the updater URLs.
The framework includes a build system, which allows to configure basic parameters for all devices, including, but not limited to, the Wi-Fi access parameters, the \textit{MQTT} connection settings and the updater URLs.
By sharing the same code, all devices ensure to have a common behavior when it comes to reporting the device status or interacting with the home-automation controller.
By sharing the same code, all devices ensure to have a common behavior when it comes to reporting the device status or interacting with the home-automation controller.
This eases configuration and allows to collect information about all devices at a central location.
This eases configuration and allows to collect information about all devices at a central location.
Each device firmware exists as a separate project and includes a link to the framework.
As development on these devices happens in cycles, older projects are missing updates of the framework and therefor do not benefit from newly added features or fixed problems.
Updating the framework version and rebuilding the firmware would often result in an easy gain of these benefits, but requires manual interaction.
More problems will arise if the application programming interface (API) of the framework changes.
As development on the devices usually happens in cycles, some of the projects would miss updates of the framework and therefor would not benefit from newly added features or fixed problems.
Regular updating of the framework version and rebuilding the firmware would often result in an easy gain of these benefits, but requires manual interaction.
Further, problems could arise if the application programming interface (API) of the framework changes.
In this situation, the device firmware must be updated to use the changed API, which can be an unpleasant and complex task and leads to higher latency for firmware updates.
In this situation, the device firmware must be updated to use the changed API, which can be an unpleasant and complex task and leads to higher latency for firmware updates.
To prevent these problems the device firmware of all devices in the hackerspace is integrated with the framework into a larger project.
To prevent these problems, the device firmware of all devices in the hackerspace is integrated with the framework into a larger project.
By doing so, each device specific code is always linked to the latest version of the framework.
By doing so, each device specific code is always linked to the latest version of the framework.
The according device type is provided as a string through a global constant at compile time and it must never be changed during operation.
The according device type is provided as a string through a global constant at compile time and it must never be changed during operation.
Device specific code is organized as a folder for each device type.
Device specific code is organized in a sub-folder for each device type.
To build the software, a \textit{Makefile}\cite{make} is used, which provides a simple way for reproducible builds.
To build the software, a \textit{Makefile}\cite{make} is used, which provides a simple way for reproducible builds.
When a new build process is started, the build system scans for all device specific folders and calls the original build process for each of them.
When a new build process is started, the build system scans for all device specific folders and calls the original build process for each of them.
The source code of the \textit{ESPer} project is published into a \textit{Git} source code repository which is provided by the hackerspace.
The source code of the \textit{ESPer} project is published into a \textit{Git} source code repository.
To avoid interferences between different build environments on developers computers and roll out new versions as early as possible, the code has been integrated into a continuous integration (CI) system.
To avoid interferences between different build environments on developers computers and roll out new versions as early as possible, the code has been integrated into a continuous integration (CI) system.
The CI, which is based on \textit{drone}\cite{drone} and provided as part of the hackerspace infrastructure, allows to execute commands on each version published into the \textit{Git} repository.
The CI, which is based on \textit{drone}\cite{drone} and provided as part of the hackerspace infrastructure, allows to execute commands on each version published into the \textit{Git} repository.
In the specific case, whenever a new version gets checked-in into the release branch, the \textit{Makefile} is used to build the binary firmware files and publish them to the firmware repository server.
Therefore, a \textit{drone} configuration file as shown in Listing~\ref{lst:drone} has been added to the source code as \texttt{.drone.yml}.
Therefore, a \textit{drone} configuration file as shown in Listing~\ref{lst:drone} has been added to the source code as \texttt{.drone.yml}.
\subsection{Device setup and flash layout}
\subsection{Device setup and flash layout}\label{flashlayout}
Microcontroller boards based on the \textit{ESP8266} MCU are mostly following the same layout: the MCU is attached to a flash chip which contains the bootloader, firmware and other application data.
Microcontroller boards based on the \textit{ESP8266} MCU are mostly following the same layout: the MCU is attached to a flash chip which contains the bootloader, firmware and other application data.
The memory mapping mechanism of the MCU allows only a single page of 1 MB of flash to be mapped at the same time \cite{ESP8266_Memory_Map} and the selected range must be aligned to 1 MB blocks.
The memory mapping mechanism of the MCU allows only a single page of 1 MB of flash to be mapped at the same time \cite{ESP8266_Memory_Map} and the selected range must be aligned to 1 MB blocks.
@ -58,17 +66,6 @@ To avoid this, we chose the memory layout shown in Figure~\ref{fig:memory_layout
The flash is split into half to contain two firmware ROMs with different versions, one being executed and one which is being downloaded.
The flash is split into half to contain two firmware ROMs with different versions, one being executed and one which is being downloaded.
This standby firmware also acts as a safety mechanism if the download fails or is interrupted as the previous version stays intact and can still be used (refer to requirement \ref{req2}).
This standby firmware also acts as a safety mechanism if the download fails or is interrupted as the previous version stays intact and can still be used (refer to requirement \ref{req2}).
In case of an error the old firmware is kept unchanged and will be used until the successful download of a newer firmware succeeds.
In case of an error the old firmware is kept unchanged and will be used until the successful download of a newer firmware succeeds.
In addition to the two firmware ROMs, the flash must provide room for the bootloader and its configuration.
\subsection{Publish device information}
For monitoring and maintenance purposes, each device publishes a set of information to a well-known \textit{MQTT} topic after connecting to the network.
Beside already existing dates like device type, chip and flash ID, the information block has been extended with details about the bootloader, SDK and firmware version as well as relevant details from the bootloader configuration, like the currently booted ROM slot and the default ROM slot to boot from.
This allows administrators to find devices with outdated bootloaders and helps to find missing and failed updates.
In addition to the two firmware ROMs, the flash provides room for the bootloader and its configuration.
Sven Reissmann
8 years ago