@ -9,9 +9,9 @@ For the component's hardware, boards based on the \textit{ESP8266} micro-control
These boards feature a small and robust design, achieve very low power consumption and integrate WiFi without requiring any extra components.
These boards feature a small and robust design, achieve very low power consumption and integrate WiFi without requiring any extra components.
It integrates a Tensilica L106 32-bit micro controller unit (MCU) with a maximum CPU performance of 160 MHz, 64 kB instruction memory and another 96 kB of main memory.
It integrates a Tensilica L106 32-bit micro controller unit (MCU) with a maximum CPU performance of 160 MHz, 64 kB instruction memory and another 96 kB of main memory.
According to the manufacturer, the ESP8266 is among the most integrated WiFi-capable chips in the industry.
According to the manufacturer, the ESP8266 is among the most integrated WiFi-capable chips in the industry.
While at the beginning of this research, mostly \textit{ESP-01s}\cite{ESP-01s} boards in combination with self-developed power supplies and use-case specific hardware components were deployed, we quickly integrated \textit{Sonoff}\cite{sonoff} wireless smart switches product series offered by \textit{ITEAD}.
While at the beginning of this research, mostly \textit{ESP-01s}\cite{ESP-01s} boards in combination with self-developed power supplies and use-case specific hardware components were deployed, \textit{Sonoff}\cite{sonoff} wireless smart switches product series offered by \textit{ITEAD} have been integrated quickly.
The firmware for all of the \textit{ESP8266}-based devices in the hackerspace has been implemented using on our common software platform, referred to as \textit{ESPer}.
The firmware for all of the \textit{ESP8266}-based devices in the hackerspace has been implemented using a common software platform, referred to as \textit{ESPer}.
\textit{Sming}, which in turn is based on the open-source SDK for \textit{ESP8266}, provides the base library for this framework.
\textit{Sming}, which in turn is based on the open-source SDK for \textit{ESP8266}, provides the base library for this framework.
It integrates a lot of other software components and provides all kinds of functionality shared by all devices, allowing to reuse parts of the source code in multiple devices.
It integrates a lot of other software components and provides all kinds of functionality shared by all devices, allowing to reuse parts of the source code in multiple devices.
@ -10,7 +10,7 @@ If a new firmware version is published for a type of devices, the target devices
\subsubsection{}\label{req2}
\subsubsection{}\label{req2}
To ensure minimal maintenance effort, the update process should be insusceptible to errors as much as possible.
To ensure minimal maintenance effort, the update process should be insusceptible to errors as much as possible.
Even if the installation of an update fails in the middle of reprogramming the device, the system should continue to work fully functional immediately and after reboot.
Even if the installation of an update fails while reprogramming the device, the system should continue to work fully functional immediately and after reboot.
\subsubsection{}\label{req3}
\subsubsection{}\label{req3}
Firmware downloads must be possible over the same WiFi connection as used during normal operation.
Firmware downloads must be possible over the same WiFi connection as used during normal operation.
@ -38,13 +38,13 @@ Device specific code is organized in a sub-folder for each device type.
To build the software, a \textit{Makefile}\cite{make} is used, which provides a simple way for reproducible builds.
To build the software, a \textit{Makefile}\cite{make} is used, which provides a simple way for reproducible builds.
Whenever a new build is started, the build system scans for all device specific folders and calls the build process for each of them.
Whenever a new build is started, the build system scans for all device specific folders and calls the build process for each of them.
After the build of the firmware has finished, the build system also creates a file for each device type, containing the build version and cryptographic signatures of the corresponding firmware images.
After the build of the firmware has finished, the build system also creates a file for each device type, containing the build version and cryptographic signatures of the corresponding firmware images.
To avoid interferences between different build environments, and to roll out new versions as quickly as possible, the code has been integrated into a continuous integration (CI) system which is also responsible for publishing the resulting firmware images to the firmware server queried during updates, and for notifying the devices to check for an update.
To avoid interferences between different build environments, and to roll out new versions as quickly as possible, the code has been integrated into a CI system which is also responsible for publishing the resulting firmware images to the firmware server queried during updates, and for notifying the devices to check for an update.
\subsection{Device setup and flash layout}\label{flashlayout}
\subsection{Device setup and flash layout}\label{flashlayout}
Microcontroller boards based on the \textit{ESP8266} MCU are mostly following the same layout: the MCU is attached to a flash chip which contains the bootloader, firmware and other application data.
Microcontroller boards based on the \textit{ESP8266} MCU are mostly following the same layout: the MCU is attached to a flash chip which contains the bootloader, firmware and other application data.
The memory mapping mechanism of the MCU allows only a single page of 1 MB of flash to be mapped at the same time \cite{ESP8266_Memory_Map} and the selected range must be aligned to 1 MB blocks.
The memory mapping mechanism of the MCU allows only a single page of 1 MB of flash to be mapped at the same time \cite{ESP8266_Memory_Map} and the selected range must be aligned to 1 MB blocks.
As the image to download and flash possibly exceeds the size of free memory heap space, the received data must be written to flash directly.
As the firmware image to download and install possibly exceeds the size of free memory heap space, the received data must be written to flash directly.
In contrast, executing the code from the memory mapped flash while writing the same area with the downloaded update leads to unexpected behavior, as the executed code changes immediately to the updated one.
In contrast, executing the code from the memory mapped flash while writing the same area with the downloaded update leads to unexpected behavior, as the executed code changes immediately to the updated one.
To avoid this, the flash is split into half to contain two firmware ROM slots with different versions, one being executed and one which is being downloaded (see Figure~\ref{fig:memory_layout}).
To avoid this, the flash is split into half to contain two firmware ROM slots with different versions, one being executed and one which is being downloaded (see Figure~\ref{fig:memory_layout}).
This standby ROM slot also acts as a safety mechanism if the download fails or is interrupted as the previous version stays intact and can still be used (refer to requirement \ref{req2}).
This standby ROM slot also acts as a safety mechanism if the download fails or is interrupted as the previous version stays intact and can still be used (refer to requirement \ref{req2}).
@ -64,7 +64,7 @@ For calculating and verifying the signatures of a firmware image, the \textit{SH
The cryptographic signature for each of the two firmware images is created by the continuous integration system during build time and is provided as meta-information along with the firmware images.
The cryptographic signature for each of the two firmware images is created by the continuous integration system during build time and is provided as meta-information along with the firmware images.
Therefore, the CI system must be equipped with the private key used to create the signatures.
Therefore, the CI system must be equipped with the private key used to create the signatures.
In contrast, the micro controller only needs to know the according public key, to be able to verify the cryptographic signature.
In contrast, to be able to verify the cryptographic signature the micro controller only needs to know the according public key.
For the same reason as stated in Section \ref{flashlayout}, the signature of the new firmware image can not be verified before it is written to flash.
For the same reason as stated in Section \ref{flashlayout}, the signature of the new firmware image can not be verified before it is written to flash.
Therefore, the calculation of the \textit{SHA-256} checksum required for the signature check is done while the update is downloaded and written to flash.
Therefore, the calculation of the \textit{SHA-256} checksum required for the signature check is done while the update is downloaded and written to flash.
@ -6,7 +6,7 @@ Within this configuration file, settings relevant to the build process are provi
First, the \texttt{CONFIG=maglab} option lets the build system use an additional configuration file (\texttt{Configurion.mk.maglab}), which is stored inside the framework repository and provides environment specific information, such as the WiFi SSID.
First, the \texttt{CONFIG=maglab} option lets the build system use an additional configuration file (\texttt{Configurion.mk.maglab}), which is stored inside the framework repository and provides environment specific information, such as the WiFi SSID.
To keep secrets like the WiFi password and the private key unexposed, it is not written down in the configuration file.
To keep secrets like the WiFi password and the private key unexposed, it is not written down in the configuration file.
Instead, to include secrets into a build process while allowing to keep the configuration public, \textit{drone} allows to encrypt these with a repository specific key.
Instead, to include secrets into a build process while allowing to keep the configuration public, \textit{drone} allows to encrypt these with a repository specific key.
Using this method, the password is stored as \texttt{.drone.sec} file inside the repository from where it is injected into the build environment.
Using this method, the secrets are stored as \texttt{.drone.sec} file inside the repository from where they are injected into the build environment.
Also noticeable in Listing~\ref{lst:drone} is the firmware version, which is configured to be the first 8 letters of the \textit{Git} commit hash uniquely identifying a version of the source code.
Also noticeable in Listing~\ref{lst:drone} is the firmware version, which is configured to be the first 8 letters of the \textit{Git} commit hash uniquely identifying a version of the source code.
\begin{lstlisting}[language=,
\begin{lstlisting}[language=,
@ -31,8 +31,8 @@ publish:
\end{lstlisting}
\end{lstlisting}
For deployment, only the master branch is considered.
For deployment, only the master branch is considered.
After a successful build, all distribution files (the firmware image and meta-information files) of all device types are copied to the repository server, where they are served by a \textit{HTTP 1.1}\cite{HTTP_1.1} server.
The configuration file (\texttt{Configurion.mk.maglab}) references this repository server as the source for updates.
After a successful build, all distribution files (the firmware image and meta-information files) of all device types are copied to the repository server, from where they are served by a \textit{HTTP 1.1}\cite{HTTP_1.1} server.
The configuration file (\texttt{Configurion.mk.maglab}) references exactly this repository server as the source for updates.
Support for multiple devices of different type is implemented in both, the \textit{ESPer} framework itself and the build system.
Support for multiple devices of different type is implemented in both, the \textit{ESPer} framework itself and the build system.
The framework keeps control over the application life-cycle.
The framework keeps control over the application life-cycle.
@ -71,7 +71,7 @@ In addition, the suffix \texttt{/flash} can be used to flash a specific firmware
While building the firmware images for a device, the build environment provides some constants, which are baked into the resulting firmware image.
While building the firmware images for a device, the build environment provides some constants, which are baked into the resulting firmware image.
Beside the environmental configuration like the WiFi credentials, \textit{MQTT} topics and other configurable tweaks, the current device and version identifiers are provided as compile time constants.
Beside the environmental configuration like the WiFi credentials, \textit{MQTT} topics and other configurable tweaks, the current device and version identifiers are provided as compile time constants.
In addition, the public key used to verify firmware signatures during updates is derived from the private key and provided as a linker object, which is linked into each firmware image (Listing~\ref{lst:public_key_object}).
In addition, the public key used to verify firmware signatures during updates is derived from the private key and provided as a object file, which is linked into each firmware image (Listing~\ref{lst:public_key_object}).
This allows to use all the information inside the code without any overhead while being configurable during build time.
This allows to use all the information inside the code without any overhead while being configurable during build time.
As the \textit{ESP-01s} is only equipped with 1 MB of flash, this means that the whole memory is mapped to a contiguous address space (refer to Section \ref{flashlayout}).
As the \textit{ESP-01s} is only equipped with 1 MB of flash, this means that the whole memory is mapped to a contiguous address space (refer to Section \ref{flashlayout}).
\subsubsection{Verifying the cryptographic signature}
\subsubsection{Verifying the cryptographic signature}
While the image is being downloaded, each chunk received in the download stream is used to update the \textit{SHA256} hash before it is written to the flash.
While the image is being downloaded, each chunk received in the download stream is used to update the \textit{SHA256} hash before it is written to the flash.
When the write has been finished, the next chunk is received and the process continues until all chunks have been received.
When the write has been finished, the next chunk is received and the process continues until all chunks have been received.
After the download of a new ROM has been finished successfully, the calculated hash is checked against the cryptographically signed hash provided in the meta-information file triggering the update using the \textit{curve25519} public key provided by the running firmware.
Only if the firmware is considered valid, the bootloader configuration is altered to boot into the new ROM slot and the device is rebooted.
After the download of the new firmware image has been finished successfully, the calculated hash is checked against the cryptographically signed hash provided in the meta-information file triggering the update using the \textit{Curve25519} public key provided by the running firmware.
Only if the checksum matches the provided signature, the firmware is considered valid and the process is continued.
\subsubsection{Reconfiguring the boot process}
\subsubsection{Reconfiguring the boot process}
For the bootloader, \textit{rBoot}\cite{rBoot} has been choosen as it is integrated within the \textit{Sming} framework and allows to boot to multiple ROM slots.
For the bootloader, \textit{rBoot}\cite{rBoot} has been choosen as it is integrated within the \textit{Sming} framework and allows to boot to multiple ROM slots.
@ -6,15 +6,15 @@ All of the devices running the OTA-enabled firmware have undergone multiple majo
This includes a major network configuration change and an important stability fix for the network communication stack.
This includes a major network configuration change and an important stability fix for the network communication stack.
All devices applied the update successfully and started to work without any manual interaction required afterwards.
All devices applied the update successfully and started to work without any manual interaction required afterwards.
While the devices from various manufacturers in the hackerspace are all delivered with a pre-installed firmware, which is thought ready for the smart home, none of them was provided with updates so far. The update infrastructure has been the crucial point for most of our members towards the framework.
While the devices from various manufacturers in the hackerspace are all delivered with a pre-installed firmware, which is thought ready for the smart home application, none of them has been provided with updates by the manufacturer so far. The update infrastructure has been the crucial point for most of our members towards the framework.
Enabling the developers to do updates in combination with the shared configuration and behavior provided by the framework resulted in a massive speedup when it comes to project deployment.
Enabling the developers to do updates in combination with the shared configuration and behavior provided by the framework resulted in a massive speedup when it comes to project deployment.
Before that, the cost for applying changes after deployment was estimated so high, that most projects tend to delay deployment until all required and wanted features were implemented.
Before that, the cost for applying changes after deployment was estimated so high, that most projects tend to delay deployment until all required and wanted features were implemented.
Now, as the devices are deployed as soon as the hardware is considered stable, these devices start to provide functionality early and therefore the developers can get better feedback on the provided functionality.
Now, as the devices are deployed as soon as the hardware is considered stable, these devices start to provide functionality early and therefore the developers can get better feedback on the provided functionality.
The project will be continued to extend the functionality and security with features already being in development.
The project will be continued to extend the functionality and security with features already being in development.
The latest development includes further security enhancements by implementing checksum verification during startup where the hash of the firmware image is checked on each boot by the bootloader.
The latest development includes further security enhancements by implementing checksum verification during startup where the hash of the firmware image is checked on each boot by the bootloader.
It also considers including the device identifier into the signature to prevent confounding of images for different device types.
Last, the standby ROM slot can be updated right after each successful update to be more failsafe.
It also considers including the device identifier into the signature to prevent confounding of images between different device types.
Last, the standby ROM slot will be updated right after each successful update to be more failsafe.
In addition, the information provided by the device about the firmware status will be enhanced to allow better control and reduce maintenance effort even more.
In addition, the information provided by the device about the firmware status will be enhanced to allow better control and reduce maintenance effort even more.
A web interface to review the published information is currently in development.
A web interface to review the published information is currently in development.
Over the last years, a rapidly growing number of IoT devices is found on the market, especially in the area of the so-called smart home.
Over the last years, a rapidly growing number of IoT devices is found on the market, especially in the area of the so-called smart home.
These devices, which are deployed in vast numbers are frequently in use over many years and pose a risk to the users privacy and to the internet as a whole if not provided regularly with security patches.
These devices, which are deployed in vast numbers, are frequently in use over many years.
They pose a risk to the users privacy and to the internet as a whole if not provided regularly with security patches.
Hence, a fully automated process for large-scale software updates of such embedded devices must be considered.
Hence, a fully automated process for large-scale software updates of such embedded devices must be considered.
In this article, we present an implementation of a durable and stable system for building and publishing cryptographically secure firmware updates for embedded devices based on \textit{ESP8266} microcontrollers.
In this article, we present an implementation of a durable and stable system for building and publishing cryptographically secure firmware updates for embedded devices based on \textit{ESP8266} microcontrollers.
This includes mechanisms to build the updates from source and automatically sign, distribute and install them on the target devices.
This includes mechanisms to build the updates from source and automatically sign, distribute and install them on the target devices.
