esper-ota/6-1-build_and_deploy.tex

\subsection{Build infrastructure and automatic deployment}

The CI system, which is based on \textit{drone} \cite{drone} allows to execute commands, whenever a new version is published into the projects \textit{Git} repository.
A corresponding \textit{drone} configuration called \texttt{.drone.yml} exists beside the source code (Figure~\ref{lst:drone}).
Within this configuration file, settings relevant to the build process are provided to the build environment.
First, the \texttt{CONFIG=maglab} option lets the build system use an additional configuration file (\texttt{Configurion.mk.maglab}), which is stored inside the framework repository and provides environment specific information, such as the WiFi SSID.
To keep secrets like the WiFi password and the private key unexposed, it is not written down in the configuration file.
Instead, to include secrets into a build process while allowing to keep the configuration public, \textit{drone} allows to encrypt these with a repository specific key.
Using this method, the secrets are stored as \texttt{.drone.sec} file inside the repository from where they are injected into the build environment.
Also noticeable in Figure~\ref{lst:drone} is the firmware version, which is configured to be the first 8 letters of the \textit{Git} commit hash uniquely identifying a version of the source code.
For deployment, only the master branch is considered.
After a successful build, all distribution files (the firmware image and meta-information files) of all device types are copied to the repository server, from where they are served by a \textit{HTTP 1.1} \cite{HTTP_1.1} server.
The configuration file (\texttt{Configurion.mk.maglab}) references exactly this repository server as the source for updates.

\begin{figure}[htbp]
\begin{lstlisting}[basicstyle=\ttfamily\scriptsize]
build:
  image: maglab/sming
  environment:
    - CONFIG=maglab
    - WIFI_PWD=$$WIFI_PWD
    - VERSION=$${COMMIT:0:8}
  commands:
    - make clean && make
\end{lstlisting}
  \caption{The \textit{drone} configuration for the \textit{ESPer} project.}
  \label{lst:drone}
\end{figure}

Support for multiple devices of different type is implemented in both, the \textit{ESPer} framework itself and the build system.
The framework keeps control over the application life-cycle.
It ensures that device unspecific code is executed at the right time and provides an API for device specific functionality. 
For this, a simple interface is specified by the framework, which must be implemented by each device.
A single function \texttt{Device* getDevice()} must be defined exactly once in each device specific folder.
To implement this interface, a static instance of \texttt{Device} is created and returned.
Each \texttt{Device} is populated with device specific \texttt{Feature} instances.
While the \texttt{Feature}-API leverages common run time polymorphism to share functionality between features, the initial \texttt{Device} creation uses compile time polymorphism, which reduces the need for memory management and increases performance by avoiding virtual function tables.
Figure~\ref{lst:create_device_socket} shows the complete device specific code used for a simple power socket, which is mainly confined to the device type and its capabilities (e.g., the GPIO pin numbers to use).

\begin{figure}[htbp]
\begin{lstlisting}[basicstyle=\ttfamily\scriptsize]
constexpr const char NAME[] = "socket";
constexpr const uint16_t GPIO = 12; // General purpose I/O

Device device;
OnOffFeature<NAME, 12, false, 1> socket(&device);

Device* getDevice() { return &device; }
\end{lstlisting}
\caption{Device specific code for a socket driver.}
\label{lst:create_device_socket}
\end{figure}

The actual compilation of the source code is mainly controlled using two \textit{Makefiles}.
The first one is a helper \textit{Makefile} built to accept a parameter for device type identifiers called \texttt{DEVICE}, and to create its whole output inside a subdirectory specific to the device type.
In addition, the primary \textit{Makefile} scans a project subdirectory and uses each directory in there as a container for device specific code.
For each of these directories, the helper \textit{Makefile} is called and the subdirectories name is used as the value of the \texttt{DEVICE} parameter.
By splitting the build and recompiling the framework each time before intermixing it with the device specific code, the device type identifier can be used inside the shared framework code.
While building a devices firmware, the meta-information file used during updates is also created and stored beside the firmware image.
For development, each device can be build separately by using the device type identifier as \textit{Makefile} target.
In addition, the suffix \texttt{/flash} can be used to flash a specific firmware to the device.

While building the firmware images for a device, the build environment provides some constants, which are baked into the resulting firmware image.
Beside the environmental configuration like the WiFi credentials, \textit{MQTT} topics and other configurable tweaks, the current device and version identifiers are provided as compile time constants.
In addition, the public key used to verify firmware signatures during updates is derived from the private key and provided as a object file, which is linked into each firmware image (Figure~\ref{lst:public_key_object}).
This allows to use all the information inside the code without any overhead while being configurable during build time.

As the \textit{ESP-01s} is only equipped with 1 MB of flash, this means that the whole memory is mapped to a contiguous address space (refer to Section \ref{flashlayout}).
Therefore, the second ROM slot can not be re-mapped to have the same start address as the first ROM slot.
While the firmware is executed without any dynamic linking mechanism and the chip does not support position independent code, the addresses used in the ROM slots are dependent to the offset at which the firmware is stored.
This arises the need for building two firmware images, one for each target location.
To do so, a linker script for each of the two ROM slots was created, which is used to create two variations of the same firmware, only differing in ROM placement.
The two resulting firmware image files are both provided for download via \textit{HTTP 1.1} - which one to download depends on the target ROM slot and is selected by the device during the update process.
Figure~\ref{lst:linker_script} shows the only difference between the two linker scripts, where \texttt{\$\{SLOT\}} is replaced with the slot number according to the current build.

\begin{figure}[htbp]
\begin{lstlisting}[basicstyle=\ttfamily\scriptsize]
update_key_pub.bin:
    echo "$(UPDATE_KEY)" | ecdsakeygen -p | xxd -r -p > "$@"

update_key_pub.o: update_key_pub.bin
    $(OBJCOPY) -I binary $< -B xtensa -O elf32-xtensa-le $@
\end{lstlisting}
\caption{Creating the linker object containing the public key.}
\label{lst:public_key_object}
\end{figure}

The build process will create the two firmware images, one for each ROM slot, and the meta-information file.
To create the meta-information file, the current version identifier is written to the \texttt{.version} file.
After the build, the signatures for both firmware images are created and attached to the file.
Due to modern compilers doing link time optimization, the resulting firmware images include only code needed according the actual configuration.

\begin{figure}[htbp]
\begin{lstlisting}[basicstyle=\ttfamily\scriptsize]
irom0_0_seg : 
  org = ( 0x40200000          // The memory mapping address
        + 0x2010              // Bootloader code and config
        + 1M / 2 * ${SLOT} ), // Offset for the ROM slot
  len = ( 1M / 2 - 0x2010  )  // Half ROM size excl. offset
\end{lstlisting}
\caption{Linker script to build firmware for two ROM slots.}
\label{lst:linker_script}
\end{figure}