Implementing an update / upgrade system for embedded Linux devices

Question

I have an application that runs on an embedded Linux device, and from time to time there are changes in the software, and sometimes in the root file system or even in the installed kernel.

In the current update system, the contents of the old application directory are simply deleted, and new files are copied over it. When changes to the root file system have been made, new files are delivered as part of the update and simply copied over the old ones.

Now there are several problems with the current approach, and I'm looking for ways to improve the situation:

• The root target file system used to create the file system images is not a version (I don’t think we even have the original rootfs).
• The rootfs that are included in the update are manually selected (instead of diff)
• Update is constantly growing and becoming pita. There is currently a separation between upgrade / upgrade where the update contains larger rootfs changes.
• I get the impression that consistency checks in an update are pretty fragile, if implemented at all.

Requirements:

• The application service pack should not be too large, and it should also be able to change the root file system in case changes were made.
• The update can be much larger and contains only material that is included in the root file system (for example, new libraries, the kernel, etc.). An update may require an update that must be installed.
  Can the update contain the entire root file system and just make dd on the target flash drive?
• Creating service packs / updates should be as automatic as possible.

I absolutely need a way to control the root file system versions. This should be done in such a way that I can calculate from it some diff that can be used to update the rootfs of the target device.

I have already studied Subversion, as we use this for our source code, but this is inappropriate for the Linux root file system (file permissions, special files, etc.).

Now I have created some shell scripts that can give me something similar to svn diff , but I would really like to know if there is already an existing and tested solution for this.

Using such a diff , I assume that Upgrade will simply turn into a package containing incremental updates based on the known state of the root file system.

What are your thoughts and ideas on this? How would you implement such a system? I prefer a simple solution that can be implemented in too little time.

Answer 1

I believe that you are wrong in this problem: any update that is not atomic (for example, a dd image of a file system, replaces files in a directory) is violated by design - if the power is turned off in the middle of the update, the system is a brick and an embedded system, power may go in the middle of an update.

I wrote a white paper on how to properly update / update Linux embedded systems [1]. He was introduced to OLS. You can find the article here: https://www.kernel.org/doc/ols/2005/ols2005v1-pages-21-36.pdf

[1] Ben Yosef, Gilad. "Building embedded Linux systems compatible with Murphy." Symposium Linux. 2005.

Answer 2

I absolutely agree that the update should be atomic - I recently started an open source project, the purpose of which is to provide a safe and flexible way to manage software with local and remote updates. I know that my answer comes very late, but it may help you in the following projects.

You can find sources for "swupdate" (project name) at github.com/sbabic/swupdate .

Stefano

Answer 3

Atomicity is crucial for embedded devices, one of the reasons in question is power loss; but there may be others, such as problems with the equipment / network.

Atomicity is perhaps a little misunderstood; this is the definition I use in the context of updates:

• Updates always complete completely or not at all.
• No software component other than an update ever sees an updated update.

A complete image update using the dual A / B partition structure is the easiest and most proven way to achieve this.

For embedded Linux, there are several software components that you may want to update and select different options; there is a newer article here: https://mender.io/resources/Software%20Updates.pdf

If you're working with the Yocto project, you might be interested in Mender.io , the open source project I'm working on. It consists of a client and a server, and the goal is to make it much faster and easier to integrate the update into an existing environment; without having to reverse engineer too much or wasting time on regular / homegrown coding. It will also allow you to centrally manage server updates.

Answer 4

Currently, there are many tools for updating embedded Linux with open source, with different focuses.

Another worth mentioning is RAUC , which focuses on handling the safe and atomic installations of signed service packs at your target, being very flexible in how you adapt it to your application and environment. Sources are on GitHub: https://github.com/rauc/rauc

All in all, a good overview and comparison of current upgrade solutions that you can find on the Wiki Yocto Project system updates page:

https://wiki.yoctoproject.org/wiki/System_Update

Answer 5

You can publish the update and split the flash update into two slots. A power failure always returns you to the current executable slot. The final step is to change the value of the log. Not atomic and not a way to make it brick. Even if it fails at the time of writing the flags of the magazine. There is no such thing as atomic update. Ever. Never seen this in my life. Iphone, adroid, my network switch - none of them are atomic. If you don’t have enough space for such a design, correct the design.