Subject: [Suit] Transactional updates

From: Phillip Hallam-Baker <[email-masked]>
Date: Tue, 24 Oct 2017 09:30:57 -0400

I don't know about you, but I am fed up of the time it takes for
software updates on my desktop. First the update has to be downloaded
and then it has to be installed. Why can't this be instantaneous?

What I would like is to download the software update and then tell the
O/S to simply overlay the update on top of the file system as an
atomic operation. So installing a software update takes a millisecond,
no more.

Rolling back a software update is just a matter of telling the O/S to
stop applying the overlay.

It seems to me that this sort of capability would be very useful at
the device level as well. Even more so as the chief concern in any
software update scheme is that you end up bricking a device. Building
the ability to roll updates forwards and backwards is powerful.

I would also like this to be integrated into software signing and in
an intelligent fashion so that the signature encompasses both the code
and the data and it is possible to validate components independently.

There is of course going to be a cost and it is the need for storage.
The traditional file system organization is a response to highly
constrained storage. I have not yet fully considered space
optimizations because I have been thinking about the constrained
device but I think my approach could be modified for a fairly
constrained device. My approach is not going to be appropriate for
your PIC controllers but it is definitely feasible at the Arduino
level and above.

My approach is based on a Merkle Tree based container format. This may
sound intimidating but it shouldn't be. I designed, documented and
implemented the scheme in under a week:

https://tools.ietf.org/html/draft-hallambaker-jbcd-container-00

or in HTML with diagrams at

http://prismproof.org/Documents/draft-hallambaker-jbcd-container.html

The draft describes a container format that consists of a sequence of
variable length chunks that has the property that the sequence can be
traversed with equal efficiency in the forward and reverse directions.
This can optionally be indexed via a binary tree to allow random
access to any frame in log(n) time. Chain and (Merkle) tree digests
may be constructed which may be signed.

Each frame consists of a header and an optional payload. The payload
may be encrypted and/or signed using JOSE. The only fixed requirement
for the header encoding is that frame 0 has to have a header encoded
in JSON (for interop). The code supports use of JSON, JSON-B, JSON-C
and ASN.1 for header encoding, the container uses JSON-B frame
encoding because it has to support the read in either direction
property.

The original concept here was to provide an append only container
format that could be used as the basis for synchronizing Mesh portal
logs between nodes and also as an encryption format for end-to-end
encrypted Web sites. Static content is straightforward, but people
expect to be able to support applications like Web forums. The idea
was to see if we could support a Web forum run on a cloud server that
does not have the ability to read any of the content (yes).

The authentication scope is intentionally limited to the payloads, and
the signature parameters. The headers are not authenticated which
allows them to be mutable. A device may store the whole update as a
single file or break it apart into attachments. The latter approach
making garbage collection of unused attachments possible without a
separate sweep/collect cycle.

I am not sure that the encryption capability is needed for software
update but it might turn out to be useful allowing devices to be
shipped with code for a wide range of applications and only enabling
the specific modules required. I do not like devices to have more code
than is necessary for their purpose. Authentication is obviously
necessary.

The way I would see this applied to software update is that the
device/desktop would download and verify the incremental updates as a
background task (when bandwidth permits). It would then request user
permission to apply the update (if required) and set a flag internally
telling the loader to use the new version in place of the old. Any
processes that needed to be restarted would be restarted.

This could be used to effect updates across multiple devices at the
same time. It could also allow support for features such as testing
the configuration and performing an automatic rollback if the test
failed.

Besides providing a more secure process, this provides a better user experience.

The code required to support the container format is comparable to
that required to implement a SHA-2 or the like. implementation in a
TPM or such would be entirely feasible.

---

From: Hannes Tschofenig <[email-masked]>
Date: Tue, 24 Oct 2017 14:44:13 +0000

Hi PHP

Thanks for the contribution.

I am trying to find out what problem you are primarily trying to solve.  You list lots of topics below. Let me start with one item where you write:

"I am fed up of the time it takes for software updates on my desktop. First the update has to be downloaded and then it has to be installed. Why can't this be instantaneous?"

Clearly, you cannot install software / firmware that has not been downloaded yet. You are not complaining about the slow transfer rate (although this might be a serious issue with many low power radio technologies). You aren't talking about differential updates or updates of individual libraries. Finally, you are not asking for a hitless update either.

I feel like I am reading that you want to write the downloaded code directly over the code that is being executed. That cannot be true either.

So, could you explain your first point in more detail?

Ciao
Hannes

-----Original Message-----
From: Suit [mailto:[email-masked]] On Behalf Of Phillip Hallam-Baker
Sent: 24 October 2017 15:31
To: [email-masked]
Subject: [Suit] Transactional updates

I don't know about you, but I am fed up of the time it takes for software updates on my desktop. First the update has to be downloaded and then it has to be installed. Why can't this be instantaneous?

What I would like is to download the software update and then tell the O/S to simply overlay the update on top of the file system as an atomic operation. So installing a software update takes a millisecond, no more.

Rolling back a software update is just a matter of telling the O/S to stop applying the overlay.

It seems to me that this sort of capability would be very useful at the device level as well. Even more so as the chief concern in any software update scheme is that you end up bricking a device. Building the ability to roll updates forwards and backwards is powerful.

I would also like this to be integrated into software signing and in an intelligent fashion so that the signature encompasses both the code and the data and it is possible to validate components independently.

There is of course going to be a cost and it is the need for storage.
The traditional file system organization is a response to highly constrained storage. I have not yet fully considered space optimizations because I have been thinking about the constrained device but I think my approach could be modified for a fairly constrained device. My approach is not going to be appropriate for your PIC controllers but it is definitely feasible at the Arduino level and above.

My approach is based on a Merkle Tree based container format. This may sound intimidating but it shouldn't be. I designed, documented and implemented the scheme in under a week:

The draft describes a container format that consists of a sequence of variable length chunks that has the property that the sequence can be traversed with equal efficiency in the forward and reverse directions.
This can optionally be indexed via a binary tree to allow random access to any frame in log(n) time. Chain and (Merkle) tree digests may be constructed which may be signed.

Each frame consists of a header and an optional payload. The payload may be encrypted and/or signed using JOSE. The only fixed requirement for the header encoding is that frame 0 has to have a header encoded in JSON (for interop). The code supports use of JSON, JSON-B, JSON-C and ASN.1 for header encoding, the container uses JSON-B frame encoding because it has to support the read in either direction property.

The original concept here was to provide an append only container format that could be used as the basis for synchronizing Mesh portal logs between nodes and also as an encryption format for end-to-end encrypted Web sites. Static content is straightforward, but people expect to be able to support applications like Web forums. The idea was to see if we could support a Web forum run on a cloud server that does not have the ability to read any of the content (yes).

The authentication scope is intentionally limited to the payloads, and the signature parameters. The headers are not authenticated which allows them to be mutable. A device may store the whole update as a single file or break it apart into attachments. The latter approach making garbage collection of unused attachments possible without a separate sweep/collect cycle.

I am not sure that the encryption capability is needed for software update but it might turn out to be useful allowing devices to be shipped with code for a wide range of applications and only enabling the specific modules required. I do not like devices to have more code than is necessary for their purpose. Authentication is obviously necessary.

The way I would see this applied to software update is that the device/desktop would download and verify the incremental updates as a background task (when bandwidth permits). It would then request user permission to apply the update (if required) and set a flag internally telling the loader to use the new version in place of the old. Any processes that needed to be restarted would be restarted.

This could be used to effect updates across multiple devices at the same time. It could also allow support for features such as testing the configuration and performing an automatic rollback if the test failed.

The code required to support the container format is comparable to that required to implement a SHA-2 or the like. implementation in a TPM or such would be entirely feasible.

_______________________________________________
Suit mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/suit
IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.

---

From: Phillip Hallam-Baker <[email-masked]>
Date: Tue, 24 Oct 2017 11:16:25 -0400

> Clearly, you cannot install software / firmware that has not been downloaded yet. You are not complaining about the slow transfer rate (although this might be a serious issue with many low power ra…

Lets look at the sequence of events

1) Download the software update
2) Unpack and write files to directories
3) Stop and restart processes or reboot.

Each of these need to occur obviously and in many cases user
permission is required at at least one point in the process. But the
user experience for the desktop right now is:

1) Informed there is an update
2) Pestered into downloading the update, promised it will be applied
automatically
3) Open machine to discover that the update could not be applied
because software I don't care about was running.
4) Apply the update manually
5) Wait 15 minutes
6) Use machine.

Another vendor has a different approach. I turn the machine on first
thing in the morning and it tells me it needs to start applying
updates.

Neither of these approaches is going to be remotely acceptable for IoT
because the whole point is to save time and you can't save time if you
are giving people systems administration tasks. IoT is the harder use
case with lower capability devices.

It has to be possible for the process to take place automatically and
transparently with the only input from the user (if any) being to
authorize roll forward to the new software or to force a rollback to
the old. So my preferred implementation would be

1) Detect and download update. This is done under prior user permission.
1a) Authenticate and validate.
2) Request user permission (if required, may be granted remotely)
3) Apply update atomically (may be scheduled)

4) Test (optional) Rollback if failed.

> I feel like I am reading that you want to write the downloaded code directly over the code that is being executed. That cannot be true either.

No. What I am saying is that the device acquires the updates and
stores them somewhere. Then it compiles whatever local index is
required to access them and switches from one index to the other as an
atomic operation.

We seem to be fixed in the 1970s notion of files in directories. This
is merely an abstraction what we have in storage is a sequence of
bits. If all we require is read access, we can store data in a zip
file and read it as efficiently as if it was a set of separate files
broken out on disk.

We have to run the reader code on the constrained device of course.
But file creation can take place on a real machine with real
resources.

---

From: Hannes Tschofenig <[email-masked]>
Date: Tue, 24 Oct 2017 15:58:41 +0000

I think it is worthwhile to point out that there are a lot of things that need to be done in order to develop a fully functional firmware update solution for IoT. This is probably why it took us 2 years to develop our solution and not just a day.
Luckily, most of them are outside the realm of the IETF.

With respect to the standardization effort being proposed here we are talking about a somewhat smaller scope to get started, namely standardizing the meta data conveyed of firmware along with the security mechanism. You correctly noted that providing authentication and integrity protection will be key. Our partners also tell us that they want encryption to make it more difficult for attackers to reverse engineer the code.

The question about user permission depends a lot on the type of environment we are talking about. For end user products (like in your home) you, as an end customer, may get asked about the firmware update but in an industrial setting the story is obviously different since a single person or a small group (call him/her/them administrator(s)) decides about a large number of devices and then triggers the update. The update may not happen instantly but may only start at a specific point in time or staged. Luckily this authorization step is also outside the scope of this work and also outside the scope in general of the IETF (since it is largely a UI issue).

For rollbacks embedded devices often use a multiple, most often two, flash images. If you brick one then there is still a second one. This is quite common these days.

Regarding the file system story presented below, I should note that most low end embedded devices don't even have a file system. File systems are mostly used on external storage, such as SD cards. Often the binary that is transmitted with the firmware update mechanism ends up on flash and it is the same code that is executed. "Compression", if used, is often done using differential updates -- not via zip or similar tools. What you see in these articles or videos about hacking IoT devices typically refers to some Linux-based device and then you see how the hacker uses binwalk or some other tools to extract the different parts of the firmware, including the file system. That's not what we are talking about here. This is what I refer to as high-end devices.

To put the manifest (which includes the security mechanism) into a larger context we have actually written an entire architecture document, see https://tools.ietf.org/html/draft-moran-fud-architecture-00.

In a nutshell, I believe we are solving the issues you raise but we focus on low end IoT devices. As you can see, there are some differences between general purpose operating systems and (real-time) OSs that run on low end IoT devices. So, in some sense that's good news.

Ciao
Hannes

PS: I can take an ARM Cortex M class device with me to Singapore, and you can implement your ideas in an IoT OS (preferable ours, the mbed OS). That might be fun.

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Phillip Hallam-Baker
Sent: 24 October 2017 17:16
To: Hannes Tschofenig
Cc: [email-masked]
Subject: Re: [Suit] Transactional updates

Each of these need to occur obviously and in many cases user permission is required at at least one point in the process. But the user experience for the desktop right now is:

1) Informed there is an update
2) Pestered into downloading the update, promised it will be applied automatically
3) Open machine to discover that the update could not be applied because software I don't care about was running.
4) Apply the update manually
5) Wait 15 minutes
6) Use machine.

Another vendor has a different approach. I turn the machine on first thing in the morning and it tells me it needs to start applying updates.

Neither of these approaches is going to be remotely acceptable for IoT because the whole point is to save time and you can't save time if you are giving people systems administration tasks. IoT is the harder use case with lower capability devices.

It has to be possible for the process to take place automatically and transparently with the only input from the user (if any) being to authorize roll forward to the new software or to force a rollback to the old. So my preferred implementation would be

> I feel like I am reading that you want to write the downloaded code directly over the code that is being executed. That cannot be true either.

No. What I am saying is that the device acquires the updates and stores them somewhere. Then it compiles whatever local index is required to access them and switches from one index to the other as an atomic operation.

We seem to be fixed in the 1970s notion of files in directories. This is merely an abstraction what we have in storage is a sequence of bits. If all we require is read access, we can store data in a zip file and read it as efficiently as if it was a set of separate files broken out on disk.

We have to run the reader code on the constrained device of course.
But file creation can take place on a real machine with real resources.