Subject: [core] Core Link Format

From: Hannes Tschofenig <[email-masked]>
Date: Thu, 7 Nov 2019 09:53:35 +0000

Hi all,

In previous discussions it was pointed out that Core Link Format has a couple of deficiencies, which motivated the work on Coral.
I was wondering whether there is a write-up available about those problems.

The reason for my question is simple:
(a) we are using Core Link Format in LwM2M and we would like to understand whether it is useful to switch to something else.
(b) We have successfully tested Core Link Format in various LwM2M testfests and it worked well for us.

Ciao
Hannes

IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.

---

From: Klaus Hartke <[email-masked]>
Date: Mon, 11 Nov 2019 16:35:13 +0100

Hi Hannes,

> I was wondering whether there is a write-up available about those problems.

I don't have a comprehensive write-up, but a few items from the top of
my head below.

The limitations of Link Format most frequently show up in CoRE WG
meetings where draft authors present exciting new ways to use Link
Format for things it was never intended for. Link Format comes with a
bunch of weirdnesses and complexities, which makes it difficult to
make it accommodate new use cases. CoRAL is more motivated by the need
for more expressiveness than the weirdnesses of Link Format. But those
weirdnesses exist, to the point, for instance, that
draft-ietf-core-resource-directory defines its own Link Format [1]. If
you'd like to avoid these weirdnesses and gain more expressiveness (in
particular for future evolution), then I think CoRAL would be really
interesting for LwM2M.

Some issues:

* The rules for the context of a link ("...origin of the target
URI...") [2] are a constant source of unawareness/confusion and
mis-implementation.

* The "hosts" link relation type [3] has really, really weird
semantics (a featureless relation between a host and a resource, where
determining the host relies on the weird rules for the link context).

* Despite the appearance of simplicity, parsing Link Format is often
more difficult than anticipated by implementers. For example, I've
seen implementations that just split the input by ';' and assume that
each resulting substring must be one link, regardless of where the ';'
occurs. And don't get me started on link attributes with
internationalized string values ;-)
title*=UTF-8'de'n%c3%a4chstes%20Kapitel

* Processing URI references correctly is also really hard to get
right. On constrained devices, implementations often don't want to pay
the price for a full URI parser and just do some simple string
manipulation instead. For example, I've seen implementations that
completely disregard the existence of percent encoding and dot
segments. URI parsers are also a constant source of fun^H^H^Hsecurity
issues [4].

* Link relation types must be registered in the IANA Link Relation
Types registry [5]. However, of those registered so far, few are
actually useful in the context of IoT - and we probably won't get far
flooding the registry with IoT-specific vocabulary. Of course, there
are extension link relation types [6], but these are not exactly
compact on the wire.

* Link target attributes don't even have an IANA registry.

* Link target attributes provide information only about link targets.
However, it is often very useful to be able to provide also
information about the represented resource at the same time.

* Link Format only has links that describe how a resource is related
to another resource. However, it is often very useful to be able to
describe also what you can do with a resource, what parameters are
required, etc., when you're extending an API over time.

Best regards,
Klaus

[1] https://tools.ietf.org/html/draft-ietf-core-resource-directory-23#appendix-C
[2] https://tools.ietf.org/html/rfc6690#section-2.1
[3] https://tools.ietf.org/html/rfc6690#section-2.2
[4] https://www.blackhat.com/docs/us-17/thursday/us-17-Tsai-A-New-Era-Of-SSRF-Exploiting-URL-Parser-In-Trending-Programming-Languages.pdf
[5] https://www.iana.org/assignments/link-relations/link-relations.xhtml
[6] https://tools.ietf.org/html/rfc8288#section-3.3

---

From: Carsten Bormann <[email-masked]>
Date: Mon, 11 Nov 2019 17:07:43 +0100

> I don't have a comprehensive write-up, but a few items from the top of

Hi Klaus,

thanks for providing the details.

My view of the high-level summary is that link-format is still fine to do the things that it originally was designed to do: Provide a self-description of a device by listing its resources, enable the aggregation of these self-descriptions in a resource directory.

When the potential became apparent to use link-format as a more general application format, the idiosyncrasies started to be a problem.  We noticed this when we tried to standardize links-json, which originally was intended to be bug-for-bug compatible with link-format but started morphing away from that.  Because we didn’t want to “break” link-format by creating “an almost but not entirely unlike” derivative, we stopped working on links-json and started to pay attention to CoRAL for the next step towards a general application format.

So the assumption is that CoRAL will serve as the improved basis for future applications that will include those now handled by link-format, but since this is a new format, there is no need to break the existing applications of link-format.

Grüße, Carsten

---

From: Esko Dijk <[email-masked]>
Date: Wed, 13 Nov 2019 11:15:31 +0000

Another item Klaus' list: the issue when doing unsecured (coap://) discovery where the Link Format should point to available secured (coaps://) resources, which is a very common case.
(Though for LWM2M specifically this problem does not exist, as there is no unsecured discovery I believe.)

Example: request to coap:// port 5683 would ideally look like this
  REQ: GET /.well-known/core
  RES: 2.05 Content
   </s/te>;rt="temperature-c",
   </s/li>;rt="light-lux"

But unfortunately the resources are typically hosted not on the same endpoint but on a secure coaps:// server of the same node. So the response cannot use the above "hosts" relation type because RFC 6690 states "The target URI MUST be a relative URI of the context URI" in case the implicit "hosts" relation like above is used. So that excludes cases where the Context URI is coap:// and the Target URIs (resources) are on coaps:// . 

Rather the example could now look like this in practice:
  REQ: GET /.well-known/core
  RES: 2.05 Content
   <coaps://[2001:db8:85a3::8a2e:0370:7334]/s/te>;rel="hosts";rt="temperature-c",
   <coaps://[2001:db8:85a3::8a2e:0370:7334]/s/li>;rel="hosts";rt="light-lux"

The size of the payload is now tripled due a lot of unneeded or verbose information (IPv6 addresses; "hosts" relation type, scheme). Overall, inefficient.
This is based on my understanding of the specs - anyone please feel free to clarify if I am wrong here.

Best regards
Esko

-----Original Message-----
From: core <[email-masked]> On Behalf Of Klaus Hartke
Sent: Monday, November 11, 2019 16:35
To: Hannes Tschofenig <[email-masked]>
Cc: [email-masked] WG <[email-masked]>
Subject: Re: [core] Core Link Format

Hi Hannes,

> I was wondering whether there is a write-up available about those problems.

I don't have a comprehensive write-up, but a few items from the top of my head below.

The limitations of Link Format most frequently show up in CoRE WG meetings where draft authors present exciting new ways to use Link Format for things it was never intended for. Link Format comes with a bunch of weirdnesses and complexities, which makes it difficult to make it accommodate new use cases. CoRAL is more motivated by the need for more expressiveness than the weirdnesses of Link Format. But those weirdnesses exist, to the point, for instance, that draft-ietf-core-resource-directory defines its own Link Format [1]. If you'd like to avoid these weirdnesses and gain more expressiveness (in particular for future evolution), then I think CoRAL would be really interesting for LwM2M.

Some issues:

* The rules for the context of a link ("...origin of the target
URI...") [2] are a constant source of unawareness/confusion and mis-implementation.

* The "hosts" link relation type [3] has really, really weird semantics (a featureless relation between a host and a resource, where determining the host relies on the weird rules for the link context).

* Despite the appearance of simplicity, parsing Link Format is often more difficult than anticipated by implementers. For example, I've seen implementations that just split the input by ';' and assume that each resulting substring must be one link, regardless of where the ';'
occurs. And don't get me started on link attributes with internationalized string values ;-) title*=UTF-8'de'n%c3%a4chstes%20Kapitel

* Processing URI references correctly is also really hard to get right. On constrained devices, implementations often don't want to pay the price for a full URI parser and just do some simple string manipulation instead. For example, I've seen implementations that completely disregard the existence of percent encoding and dot segments. URI parsers are also a constant source of fun^H^H^Hsecurity issues [4].

* Link relation types must be registered in the IANA Link Relation Types registry [5]. However, of those registered so far, few are actually useful in the context of IoT - and we probably won't get far flooding the registry with IoT-specific vocabulary. Of course, there are extension link relation types [6], but these are not exactly compact on the wire.

* Link target attributes provide information only about link targets.
However, it is often very useful to be able to provide also information about the represented resource at the same time.

* Link Format only has links that describe how a resource is related to another resource. However, it is often very useful to be able to describe also what you can do with a resource, what parameters are required, etc., when you're extending an API over time.

Best regards,
Klaus

_______________________________________________
core mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/core

---

From: Klaus Hartke <[email-masked]>
Date: Sun, 17 Nov 2019 08:08:49 +0100

I tried to remember where I had seen those broken Link Format
implementations. I've found some of them now again. It's Leshan and

> URI...") [2] are a constant source of unawareness/confusion and

Wakaama seems to completely ignore the existence of the 'anchor' attribute. [1]

> determining the host relies on the weird rules for the link context).

Wakaama seems to completely ignore the existence of link relation types. [1]

> occurs.

(Of course, links are separated by ',' and not ';'.)

Leshan splits by ',', regardless of where the ',' occurs. [2]

> segments.

Wakaama doesn't seem to do any kind of percent encoding or dot segment
processing. [1]

Overall, it seems that LwM2M implementations are interoperable on the
basis of an informally-specified subset of Link Format. That might
work out for LwM2M, but I think it highlights exactly the problem with
Link Format. :-)

Klaus

[1] https://github.com/eclipse/wakaama/blob/31d64c0c41fae9653c1fa53ef58d1a44e49017fa/core/registration.c#L1279-L1447

[2] https://github.com/eclipse/leshan/blob/f512b574797e852f8e883641ab0df852bebb7fcd/leshan-core/src/main/java/org/eclipse/leshan/Link.java#L119-L160