Subject: [core] Unnecessary "anchor" attributes in draft-ietf-core-resource-directory-25 ?

From: Esko Dijk <[email-masked]>
Date: Mon, 12 Oct 2020 11:25:10 +0000

Dear all,

I’ve created a Github issue for RD: https://github.com/core-wg/resource-directory/issues/281 , about “anchor” attributes in Link Format that seem unnecessary. Hoping this helps to make Link Format documents smaller in size.
This is something I found after reviewing RD again after some years I last looked at it.    (Maybe there were new insights that caused this change, in which case I must have missed these …  apologies if so)

Best regards
Esko

IoTconsultancy.nl  |  Email/Teams: [email-masked]<mailto:[email-masked]>

---

From: Christian Amsüss <[email-masked]>
Date: Tue, 20 Oct 2020 12:31:59 +0200

Hello Esko,

(picking this to go first as it's the most visible of your comments, and
thanks for all of them),

On Mon, Oct 12, 2020 at 11:25:10AM +0000, Esko Dijk wrote:
> [...] about “anchor” attributes in Link Format that seem unnecessary.
> Hoping this helps to make Link Format documents smaller in size.
>
> This is something I found after reviewing RD again after some years I
> last looked at it.

The issue with the redundancy between anchor and target is that the
RFC6690 link format details about target and anchor are ... well we can
argue about whether they are ambiguous, but they do conflate concepts
(context and base), are hard to read, subtly different from the Link
header and, most importantly, nobody ever implemented them right[1].

Consequently, a compatible subset of link-format and Link headers, that
would be consumable by a wider range of devices, was added as Limited
Link Format (in the RD appendix B). It's admittedly not great at
compression, but at least it works with how people use link-format.

Back when this was added, the hope was that nobody would practically use
text-based link format anyway because links-json[2] (actually, -cbor)
was on the brink of being published. It wouldn't have removed all the
warts, but at least the most pressing ones about resolution. (It would
still have kept the ASCII-delimited arrays for rt and if, and would not
have had a way to set an inline base address, as you'd probably want to
do when returning more than two links from a single endpoint). The
document was withdrawn to avoid churn anticipating a more powerful
solution to the problem (CoRAL), which hasn't landed yet.

(Avoiding too many formats war also why Limited Link Format was born: We
could have done a 6690bis, but feared it'd delay RD too much (hah,
anticipating a publication in 2019), and it would only delay a proper
solution that doesn't need to be held back by the link-format
decisions.)

Does this, to you, explain and justify the choices made about the
link-format representations?

KR
Christian

[1]: https://mailarchive.ietf.org/arch/msg/core/D_kuIq6QoBZ86FNniN7G_6d1Rtg/
[2]: https://datatracker.ietf.org/doc/draft-ietf-core-links-json/

-- 
To use raw power is to make yourself infinitely vulnerable to greater powers.
  -- Bene Gesserit axiom

---

From: Esko Dijk <[email-masked]>
Date: Wed, 21 Oct 2020 09:46:46 +0000

Hello Christian,

Thanks for the background information. What I don't understand is that you seem to say that Limited Link Format is the cause of a less efficient encoding of link format payloads.
When I look into Appendix C (Github master branch .md) I don't see any rules that there must be an anchor present in particular cases. And I don't see rules that change or simplify Section 2.1 of RFC 6690.

I agree that Link Format can be tremendously complex when all its features are being used, and it is hard to get it right in this case. And yes it has particular flaws (e.g. tremendously inefficient when indicating over coap:// resources hosted on the same host under coaps:// ) 
But I also see that the basic idea of link format for constrained devices is just to use mostly, or exclusively, the "hosts" relation which is the default.
In this case you get nothing more or less than a list of resources, with some optional attributes with it that describe e.g. content format and resource type semantics.

Such a list of resources is quite straightforward to generate, to parse, and to get right I believe.

So what the Limited Link Format Appendix could define in addition is the following rule:
* "anchor" attribute MUST NOT be present for a link with the "hosts" relation   (i.e. the default relation if none specified)

This will lead to the desired, and originally intended, solution for IoT devices that nothing more than a list of URIs is communicated with optional semantic attributes.

So a device serving its own links would respond relative links e.g.
</sensors/temp>;rt=temperature-c;if="sensor",
</sensors/light>;rt=light-lux;if="sensor"

This is simple to interpret as "resources are hosted on the same CoAP endpoint".

A device like an RD serving other's links would respond with absolute links e.g.
<coap://new.example.com/sensors/temp>;rt=temperature-c;if="sensor",
<coap://new.example.com/sensors/light>;rt=light-lux;if="sensor"

This is also simple to interpret as "resources are hosted at the specified CoAP endpoint".
Adding an "anchor" attribute to the above list would only confuse things and the requesting client -- if an IoT device -- wouldn't anyhow use this information! See Section 2.3 of RFC 6690.

To me, it seems we can make Link Format more limited, and simpler, and more efficient at the same time, by the above. And it would also be a true subset of RFC 6690 Link Format.

To make things more clear , and more subset, one can also consider to add the below rule:
* "anchor" attribute MUST be present for a link that specifies a relation ("rel") attribute.   (I.e., that includes a "rel" attribute - and does not use the implicit "hosts" relation.)

So this makes it clear there are 2 types of links possible:
1) Simple "hosts" links that just specify the resource (relative or absolute) - also IoT devices can parse these as a list of links/resources without forming semantic RDF triples internally :)
2) More complex links ( 'subject' <relation> 'object' - semantic statements) that IoT devices are not expected to parse, but more powerful devices could. E.g. a Commissioning Tool.  And there would not be confusion as to what the 'subject' is because that's always specified in the link.

Overall I do not see yet a good reason why the "anchor" attribute is there in the RD draft - given that the simple IoT client may just ignore it (Section 2.3 of RFC 6690) and given that the Appendix C doesn't change the interpretation rules (e.g. Section 2.1 of RFC 6690).
Could you please clarify why it would be useful or good to have the "anchor" attribute there for the "hosts" relation?   Including this seems to make it more rather than less complex I fear.

Of course if we expect this Limited Link Format isn't going to be used much because Coral will eventually replace it, our discussion may be less useful. 

Best regards
Esko

-----Original Message-----
From: Christian Amsüss <[email-masked]> 
Sent: Tuesday, October 20, 2020 12:32
To: Esko Dijk <[email-masked]>
Cc: [email-masked]
Subject: Re: [core] Unnecessary "anchor" attributes in draft-ietf-core-resource-directory-25 ?

Hello Esko,

KR
Christian

---

From: Christian Amsüss <[email-masked]>
Date: Wed, 21 Oct 2020 13:37:36 +0200

Hello Esko, hello CoRE,

You're right, this does not only come from Limited Link Format, but is
an explicit additional requirement on the lookup side. As I remember,
this was intruduced for two reasons:

* There was little trust in implementations following the RFC6690 rules
  -- they might just as well work under Link Header rules.

* The 6690 rules would require that any present relative anchor would
  need resolution, and in some absent cases, an absent anchor would need
  to be set to a (full) URI.

  Given the varying interpretations of 6690, it would be difficult to
  come up with sharp rules when the anchor could be elided.

Re-reading the 6690 rules, I see that some of my own interpretations of
the resolution process (that, apart from discussions on whether or not
the relation type can have a bearing on the resolution process[1],
seemed to be largely agreed on) can easily have been wrong -- and that
reading played into the "always absolute in lookup" rules.

If what how I think you interpret 6690[2] is the accepted reading of
6690, the explicit anchor can indeed be elided for many cases.

The changes would be manageable on a prescriptive level (but affect a
lot of lookup examples -- removing text from the wire so that's a very
good thing, but still a large change). The three reasons I'm not jumping
to just doing it but ask the WG to chime in are:

* We're quite late in the process.
* If we pick [2] as The Reading of 6690, we should be sure about it, and
  I've been wrong about details of the 6690 process too often to jump to
  conclusions here.
* Very few people using link-format seem to care about the semantics of
  the links in link-format, using it more for the list of resources
  (like, we could have used some resource description format in the
  first place). This makes reviews on preserving the actual link
  semantics -- which we have to to not break the few cases where it does
  matter -- hard to come by.

Chairs, can we even still make such a change?

Esko, does [2] capture how you read the resolution process?

Group, can we find agreement on that being the right interpretation?

KR
Christian

[1]:
> * "anchor" attribute MUST NOT be present for a link with the "hosts"
> relation   (i.e. the default relation if none specified)

This is relatively hard to check -- sure you can test for the presence,
but how about if someone made it explicit as rel=hosts? What about
rel="managed-by hosts"?

[2]: I've read and written too many phrasings of that, maybe this is
less ambiguous -- the RFC6690 algorithm as I think you understand it:

```
docbase = the RFC3986 base URI (from the encapsulating entity, as
  link-format has no base embedded in the content)
href = string part between the angular brackets
anchor = string value of the anchor attribute, or null if absent
target = resolve href against docbase
if anchor == null:
    context = resolve anchor against docbase
else:
    context = origin of target
```

---

From: Esko Dijk <[email-masked]>
Date: Wed, 21 Oct 2020 12:59:59 +0000

Hello Christian,

Thanks for the helpful response. I've swapped two lines in your item [2] plus added some comments for clarification; below is my understanding:

```
docbase = the RFC3986 base URI (from the encapsulating entity, as link-format has no base embedded in the content)
href = string part between the angular brackets
anchor = string value of the anchor attribute, or null if absent
target = resolve href against docbase
if anchor == null:
        context = origin-of ( target )             # Section 2.1 rule (b) and (c). If href is ABNF 'URI' then this is origin-of(href), if href 'relative-ref' then origin-of(docbase).
else:
    context = resolve anchor against docbase       # Section 2.1 rule (a) - if anchor is ABNF 'URI' then result of resolution is again 'anchor'. If ABNF 'relative-ref' then result is "docbase / relative-ref"
```

Because this may be multi-interpretable, my idea for Limited Link Format was to simplify the parsing by splitting into two possible cases: 1) implicit rel=hosts without 'anchor' ; and 2) any other link with 'anchor'.
Then for parsers that would accept category 2) they can remove the above "anchor==null" case in the parsing, which simplifies it.  And IoT devices could just ignore any such 'complex' links that specify a 'rel' and an 'anchor'. (As is also expected per RFC 6690); and only parse case 1) links.
Case 1) is the simple "list of links" which is easier to interpret than the more general case 2).

Base on your comment [1] I see that my proposed rule needs some update to do this properly:  what about 2 rules:
1* "anchor" attribute MUST NOT be present for any link without the "rel" attribute   (this implies that this link uses the implicit rel=hosts semantics only)
2* "anchor" attribute MUST be present for any link with the "rel" attribute                  (this could be any type of link, possibly using rel="managed-by hosts" or rel="hosts" or rel=hosts )

So then the only case where "anchor" can be elided is the case of using the implicit 'rel=hosts' relation, achieved by leaving out the "rel" attribute.

There is one more consideration for the current RD draft - in case we decide to keep the explicit "anchor" attribute in all the examples. Because RFC 6690 is a normative reference, the assumption of the reader currently should still be that the "anchor" attribute that's added in all the example is just superfluous i.e.
* an RD implementation may elide it , following RFC 6690 rules  (in the interpretation as coded at the top of this email.) and Section 2.3 of 6690
* an RD client doing lookups may ignore it, following RFC 6690 rules in Section 2.3

So we still will be faced with interoperability issues if RD implementers interpret RFC 6690 differently.   What is needed in this case is that we *update* RFC 6690 rules explicitly e.g. by making "anchor" mandatory in all cases.  (Whether we formally update RFC 6690 or just define it as part of Limited Link Format doesn't matter that much I think.)

Best regards
Esko

-----Original Message-----
From: Christian Amsüss <[email-masked]> 
Sent: Wednesday, October 21, 2020 13:38
To: Esko Dijk <[email-masked]>
Cc: [email-masked]
Subject: Re: [core] Unnecessary "anchor" attributes in draft-ietf-core-resource-directory-25 ?

Hello Esko, hello CoRE,

KR
Christian

```

---

From: Esko Dijk <[email-masked]>
Date: Wed, 21 Oct 2020 13:21:42 +0000

PS one additional remark to my previous email.

In the present link examples obtained from RD we may have:

<coap://[2001:db8:3::123]:61616/temp>;rt="tag:example.org,2020:temperature";anchor="coap://[2001:db8:3::123]:61616"

I was wondering what parsing problems would arise in the client if it were returned without the 'anchor':

<coap://[2001:db8:3::123]:61616/temp>;rt="tag:example.org,2020:temperature"

That seems clear; from this answer - how could anyone be mistaken about the anchor i.e. which server is hosting the '/temp' resource ? There's only one server mentioned so that's the one right :)    
And for the rel=hosts relation it would not make much sense to proclaim e.g. "coap://server1  hosts  coap://server2/resource"  ... 

Best regards
Esko

-----Original Message-----
From: core <[email-masked]> On Behalf Of Esko Dijk
Sent: Wednesday, October 21, 2020 15:00
To: Christian Amsüss <[email-masked]>
Cc: [email-masked]
Subject: Re: [core] Unnecessary "anchor" attributes in draft-ietf-core-resource-directory-25 ?

Hello Christian,

```

Best regards
Esko

-----Original Message-----
From: Christian Amsüss <[email-masked]> 
Sent: Wednesday, October 21, 2020 13:38
To: Esko Dijk <[email-masked]>
Cc: [email-masked]
Subject: Re: [core] Unnecessary "anchor" attributes in draft-ietf-core-resource-directory-25 ?

Hello Esko, hello CoRE,

KR
Christian

```

-- 
To use raw power is to make yourself infinitely vulnerable to greater powers.
  -- Bene Gesserit axiom
_______________________________________________
core mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/core

---

From: Christian Amsüss <[email-masked]>
Date: Wed, 21 Oct 2020 15:32:07 +0200

On Wed, Oct 21, 2020 at 01:21:42PM +0000, Esko Dijk wrote:
> That seems clear; from this answer - how could anyone be mistaken
> about the anchor i.e. which server is hosting the '/temp' resource ?
> There's only one server mentioned so that's the one right :)    
>
> And for the rel=hosts relation it would not make much sense to
> proclaim e.g. "coap://server1  hosts  coap://server2/resource"  ... 

That's intuitively clear for coap://server1 and coap://server2, but a
statement like coap://server1 hosts coap+sms://+15551234567/ might make
a bit more sense, and the tautological definition of the relation
doesn't help much there. As the meaning is so vague, I'm loathe to make
precise rules depend on it.

The description in RFC6690 Section 2.2 doesn't help either; it
potentially makes it worse by claiming that hosts only works with
relative references in the target. That statement is probably what
fueled my (probably mis?)conception about the target being resolved
relative to the anchor, as only then it makes sense. (Salvaging it by
saying there can't be an anchor with a hosts relation would just make it
worse by breaking RD).

KR
c

---

From: Christian Amsüss <[email-masked]>
Date: Wed, 21 Oct 2020 15:59:46 +0200

Hello Esko,

On Wed, Oct 21, 2020 at 12:59:59PM +0000, Esko Dijk wrote:
> I've swapped two lines in your item [2] plus added some comments for
> clarification

The swapping resolved a refactoring error of mine -- so we are aligned
there.

> Because this may be multi-interpretable, my idea for Limited Link
> Format was to simplify the parsing by splitting into two possible
> cases: 1) implicit rel=hosts without 'anchor' ; and 2) any other link
> with 'anchor'.

It the above reading is agreed on, the rule can even be simpler: "If
anchor was absent originally, it can stay absent". That would catch the
same cases and some more, not depend on understanding rel, and upset the
same set of implementations (those based on the 2018 understanding, and
AFAIK that means it's only mine).

> 2* "anchor" attribute MUST be present for any link with the "rel" attribute                  (this could be any type of link, possibly using rel="managed-by hosts" or rel="hosts" or rel=hosts )

If we go that route (and at least a brief chair chat earlier today
sounded a bit reluctant), I'd revisit the known implementations. My
expectation is that these restrictions won't even be necessary.

The main offender that necessitated the full-anchor-everywhere was my
2018 understanding. The change would be to reiterate in Limited Link
Format that unlike in the Link header, an absent anchor means the
context is origin-of(resolved-target), and then no further conditions on
rel need apply.

So probably it'd be either leave-it-and-say-why-it's-like-that, or going
even a step further than you are suggesting.

> There is one more consideration for the current RD draft - in case we
> decide to keep the explicit "anchor" attribute in all the examples.
> Because RFC 6690 is a normative reference, the assumption of the
> reader currently should still be that the "anchor" attribute that's
> added in all the example is just superfluous i.e.
>
> * an RD implementation may elide it , following RFC 6690 rules  (in
>   the interpretation as coded at the top of this email.) and Section
>   2.3 of 6690

I disagree here -- just because we normatively use 6690 doesn't mean we
can't profile it and prescribe that a particular serialization needs to
be used.

> * an RD client doing lookups may ignore it, following RFC 6690 rules

... and here: 6690 says that "A consuming implementation can, however,
choose to ignore such links." (and not such attributes). The client woud
just ignore all resource lookup results, and the implementer would thus
need to implement (at least rudimentary) support for the anchor
attribute.

KR, and thanks for all your input
Christian

---

From: Esko Dijk <[email-masked]>
Date: Mon, 26 Oct 2020 09:28:43 +0000

Hello Christian,

> a bit more sense, and the tautological definition of the relation

To me this looks as if not intended by RFC 6690 , because the server coap://server1 is a different server than the one on coap+sms://+15551234567 .  At least per RFC 7252, one server is one endpoint. So different endpoint is always a different server, or not?
(Interesting to note is that for clients it is similarly defined in RFC 7252, if a CoAP client changes its outgoing UDP port it becomes a different client.)

> relative references in the target. That statement is probably what

Agree that the MUST statement there is problematic - I initially interpreted it as #1 below: 

** Interpretation #1
"resource indicated by the target URI MUST be a URI that can be potentially described as a relative reference with respect to context-URI used as base URI"
-> then 'coap://server1 hosts /test' would be valid.
-> then 'coap://server1 hosts coap://server1/test' would be valid.
-> then 'coap://server2 hosts coap://server1/test' would be invalid.

** Interpretation #2
But it could also be interpreted as
"target URI MUST be a URI of the ABNF form 'relative-ref' , and the target URI MUST be a relative reference with respect to context-URI used as base URI"
-> then 'coap://server1 hosts /test' would be valid.
-> then 'coap://server1 hosts coap://server1/test' would be invalid.
-> then 'coap://server2 hosts coap://server1/test' would be invalid.

If a majority of people interprets like #1, then we don't have to update/correct RFC 6690 and we can define a profile of Limited Link Format. And clarify a bit how we should read 6690.
If however most people interpret as #2, then current RD usage of rel=hosts for absolute target URI is incorrect and we would need to formally update RFC 6690 I think to make that possible.

> relative to the anchor, as only then it makes sense. (Salvaging it by

I assume for the moment the interpretation #1 as above. The target needs to be resolved relative to the context URI as per the Section 2.2 MUST requirement. The
context URI could be an explicit URI given by the 'anchor' attribute but it may also use the default, if that's not given.  If the target URI is an absolute URI
form then the default context URI becomes the origin of that target URI. So the MUST relation always holds true if the target URI is written in
an absolute URI form.   So in this case it (to me) makes sense that including any anchor parameter is not necessary in that case, even superfluous.

How to proceed with the issue of the different interpretations #1/#2?

Best regards
Esko

-----Original Message-----
From: Christian Amsüss <[email-masked]> 
Sent: Wednesday, October 21, 2020 15:32
To: Esko Dijk <[email-masked]>
Cc: [email-masked]
Subject: Re: [core] Unnecessary "anchor" attributes in draft-ietf-core-resource-directory-25 ?

KR
c

---

From: Esko Dijk <[email-masked]>
Date: Mon, 26 Oct 2020 10:01:09 +0000

> AFAIK that means it's only mine).

The proposed rule sounds good. It's easier to understand and apply for implementers than what I proposed.
So if the registering client sends its list of relative links without any 'rel' attribute in there nor 'anchor', the RD serves them back to lookup clients as absolute URIs without any 'rel' nor 'anchor' - unmodified statements, except for the change of relative URI to absolute URI.

> context is origin-of(resolved-target), and then no further conditions on

Yes, agree.  That would be good to clarify/profile.

> even a step further than you are suggesting.

Okay, I'm hoping the outcome would be the one where the 'anchor' attributes are not added by the RD. (Per above text)

> can't profile it and prescribe that a particular serialization needs to

Agree that we can make a profile of RFC 6690 and not update it!  (as long as we don't violate any existing 6690 rules).
Sorry if I wasn't fully clear here.  In the case of the current RD draft text, the profile would need to say e.g. "the Limited Link 
Format serialization created by RD-lookup response must include the 'anchor' attribute in each link".
This is needed to clarify why all the examples use the 'anchor' while 6690 doesn't require it. And also to clarify that
RD implementations cannot use the shorter variant without 'anchor' even though 6690 would allow it.

Best regards
Esko

-----Original Message-----
From: Christian Amsüss <[email-masked]> 
Sent: Wednesday, October 21, 2020 16:00
To: Esko Dijk <[email-masked]>
Cc: [email-masked]
Subject: Re: [core] Unnecessary "anchor" attributes in draft-ietf-core-resource-directory-25 ?

Hello Esko,