Subject: [core] Questions on draft-ietf-core-resource-directory-12

From: Jim Schaad <[email-masked]>
Date: Sun, 1 Apr 2018 11:37:39 -0700

Here are some more questions on the document.

1.  The registration text in section 5.3 says that the interface is
idempotent, however it is not clear to me just how idempotent this is
supposed to be.  It is obvious that the goal is to make sure that a
different resource is not created, however I am not clear what is supposed
to happen with some of the parameters such as the 'lt' parameter.  One way
to implement this is to just make it a POST to the previously existing
resource which would inherit old attributes on the endpoint.  The other way
is to say we are going to create a new resource, it just has the same
address.  It is not clear which, if any, of these two methods is to be used.

Never mind I found the answer to this one.  Need to think if I want clearer
language.

2.  When doing an endpoint lookup, I assume that the 'lt' parameter would be
the same as the one registered at creation.  Is there/should there be a
parameter which is a TTL value?

3.  How are href comparisons done.  
	a) compare against what was registered
	b) Resolve what was registered and the href in the current context
and compare
	c) something I did not think of

4.  compare on group for remote endpoint.  Should this do the remote lookup
or can it just be ignored for attribute comparison

5.  Value of lt = should it be the set parameter or the time left - how
would time left be represented outherwise - should it be represented.

6.  Inferred context based on source IP when an update is done.  If no con
is provided, is that supposed to be checked again to see if it is "right"?
Only if we did the infer to begin with?

7.  I don't understand the presence of the content-format for the empty
message posted for simple registration.  This seems to be odd if the content
is empty.

8  Is there supposed to be a way for simple registration to return an error
message?  As written this is not necessarily done.  Specifically, the
response to the post can be executed before the query to /.well-known/core
is done and the registration is attempted.

---

From: peter van der Stok <[email-masked]>
Date: Wed, 04 Apr 2018 11:21:09 +0200

Hi Jim,

thanks for the questions. Se my answers below.

> Never mind I found the answer to this one.  Need to think if I want 

I hope we find the same answer.
The text says "registering with the same endpoint parameters , ep and d, 
does not create multiple resources.
Consequently, registering with existing ep and d removes existing 
registration and creates new registration with new location. Registering 
with a different ep and same d, or different d and same ep, creates new 
registration next to the existing one. The values of the other 
parameters are irrelevant.

is additional text like above needed?
> parameter which is a TTL value?

I don't understand the question. TTL is about the lifetime of a packet 
in the network (expressed in hops)
lt is about the lifetime of the registration expressed in seconds
> 	c) something I did not think of

probably someone else wants to react as well.
The model in the RD stores the target as specified in /.well-known/core 
of the source.
At lookup the links are resolved against con.
Consequently, the comparison of href is the comparison versus the target 
as stored in the RD and as copied from /.well-knwon/core.

Is that reasonable for you? needs extra text in the RD spec.?

> or can it just be ignored for attribute comparison

I am not quite sure what you mean by attribute comparison.
When it is about registration attributes, no remote lookup seems 
necessary to me.

> would time left be represented outherwise - should it be represented.

Right on the nail. This is very vaguely specified. The unit is specified 
as seconds.
In my expectation the returned lt value is the time in seconds from 
lookup till end.

Other opinions by my co-authors?  If we all agree, we specify it as such 
in the RD spec.
> Only if we did the infer to begin with?

My interpretation of the current text is:
When a  registration is done with the same ep and d values, the old 
registration is removed and
a new one with the new parameter values is created at a new location. 
That means that the value of con
is specified in the registration without any memory of earlier values.
> message posted for simple registration.  This seems to be odd if the 

content-format is specified such that RD knows what content-format to 
request.
> is done and the registration is attempted.

I see what you mean: also the return of 2.04 shows in the example but 
not in the specification.
This warrants additional text to specify the simple registration text.
Many thanks for these questions.
Looking forward to your answer,

Greetings,

peter
> 
> _______________________________________________
> core mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/core

---

From: Jim Schaad <[email-masked]>
Date: Wed, 4 Apr 2018 08:56:50 -0700

> -----Original Message-----
> From: peter van der Stok <[email-masked]>
> Sent: Wednesday, April 4, 2018 2:21 AM
> To: Jim Schaad <[email-masked]>
> Cc: [email-masked]; 'core' <[email-masked]>
> Subject: Re: [core] Questions on draft-ietf-core-resource-directory-12
> 
> Hi Jim,
> 
> > different resource is not created, however I am not clear what is
> > supposed to happen with some of the parameters such as the 'lt'
> > parameter.  One way to implement this is to just make it a POST to the
> > previously existing resource which would inherit old attributes on the
> > endpoint.  The other way is to say we are going to create a new
> > resource, it just has the same address.  It is not clear which, if
> > any, of these two methods is to be used.
> >
> > Never mind I found the answer to this one.  Need to think if I want
> > clearer language.
> 
with
> a different ep and same d, or different d and same ep, creates new
> registration next to the existing one. The values of the other parameters
are
> irrelevant.

I believe that if you register with existing ep and d, it will remove the
existing registration and create a new registration with the SAME location.
This is part of being idempotent.

I agree w/ the endpoint/d mismatch.

> > 2.  When doing an endpoint lookup, I assume that the 'lt' parameter
> > would be the same as the one registered at creation.  Is there/should
> > there be a parameter which is a TTL value?
> 
> I don't understand the question. TTL is about the lifetime of a packet in
the
> network (expressed in hops) lt is about the lifetime of the registration
> expressed in seconds

In this case, TTL would be the number of seconds until the registration
expires.   I have seen TTL used in non-packet situations to describe the
exact same thing - how long is this going to be alive.  I have debated using
the smallest TTL value when I return a result as the option Max-Age, but
that does not quite mesh with how that option is defined.

> The model in the RD stores the target as specified in /.well-known/core of
> the source.
as
> stored in the RD and as copied from /.well-knwon/core.
> 

This is how I read the current text, but it was not the only possible
answer.  If the answer is not b, then I don't think extra text is needed.

> 
> >
> > 4.  compare on group for remote endpoint.  Should this do the remote
> > lookup or can it just be ignored for attribute comparison
> 
> When it is about registration attributes, no remote lookup seems necessary
> to me.

If I do the lookup of 
/rd/gp-lookup?rt=foobar

And one of the groups has the following

/rd-group/xxx
https://otherrd/rd-group/yyy

I would follow the local group definition to find out if the resource type
is defined on some resource in the group, however the question is do I need
to go and query the other group which is not local in order to do the same
tree walking when doing attribute comparisons.  Not that this applies to
remote endpoint registrations in a group as well.

> 
> >
> > 5.  Value of lt = should it be the set parameter or the time left -
> > how would time left be represented outherwise - should it be
represented.
> Right on the nail. This is very vaguely specified. The unit is specified
as
> seconds.
> In my expectation the returned lt value is the time in seconds from lookup
till
> end.
in the
> RD spec.

This is a dup of the TTL question above.  Sorry that I missed that.

> >
> > 6.  Inferred context based on source IP when an update is done.  If no
> > con is provided, is that supposed to be checked again to see if it is
> > "right"?
> registration is removed and a new one with the new parameter values is
> created at a new location.

This is not the sequence that I was looking for.

POST /rd?ep=node1
   - payload contains the set of resources the register.
   - returns a location of /rd/yyyy

POST /rd/yyyy?ep=node1
   - payload is empty.

In the first post, the rd will infer a context based on how the registration
is done.  The second post just says that the TTL value is to be refreshed
back to the life-time value.  However, the inferred context could also be
changed.

> >
> > 7.  I don't understand the presence of the content-format for the
> > empty message posted for simple registration.  This seems to be odd if
> > the content is empty.
> 

Would accept be a better option to use?  This would allow for more than one
value to be specified.

> >
> > 8  Is there supposed to be a way for simple registration to return an
> > error message?  As written this is not necessarily done.
> > Specifically, the response to the post can be executed before the
> > query to /.well-known/core is done and the registration is attempted.
> 
> I see what you mean: also the return of 2.04 shows in the example but not
in
> the specification.

---

From: peter van der Stok <[email-masked]>
Date: Thu, 05 Apr 2018 09:51:25 +0200

Hi Jim,

some additional questions and remark,

>> parameters

Idempotency says that applying the same operation multiple times should 
lead to the same result independent of the number of times.
The problem is: what is the same operation; If only ep and d are 
relevant then applying multiple times an operation (registration) with 
the same ep and d values indeed leads to the same result ignoring all 
other attribute values (including location)

If one says that the same operation implies that all other specification 
parameters are unchanged, then indeed location should not be changed in 
that case.

What view do you adhere to?

And do you suggest we should insert clarifying text?

>> necessary

Thanks for the example.
Indeed following the latest text, the comparison should include the 
remote registration.

The text could include words like tree walking is done locally only; 
personally I would not be too happy about that.
Apparently the text should say something about remote groups and tree 
walking. I will add the issue to github.
>> lookup
Also an issue for github to get more reactions
> back to the life-time value.  However, the inferred context could also 

Thanks for the example; it would take me long to figure this out 
otherwise.

Reading the update specification, the ep=node1 is illegal in the update. 
Specifying ep means creating a new registration.

sending the update
POST /rd/yyy will update the lt value.
It is allowed to change the value of con in the update POST 
specification.
The question is:
When an updating ep, different from the registering one, sends the 
update of lt, does the context change to the hosts relation value of the 
updating ep.

My gut reaction is no. it does not change the context value in the RD 
because that needs an explicit ?con= in the update specification.

A warning in the RD text to this effect looks reasonable.
> Would accept be a better option to use?  This would allow for more than 

Agree.
>> not

will add the simple registration template to the text.

many thanks for your additional explanations.
One issue of idempotency is left.

Peter

---

From: Christian Amsüss <[email-masked]>
Date: Fri, 13 Apr 2018 20:29:47 +0200

Hello Jim, Peter,

On Thu, Apr 05, 2018 at 09:51:25AM +0200, peter van der Stok wrote:
> Idempotency says that applying the same operation multiple times should lead
> to the same result independent of the number of times.
> The problem is: what is the same operation; If only ep and d are relevant
> then applying multiple times an operation (registration) with the same ep
> and d values indeed leads to the same result ignoring all other attribute
> values (including location)
> 
> parameters are unchanged, then indeed location should not be changed in that
> case.
> 

We should look at why we are requiring idempotency here.

From what I can tell, this is because CoAP servers and clients are
easier to implement if they can rely on the operation to be idempotent,
but they are looking at the very same request being sent again in a
relatively short time frame.

An additional use case here is stability in presence of "semi-simple"
clients that POST their .well-known/core to the directory resource every
few hours and never try to parse the Location returned. If those change
the registration resource address every time, an observation on the
endpoint lookup becomes needlessly noisy.

As long as those are the cases we consider, I think that it is
sufficient to specify that the registration operation must return the
same response to a CoAP request with the same requester / security
context, code, options and payload within EXCHANGE_LIFETIME.

The concern for observation stability is more a matter of implementation
quality, where me can recommend but need not specify.

> Consequently, the comparison of href is the comparison versus the target as

I think the more convenient thing to do is to say that resources are
compared against the full path, and registrations and group resources
(they can be matched too) are matched against whatever is returned in
endpoint lookup.

That would be more intuitive to the user of the lookup (matching against
what is returned).

> > I would follow the local group definition to find out if the
> > resource type is defined on some resource in the group, however the
> > question is do I need to go and query the other group which is not
> > local in order to do the same tree walking when doing attribute
> > comparisons.  Not that this applies to remote endpoint registrations
> > in a group as well.
> 
> Thanks for the example.
> Indeed following the latest text, the comparison should include the remote
> registration.
> 

I think that this is out of scope for RD itself. What I'd like to have
in RD is the text that ensures that clients will not get upset when they
see absolute URIs in group or endpoint lookups.

We are not actually describing a distributed RD here, this is only to
keep the door open for such extensions.

Would it help if we changed the example to say

GET coap://rd.example.com/ep?gp=lights1
Res: 2.05 Content
<coap+tcp://rd.example.com/rd/abcd>;con="coap://[2001:db8:3::123]:61616";
ep="node1";et="oic.d.sensor";ct="40";lt="600",
</rd/efgh>;con="coap://[2001:db8:3::124]:61616";
ep="node2";et="oic.d.sensor";ct="40";lt="600"

? It would just indicate that the registration resource is hosted by the
same RD but on another protocol (presumably because the endpoint
registered using coap+tcp, and that Location can't just jump protocols).

> > how would time left be represented outherwise - should it be represented.
> 
> Right on the nail. This is very vaguely specified. The unit is
> specified as seconds.
> In my expectation the returned lt value is the time in seconds
> from lookup till end.
> 
> Other opinions by my co-authors?  If we all agree, we specify it as
> such in the RD spec.

My expectation is that lt= is the registered value and never decreases.
(For an additional value that indicates where the registration is in its
lifetime, I've suggested lt-age in draft-amsuess-core-rd-replication-01
-- but only because there's an application for it there). Having lt
decrease is problematic with caching (admittedly, so is lt-age).

What use cases do we have for exposing a lifetime in the endpoint lookup
at all?

(If we have none, an RD might just as well not display a lifetime).

> > In the first post, the rd will infer a context based on how the
> > registration is done.  The second post just says that the TTL value
> > is to be refreshed back to the life-time value.  However, the
> > inferred context could also be changed.
> 
> Thanks for the example; it would take me long to figure this out otherwise.
> 

ep=node1 would be ...well not technically illegal (the template would
put the ep in extra-attrs, and the server hopefully not store it there);
it is an attribute that is not changing and therefore SHOULD NOT be
included in the update by the client.

A changed ep would not create a new registration (we're at a
registration resource already), but simply be refused.

> It is allowed to change the value of con in the update POST specification.
> The question is:
> When an updating ep, different from the registering one, sends the update of
> lt, does the context change to the hosts relation value of the updating ep.
> 

The text is already pretty explicit:

    "If the parameter is set in an update, it is stored by the RD as the
    new Base URI under which to interpret the links of the registration,
    following the same restrictions as in the registration.  If the
    parameter is not set and was set explicitly before, the previous
    context value is kept unmodified.  If the parameter is not set and
    was not set explicitly before either, the source address and source
    port of the update request are stored as the context."

This allows for clients that do not keep a good eye on their addresses
but have their OS pick the latest one.

Note that nobody other than the endpoint (or CT if created by one) may
manipulate a registration resource.

(If we wanted to allow other hosts to manipulate a different endpoint's
registration, we'd need something like a ?ct=keep or change in semantics
to allow that -- but I'd like to have a good reason to do that in the
first place).

> [...]
> Would accept be a better option to use?  This would allow for more than
> one value to be specified.

It should be neither; that request is empty and so is the response.

The RD will need to probe for the content format of its own (possibly
trying its preferred content format first, falling back to an empty
accept or 40).

The best equivalent of "and look there expecting this content format"
would be HTTP's link header (a la 'POST /.w-k/c?ep=node1, Link:
<coap://myself/.well-known/core>;ct="40 64 504"'), and we can't do that
in CoAP, at the very least not in a simple registration.

> > I see what you mean: also the return of 2.04 shows in the example
> > but not in the specification.

I think that the more important question behind is of whether the RD
should (or may) wait with a response until it has fetched the .w-k/c of
the endpoint. The example shows "respond first, then ask details", which
we might consider prescribing given that highly constrained clients
might not manage to multi-task the request and the response. (Dealing
5.03 Sorry I'm Busy until their pending request is answered or expired).

Best regards
Christian

(updating the issue tracker in parallel)

-- 
To use raw power is to make yourself infinitely vulnerable to greater powers.
  -- Bene Gesserit axiom

---

From: Jim Schaad <[email-masked]>
Date: Sun, 15 Apr 2018 13:49:38 -0700

> From: Christian Amsüss <[email-masked]>
> Sent: Friday, April 13, 2018 11:30 AM
> To: [email-masked]
> Cc: Jim Schaad <[email-masked]>; draft-ietf-core-resource-
> [email-masked]; 'core' <[email-masked]>
> > Idempotency says that applying the same operation multiple times
> > should lead to the same result independent of the number of times.
> > If one says that the same operation implies that all other
> > specification parameters are unchanged, then indeed location should
> > not be changed in that case.
> >
> From what I can tell, this is because CoAP servers and clients are easier
to
> implement if they can rely on the operation to be idempotent, but they are
> looking at the very same request being sent again in a relatively short
time
> frame.
few
> hours and never try to parse the Location returned. If those change the
> registration resource address every time, an observation on the endpoint
> lookup becomes needlessly noisy.
> 
> As long as those are the cases we consider, I think that it is sufficient
to
> specify that the registration operation must return the same response to a
> CoAP request with the same requester / security context, code, options and
> payload within EXCHANGE_LIFETIME.
> 

I am less worried about the observation jitter problems than I am about the
turnover on groups.  Given that groups are defined in terms of the href
returned from an endpoint registration this means that groups are going to
be updated every time.  I really think that the location needs to not change
for the lifetime of the registration not just the EXCHANGE_LIFETIME.  I am
not sure that idempotent means anything useful when looking at doing
registrations because I don't really know what it would mean to say
something is idempotent here.   Does it mean that if nothing changes then
nothing happens or just that if you register the same thing then the same
set of events should happen w/ the same outcomes.

> > The model in the RD stores the target as specified in
> > /.well-known/core of the source.
> > Consequently, the comparison of href is the comparison versus the
> > target as stored in the RD and as copied from /.well-knwon/core.

I am not sure what you are saying.  I do comparison followed by
serialization at the moment.  I do the resolution during serialization and
not during comparison.  This is the reason that I was asking the question.

> I think the more convenient thing to do is to say that resources are
compared
> against the full path, and registrations and group resources (they can be
> matched too) are matched against whatever is returned in endpoint lookup.
> 

This is understandable and reasonable.  And would mean that I need to change
my code.

> I think that this is out of scope for RD itself. What I'd like to have in
RD is the
> text that ensures that clients will not get upset when they see absolute
URIs
> in group or endpoint lookups.
> 
> We are not actually describing a distributed RD here, this is only to keep
the
> door open for such extensions.
> 
same
> RD but on another protocol (presumably because the endpoint registered
> using coap+tcp, and that Location can't just jump protocols).

Yeah, that whole question jumps back to the question - if it is on a
different schema is it the same resource or a different resource.   I think
that the example with the explanation would be better.

> (For an additional value that indicates where the registration is in its
lifetime,
> I've suggested lt-age in draft-amsuess-core-rd-replication-01
decrease is
> problematic with caching (admittedly, so is lt-age).
> 
at
> all?

All of the cases that I can think of can be done with observe.  I think that
this probably does not need to be discussed except to say that max-age
should not be set to this value.

otherwise.
> ep=node1 would be ...well not technically illegal (the template would put
the
> ep in extra-attrs, and the server hopefully not store it there); it is an
attribute
> that is not changing and therefore SHOULD NOT be included in the update by
> the client.
> 
> A changed ep would not create a new registration (we're at a registration
> resource already), but simply be refused.
> 
> > sending the update
specification.
> This allows for clients that do not keep a good eye on their addresses but
> have their OS pick the latest one.
> 
to
> allow that -- but I'd like to have a good reason to do that in the first
place).

I missed this text.  It answers the question that I had.

> > > empty message posted for simple registration.  This seems to be odd
> > > if the content is empty.
> >
> > [...]
> >
> > Would accept be a better option to use?  This would allow for more
> > than one value to be specified.
> 
trying
> its preferred content format first, falling back to an empty accept or
40).
> <coap://myself/.well-known/core>;ct="40 64 504"'), and we can't do that in
> CoAP, at the very least not in a simple registration.
> 
> 
> > > > 8  Is there supposed to be a way for simple registration to return
> > > > an error message?  As written this is not necessarily done.
should
> (or may) wait with a response until it has fetched the .w-k/c of the
endpoint.
> The example shows "respond first, then ask details", which we might
> consider prescribing given that highly constrained clients might not
manage
> to multi-task the request and the response. (Dealing

I would have done an ACK (to stop resends) followed by a response when the
resources had been queried rather than returning two different responses
that are different and would need to have a pending.  If the client cannot
wait for the response, it knows that the message is being processed.
However it would know not to shutdown until it has gotten a final answer.  I
don't see any need to return some type of busy.  The constrained client
should be able to respond to questions while it is in the middle of asking
one.

Jim

---

From: peter van der Stok <[email-masked]>
Date: Mon, 16 Apr 2018 09:46:27 +0200

> included in the update by the client.

I do not follow.
Registration template:
EP->RD
POST
URI template {+rd}{?ep,d,lt,con,extra-attrs*}

Update template
EP->RD
POST
URI template {+rd}{?lt,con,extra-attrs*}

What distinguishes Update from registration? => the presence of ?ep,d
saying that extra-attrs* can include ep makes life really difficult IMO.

sending a POST to /rd/123 can create a new registration at /rd/123/567, 
according to me, we did not distinguish resources as registration 
resource or general rd resource.

Am I missing something?

>> update of

Do we understand that the hosts relation does not set the parameter?

> manipulate a registration resource.

but difficult to check or enforce. the registration has no memory of the 
registering ep.

---

From: Christian Amsüss <[email-masked]>
Date: Tue, 17 Apr 2018 09:23:21 +0200

Hello Peter,

> saying that extra-attrs* can include ep makes life really difficult IMO.

What distinguishes them is the first part of the template:

The registration goes to /rd.

The update goes to /reg/1234 (and should be {+location}).

(Example paths used without loss of generality.)

> according to me, we did not distinguish resources as registration resource
> or general rd resource.

I do not think so. It is nowhere that we indicate that the Registration
Update operation on the location (POST /reg/1234) creates anything or
returns a Location.

(General note on POST: POST doesn't necessarily mean "create this as a
new resource under", its general meaning is "Enact whatever your thing is
with this data". That can often be "store in a new resource" as it is
with the /rd directory resource.)

> Do we understand that the hosts relation does not set the parameter?

Where does rel="hosts" come into play here?

> but difficult to check or enforce. the registration has no memory of the

It doesn't need to be enforced, only to be trusted.

Best regards
Christian

---

From: Christian Amsüss <[email-masked]>
Date: Tue, 17 Apr 2018 15:18:04 +0200

Hello Jim, hello RD authors,

> set of events should happen w/ the same outcomes.

But the location does not change over the lifetime of the registration
(see also my parallel response to Peter at [1]) unless I got something
very wrong.

Only if an endpoint re-registers (ie. sends the same POST to /rd),
long-term idempotency does become the matter. But I'd argue that this is
close enough to situations of "the EP did not update its registration in
time and already got kicked out" than to regular updates.

Not that we'd already have an answer to how group management interacts
with temporarily timed-out clients; maybe we should solve that first and
then come back to the topic of idempotency.

[1]: https://mailarchive.ietf.org/arch/msg/core/IYkEs59MMdPWPH5nOjiYyO6juFY

> not during comparison.  This is the reason that I was asking the question.

I'd say "compare the resolved absolute references", in other words
"resolve, then compare, then serialize". That's more work for the RD
(and for you as an implementor), but matches more closely the
expectations of the client, and should be better suited for later steps
with protocol-negotiation.

> > We are not actually describing a distributed RD here, this is only
> > to keep the door open for such extensions.
> > 
> > ? It would just indicate that the registration resource is hosted by
> > the same RD but on another protocol (presumably because the endpoint

Unless something explicitly says that it is the same, it is not.

Text proposal at [2] for review by the other authors.

[2]: https://github.com/core-wg/resource-directory/pull/129

> > What use cases do we have for exposing a lifetime in the endpoint
> > lookup at all?
> > 

There is always a Max-Age, it has a default value of 60 (seconds).

> > should (or may) wait with a response until it has fetched the .w-k/c
> > of the endpoint.  The example shows "respond first, then ask
> > details", which we might consider prescribing given that highly
> > constrained clients might not manage to multi-task the request and
> > the response. (Dealing 5.03 Sorry I'm Busy until their pending
> > request is answered or expired).
> 

I was not so much thinking about returning "busy" than about returning
"Success" in the sense of "I've got your request; whatever happens
further is out of your control".

Given that two RDs are behaving observably different here, I'm opening
this as an issue (basically "should we manadate your behavior or is
either fine, and if the latter do we need to warn simple client
implementors?") at [3].

[3]: https://github.com/core-wg/resource-directory/issues/130

Thanks for your input
Christian