Subject: [apps-discuss] Using the .well-known registry for SRV (and other purposes)

From: Phillip Hallam-Baker <[email-masked]>
Date: Thu, 17 Jan 2013 17:20:33 -0500

When setting up a web service it would be really nice to be able to use SRV
records for discovery.

That then sets up the question of where to look for the service on the HTTP
server which might be dedicated to just one web service or support more
than one.

There is a DNS URI scheme that is intended to address this issue but this
gives more flexibility than we actually need. The idea of using something
like the URI scheme for discovery is so that we can tell someone about our
web service by specifying no more than the domain name and the service name
- e.g. "example.com" and "omnibroker". We don't need to be able to put that
anywhere on the web server, just somewhere that it isn't going to conflict.

Now chances are that the .well-known string and the SRV string (if
assigned) are going to be the same. So how about we could use the
.well-known registry for BOTH purposes?

So imagine my Web Service name is omnibroker

SRV Prefix: _omnibroker._well-known.example.com
URI stem: http://example.com/.well-known/omnibroker/

This would seem to cover all the bases. We can argue as to what the SRV
prefix should be. _well-known has two types of separator, maybe _ws would
be simpler.

Finally, we might want to also define a default URI scheme at the same
time, options for this would be:

omnibroker:example.com
ws:omnibroker:example.com

The first is a lot cleaner but probably involves more IETF plumbing than
people are going to accept. The second is pretty simple and has the
advantage that code knows that what is being dealt with here is a web
service type thing.

Note that if SRV discovery is being used there is no need to specify a
port. If SRV discovery is not to be used then the web service endpoint
would specify the port directly:

wks:omnibroker:example.com:80

This approach would effectively collapse what are currently three separate
requests into a single request. It would also encourage every web service
to make use of URI format identifiers.

-- 
Website: http://hallambaker.com/

---

From: Nico Williams <[email-masked]>
Date: Thu, 17 Jan 2013 16:39:26 -0600

On Thu, Jan 17, 2013 at 4:20 PM, Phillip Hallam-Baker <[email-masked]> wrote:
> Finally, we might want to also define a default URI scheme at the same time,
> options for this would be:
>
> omnibroker:example.com

The latter.  For the reasons you surmise.

> Note that if SRV discovery is being used there is no need to specify a port.
> If SRV discovery is not to be used then the web service endpoint would
> specify the port directly:
>

Well, we might use well-known to find the port number by asking the
server at port 80.

Would this say anything to the user-agent about how to authenticate
the service?  For example, when using Kerberos this might indicate to
the user-agent that the service principal should be
omnibroker/<server>.  But we might need to be able to specify
naming/security options for other mechanisms (and even for Kerberos,
we might want to say what the realm is).

Anyways, I think I like this.  We badly need service discovery!

Nico
--

---

From: Phillip Hallam-Baker <[email-masked]>
Date: Thu, 17 Jan 2013 18:35:41 -0500

> The latter.  For the reasons you surmise.

Yeah, I suspected as much.

> port.

I am not sure what you mean there. It is certainly possible to do an
internal redirect or an explicit redirect.

The idea I had for resolving the URI was as follows:

1) If the port is specified explicitly, use the specified port and domain
2) Otherwise, try to retrieve an SRV record at the specified domain
3) Otherwise, default to port 80

Which now strongly suggests that what we actually need are wk: and wks:
prefixes.

wk: use http, default port is 80
wks: use http over tls, default port is 443

> we might want to say what the realm is).

I don't see a need to specify that inband. We could certainly specify a
mechanism as part of HTTP authentication but I don't think we need to bring
that forward into the URI.

---

From: Nico Williams <[email-masked]>
Date: Thu, 17 Jan 2013 17:41:54 -0600

> that forward into the URI.

But since we're doing service discovery, the discovery service can
tell us information we need *before* we start authentication.  For
example, it might tell us what CA issues the real servers' certs --
nothing else could do that (well, DNSSEC, that is, other discovery
mechanisms could) as we couldn't trust a server to tell us how to
authenticate it.

Nico
--

---

From: Phillip Hallam-Baker <[email-masked]>
Date: Thu, 17 Jan 2013 18:58:12 -0500

> mechanisms could) as we couldn't trust a server to tell us how to

Oh you mean add the CA key fingerprint to the service URI?

wks:omnibroker:example.com:ca=
6ac3c336e4094835293a3fed8a4b5fedde1b5e2626d9838fed50693bba00af0e

Or similar DANE like things.

We could do that but the amount of arguing involved would be significant.

---

From: Nico Williams <[email-masked]>
Date: Thu, 17 Jan 2013 18:03:14 -0600

> We could do that but the amount of arguing involved would be significant.

What an optimist!  :)

---

From: Patrik Fältström <[email-masked]>
Date: Fri, 18 Jan 2013 06:32:34 +0100

On 17 jan 2013, at 23:20, Phillip Hallam-Baker <[email-masked]> wrote:

> There is a DNS URI scheme that is intended to address this issue but this gives more flexibility than we actually need. The idea of using something like the URI scheme for discovery is so that we can tell someone about our web service by specifying no more than the domain name and the service name - e.g. "example.com" and "omnibroker". We don't need to be able to put that anywhere on the web server, just somewhere that it isn't going to conflict. 

Can you expand on this please.

When reading what you write, I thought this was exactly what you described:

_omnibroker._well-known.example.com. IN URL ....

   Patrik

---

From: Patrik Fältström <[email-masked]>
Date: Fri, 18 Jan 2013 06:39:34 +0100

> _omnibroker._well-known.example.com. IN URL ....

Sigh, before morning coffee...

_omnibroker._well-known.example.com. IN URI 0 0 "scheme://host:port/more_stuff"

   Patrik

---

From: Phillip Hallam-Baker <[email-masked]>
Date: Fri, 18 Jan 2013 01:02:32 -0500

My point is that in the context of mashing the .well-known registry and the
DNS prefix registry together, we have already decided that the deployer
does not get to choose where the endpoint goes on the Web server.

So URI gives us more flexibility, but not flexibility that we necessarily
need. We certainly don't need the URI stem part at any rate.

What would have value though is being able to specify the protocol. For
example:

_omnibroker._well-known.example.com. IN URI 0 0 "
http://w1.example.com/.well-known/omnibroker/"
_omnibroker._well-known.example.com. IN URI 0 0 "https://
w1.example.com/.well-known/omnibroker/"
_omnibroker._well-known.example.com. IN URI 0 0 "coap://
w1.example.com/.well-known/omnibroker/"

That lets us extend the signalling to give the transport and in particular
specify whether to use http or http over SSL.

Switching to http2 could also be signaled in the same way.

So I didn't see a use for URI for the stem part alone, but the scheme part
is certainly useful. And if we are using URI then of course we would let
the URI specify the stem and this would be an eventual route out of
.well-known

So the discovery would be

1) If there is a port specified on the URI (wk:omnibroker://example.com:80/)
then use the speficied domain and port

2) Otherwise use URI for discovery

3) Otherwise try the default port and http for wk or https for wks

What I do think is that we should either choose SRV or URI not have a
choice or both.

On Fri, Jan 18, 2013 at 12:39 AM, Patrik Fältström <[email-masked]> wrote:

> >> There is a DNS URI scheme that is intended to address this issue but
> this gives more flexibility than we actually need. The idea of using
> something like the URI scheme for discovery is so that we can tell someone
> about our web service by specifying no more than the domain name and the
> service name - e.g. "example.com" and "omnibroker". We don't need to be
> able to put that anywhere on the web server, just somewhere that it isn't
> going to conflict.
> >

---

From: Phillip Hallam-Baker <[email-masked]>
Date: Fri, 18 Jan 2013 01:06:40 -0500

---

From: Cyrus Daboo <[email-masked]>
Date: Fri, 18 Jan 2013 09:43:29 -0500

Hi Phillip,

--On January 18, 2013 1:06:40 AM -0500 Phillip Hallam-Baker 
<[email-masked]> wrote:

> My point is that in the context of mashing the .well-known registry and
> the DNS prefix registry together, we have already decided that the
> deployer does not get to choose where the endpoint goes on the Web server.
>
>

Please see draft-daboo-srv-caldav (in RFC ed queue for 800+ days now). That 
spec uses new IANA service types to allow SRV->web-service mapping with 
.well-known mappings also registered. Initially the spec just mapped 
SRV->.well-known. During final reviews there was a TSV-Dir review that 
requested to make it more flexible and that resulted in the addition of an 
option to have an accompanying TXT record with a path= attribute that 
effectively defined the URI path segment (stem part in your terminology) to 
be defined. I was never keen on that, and as it has turned out I am not 
aware of anyone who has deployed with a TXT (and indeed the additional spec 
reference required by that is the one that has held up publication for so 
long). Anyway, there was a lot of discussion on this at the time on 
[email-masked] so you can see why it ended up the way it did by reviewing 
those if you care.

There was also discussion on whether Patrik's URI RR should be used instead.

In any case, buy-in from TSV/DNS folks will be needed to move your idea 
forward, and having that done earlier rather than later would be best...

I should note that there are other drafts using is similar 
SRV/TXT/.well-known approach too: draft-douglass-timezone-service, 
draft-desruisseaux-ischedule, and draft-daboo-aggregated-service-discovery. 
The idea of putting all the registered .well-known services "automatically" 
under a DNS namespace (by defining _well-known as the service type) seems 
attractive if we expect a lot of folks to want to do SRV->.well-known. The 
one downside is not being able to encode the scheme via a single service 
type - instead we had to have _caldav and _caldavs for http and https 
mappings. That is something where the URI RR has an advantage...

Another thing to point out: Section 1.1 in RFC5785 indicates that 
.well-known is not meant for "general information retrieval". That 
statement required srv-caldav to state that the .well-known URI could not 
be the actual service endpoint, and instead an HTTP redirect to the actual 
service would need to be done. So the original intent of .well-known needs 
to be considered before opening the flood gates for any/all web services to 
be exposed that way.

-- 
Cyrus Daboo

---

From: Phillip Hallam-Baker <[email-masked]>
Date: Fri, 18 Jan 2013 11:22:20 -0500

On Fri, Jan 18, 2013 at 9:43 AM, Cyrus Daboo <[email-masked]> wrote:

> Hi Phillip,
>
>
> --On January 18, 2013 1:06:40 AM -0500 Phillip Hallam-Baker <
> [email-masked]> wrote:
>
> Please see draft-daboo-srv-caldav (in RFC ed queue for 800+ days now).
> That spec uses new IANA service types to allow SRV->web-service mapping
> with .well-known mappings also registered. Initially the spec just mapped

800+ days in the queue strongly suggests that is what you should do.

It sounds like the draft is blocked because

1) People insisted on more flexibility
2) The mechanism proposed to achieve the additional flexibility was not
acceptable
3) The issue was not raised with sufficient force in the places that a
decision might be made

On issues like this where the only thing that really matters is that the
IETF just pick one way to do something, it is a process failure if we end
up with delays of 90 days let alone three years.

Directorates have no formal standing in IETF. They can comment on a
document during last call but those comments have no more force than any
other reviewer. Only an AD can block a draft. So your problem would be with
the AD rather than the directorate.

> forward, and having that done earlier rather than later would be best...

No, I think the thing to do here is to put a solution on the table that
anticipates their position based on the already exhaustive discussions they
have had.

The whole point of making the proposal in the first place is that the
current scheme has too many moving parts. If the IAB is going to have
architecture in its name then this is the sort of issue where they should
be taking a lead if a deadlock occurs.

If architecture is going to make any sense there should be one discovery
mechanism that we are heading towards.

As I see it there are two options for going forward on CALDAV:

1) Adopt URI as a replacement for SRV
2) Drop the TXT record hack

Given the installed base of CALDAV the de facto standard is to only
retrieve SRV records and any site that attempts to advertise a service that
is not backwards compatible is going to break.

So I think we have to consider CALDAV as a legacy case. We already have a
huge amount of well known port legacy that is going to hand around for 20
years. Having CALDAV do SRV instead of URI is hardly going to hurt much.

Note that under my proposed scheme there would also be a wk:caldav: URI
scheme that would be available for use if this was desirable.

> SRV/TXT/.well-known approach too: draft-douglass-timezone-**service,
> draft-desruisseaux-ischedule, and draft-daboo-aggregated-**service-discovery.

I saw that as well. There is a slight difference in the semantics. The
caldavs: has an explicit 'must use TLS' policy that is bound to the
identifier. Putting that information in the DNS means that the policy is
acquired during discovery and is only as good as the DNS security. There is
a significant change in semantics there even if you use DNSSEC to secure
the DNS.

I think you actually need both.

> to be considered before opening the flood gates for any/all web services to

I think 'they' are confused bunnies and don't understand what the
restriction was about.

I think that the point of the restriction is to avoid conflict between Web
Services and Web Browsing. Most Web Services are at least 90% idempotent
information retrieval with no change of state. But they are implemented in
a very different way. They are not static data, they are dynamically
generated content.

Probably a good test for whether something is a Web Service or a Web
Content is whether you are likely to want to have a link to the actual Web
Service appear in a URL or not.

So one final thought here that struck me while writing this is that what we
probably want is not Patrick's URI proposal but a very slight modification
so that we have two slots, one for the URI and one for an optional policy
specification. That would then allow the service endpoint to include
relevant descriptive information such as the minimum and maximum supported
versions and credential requirements etc.

It seems much better to glue that information to the discovery record
advertising the service than any other technique:

Ownername TTL Class URI Priority Weight Target

becomes

Ownername TTL Class URI Priority Weight Target [Policy]

If we make the field optional people can still use existing configurations
(there are plenty of DNS RRs with optional fields)

So we might have something like:

_omnibroker._wk.example.com URI 1 1 https://w1.example.com "max=2"
_omnibroker._wk.example.com URI 1 1 https://w1.example.com "min=2.0"
_omnibroker._wk.example.com URI 1 1 coap://w1.example.com "ipsec"

So now when we are searching for an implementation of omnibroker, the
client can make a choice that is informed by the version of the protocol it
supports and we have a peg from which we can hang other security schemes
than TLS.

---

From: Cyrus Daboo <[email-masked]>
Date: Fri, 18 Jan 2013 14:06:40 -0500

Hi Phillip,

--On January 18, 2013 at 11:22:20 AM -0500 Phillip Hallam-Baker 
<[email-masked]> wrote:

> retrieve SRV records and any site that attempts to advertise a service
> that is not backwards compatible is going to break.
>
>

Actually the goal we have for CalDAV service discovery is to define a 
generic client account configuration service to cover all aspects of a 
typical device setup, something akin to 
draft-daboo-aggregated-service-discovery - several of us are working on 
putting together a BOF to move that forward. Now that document does 
currently define SRV->.well-known too - but that could shift to URI if 
there were consensus around that.

-- 
Cyrus Daboo

---

From: Barry Leiba <[email-masked]>
Date: Fri, 18 Jan 2013 14:11:05 -0500

> IETF just pick one way to do something, it is a process failure if we end up
> with delays of 90 days let alone three years.

To be clear: the draft has been stuck in the RFC Editor queue because
of a normative-reference mess in what the IESG has come to think of as
"that cursed Cluster 83."  See
http://www.rfc-editor.org/cluster_info.php?cid=C83

Cyrus's document has a normative reference to
draft-cheshire-dnsext-dns-sd, and that document has a normative
reference to draft-cheshire-dnsext-multicastdns.  Both of those
documents have been stuck for a long time.

I agree that this is a process failure.

Barry

---

From: Phillip Hallam-Baker <[email-masked]>
Date: Fri, 18 Jan 2013 14:32:04 -0500

On Fri, Jan 18, 2013 at 2:06 PM, Cyrus Daboo <[email-masked]> wrote:

> Hi Phillip,
>
>
> --On January 18, 2013 at 11:22:20 AM -0500 Phillip Hallam-Baker <
> [email-masked]> wrote:
>
> typical device setup, something akin to draft-daboo-aggregated-**service-discovery
> - several of us are working on putting together a BOF to move that forward.
> Now that document does currently define SRV->.well-known too - but that
> could shift to URI if there were consensus around that.

Actually, I have now moved on to wanting the policy info in the same record
so its a little more than URI. So now its WK instead of URI.

The way I would resolve this is to specify that the caldav: URI scheme is
resolved using SRV with an optional fallback to WK and the wk:caldav is
only resolved using WK alone.

This allows the wk:caldav version to be used to force use of the new
discovery scheme that provides for expressing specific endpoints and policy
etc.

The way I would manage the transition from where we are now to where we
want to be is by using a protocol like the one I propose in Omnibroker that
provides a high level interface to the DNS for service announcement.

So when a caldev server came online it would announce itself to the
omnibroker service which would know that this is a special legacy case and
so it has to mint an SRV record set as well as the WK record set that
includes additional information.

Legacy protocols using well known ports that we might want to add policy
information could be managed in the exact same way only the service
announcement service (which could be omnibroker or something else) would
then be configuring an A or AAAA record instead of the SRV record.

This would also configure things like the firewall/NAT box so that the
appropriate ports were opened up.

The basic idea here is that discovery is currently a mess that lacks
consistency. So to get to a consistent place we first of all define one
discovery mechanism that can support the whole range of needs, then we put
all the complexity and special case handling into the interface to that
mechanism.