Subject: [weirds] URI templates and WEIRDS

From: "Murray S. Kucherawy" <[email-masked]>
Date: Sun, 5 Aug 2012 19:40:00 -0700

Colleagues,

Speaking as a participant:

As noted in the minutes, I presented in Vancouver on the notion of
using URI templates.  The consensus in the room was, as I recall, that
this is "an elegant solution to a problem we don't have."  I'm not
trying to force this on the working group, but  for the sake of more
detail, I'd like to cover this briefly in a weirds thread.  The same
debate is already occurring or will occur elsewhere, so I think it
would be helpful to have it on the record (we've only dealt with it
verbally so far) so that others can refer to it as well.  For example,
the same issue is still open in the REPUTE working group, but that
group is fairly quiet lately so I think it's more useful to discuss it
here.

In summary, I proposed the use of URI templates (RFC6570) via a
well-known URI at the weirds server as a means of instructing a weirds
client how to form a query URI.  The advantages to this are:

1) If, in the end, name and number queries differ in important ways, a
client wouldn't need to know which one it was about to address a
priori; rather, the server tells it how it wants the query URI to be
constructed.

2) If at some point a server undergoes a change in architecture and
wants the query to be formatted differently, it only needs to alter
the template rather than being forced to accommodate a standard.  The
client then adjusts immediately and automatically.

3) Consensus among web architecture experts (one of whom is Cc'd here
and hopefully will comment) is that reserving part of the URI
namespace in a standard is bad practice.  As I heard it stated,
"servers own URI syntax, not standards."

4) Existing implementations don't need to change, other than to
declare a template for the syntax they're already using.  We would
simply need to standardize the template tokens and what values they
should have.

As I recall the resistance points:

a) This is unneeded complexity.  Further, pushing complexities of a
protocol into clients is a bad idea.

b) We already have mod_rewrite (or equivalent), why not use it?

c) Clients already need to know something about the server they're
going to hit, so URI syntax could be included in that knowledge.

Please say so if there are other points I've missed on either list.

I can say that I implemented a first REPUTE system using URI templates
and the implementation wasn't terribly difficult.  The C library I
used is now open source, and numerous other implementations also exist
in a variety of languages.  Much of that code could easily be recycled
into weirds clients, removing much of the implementation complexity.

Thanks for indulging me on this.  I think this is an important thing
to resolve openly so that other groups can benefit from the
discussion.

-MSK

---

From: Byron Ellacott <[email-masked]>
Date: Mon, 6 Aug 2012 17:56:43 +1000

Hi Murray-as-participant,

> Please say so if there are other points I've missed on either list.

d) It's twice the number of HTTP requests for a simple lookup.

e) Clients might just bypass it anyway; in the numbers space, whatever URI structure ARIN uses has a high likelihood of being hard coded into many clients so they can skip the never changing step of fetching a URI template to do exactly the same transform every time.  This might happen for .com's URI structure, particularly if there's a reference server that a number of common TLDs all operate.

f) They complicate redirection - if we use an HTTP redirection code, a client would fetch the URI templates list, create the URI for the resource they want, issue a query, and get back (after redirection) another URI templates list.  Or the redirecting server is forced to act as a proxy, introducing a whole new class of error conditions and apparent quality of service based on a third party operator.

I don't think I raised these at the mic, though.

> into weirds clients, removing much of the implementation complexity.

It's not hard, by any stretch of the imagination, but it is double the work for the client:

--- with URI templates, you start here ---
1. Fetch the URI template resource
2. Parse the URI template resource
3. Select the appropriate template, and parse that
--- without URI templates, you start here ---
4. Fetch the data resource
5. Parse the data resource
6. Do something useful with the data resource

While the parsing of the template is presumably to be done by a library, this is still a lot of boilerplate to stitch container parsing, template selection, and library call together, given how trivial the rest of the client would be.

I spoke at the microphone about URI templates solving a problem we don't really have.  By that, I mean we don't have a mapping problem, or a (client-server) configuration problem, and we don't have such a complex discovery problem that we need this solution for it.  I find the arguments in favour of URI templates to be non-compelling:

1) If names and numbers diverge, a client is going to need to know how to handle two different responses.  You cannot gloss over differences with a template for the query string.

2) A server's architectural change is bound to the same number of moving parts in its URI space as before, to accommodate the standard's description of what variables might be found in a template.  What changes do you think could meaningfully be made to the noise around "example.com" that wouldn't just be deck chair rearrangements?  Isn't best current web software architecture practice to loosely couple URLs to service modules?

3) All other behaviour once inside the protocol URI space is dictated by the standard.  I could say "servers own response data formats, not standards" but it would be equally meaningless.  The protocol owns the conversation between clients and servers, and both how to ask and how to respond are part of the protocol.

4) There are no existing implementations of this as yet unspecified protocol.  Existing port 80 rest-like registration data directory services do not return results in a compliant format, and may not provide query entry points for the same aggregate roots; the URL structure is the least of the impacts on similar existing services.

I am not, I admit, a web architecture expert, but to my layperson's eyes the listed benefits to servers seem nebulous and more about technical purity than technical benefit, while the costs to clients are real, and relatively large.

  Byron

---

From: Andy Newton <[email-masked]>
Date: Mon, 6 Aug 2012 10:02:38 +0000

I think Byron covered a lot of points and I agree with them all.
I just want to add one more counter to one of your assertions.

> priori; rather, the server tells it how it wants the query URI to be

Today, generic Whois clients do make a distinction between IP addresses, AS numbers, and domain names. I'd see no reason why they would not do this in the future.

Specific built clients would likely not be as full-blown and just as likely to hardcode URI variables as they would use a good-enough redirect server. Knowing what I know about queries that are thrown at our server, a lot of people wouldn't do anything much more complicated that wget URL | jsawk.

-andy

---

From: SM <[email-masked]>
Date: Mon, 06 Aug 2012 03:02:50 -0700

>d) It's twice the number of HTTP requests for a simple lookup.

I don't see any compelling reason to double the number of HTTP requests.

>e) Clients might just bypass it anyway; in the numbers space, 
>whatever URI structure ARIN uses has a high likelihood of being hard 
>coded into many clients so they can skip the never changing step of 
>fetching a URI template to do exactly the same transform every 
>time.  This might happen for .com's URI structure, particularly if 
>there's a reference server that a number of common TLDs all operate.

Yes.

Regards,
-sm

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Mon, 6 Aug 2012 11:01:43 -0700

> I don't see any compelling reason to double the number of HTTP requests.

There's an HTTP header field that tells the client how long to
remember a result.  This applies to the template as well, so only the
first query to a given server has the double round-trip.

-MSK

---

From: SM <[email-masked]>
Date: Mon, 06 Aug 2012 13:28:01 -0700

Hi Murray,

Yes.

I thought of the caching aspect when I posted the comment.  I would 
not implement that in the client as it is more code.  To be fair, the 
double round-trip is not as bad as it sounds as we talking about 
Whois instead of content where you cannot afford the expense of that 
double round-trip.

Regards,
-sm

---

From: "John Levine" <[email-masked]>
Date: 7 Aug 2012 01:40:22 -0000

I took another look at URI templates, and they're still an elegant
(albeit underspecified) solution to a problem that has no relation to
what WEIRDS is doing.

The sets of questions that one can ask a WHOIS server are really quite
small and well understood, even though there is a haze of nits
surrounding them.  The questions today are the same today as they were
in the 1990s.  Once we define the URL syntax for those questions, we
are done.  As I hardly need remind you, the fewer options a standard
gives implementers, the better interoperation you get, and I really do
not want more than one way to ask who owns a domain, or what is the
abuse contact for an IP address.

For your specific points:

Constructing the URL is about 2% of what a WEIRDS client does.  It
needs to know what the question is (which is more than the datatypes
of the arguments, e.g., queries for registered domains and for name
servers both take a domain name argument), figure out what credentials
are relevant and how to pass them, deal with errors and redirects,
parse the result, and pick out whatever parts the client application
needs.  Other than dealing with the errors and redirects, and maybe
parsing some sub-chunks of the result, name and number queries have
nothing in common.

Not our problem.  As has often been noted, if you want mod_rewrite,
you know where to find it.  We all expect the number of clients to
vastly exceed the number of servers, so we want to make the clients
simpler.

In our context, that's just wrong.  We are not trying to lay a common
interface on top of an existing set of web services, we are defining a
web front end to a (perhaps) existing set of non-web databases.

  4) Existing implementations ...

Don't exist.  The prototypes are certainly useful, but they are
not anything we need to work around.

Security considerations: Templates give you a way to introduce
arbitrary strings of arbitrary size into client applications.  When
looking at RFC 6570 and the test suites, I see lots of tests to see
whether reasonable templates with reasonable data expand correctly,
but no stress tests to see what happens with unreasonable templates
and data created by unreasonable people (like, say, me).  By my
reading of 6570, a 100K character variable name is perfectly legal.
What will your C code do with megabyte templates with 100K variable
names?  Do we really want to introduce a new source of buffer overflow
exploits?

R's,
John