Subject: rel="shortlink" proposal for advertising short URLs in HTML/HTTP

From: Sam Johnston <[email-masked]>
Date: Tue, 14 Apr 2009 20:53:49 +0200

Evening all,

I think it's time to call the grown-ups into a discussion started by a
handful of web developers trying to kill off URL shorteners and
associated linkrot, opaque URLs, etc. Perhaps it's just accelerating
change but the decidedly ordinary rev="canonical" proposal has already
taken a life of its own, springing up a blog[1], a "30 minutes or
less" PoC[2], a busy twitter hashtag, even a slashdot article[3]
(submitted by the "inventor" himself no less) - but more worryingly a
handful of high profile implementations at sites like Ars Technica
(even if there's no clients yet that will read them).

The concept is simple: rather than forcing users to rely on third
parties like bit.ly, tinyurl.com etc. for short links the publishers
themselves can suggest link(s) in the HTTP headers and/or HTML code.
The resulting URL (e.g. http://example.com/promo) will generally live
as long as the content does and won't vanish when the redirector
disappears (as some invariably will - there's at least a 3 figure
count of them now and probably a dozen new ones every day). It's also
a good deal more useful to users as it can show the source (domain)
and subject (path), and when the link is exposed via HTTP HEAD the
performance is at least as good as third party shorteners.

A bunch of alternatives to rev="canonical" have been proposed
including rel="short|shorter|shortcut|short_url|short_url|alternate|self|...",
but for various reasons[4] I don't think any of them are suitable. I
think rel="shortlink" would work nicely and is impossible to confuse
with anything else (I got here via rel="short" and rel="shortcut").
I've roughed up a specification[5] so you can see the technical
details (copied below).

I'm not necessarily all that fussed about this - the concept looks
good but I was basically just driving by on the weekend and saw an
accident about to happen (e.g. people confusing rel and rev and
knocking sites like Ars out of the search engines). Seems it caught
mnot's eye too[6] and I should point out that I had every intention[7]
of bringing this to your attention after having extracted consensus in
the group[8]. I think link relations come relatively cheap and unless
there's something I've missed helping to put another nail in the
coffin of the URL shorteners is arguably a good thing.

Thoughts?

Sam

1. http://revcanonical.wordpress.com/
2. http://revcanonical.appspot.com/
3. http://developers.slashdot.org/article.pl?sid=09/04/12/1834205
4. http://code.google.com/p/shortlink/wiki/Alternatives
5. http://code.google.com/p/shortlink/wiki/Specification
6. http://www.mnot.net/blog/2009/04/14/rev_canonical_bad
7. http://samj.net/2009/04/introducing-relshort-better-alternative.html
8. http://groups.google.com/group/shortlink

shortlink Specification

Technical specification for the HTML/HTTP "shortlink" relation

Introduction

The shortlink relation allows webmasters to specify a short link to
use for the resource, thereby avoiding having to obtain one from a
potentially unreliable third party URL shortening service such as
tinyurl.com.

Such links are useful for space-constrained applications (e.g.
microblogging including Twitter and mobile Internet) as well as any
time URLs need to be manually entered (e.g. when they are printed or
spoken).

Note: Until such time as shortlink is officially standardised
http://purl.org/net/shortlink should be used for standards compliance.

Details

The shortlink appears in two places:

within the HEAD section of the HTML document:

<link rel="shortlink" href="http://example.com/promo">

in the Link: HTTP header:

Link: <http://example.com/promo>; rel=shortlink

Implementation

Servers

Servers should implement both HTML and HTTP links for efficiency and
performance reasons.

The shortlink should default to an automatically generated stable URI
based on an existing unique identifier (e.g. http://example.com/123).
Such identifiers may be compressed using base32 or similar (e.g.
http://example.com/3r). URIs should be case-insensitive and avoid
symbols that look or sound similar (e.g. 1 vs l), particularly when
manual entry will be required (e.g. printed, spoken).

Publishers should also be given the option to specify a human-friendly
slug (e.g. http://example.com/promo), as users should be able to
derive information about the resource (path) and its source (domain)
from the URL.

Where a shortlink is changed the previous URL should not be broken as
it may have been stored by users. Typically this requires maintaining
a register of mappings.

Clients

Clients that have already retrieved the document (e.g. web browsers,
news readers) should parse it to discover the link rel="shortlink"
element(s) and extract the href attribute from each.

Clients that have the URL but not the document (e.g. microblogging
software) should conduct a HTTP HEAD request and extract any Link:
headers from the response. Clients should not retrieve and parse the
document unless the user specifically requests it.

In the event that there are multiple shortlinks then the client may
choose one itself or offer the user the choice (e.g. in a drop-down
list). If the client chooses one it may do so randomly, by order
(first vs last) or by some quality of the URL (length, readability,
etc.).

---

From: Keith Moore <[email-masked]>
Date: Tue, 14 Apr 2009 19:52:59 -0400

I think it's interesting that PURLs started out with the notion that
URLs via a 3rd party referrer could be more persistent than URLs from
the original domain, and now we're proposing that an original domain's
URLs might be more persistent than those through a 3rd party service. 
Both are correct, of course, depending on circumstances.

In general I think that the fewer human meaningful components to an
identifier, the more persistent you can make the binding between that
identifier and whatever it originally referred to.  Having the
components of the identifier be meaningless reduces the temptation to
change the binding associated with an identifier (from that originally
established) to something that is more "current".  (There are lots of
versions of this.  If you use file and directory names in URLs, at some
point you inevitably feel the need to reorganize the directory tree -
which tends to invalidate URLs based on the old tree.)

What's not immediately clear to me is why you'd use anything other than
the stable identifier in any links within documents.  I can understand
wanting to keep a set of "human friendly" identifiers that are easy for
users to remember, that might change over time, but I have a more
difficult time understanding why you'd want to use those in links.

So anyway, I think that having a "stable" or "persistent" link in the
HTTP and/or document header is a good idea.  (Note: putting this in the
HTTP header is more general: don't restrict this to just HTML!  And
putting it in the document header is somewhat risky as the link might
need to change (or not) when the document is updated - existing tools
certainly won't do the right thing in all cases.) 

Ideally, software would recognize these headers and use the stable URL
in preference to other URLs when appropriate. So that when you create a
bookmark to a web page, the bookmark points to the stable link by
default. Or when you are editing a document and you create a link to
another document, you should get the stable link by default.  I think it
should be possible to override the defaults, but it ought to be clear to
the user/editor that he's not using the stable URL.

I don't think this mechanism can work well without some sort of content
management system (it can be a very primitive one) that automatically
creates unique, persistent URLs for things.  So it's important for web
servers to _not_ generate these headers by default, but rather, to do so
only when explicitly configured (presumably by a content management system).

Of course, this is trying to solve the same problems for which URNs were
invented, and I still think that URNs are a good approach.  I'd like to
see any mechanism for advertising a persistent identifier be compatible
with URNs.  But it doesn't bother me that people are still interested in
using URLs for this purpose.

Keith

Sam Johnston wrote:
> _______________________________________________
> Apps-Discuss mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/apps-discuss

---

From: Sam Johnston <[email-masked]>
Date: Wed, 15 Apr 2009 14:08:12 +0200

Keith,

Oh my, it seems we have yet another type of link relation to cater for:
immutable unique identifiers ala atom:id :)

> Both are correct, of course, depending on circumstances.

Persistency is not the key requirement here - shortness is (and for some
applications, human-friendliness). Canonical URLs (which are generally long
and stuffed with SEO juice) are very much subject to change. A human
friendly shortlink like http://nike.com/just-do-it is less likely to change
while shortlinks based on immutable identifiers like
http://nike.com/123should never change (though the resource they point
at may).

> which tends to invalidate URLs based on the old tree.)

Agreed, the more information in the URL the less persistent it will be. URLs
committed to dead tree versions (e.g. references in an academic paper) are
by very definition immutable so these should be able to be updated to point
at the most recent/best version of the intended resource.

> difficult time understanding why you'd want to use those in links.

Say I'm microblogging or texting about some campaign, for example
Greenpeace's Save the Whales. Both me and my users are going to much prefer
seeing http://greenpeace.com/whales than
http://tinyurl.com/aZq4b<http://example.com/>- and I'm far more likely
to remember the former than the latter (even if
longer).

> certainly won't do the right thing in all cases.)

Agreed, except that I'm not sure the unique ID need be resolveable to the
resource - just that it allow differentiating between resources. I actually
think specifying rel=canonical (that's rel, not rev) solves many of the real
world problems we see today. If people want to break their own links then
that's their problem.

> the user/editor that he's not using the stable URL.

Again this sounds like a job for the canonical URL - I'd much rather see
search engines and bookmarks having URLs like
http://greenpeace.com/blog/2009/04/greenpeace-helps-save-the-whales.htmlthan
http://greenpeace.com/123.

> only when explicitly configured (presumably by a content management

Sure, for persistent IDs.

> using URLs for this purpose.

I personally prefer UUIDv4 (random) based URNs for this, but I think it's a
completely separate need from that of shortlinks. Does that make sense?

Sam

Sam Johnston wrote:
> > including
> rel="short|shorter|shortcut|short_url|short_url|alternate|self|...",

---

From: Keith Moore <[email-masked]>
Date: Wed, 15 Apr 2009 08:37:18 -0400

Sam Johnston wrote:
> Keith,
>
> Oh my, it seems we have yet another type of link relation to cater
> for: immutable unique identifiers ala atom:id :)
>
> On Wed, Apr 15, 2009 at 1:52 AM, Keith Moore <[email-masked]
> <mailto:[email-masked]>> wrote:
>
> Persistency is not the key requirement here - shortness is (and for
> some applications, human-friendliness). Canonical URLs (which are
> generally long and stuffed with SEO juice) are very much subject to
> change. A human friendly shortlink like http://nike.com/just-do-it is
> less likely to change while shortlinks based on immutable identifiers
> like http://nike.com/123
>
> should never change (though the resource they point at may).
>
> Agreed, the more information in the URL the less persistent it will
> be. URLs committed to dead tree versions (e.g. references in an
> academic paper) are by very definition immutable so these should be
> able to be updated to point at the most recent/best version of the
> intended resource.
Not clear;  it depends on the nature of the reference.  Sometimes it's
important to point to the exact same version of the resource that the
referrer used. 

It can actually make sense to have multiple persistent names for a
resource, e.g. one for the version cited, another for the most recent
version.  The trick is keeping all of the different references straight,
using the right one in the right circumstance, and maintaining them all.
> Greenpeace's Save the Whales. Both me and my users are going to much
> prefer seeing http://greenpeace.com/whales than
> http://tinyurl.com/aZq4b <http://example.com/> - and I'm far more
> likely to remember the former than the latter (even if longer).
That strikes me as a UI issue.  The HREF link in an A element should
point to a URL that won't break.   Perhaps, on visiting that page, the
user should see a more friendly URL is one is available.  But that can
also be misleading in subtle ways as the user isn't seeing the link that
got him to the page he is reading. 
> Agreed, except that I'm not sure the unique ID need be resolveable to
> the resource - just that it allow differentiating between resources.
To me it seems that if the ID isn't resolvable to the resource, you
really haven't solved the problem of links breaking - you've just moved it.
> I actually think specifying rel=canonical (that's rel, not rev) solves
> many of the real world problems we see today. If people want to break
> their own links then that's their problem.
>  
>
> Again this sounds like a job for the canonical URL - I'd much rather
> see search engines and bookmarks having URLs like
> http://greenpeace.com/blog/2009/04/greenpeace-helps-save-the-whales.html
> than http://greenpeace.com/123.
I think I see your point - but it's not immediately clear to me which
form is more likely to be persistent.
> I personally prefer UUIDv4 (random) based URNs for this, but I think
> it's a completely separate need from that of shortlinks. Does that
> make sense?
I understand that there's a need for shortlinks independent of
persistence.  But I don't think the two issues are easily teased apart.

(one nice thing about non-random URNs is that there is some potential of
being able to resolve them...though that's rarely worked as well as we
hoped.)

Keith

---

From: Eric Burger <[email-masked]>
Date: Fri, 17 Apr 2009 07:02:02 +0800

Maybe I'm being an idiot, but why does the server need to know that  
this thing masquerading as a URL is really a URN in URL syntax?

Why does the client need to know this thing advertised as a URL is not  
really a URL?

To me, it is just an opaque string. The fact the server advertises the  
same resources with two opaque strings really is not my business, is it?

---

From: Sam Johnston <[email-masked]>
Date: Fri, 17 Apr 2009 08:09:53 +0200

> important to point to the exact same version of the resource that the

Right, pointing at a Wikipedia article would be a good example of where this
would be useful. Can we hijack the existing threading relations I wonder? I
can't see why not...

> using the right one in the right circumstance, and maintaining them all.

The most common use case (and the case for "shortlink" - though that is more
of an implementor concern) is to return the latest version.

> got him to the page he is reading.

The user should see the canonical URL when they visit the page (other cruft
like search parameters can be grey so it's there and editable but not
confusing). This is easy to implement when you tell the user agent what
rel="canonical" is.

Consider the use case of a text message or TV advertisement - you're going
to want to have a short link that tells you something about what you're
visiting but when you access that URL you're going to be more pleasantly
surprised than confused or annoyed when it gets expanded out to something
more meaningful.

<snip>
> being able to resolve them...though that's rarely worked as well as we

OTOH if it's not resolveable then it's going to be a lot less likely to
break...

Sam

---

From: Barry Leiba <[email-masked]>
Date: Wed, 22 Apr 2009 15:25:09 -0400

Hm.

The problem, as I see it, with defining "persistence" is that one
wants different things in different contexts.  Consider these three
cases of things I might write:

1 ---------------------------------------------------
Note that any comments you post here are subject to my host's
copyright policy: [link to current copyright policy at time of
clicking].
------------------------------------------------------

2 ---------------------------------------------------
Note that the comment you just made is subject to my host's copyright
policy: [link to version of copyright policy in effect at the time of
writing].
------------------------------------------------------

3 ---------------------------------------------------
Check out the stupid typo in my host's copyright policy:
[link to immutable snapshot of copyright policy at the time of writing].
------------------------------------------------------

In case 1, I want a pointer that will always get me to the current
policy at the time you click the link.

In case 2, I want a pointer that will always get me to the policy
that's in effect when I wrote the text, regardless of subsequent
policy changes.

In case 3, I want a pointer that will always get me to the version
with the typo, even if they go and fix the typo without issuing a new
policy version.

With URL shorteners, you have no idea what you'll get, even when the
link isn't broken.  There's no guarantee that the content the link
points to has any relation whatever to what was there when the short
URL was made (if, say, the URL was set up to point to the current
front page of the newspaper).

With "short URLs" created by the content provider, you have a chance
of at least knowing what you're getting.  But what is it that you want
to get, and how do you specify it.  There are at least the three use
cases above, and perhaps others.

Barry

---

From: Sam Johnston <[email-masked]>
Date: Wed, 22 Apr 2009 22:39:56 +0200

G'day Barry,

Is any of this specific to short links or does it apply across the
board? I would suggest the latter.

Sam on iPhone

---

From: Barry Leiba <[email-masked]>
Date: Wed, 22 Apr 2009 16:44:41 -0400

> board? I would suggest the latter.

It does, but the connection to short links is that if a page offers a
"short link" (or a client asks for one), there probably has to be some
way to communicate the scope.  Otherwise, the client or user won't
know what to rely on the short link for.

Barry

---

From: Nicolas Williams <[email-masked]>
Date: Wed, 22 Apr 2009 15:55:21 -0500

> know what to rely on the short link for.

Arguably the scope should be the same as for the long link.

Nico
--