Subject: Strawman: PTTH

From: "Thomson, Martin" <[email-masked]>
Date: Tue, 24 Mar 2009 04:06:52 -0500

Folks,

I've been interested in solving the HTTP problem discussed today for quite some time.  I've reviewed all the proposals reviewed.  Two things strike me:

 - BOSH and Bayeux both address the real problem.  Namely: how this works in the world that exists right now.  They do this with long polling.  This is a good solution that we know works.  rHTTP and Websockets are new protocols that might work, but I have my reservations.

 - BOSH and Bayeux seem to be way over-engineered.  Channels and sessions and all that stuff isn't HTTP - HTTP is, when you get right down to it, pretty dumb.  That's a virtue that is lost.

I am truly surprised that no-one has suggested this particular idea, so I'll do a brief description and hopefully I can then sleep.

~~~

===PTTH; or HTTP, over easy (on this note, I'd like to find a nasty backronym for PTTH, but Protocol That Tweaks/Twists/Turns HTTP kinda sucks) 

A frighteningly simple idea: long polling + back-to-front HTTP requests inside other HTTP requests.

==Terminology:

Client and server can be misleading in this context.  RFC 3080 deals with this problem already - differentiating between connection establishment and request/response exchanges.  I'll borrow those terms.

Initiating Peer - the "classical" HTTP client, the peer that establishes the TCP connection.
Listening Peer - the "classical" HTTP server, the peer that listens for TCP connections.
Client Peer - the peer that is making PTTH requests (HTTP server)
Serving Peer - the peer that is serving PTTH requests (HTTP client)

==Overview of operation

Application semantics establish the need for an HTTP peer to establish a PTTH "session".  A part of this includes a URI that the application feeds to the initiating peer with the expectation that a PTTH session is established.  The initiating peer opens a connection to the listening peer and sends an HTTP POST request:

  POST /ptth/path HTTP/1.1
  Host: server.name.example
  Content-Length: 0

Long polling ensues.  On timeout the listening peer sends an HTTP response that doesn't contain an HTTP request, that response is discarded by the initiating peer and a new, empty request is sent.  

If the client peer needs to send something it sends an HTTP response that contains an HTTP request:

  HTTP/1.1 200 OK
  Content-Type: application/http
  Content-Length: ...

  GET /index.html HTTP/1.1
  Host: 

The serving peer includes the response in the next request it sends.  This also serves as the means by which the next request might be acquired:

  POST /ptth/path HTTP/1.1
  Host: server.name.example
  Content-Type: application/http
  Content-Length: ...

  HTTP/1.1 200 OK
  Content-Type: application/xhtml+xml

  ...content...

==Protocol pieces

A new MIME type: application/http or some such to distinguish message bodies.

Possibly: an HTTP header that indicates that an HTTP request is an initiating message. However, I consider it likely that this is not actually necessary.  The expectation is set by the application, and the MIME-type is a further indication that this behaviour is desired.

==Limitations and things to look into further

Pipelining is more or less impossible with this approach.  Since a response from the serving peer is expected on the next HTTP request received by the client peer, the serving/initiating peer cannot pipeline requests.  This might need to be looked into if the problem is considered as being worth solving.  This is the only major concern I've run into.

Tuning the timeout is something that could be looked at, but this is a problem that applies in general to all long-polling approaches.

I haven't done a thorough check of HTTP headers, but I don't actually think that there is any need for anything more complicated than this.  Listening peers might like to use Cache-Control: private in responses to prevent caching, and so forth, but I don't actually think that there is much more required here.  Some headers are pointless on the nested request, such as proxy authentication headers or transfer-encoding, but - again - those are secondary issues.  I need to go through Lisa's mini using-http checklist (BCP56bis) - there are probably other clues there as to what else needs to be considered.

==Implementation

This is easy to implement.  The listening/client peer is very easy.  This is especially true with things like Jetty continuations where you can sit waiting almost indefinitely.

The initiating/serving peer can be implemented by ramming an HTTP client together with an HTTP server.  The HTTP client makes the long polls.  If a response comes back with an application/http body, it extracts that response and sends that as a request to the HTTP server.  The response from the HTTP server is sent to the listening/client peer in the next request.

~~~

I can't help but think that this is something like what Mark Nottingham was hinting at with his comments at the mic today.
	
No doubt there are drawbacks, because this is almost too simple.  Of course, it inherits a great deal of the characteristics of HTTP, both good and bad.  I'm not sure whether the pipelining thing is a showstopper, or whether there are other things that I've missed. 

I thought it worth sharing, if only to enrich the discussion.  

From deep in the twilight zone,
Martin

------------------------------------------------------------------------------------------------
This message is for the designated recipient only and may
contain privileged, proprietary, or otherwise private information.  
If you have received it in error, please notify the sender
immediately and delete the original.  Any unauthorized use of
this email is prohibited.
------------------------------------------------------------------------------------------------
[mf2]

---

From: "Thomson, Martin" <[email-masked]>
Date: Tue, 24 Mar 2009 04:40:49 -0500

...still not sleeping...

On consideration: ignore what I said about pipelining; I worked it out.  The listening/client peer can just ignore requests that don't contain the application/http MIME type.  Ordering then remains the sole means of request/response correlation.  No headers needed.

On connection close:  the listening/client peer needs a way of unambiguously telling the initiating/serving peer that it's finished.  This might be an application-layer thing, but as a suggestion, Connection: close would probably work well enough.  The client should probably keep trying until it sees one of these because it can't be sure that a connection drop isn't for other reasons (e.g. a proxy timeout).

Insomniacally,
Martin

> -----Original Message-----
> From: [email-masked] [mailto:apps-discuss-
> [email-masked]] On Behalf Of Thomson, Martin
> Sent: Tuesday, 24 March 2009 2:07 AM
> To: [email-masked]
> Subject: Strawman: PTTH
> 
> Folks,
> 
> I've been interested in solving the HTTP problem discussed today for
> quite some time.  I've reviewed all the proposals reviewed.  Two things
> strike me:
> 
>  - BOSH and Bayeux both address the real problem.  Namely: how this
> works in the world that exists right now.  They do this with long
> polling.  This is a good solution that we know works.  rHTTP and
> Websockets are new protocols that might work, but I have my
> reservations.
> 
>  - BOSH and Bayeux seem to be way over-engineered.  Channels and
> sessions and all that stuff isn't HTTP - HTTP is, when you get right
> down to it, pretty dumb.  That's a virtue that is lost.
> 
> I am truly surprised that no-one has suggested this particular idea, so
> I'll do a brief description and hopefully I can then sleep.
> 
> ~~~
> 
> ===PTTH; or HTTP, over easy (on this note, I'd like to find a nasty
> backronym for PTTH, but Protocol That Tweaks/Twists/Turns HTTP kinda
> sucks)
> 
> Client and server can be misleading in this context.  RFC 3080 deals
> with this problem already - differentiating between connection
> establishment and request/response exchanges.  I'll borrow those terms.
> 
> Application semantics establish the need for an HTTP peer to establish
> a PTTH "session".  A part of this includes a URI that the application
> feeds to the initiating peer with the expectation that a PTTH session
> is established.  The initiating peer opens a connection to the
> listening peer and sends an HTTP POST request:
> 
>   POST /ptth/path HTTP/1.1
> Long polling ensues.  On timeout the listening peer sends an HTTP
> response that doesn't contain an HTTP request, that response is
> discarded by the initiating peer and a new, empty request is sent.
> 
> The serving peer includes the response in the next request it sends.
> This also serves as the means by which the next request might be
> acquired:
> 
>   POST /ptth/path HTTP/1.1
> Possibly: an HTTP header that indicates that an HTTP request is an
> initiating message. However, I consider it likely that this is not
> actually necessary.  The expectation is set by the application, and the
> MIME-type is a further indication that this behaviour is desired.
> 
> Pipelining is more or less impossible with this approach.  Since a
> response from the serving peer is expected on the next HTTP request
> received by the client peer, the serving/initiating peer cannot
> pipeline requests.  This might need to be looked into if the problem is
> considered as being worth solving.  This is the only major concern I've
> run into.
> 
> Tuning the timeout is something that could be looked at, but this is a
> problem that applies in general to all long-polling approaches.
> 
> I haven't done a thorough check of HTTP headers, but I don't actually
> think that there is any need for anything more complicated than this.
> Listening peers might like to use Cache-Control: private in responses
> to prevent caching, and so forth, but I don't actually think that there
> is much more required here.  Some headers are pointless on the nested
> request, such as proxy authentication headers or transfer-encoding, but
> - again - those are secondary issues.  I need to go through Lisa's mini
> using-http checklist (BCP56bis) - there are probably other clues there
> as to what else needs to be considered.
> 
> ==Implementation
> 
> This is easy to implement.  The listening/client peer is very easy.
> This is especially true with things like Jetty continuations where you
> can sit waiting almost indefinitely.
> 
> The initiating/serving peer can be implemented by ramming an HTTP
> client together with an HTTP server.  The HTTP client makes the long
> polls.  If a response comes back with an application/http body, it
> extracts that response and sends that as a request to the HTTP server.
> The response from the HTTP server is sent to the listening/client peer
> in the next request.
> 
> ~~~
> 
> I can't help but think that this is something like what Mark Nottingham
> was hinting at with his comments at the mic today.
> 
> No doubt there are drawbacks, because this is almost too simple.  Of
> course, it inherits a great deal of the characteristics of HTTP, both
> good and bad.  I'm not sure whether the pipelining thing is a
> showstopper, or whether there are other things that I've missed.
> 
> _______________________________________________
> Apps-Discuss mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/apps-discuss

---

From: Eric Burger <[email-masked]>
Date: Tue, 24 Mar 2009 02:43:55 -0700

Again, this feels like (1) a research project and (2) a transport (NOT  
applications) problem.  Yes, we are speaking about HTTP here. No, we  
are not talking about the application running on HTTP, but the  
transport of data.  That is a transport, not applications, issue.

On Mar 24, 2009, at 2:06 AM, Thomson, Martin wrote:

> quite some time.  I've reviewed all the proposals reviewed.  Two  
> things strike me:
>
> I am truly surprised that no-one has suggested this particular idea,  
> so I'll do a brief description and hopefully I can then sleep.
>
> ~~~
>
> Application semantics establish the need for an HTTP peer to  
> establish a PTTH "session".  A part of this includes a URI that the  
> application feeds to the initiating peer with the expectation that a  
> PTTH session is established.  The initiating peer opens a connection  
> to the listening peer and sends an HTTP POST request:
>
>  POST /ptth/path HTTP/1.1
> The serving peer includes the response in the next request it  
> sends.  This also serves as the means by which the next request  
> might be acquired:
>
>  POST /ptth/path HTTP/1.1
> actually necessary.  The expectation is set by the application, and  
> the MIME-type is a further indication that this behaviour is desired.
>
> pipeline requests.  This might need to be looked into if the problem  
> is considered as being worth solving.  This is the only major  
> concern I've run into.
>
> Tuning the timeout is something that could be looked at, but this is  
> a problem that applies in general to all long-polling approaches.
>
> I haven't done a thorough check of HTTP headers, but I don't  
> actually think that there is any need for anything more complicated  
> than this.  Listening peers might like to use Cache-Control: private  
> in responses to prevent caching, and so forth, but I don't actually  
> think that there is much more required here.  Some headers are  
> pointless on the nested request, such as proxy authentication  
> headers or transfer-encoding, but - again - those are secondary  
> issues.  I need to go through Lisa's mini using-http checklist  
> (BCP56bis) - there are probably other clues there as to what else  
> needs to be considered.
>
> ==Implementation
>
> This is especially true with things like Jetty continuations where  
> you can sit waiting almost indefinitely.
>
> extracts that response and sends that as a request to the HTTP  
> server.  The response from the HTTP server is sent to the listening/ 
> client peer in the next request.
>
> ~~~
>
> I can't help but think that this is something like what Mark  
> Nottingham was hinting at with his comments at the mic today.
> 	
> course, it inherits a great deal of the characteristics of HTTP,  
> both good and bad.  I'm not sure whether the pipelining thing is a

---

From: Tony Garnock-Jones <[email-masked]>
Date: Tue, 24 Mar 2009 13:37:48 +0000

> I am truly surprised that no-one has suggested this particular idea, so I'll do a brief description and hopefully I can then sleep.

Check out http://www.reversehttp.net/reverse-http-spec.html.

Abstract:

   "This document describes a dynamic, ReST-style means of enrolment and
   participation in an HTTP network. The message/http and
   application/http MIME types defined by RFC 2616 are used to build a
   dynamically-configurable "Remote CGI" service.

   "Joining the World Wide Web as an HTTP server has been an ad-hoc,
   manual process. By using the protocol defined here, programs can
   provide services to the Web just as easily as they request services
   from the Web."

It's almost exactly what you describe.

An implementation is available: http://www.reversehttp.net/download.html
There are online demos of web-server-in-a-browser using it:
http://www.reversehttp.net/demos/

Other demos of AMQP-style relaying and queueing are not yet polished
enough for release, but are lurking the git repo for those curious enough.

Regards,
  Tony
-- 
 [][][] Tony Garnock-Jones     | Mob: +44 (0)7905 974 211
   [][] LShift Ltd             | Tel: +44 (0)20 7729 7060
 []  [] http://www.lshift.net/ | Email: [email-masked]

---

From: Tony Garnock-Jones <[email-masked]>
Date: Tue, 24 Mar 2009 13:52:52 +0000

Eric Burger wrote:
> applications) problem.  Yes, we are speaking about HTTP here. No, we are
> not talking about the application running on HTTP, but the transport of
> data.  That is a transport, not applications, issue.

You have a point. I think there are two important issues being addressed
here, one a transport issue and one an applications issue.

The issues, as I see them:

 (1) - Given that a program wants to provide HTTP services to the world,
       how does it do so? (TRANSPORT)

 (2) - Once a program can both issue *and respond to* HTTP requests,
       without undue effort, what kinds of new applications arise?
       (APPLICATIONS)

Reverse HTTP in all its forms -- the Upgrade-header based one and the
JSON encoding of HTTP that Lentczner and Preston have been working with,
as well as the message/http-based one that Martin Thompson proposed just
before and that I've been working with for a while -- solves a very
low-level issue: how to get HTTP requests out from, and HTTP responses
in to, a Point Of Attachment for an *HTTP* network.

The spec I linked to in my previous message also addresses a closely
related issue: *enrolment* in an HTTP network; that is, how the rHTTP
service is assigned a portion of URL-space that it is responsible for.
One might think of this aspect of it as providing a win equivalent to
that given by DHCP when compared to static configuration of IP
addresses. Using the enrolment protocol I describe allows a form of
address autoconfiguration for HTTP networks that to my knowledge doesn't
exist anywhere else yet. The alternative, the status quo, is equivalent
to static IP address assignment: the manual DNS and firewall
configuration, the manual installation of apache or equivalent.

So, given all that: is this the right forum for discussion of rHTTP and
friends?

Regards,
  Tony

---

From: Salvatore Loreto <[email-masked]>
Date: Tue, 24 Mar 2009 17:13:36 +0200

I would tend to agree that  the  websocket  proposal is more a transport 
issue that an application one:
it try to solve the problem of how to open a socket in a Web Browser or 
generally in a Web environment using HTTP,
but the other one are all about how to transport  data, in both the 
direction, using  HTTP
that means using HTTP as a transport

Sal

---

From: Lisa Dusseault <[email-masked]>
Date: Tue, 24 Mar 2009 11:17:25 -0700

2009/3/24 Eric Burger <[email-masked]>:
> applications) problem.  Yes, we are speaking about HTTP here. No, we are not
> talking about the application running on HTTP, but the transport of data.
>  That is a transport, not applications, issue.

1.  It really is not a research project.  At least three different
systems are deployed and working and have been for a while.

2.  It really belongs in APPS, not because of an
increasingly-inaccurate layering model for areas, but because APPS
area collects HTTP experts.

Lisa

---

From: Eric Burger <[email-masked]>
Date: Tue, 24 Mar 2009 13:56:55 -0700

Compromise?  If we do do something, can we have a TSV expert assigned  
to the group, just as we have Security Experts assigned to a group?

---

From: Salvatore Loreto <[email-masked]>
Date: Tue, 24 Mar 2009 22:59:06 +0200

good idea/compromise !

/Sal

Eric Burger wrote:
>>> are not
> ------------------------------------------------------------------------
>

---

From: Lisa Dusseault <[email-masked]>
Date: Tue, 24 Mar 2009 15:48:23 -0700

You bet!
Lisa

On Tue, Mar 24, 2009 at 1:56 PM, Eric Burger <[email-masked]> wrote:
> Compromise?  If we do do something, can we have a TSV expert assigned to the
> group, just as we have Security Experts assigned to a group?
>
>>> not