Subject: Re: [httpwg/http-extensions] Algorithm to parse lists doesn't allow parameterised values (#443)

From: HTTP issue updates <[email-masked]>
Date: Thu, 01 Feb 2018 23:12:15 +0000 (UTC)

Flipping to design, because this has some design implications.

One of my goals for processing is that a generic processor should be able to take any input and produce the appropriate structure (this isn't true at the very top level, which probably deserves a separate issue).

So, we need a way to distinguish parameterised labels from "ordinary" labels for parsers.

The most obvious way to do this would be to use a sigil, and the most obvious candidate is ":" -- e.g.,

    PLabelExample: ;foo; bar=baz; bat

That would make lists of plabels look like:

    PLabelListExample: ;foo;bar, baz;bat=1

I'm not crazy about the leading `;`, but I don't see any way to avoid having something there. Alternative approaches, or alternative syntax anyone?

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/httpwg/http-extensions/issues/443#issuecomment-362433302

---

From: HTTP issue updates <[email-masked]>
Date: Sun, 04 Feb 2018 03:21:13 +0000 (UTC)

If you mark the entire list (so either: all its members are plabels, or all its members are items) that doesn't match the current design, which allows a mix of item / parameterised within the one list.  This is a reduction in capability, but not sure if YAGNI.

Alternatively we could make the indication on a per-item basis; either as an extension to the list syntax (effectively marking up the separator), or modifying the types (so label and plabel are always unambiguously distinct.)  Which they are according the the ABNF, though not easily distinguished by the algorithm.

So another alternative is: don't provide algorithms. Or at least, not algorithms that are drastically different from the ABNF (branching on prefixes vs whole pattern matching.)  This is unambiguous, if ugly as sin, and stuff we've been dealing with for decades:

```
structured_header = dictionary / list / parameterised / item

dictionary        = dictionary_member *1023( OWS "," OWS dictionary_member )
dictionary_member = label "=" item

list              = list_member 0*1023( OWS "," OWS list_member )
list_member       = item / parameterised

parameterised     = label *256( OWS ";" OWS label [ "=" item ] )

item              = integer / float / string / label / binary

integer           = ["-"] 1*19DIGIT

float             = ["-"] (
                      ; not copy-pasting the whole thing
                    )
string            = DQUOTE 0*1024(char) DQUOTE
char              = unescaped / escape ( DQUOTE / "\" )
unescaped         = %x20-21 / %x23-5B / %x5D-7E
escape            = "\"

label             = lcalpha *255( lcalpha / DIGIT / "_" / "-"/ "*" / "/" )
lcalpha           = %x61-7A ; a-z

binary            = "*" 0*21846(base64) "*"
base64            = ALPHA / DIGIT / "+" / "/"
```

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/httpwg/http-extensions/issues/443#issuecomment-362877310

---

From: HTTP issue updates <[email-masked]>
Date: Sun, 04 Feb 2018 22:48:52 +0000 (UTC)

Ah, sorry, my intent was to mark each plabel, but I messed up the list example.

I'm not excited about dropping the algorithms; we've been dealing with the mess for a long time, yes, but it *is* a mess. But we'll see; for the time being, having the algorithms helps us clarify our intent really well, I think.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/httpwg/http-extensions/issues/443#issuecomment-362947075

---

From: HTTP issue updates <[email-masked]>
Date: Mon, 05 Feb 2018 01:23:33 +0000 (UTC)

`PLabelListExample: ;foo;bar, ;baz;bat=1` ?

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/httpwg/http-extensions/issues/443#issuecomment-362958034

---

From: HTTP issue updates <[email-masked]>
Date: Mon, 05 Feb 2018 06:02:19 +0000 (UTC)

Yes.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/httpwg/http-extensions/issues/443#issuecomment-362989640

---

From: HTTP issue updates <[email-masked]>
Date: Sun, 04 Feb 2018 22:57:38 -0800

If we replace 2.1 with a branch ("if input_string starts with lcalpha then parse as plist, otherwise parse as item") bare labels will fall out as plabels like `[foo, {}]`. Would that hurt anything? 

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/httpwg/http-extensions/issues/443#issuecomment-362997206

---

From: HTTP issue updates <[email-masked]>
Date: Fri, 16 Feb 2018 03:10:35 +0000 (UTC)

Bare labels are also used as the keys in dictionaries and parameters, so that seems... awkward. Also, it'd force everyone using labels to say "take the first member of the tuple...".

Another solution would be to serialise the label of PLABELs as uppercase on the wire.

E.g., a mixed list could look like:

> MixedListExample: 1, "foo", bar, BAZ;bat

.. containing an integer, string, label and plabel with one empty parameter.

Does that seem too hacky? I kind of like the visual emphasis of the "main" label, but discriminating data structures based on case... hm.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/httpwg/http-extensions/issues/443#issuecomment-366133211

---

From: HTTP issue updates <[email-masked]>
Date: Fri, 16 Feb 2018 04:08:09 +0000 (UTC)

> Bare labels are also used as the keys in dictionaries and parameters, so that seems... awkward.

This is only the algorithm for lists, and lists are the only structures where a label or a plabel could exist in the same place. (?)

> Also, it'd force everyone using labels to say "take the first member of the tuple...".

Everyone using *lists of* labels.  But then, don't they already have to do some extra work in this space, since `list_member = item / parameterised`?

> Does that seem too hacky?

```
Foo-Bar: TEXT/HTML; charset=utf-8
```
:worried:

```
List-Of-Foo: TEXT/HTML; charset=utf-8, TEXT/PLAIN; charset=utf-8
Reference-To-Foo: text/html
```
:grimacing:

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/httpwg/http-extensions/issues/443#issuecomment-366139294

---

From: HTTP issue updates <[email-masked]>
Date: Fri, 16 Feb 2018 16:40:01 +0000 (UTC)

In addition to @phluid61's great points, the draft says a goal is to be useful for "revisions of existing header field specifications". Changing the spelling of parameterised labels would seem to give up on that.

Re "take the first member of the tuple", I suspect the right wording for specifications is to talk about the label and its parameter map, not the tuple that the current parsing algorithm currently returns. Web specs would likely use ["structs"](https://infra.spec.whatwg.org/#structs) for this.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/httpwg/http-extensions/issues/443#issuecomment-366288433

---

From: HTTP issue updates <[email-masked]>
Date: Sat, 17 Feb 2018 13:00:03 -0800

I think we should retain the algorithms, because if *we* cannot write concise readable code for parsing this stuff, we have already failed before we reached the door.

With respect to label and plabel I think the close naming messes with our brains: "label" is an identifier, "plabel" is more like a function call with parameters.  Maybe "label" should be renamed to "ident" ?

The fact that the parser cannot tell them apart until it sees the ';' or something else, is not different from telling integer/float apart until we spot the '.'

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/httpwg/http-extensions/issues/443#issuecomment-366470999

---

From: HTTP issue updates <[email-masked]>
Date: Sun, 18 Feb 2018 22:04:49 +0000 (UTC)

It is different, though, because a float must contain a `.` and an integer must not; a plabel (whatever it's called) can have zero parameters, in which case it's indistinguishable from an item/label.  No matter how you rationalise it, the algorithm will have to choose whether to return a scalar or a tuple as long as it could be one or the other.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/httpwg/http-extensions/issues/443#issuecomment-366552094

---

From: HTTP issue updates <[email-masked]>
Date: Sun, 18 Feb 2018 14:35:07 -0800

@jyasskin I don't think that changing the serialisation gives up on existing headers, if the existing header is case-insensitive. And, at any rate, that's an aspirational goal -- there are many existing headers where that won't be possible already (because of subtle error handling that can't be changed).

I think the heart of the issue is that all other non-top-level structures can be unambiguously identified, but because both label and plabel (or whatever we call it) can't, it's ambiguous.

I think that means that we need to either a) make them unambiguous, or b) disallow plablels (or whatever we call them) in lists.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/httpwg/http-extensions/issues/443#issuecomment-366554317

---

From: HTTP issue updates <[email-masked]>
Date: Sun, 18 Feb 2018 22:43:16 +0000 (UTC)

> or b) disallow plablels (or whatever we call them) in lists.

Or b(ii): split list into list-of-items and list-of-parameterised.

If top-level structures already have side-channel information about their type, it doesn't matter that the two lists are potentially indistinguishable; and it means we can still recreate the `Accept` header with q-values[*].  To quote myself up-thread, this is a reduction in capability, but not sure if YAGNI.

[*]: except for `*/*`

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/httpwg/http-extensions/issues/443#issuecomment-366554892

---

From: HTTP issue updates <[email-masked]>
Date: Sun, 18 Feb 2018 22:44:31 +0000 (UTC)

Good point, @phluid61. That seems viable; still not sure what I prefer :)

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/httpwg/http-extensions/issues/443#issuecomment-366554993

---

From: HTTP issue updates <[email-masked]>
Date: Mon, 19 Feb 2018 19:43:59 -0800

I feel I should point out, the two _list_ examples in the current text are for a "list of ~~labels~~ identifiers" and a "list of parameterised labels".  While theoretical future use-case loving me appreciates supporting mixed-type arrays, I wonder how far we need to take it.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/httpwg/http-extensions/issues/443#issuecomment-366859098

---

From: HTTP issue updates <[email-masked]>
Date: Mon, 19 Feb 2018 20:42:40 -0800

What do other folks think about @phluid61's @b(ii)? @bsdphk?

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/httpwg/http-extensions/issues/443#issuecomment-366865439

---

From: HTTP issue updates <[email-masked]>
Date: Thu, 22 Feb 2018 00:18:33 -0800

b(ii) WFM

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/httpwg/http-extensions/issues/443#issuecomment-367600917

---

From: HTTP issue updates <[email-masked]>
Date: Thu, 01 Mar 2018 02:43:26 +0000 (UTC)

@bsdphk @phluid61 have a quick look at the commit above? Would like to publish a new draft soon.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/httpwg/http-extensions/issues/443#issuecomment-369456080

---

From: HTTP issue updates <[email-masked]>
Date: Wed, 28 Feb 2018 18:43:57 -0800

(btw as I did this, I came to really like this approach)

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/httpwg/http-extensions/issues/443#issuecomment-369456166

---

From: HTTP issue updates <[email-masked]>
Date: Wed, 28 Feb 2018 21:14:00 -0800

Looks good to me, once you fix the number problems @jyasskin pointed out.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/httpwg/http-extensions/issues/443#issuecomment-369476189