Subject: [conex] Genart LC review: draft-ietf-conex-abstract-mech-12

From: Robert Sparks <[email-masked]>
Date: Tue, 05 Aug 2014 14:58:50 -0500

I am the assigned Gen-ART reviewer for this draft. For background on
Gen-ART, please see the FAQ at

<http://wiki.tools.ietf.org/area/gen/trac/wiki/GenArtfaq>.

Please resolve these comments along with any other Last Call comments
you may receive.

Document: draft-ietf-conex-abstract-mech-12
Reviewer: Robert Sparks
Review Date: 5-Aug-2014
IETF LC End Date: 8-Aug-2014
IESG Telechat date: Not on an upcoming telechat agenda

Summary: Ready for publication as Informational

This document handles a complex description problem in a very accessible 
way.
Thank you for the effort that has gone into creating it.

One minor point to double-check:

This document goes out of its way to push decisions about measuring in 
packets,
bytes, or other units to the concrete  encoding proposals. RFC6789 was 
explicit
about conex exposing a metric of congestion-volume measured in bytes.

RFC6789 was published a couple of years ago - has that part of it become 
stale?
If so, it would be good for this document to explicitly call that out.

If not, (most of section 4.6 goes back to -04 which predates RFC6789),
does this document need to retain the this flexibility in its description?

---

From: Matt Mathis <[email-masked]>
Date: Wed, 6 Aug 2014 17:02:29 -0700

You are correct, the section is a bit stale.  And although the authors of
6789 would like to claim the topic is closed, newer documents
(e.g. draft-ietf-tcpm-accecn-reqs-07.txt,) have found it necessary to hedge
on this very issue for pragmatic reasons.  (Note the overlapping authors
between -abstract-mech-, 6789 and -accecn-).

The core advice in section 4.6 still stands:
>
> "This document does not take a strong position on this issue.    However,
> a ConEx encoding will need to explicitly specify whether it assumes units
> of bytes or packets consistently for both congestion indications and ConEx
> markings.  (see network layer requirement E in Section 3.3)."

Some of the surrounding editorializing reflects not completely resolved
tension between the authors on this point.  I for one would prefer to
remove the presumption that 6789 and 7141 are the final answer, and make
this draft purely bytes/packets agnostic.  I partially ceded the point on
the grounds that the impracticality 6789 would doom it over the long haul,
as we have already seen in -accecn-.

It would be bad form for this document to explicitly conflict with 6789,
but I for one would object to it unequivocally endorsing 6789, and although
leaving it waffle, isn't pretty, it does accurately reflect the views of
the authors.

Thanks,
--MM--
The best way to predict the future is to create it.  - Alan Kay

Privacy matters!  We know from recent events that people are using our
services to speak in defiance of unjust governments.   We treat privacy and
security as matters of life and death, because for some users, they are.

---

From: Bob Briscoe <[email-masked]>
Date: Thu, 7 Aug 2014 09:51:08 +0100

Matt,

I've sent suggested text to you for agreement before sending to Robert.

The issue with draft-kuehlewind-tcpm-accurate-ecn 
is different. We took the (interim) decision to 
use packets not bytes for feedback on the basis 
that the sender could then approximately 
reconstruct bytes if it needed to (because the 
sender can remember the sizes of the packets it 
sent). I say 'interim', because once we do the 
implementation, we will see how possible this is. 
I'm confident we can have a crack at it Linux. 
However, I wouldn't even want to attempt it in 
FreeBSD, which doesn't keep a record of packets 
in flight, so it would require a complete redesign.

You may be basing your view on the accecn 
presentation I gave, which was brief on this 
point. In the accecn draft it says:

"   If a particular sender behaviour needed to associate AccECN's
    feedback of each ECN marking with the size of the original packet
    that picked up the marking, there is enough information in AccECN
    feedback to do so, although perhaps imperfectly.  ...

                                                  ...to apply AccECN to
    these more challenging tasks, the Data Sender would probably have to
    record the sizes and/or timings of packets in flight and combine
    AccECN feedback with the cumulative acknowledgement numbers on each
    ACK as well as selective ACK (SACK) information [RFC2018].
"

Bob

At 01:02 07/08/2014, Matt Mathis wrote:
>You are correct, the section is a bit stale. Â 
>And although the authors of 6789 would like to 
>claim the topic is closed, newer documents 
>(e.g.Â draft-ietf-tcpm-accecn-reqs-07.txt,) have 
>found it necessary to hedge on this very issue 
>for pragmatic reasons. Â (Note the overlapping 
>authors between -abstract-mech-, 6789 andÂ -accecn-).
>
>The core advice in section 4.6 still stands:Â
>"This document does not take a strong position 
>on this issue. Â  Â However, a ConEx encoding 
>will need to explicitly specify whether it 
>assumes units of bytes or packets consistently 
>for both congestion indications and ConEx 
>markings. Â (see network layer requirement E inÂ Section 3.3)."
>
>Â
>Some of the surrounding editorializing reflects 
>not completely resolved tension between the 
>authors on this point. Â I for one would prefer 
>to remove the presumption thatÂ 6789 and 7141 
>are the final answer, and make this draft purely 
>bytes/packets agnostic. Â I partially ceded the 
>point on the grounds that the impracticality 
>6789 would doom it over the long haul, as we have already seen inÂ -accecn-.
>
>It would be bad form for this document to 
>explicitly conflict with 6789, but I for one 
>would object to it unequivocally endorsing 6789, 
>and although leaving it waffle, isn't pretty, it 
>does accurately reflect the views of the authors.
>
>Thanks,
>--MM--
>The best way to predict the future is to create it. Â - Alan Kay
>
>Privacy matters! Â We know from recent events 
>that people are using our services to speak in 
>defiance of unjust governments. Â  We treat 
>privacy and security as matters of life and 
>death, because for some users, they are.
>
>
>On Tue, Aug 5, 2014 at 12:58 PM, Robert Sparks 
><<mailto:[email-masked]>[email-masked]> wrote:
><<http://wiki.tools.ietf.org/area/gen/trac/wiki/GenArtfaq>http://wiki.tools.ietf.org/area/gen/trac/wiki/GenArtfaq>.
>
>This document handles a complex description problem in a very accessible way.
>This document goes out of its way to push 
>decisions about measuring in packets,
>bytes, or other units to the concrete Â encoding 
>proposals. RFC6789 was explicit
>RFC6789 was published a couple of years ago - 
>has that part of it become stale?

________________________________________________________________
Bob Briscoe,                                                  BT

---

From: John Leslie <[email-masked]>
Date: Thu, 7 Aug 2014 08:01:10 -0400

> I've sent suggested text to you for agreement before sending to Robert.

   That sentence unwittingly summarizes the story of ConEx: Bob wants
the authors to do the work, not the WG. :^(

   This is very sad. Bob is an extremely intelligent fellow with lots
of good ideas; but he's very selective in who he works with. ConEx is
a story of Bob abandoning co-authors when they questioned him too much.

   I, for example, was purged when I pointed that accurately counting
packets had to be better than under-counting bytes, and that there were
significant areas of the Internet which are not simply byte-congestible.

   Apparently Bob is getting ready to purge Matt Mathis. :^( :^(

> sent)...

   Note the "we".

   In fact, ECN feedback _has_ to reflect packets. All the noise about
what senders "could" do is just rationalizing. :^(

>> You are correct, the section is a bit stale.  
>> (e.g.? draft-ietf-tcpm-accecn-reqs-07.txt,) have 
>> for pragmatic reasons. ? (Note the overlapping 
>> authors between -abstract-mech-, 6789 and? -accecn-).

   I respect Matt: thus when he asked me to stop posting about bytes
vs. packets, I did...

>> The core advice in section 4.6 still stands:?
>> on this issue. ?  ? However, a ConEx encoding 
>> markings. ? (see network layer requirement E in? Section 3.3)."

   Technically, it is possible for some future transport to use ECN
and count bytes instead of packets: I remain unconvinced that this
could ever _accurately_ count bytes-on-the-wire (which is what
byte-congestible _means_).

>> authors on this point. ? I for one would prefer 
>> to remove the presumption that? 6789 and 7141 
>> bytes/packets agnostic. ? I partially ceded the 
>> 6789 would doom it over the long haul, as we have already seen in? 
>> -accecn-.

   I'm afraid it's not only 6789 that's doomed. ConEx is very likely
to be shut down entirely. Right now we're being told to get on schedule;
but in essence we _can't_ stay on schedule: thus the question is whether
we'll get this draft to the IESG before we're shut down, not _whether_
we'll be shut down. :^(

>> does accurately reflect the views of the authors.

   I would still like ConEx to produce a practical system for making
congestion visible within the routing cloud. I don't see any way for
that to happen. Thus, I simply support Matt on this question.

--
John Leslie <[email-masked]>

---

From: Bob Briscoe <[email-masked]>
Date: Thu, 7 Aug 2014 15:33:36 +0100

John,

Please write your arguments down about byte-pkt in an Internet Draft. 
Then if you have a coherent argument, we will all be able to 
understand it. I cannot understand what your arguments are in all the 
emails you have sent on this subject; too much seems to be implied 
rather than stated.

More inline...

>significant areas of the Internet which are not simply byte-congestible.

In case anyone on the list interprets John's use of the passive ("I 
was purged") as implying I somehow removed him, I didn't. The 
decision was presented to me by the chairs just as it was presented 
to John. Although I don't fully know the reasons, I doubt very much 
that it was anything to do something as specific as byte-pkt arguments.

>    Apparently Bob is getting ready to purge Matt Mathis. :^( :^(

Why would I be talking offlist with Matt if it was my intention to purge him?
Everyone works off-list with their co-authors. I primarily talk 
off-list to keep the signal to noise ratio high for postings on list.

> > sender can remember the sizes of the packets it

We = the co-authors (myself, Richard Scheffenegger and Mirja Kuehlewind).

We (obviously) had a huge amount of off-list discussion before 
posting the lastest accecn draft. We put our rationale in the draft, 
and now we've published it for review and comment by the whole WG. 
Nothing unusual about any of that.

>what senders "could" do is just rationalizing. :^(

Not so, on both counts.
1) In accecn (so far), we could have reflected bytes counts (as TCP 
does with seq nos & ack nos). We chose to feedback marked packets 
because it should be sufficient to be interpreted either as bytes or 
packets - on the same basis that an ECN marking at the IP layer can 
be interpreted as either the bytes in the marked packets, or the 
number of marked packets. This also kept header overhead low enough 
to fit within existing fields by overloading - an attempt to minimise 
the chance of being tampered with by middleboxes.
2) I deferred to Mirja & Richard for determining what senders could 
do, given their respective expertise in Linux & FreeBSD implementations.

>byte-congestible _means_).

What's the problem? If an ECN mark is interpreted in bytes, you just 
add up the sizes of the marked packets, rather than counting just the 
number of marked packets. Or is your concern about which headers were 
on the packet when it was marked, that may have been stripped before 
they reach the transport?

If that is your only concern, then we will have made progress. This 
is something I have dismissed as an approximation error, particularly 
given congestion counts are universally normalised into proportions 
(marked vs total bytes, or marked vs total packets), and in general 
all the packets will have had the same headers stripped.

>we'll get this draft to the IESG before we're shut down, not _whether_

Constructive comment?

>that to happen. Thus, I simply support Matt on this question.

So do I.

Bob

---

From: John Leslie <[email-masked]>
Date: Thu, 7 Aug 2014 14:10:09 -0400

> Please write your arguments down about byte-pkt in an Internet Draft. 

   I suppose I'm willing to do that; but I'd much prefer not to make
an Internet-Draft simply to document an argument between Bob and me.

> emails you have sent on this subject; too much seems to be implied 

   Let's try a summary:

   Beyond question, it is possible for a forwarding node to count the
packets it marks or drops.

   While it is possible for a forwarding node to count bytes-on-the-
wire for packets it marks or drops, that count may not agree with such
a count at another forwarding node; and certainly will not agree with
a count of payload bytes at one end or the other.

   It is the bytes-on-the-wire which count toward congestion; and in
fact there is always _some_ per-packet overhead (though this can be
rather small near the backbone -- it is _not_ small in DOCSIS cable
modems).

   Tunnels are found in the forwarding cloud -- bytes-on-the-wire
for these is greater than a non-tunnel forwarder will see.

   Thus, I claim it rather likely that any way endpoints might count
bytes is in danger of under-counting the actual congestion.

   I have also claimed that so long as we're dealing in 1500-byte
packets, the difference between "small" packets and "full" packets
(in bytes-on-the-wire) is not great enough to justify the effort
we've been putting into trying to count bytes of congestion.

   And in practice, most transports send similar-length packets
most of the time...

> was purged") as implying I somehow removed him, I didn't.

   Passive voice _implies_ only that the writer didn't want to blame
a particular person. I trust our readers to know this.

> The decision was presented to me by the chairs just as it was presented 

   That wasn't implied either...

   But perhaps we _do_ need to review that decision. :^(

   Usually I'd sleep on it before saying things like this; but I do
think there's something wrong with removing two principal editors
without the principal author knowing the reasons. Do either of our
WGCs want to comment on _that_?

>> Apparently Bob is getting ready to purge Matt Mathis. :^( :^(

   Appearances _can_ be misleading, I admit...

> Why would I be talking offlist with Matt if it was my intention to purge 

   True, that is not Bob's normal style.

> off-list to keep the signal to noise ratio high for postings on list.

   This isn't always best.

   The list has become near-dormant, IMHO, because folks couldn't follow
the reasoning behind announced changes to the text. (I have some reason
to believe even the _authors_ sometimes couldn't follow these.)

> Nothing unusual about any of that.

   I haven't yen decided whether it's worth-while commenting on accecn.
If I do, I'll mention that the complexity of Bob's "urgent pointer"
proposal is a potentially serious drawback...

> does with seq nos & ack nos).

   Indeed, with a much longer TCP option, Bob could have proposed
counters for payload bytes. Need I remind anyone that payload bytes
is always less than bytes-on-the-wire?

> We chose to feedback marked packets because it should be sufficient
> to be interpreted either as bytes or packets - on the same basis that
> an ECN marking at the IP layer can be interpreted as either the bytes
> in the marked packets, or the number of marked packets. This also kept
> header overhead low enough to fit within existing fields by overloading
> - an attempt to minimise the chance of being tampered with by
> middleboxes.

   Frankly, this still sounds like rationalizing to me, but YMMV.

> do, given their respective expertise in Linux & FreeBSD

   Thank you.

> number of marked packets.

   Alas, Bob has always been a bit vague about what exactly he counts
when he counts bytes.

   Neither the sender nor the receiver can know what the overhead is
in terms of bytes-on-the-wire. In some cases they can't even agree
on the size of the packet as sent or received.

   I have usually guessed that Bob wishes to count payload bytes --
on which the sender and receiver _can_ agree. This, of course, would
seriously under-count bytes-on-the-wire for small packets.

   If Bob instead wants to count total-bytes-in-the-sent-packet, it
would help if he defined _how_ he wants to calculate that. (Any way
he might choose, though, will under-count bytes in some cases or
over-count bytes in most cases.)

> Or is your concern about which headers were on the packet when it
> was marked, that may have been stripped before they reach the
> transport?

   That, I suppose, is a small part of my concern...

> all the packets will have had the same headers stripped.

   If I were happy to dismiss approximation errors, I would have
pointed out that in practice approximating bytes by counting exactly
one byte per packet (or choose any other constant number) doesn't
work that badly in practice.

   Approximation errors aren't always small; and they often tend
to go in the same direction, which becomes significant over time...

>> we'll get this draft to the IESG before we're shut down, not _whether_

   I'd be happy to hear suggestions on how to restate that as such...

>> that to happen. Thus, I simply support Matt on this question.

   Perhaps there _is_ hope -- but I'm not fully convinced. :^(

---

From: Bob Briscoe <[email-masked]>
Date: Fri, 8 Aug 2014 14:50:25 +0100

John,

So your argument does seem to be about not being able to allow for 
the bytes in headers on the wire (which the endpoints cannot guess). 
But you seem to have reserved the right to have other arguments. 
That's why it would be good to see all your arguments in a draft.

The conex docs have defined their solution to this problem precisely. 
Specifically across these two docs:

<http://tools.ietf.org/html/draft-ietf-conex-abstract-mech-12#section-3.3>
    An experimental ConEx specification SHOULD describe the following
    protocol details:
       F.  If the units are bytes a definition of which headers are
           included in the size of the packet;

<http://tools.ietf.org/html/draft-ietf-conex-destopt-06#section-4>
    the number of bytes carried by
    this IP packet (including the IP header) SHOULD be accounted for
    determining congestion or credit information.

Bear in mind I have as much desire for ConEx to work as you. What 
would be the point of me definining something I didn't think would 
work? Given these definitions, here are my arguments for why the 
missing headers problem can be considered a non-problem:

1) The quantity of congestion-bytes is primarily important at policy 
devices and audit.
   1a) Audit:
       Always solely compares packets with others in the same flow,
       so any additional headers will be the same for both sides of 
the comparison.
   1b) Policing:
       If policers work in units based on a universally measureable 
definition (as above),
       it is hardly important that the actual amount of congestion 
generated was from a
       link where there were more tunnel headers or a different 
link-layer - because the
       two never have to be directly compared or related.

2) The number of additional headers that a network path adds outside 
the outermost e2e IP header during transit is not something that 
end-systems have control over. So they should not be held responsible 
for them anyway (and by argument 1 there is no reason to have to). If 
a network operator adds so many headers that it congests its links, 
well frankly it should be allowed to go bust.

Bob