Subject: [conex] Accounting of ConEx signals

From: Mirja Kühlewind <[email-masked]>
Date: Fri, 7 Oct 2011 01:15:27 +0200

Hi everybody,

Richard and I are currently working on the TCP modifications draft. And we get 
always stocked at the same question how the accounting of the ConEx signal 
should be done. I believe we assume that the ConEx signal is counted 
byte-wise that means all bytes of a marked packet are assumed to expose the 
link congestion. First of all, I have to say hat this is nowhere officially 
defined. That lead to the following couple of questions:

1. Where do we need to define this? In the IPv6 doc? 
Because it might be a more general question and it must be clear to everyone 
that if we account the signal byte-wise this has to be done in the endsystem 
as well as in every network device using this information.

2. Which bytes of a packet do we take? The IP length incl header, the IP 
payload incl. TCP header or the actual payload?
In TCP if a packet get lost (with SACK) we only know how many payload got 
lost. We do not know the number of packets/headers that were used to transmit 
this data. If we would loose one big packet and then retransmit a large 
number of small packets instead (which are all ConEx marked) that might give 
quite a different ConEx signal. On the other hand, what causes the congestion 
is the whole packet incl header(s). What is the right thing to do?

Input is very welcome!

Thx,
Mirja

---

From: John Leslie <[email-masked]>
Date: Thu, 6 Oct 2011 20:00:31 -0400

Mirja K?hlewind <[email-masked]> wrote:
> 
> Richard and I are currently working on the TCP modifications draft.
> And we get always stocked at the same question how the accounting
> of the ConEx signal should be done. I believe we assume that the
> ConEx signal is counted byte-wise that means all bytes of a marked
> packet are assumed to expose the link congestion.

   Marcello may well believe this needs an "official" answer; I am
not convinced it does.

   From an ISP standpoint, short packets don't necessarily contribute
less to congestion. And IMHO, the most important signal of ConEx is
that the sender is sending this packet despite a belief it may
contribute to congestion.

   Whether the initial upstream counts marked packets as per-byte,
per-packet, or somewhere in between doesn't seem to me to be helpful
for us to define at this stage.

> First of all, I have to say hat this is nowhere officially defined.
> That lead to the following couple of questions:
> 

   I don't believe we need to define it -- we're likely better to
define it _after_ some consensus appears.

> Because it might be a more general question and it must be clear
> to everyone that if we account the signal byte-wise this has to be
> done in the endsystem as well as in every network device using
> this information.

   That's not clear to me...

> 2. Which bytes of a packet do we take? The IP length incl header,
> the IP payload incl. TCP header or the actual payload?

   If we were to define this, we'd have to aim for bits-on-the-wire
plus some estimate of packet overhead.

   (IMHO, bits-on-the-wire isn't always clear to the sender.)

> In TCP if a packet get lost (with SACK) we only know how many
> payload got lost. We do not know the number of packets/headers
> that were used to transmit this data. If we would lose one big
> packet and then retransmit a large number of small packets instead
> (which are all ConEx marked) that might give quite a different
> ConEx signal.

   (That's not an unreasonable case.)

> On the other hand, what causes the congestion is the whole packet
> incl header(s). What is the right thing to do?

   I'd punt...

   For most purposes, I think just counting the marks will suffice;
but clearly we shouldn't prohibit experiments counting something else.

--
John Leslie <[email-masked]>

---

From: "Scheffenegger, Richard" <[email-masked]>
Date: Fri, 7 Oct 2011 11:44:48 +0100

Hi John,

I tend to agree with you - on average, there will be little difference
between counting packets, or counting bytes;

However, if we don't agree on what is the "correct" approach beforehand
and any box is free to account whatever, this may lead to exploits. On
another example, if my ISP charges for the congestion volume, it is not
irrelevant how this accounting is performed. And as soon as money is
involved, this is a high incentive to cheat...

IMHO, congestion volume (as defined) is measured in bytes, not packets;
and as Conex is a IP-layer mechanism, the natural unit would be full IP
packets (IP header + payload), which happens to be readily available by
examining the header (although not as quick as simply accounting
packets).

For the sender transmitting variable sized packets, this may lead to a
different number of conex marked packets (but more accurate
byte-congestion volume), or vice versa.

Richard Scheffenegger

> _______________________________________________
> conex mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/conex

---

From: John Leslie <[email-masked]>
Date: Fri, 7 Oct 2011 08:16:12 -0400

> between counting packets, or counting bytes;

   Thank you.

> involved, this is a high incentive to cheat...

   Perhaps... It's not clear money _will_ be involved...

   Recall the history of long-distance phone charges: there were several
orders of magnitude difference in charges between different entities.

> examining the header (although not as quick as simply accounting

   Readily available to middleboxes, that is. It won't always be readily
available to the sender (which I presume would ordinarily be the source
of any money changing hands) or receiver (the second most likely).

> byte-congestion volume), or vice versa.

   Indeed...

   But, as an ISP, I doubt I would choose to argue the charges to the
sender based on "congestion volume" as defined by some third party.

   Also, for different tunneling (a very real consideration in IPv6
at the present), the "congestion volume" may be different at different
middleboxes. :^(

   I just don't see the value in this WG "standardizing" how to charge.

---

From: Christopher Morrow <[email-masked]>
Date: Fri, 7 Oct 2011 10:45:53 -0400

>   I just don't see the value in this WG "standardizing" how to charge.

don't disagree, but I'd point out that 'congestion' (or rather packet
drops) happen not just because of full pipes, often packet RATES
matter as well. So contributing to high packet rates (even on not full
interface by volume) can cause loss.

We had a, memerable, customer who ran their interface very hot from a
pps perspective while only being ~half full, they were regularly
unhappy with loss rates on that interface.

-chris
small fast packets are often as painful as large slow ones

---

From: Mirja Kuehlewind <[email-masked]>
Date: Fri, 7 Oct 2011 17:03:00 +0200

Hi John,

I agree with you and Richard that usually it shouldn't make a big difference, 
but...

>    (That's not an unreasonable case.)
... we have to consider those kind on cases if there might be a disadvantage 
or an advantage or even the possible to cheat the policing system for some 
users.

Mirja

---

From: Bob Briscoe <[email-masked]>
Date: Fri, 07 Oct 2011 19:10:15 +0100

Mirja,

I agree we don't have to standardise how ISPs count 
congestion-volume. But your & Richard's question comes from a need to 
write a document defining the experimental ConEx behaviour of /TCP/, 
not an ISP meter.

Audit, not accounting, is the main reason a definitive standard 
answer is needed. We don't want to trigger the audit function into 
detecting an attack and dropping packets just because we haven't 
written interoperable documents. If ConEx signals balance congestion 
signals (drops or ECN marks) packet-for-packet, but the audit 
function checks for balance byte-for-byte (or vice versa), then the 
application could have a problem.

Altho Chris is right that you can get packet congestion, the digging 
behind the byte-pkt draft below found that byte-congestion is 
predominant on the Internet.
<http://tools.ietf.org/wg/tsvwg/draft-ietf-tsvwg-byte-pkt-congest/>
And the Internet is like this for good technical reasons. Designers 
of kit want the forwarding engine to still be in control when it gets 
congested, so machines are _generally_ designed so the processor can 
still cope even when the link is flat-lining with small packets.

Nonetheless, in order to push the limits on performance, some 
machines are benchmarked against a workload with a 'typical worst 
case' mix of small and large packets, rather than actual worst case 
(all small).

So I am afraid the answer is bytes: ConEx signals ought to balance 
congestion signals (drop or ECN) byte-for-byte.

This is unfortunately more difficult to implement starting from 
existing TCP code. But there's an easier answer. If you can get 
through an audit function by doing packet-for-packet signalling, when 
audit is checking byte-for-byte balance, then good luck to you. 
That's what we did in our implementation, and it seemed OK through an 
audit function checking bytes. However, we didn't yet go looking for 
scenarios that would lead to problems.

I suggest in the TCP doc we say bytes would be the ideal, but packets 
is good enough at the experimental stage. In the more general docs, I 
would ask that we say bytes, so if there are interoperability 
problems, we have a 'standard' to fall back on that is based on sound 
technical reasoning.

Finally, you are right to ask whether we should have more general 
guidance to be used when defining ConEx for other transports. We 
already have this in the two existing w-g docs (see quotes below).

<http://tools.ietf.org/html/draft-ietf-conex-concepts-uses-02>
2.1. Definitions
                 ...
    Congestion-volume:  For any granularity of traffic (packet, flow,
       aggregate, link, etc.), the volume of bytes dropped or marked in a
       given period of time.  Conceptually, data volume multiplied by the
       instantaneous congestion each packet of the volume experienced.
       Usually expressed in bytes (or MB, GB, of course).

<http://tools.ietf.org/wg/conex/draft-ietf-conex-abstract-mech-02>
3.3. Abstract Encoding
                 ...
    In both cases, the amount of congestion is signaled by the volume of
    marked data--just as the volume of lost data or ECN marked data
    signals the amount of congestion experienced.  Thus the size of each
    packet carrying a ConEx Signal is significant.
                 ...
4.4. Audit
                 ...
    To audit ConEx Signals against actual ECN markings or losses, the
    auditor could work as follows: monitor flows or aggregates of flows,
    only holding state on a flow if it first sends a ConEx-Marked packet
    (Credit or either Re-Echo marking).  Count the number of bytes marked
    with Credit or Re-Echo-ECN.  Separately count the number of bytes
    marked with ECN.  Use Credits to assure that {#ECN} <= {#Re-Echo-ECN}
    + {#Credit}, even though the Re-Echo-ECN markings are delayed by at
    least one RTT.

HTH

Bob

>or an advantage or even the possible to cheat the policing system for some

________________________________________________________________
Bob Briscoe,                                BT Innovate & Design

---

From: Christopher Morrow <[email-masked]>
Date: Fri, 7 Oct 2011 14:24:05 -0400

On Fri, Oct 7, 2011 at 2:10 PM, Bob Briscoe <[email-masked]> wrote:
> Altho Chris is right that you can get packet congestion, the digging behind
> the byte-pkt draft below found that byte-congestion is predominant on the
> Internet.
> And the Internet is like this for good technical reasons. Designers of kit
> want the forwarding engine to still be in control when it gets congested, so
> machines are _generally_ designed so the processor can still cope even when
> the link is flat-lining with small packets.

there is LOTS of kit that behaves very badly with line-rate min-len
packets... If part of the reasoning behind conex is 'cost to upgrade',
there are likely lots of places using this elderly kit which can't be
'cost effectively' upgraded.

Anyway, it seems the discussion is spiraling toward an answer and I
was only interested in noting that 'not always is the byte what
bites.'.

-chris

---

From: John Leslie <[email-masked]>
Date: Fri, 7 Oct 2011 18:57:03 -0400

Mirja Kuehlewind <[email-masked]> wrote:
> 
> I agree with you and Richard that usually it shouldn't make a big
> difference, but...
> 

   (But it is yet another case where the sender may be unaware, and
the receiver almost certainly will be unaware, of the packet overhead.)

> ... we have to consider those kind on cases if there might be a
> disadvantage or an advantage or even the possible to cheat the
> policing system for some users.

   I'd prefer we _didn't_ invite random middleboxes to police senders
they think are abusing the system. It's not necessary to the information
flow, and experience shows it becomes a game of whack-a-mole. :^(

---

From: John Leslie <[email-masked]>
Date: Fri, 7 Oct 2011 19:41:20 -0400

Bob Briscoe <[email-masked]> wrote:
> congestion-volume.

   Thank you.

> But your & Richard's question comes from a need to write a document
> defining the experimental ConEx behaviour of /TCP/, not an ISP meter.
> 
> written interoperable documents.

   Agreed: that would be bad... I'd agree that an audit system should
be fairly narrowly defined, and the points at which packet drop should
be encouraged should be limited. IMHO, packet drop only deserves to
be used where severe packet loss is encountered; and clearing
Congestion-Expected should be forbidden wherever the _actual_ source
of packets is obscure.

   The simple case -- likely the best for initial experiments -- is
to simply disallow policing (understanding that packets _will_ be
dropped anyway during DDoS).

> If ConEx signals balance congestion signals (drops or ECN marks)
> packet-for-packet, but the audit function checks for balance
> byte-for-byte (or vice versa), then the application could have a
> problem.

   Indeed!

> <http://tools.ietf.org/wg/tsvwg/draft-ietf-tsvwg-byte-pkt-congest/>

   (in which Bob recommends that "packet size should be taken into
account when transports read congestion indications", while
"packet size should not be taken into account when network equipment
creates congestion signals".)

> still cope even when the link is flat-lining with small packets.

   Yes... but...

   Not all designs survive into the products shipped. :^(

> case' mix of small and large packets, rather than actual worst case 

   Alas, yes.

   (And some particularly nasty things happen in shared-media systems
like cable-Internet and wireless.)

> congestion signals (drop or ECN) byte-for-byte.

   Strange! I read this data quite the other way: that the answer is
count-packets-only.

> scenarios that would lead to problems.

   (As is too often the case, I don't follow Bob's reasoning here.)

> is good enough at the experimental stage.

   I suggest rather saying that packets-only should be counted during
the initial experimental stage; but that byte-counting will be studied
later on.

> In the more general docs, I would ask that we say bytes, so if there
> are interoperability problems, we have a 'standard' to fall back on
> that is based on sound technical reasoning.

   "Sound technical reasoning" seems ill-advised here. Packets must
pass through a cloud of many different systems under differing
managements: essentially none of which will be ruled by "sound
technical reasoning". :^(

>       Usually expressed in bytes (or MB, GB, of course).

   Defining "congestion-volume" doesn't affect how ISPs may regulate
congestion-expected marking.

>    packet carrying a ConEx Signal is significant.

   "Amount of congestion" isn't ordinarily considered in TCP congestion
management.

>    + {#Credit}, even though the Re-Echo-ECN markings are delayed by at

   I don't particularly like this text -- though I understand why it
was drafted this way and don't wish to criticize the author.

   I suppose we'll have to discuss text like this sometime, but that
time doesn't feel like now...

---

From: Bob Briscoe <[email-masked]>
Date: Mon, 10 Oct 2011 17:14:38 +0100

John,

>Congestion-Expected should be forbidden wherever the _actual_ source

I work on the basis we can't forbid anyone to do anything, where 
policing is concerned. We can only give them good/useful ways of doing it.

>dropped anyway during DDoS).

Given the real delta ConEx offers is in the policing area, I imagine 
any experiments will be focused on policing, so it would be futile to 
disallow them.

>    Not all designs survive into the products shipped. :^(

You, me, Chris all agree here. That byte congestion is the more 
prevalent, but packet congestion is by no means irrelevant.

>like cable-Internet and wireless.)

Perhaps it would be best to say that the bar to fixing congestion is 
lower if it's due to packet processing than if it's bytes.

For instance, in some of our platforms we trigger alarms to tell us 
if packet-processor utilisation on an edge router hits 50% a few 
times. Whereas line utilisation on an edge node might be regularly 
much higher than that every day and we just consider that normal.

>    Strange! I read this data quite the other way: that the answer is

The context of draft-ietf-tsvwg-byte-pkt-congestion is creating 
congestion signals (packet-based) vs using congestion signals 
(byte-based). ConEx is all in the "using" space. ConEx doesn't touch 
the space where signals are created in the first place (AQM etc).

I'm not going to repeat the arguments in 
draft-ietf-tsvwg-byte-pkt-congestion on this list, because you need 
to sit down with a bag of frozen peas on your head and think hard 
about it. So it's best to read that draft if you want to do this. If 
you remember doing probability questions in college, you'll remember 
it's very easy to make basic mistakes.

>    (As is too often the case, I don't follow Bob's reasoning here.)

Bascially, I'm agreeing with what I think Chris said earlier (or was 
it Mirja?). Drops (or ECN) will be as likely to hit small as large 
packets (randomness). Altho ConEx signalling is meant to match drops 
byte-for-byte, if it's done packet-for-packet, because of the 
randomness, it will as often be high as low.

However, there are probably scenarios that aren't random where 
matching signals packet-for-packet will lead to a persistent 
imbalance in byte terms. If these exist (and if the imbalance leads 
to understatement rather than overstatement), we'll only find out 
during experiments, then we'll have to assess whether this is serious 
enough to mandate that TCP ConEx code has to match signals byte-for-byte.

>the initial experimental stage; but that byte-counting will be studied

Disagree. It would be wrong to say implementers SHOULD do the wrong 
thing. More correct to say they can probably get away with doing the 
wrong thing.

>technical reasoning". :^(

As above. Interoperability needs congestion-volume to be in terms of 
bytes (based on "sound technical reasoning", at least IMHO). So we 
need to say byte-for-byte is required for interop, then say that 
packet-by-packet balance will probably be sufficient to achieve 
sufficient byte-by-byte balance to be interoperable.

>congestion-expected marking.

It certainly doesn't affect what they might do. But it does say what 
they will need to do if they want to measure the economic cost of traffic.

I don't think we have to worry about operators counting congested 
packets, rather than bytes. Operators today measure the number of 
bytes you send to determine how much you have used their network. 
They don't often count how many packets you sent. You don't see usage 
limits in terms of numbers of packets.

>    "Amount of congestion" isn't ordinarily considered in TCP congestion

Indeed. And one wouldn't expect it to, because transport protocols 
generally don't do fairness or accounting (even tho for many years 
numerous researchers thought TCP did fairness, until it was pointed 
out that fairness depends on how much the user uses the transport, 
and how many transports the user creates).

See <http://www.bobbriscoe.net/projects/refb/#relax-fairness>

>time doesn't feel like now...

I often notice that you talk about sorting out text later, when we're 
doing ConEx right now, and we're already behind with milestones.

Bob

---

From: Bob Briscoe <[email-masked]>
Date: Mon, 10 Oct 2011 17:46:06 +0100

Mirja,

The thread largely overlooked your second question, which is also a good one.

We haven't got any text anywhere that talks about this.

IMO the size should include the network layer header, but not link layer.

Reason: ConEx is targeted at the IP layer because 
it is the interoperability layer across networks. 
When the bytes of a packet cause congestion 
(setting aside the discussion about packet 
processing congestion), the IP header is always 
there, so it is part of the size that always needs to be counted.

So, when we add ConEx to TCP, byte-for-byte 
balance should strictly take account of IP 
headers. But as you rightly point out, TCP often 
doesn't know how many IP headers are involved (or 
how many TCP headers were involved the last time 
round the loop). So again, we can only say that 
the practical approach when coding TCP is to make 
assumptions about IP and TCP headers (e.g. assume 
there will always be about the same amount of IP 
and TCP header size added to the same amount of TCP payload).

This should be good enough at this experimental 
stage. If it raises problems, we will have to deal with them.

Bob

PS. Strictly, congestion at the link layer 
includes the size of link layer headers, and 
congestion in a tunnel includes tunnel headers, 
etc, etc. But for simplicity, from the 
transport's viewpoint, I think it's reasonable to 
use the size of the packet including the one IP 
header that the transport can assume. This is a 
bit like the transport layer checksum calculation 
that includes an IP pseudo-header. But in this 
case, the transport just adds a pseudo-size for the IP and TCP headers.

---

From: John Leslie <[email-masked]>
Date: Mon, 10 Oct 2011 15:37:33 -0400

> policing is concerned. We can only give them good/useful ways of

   We most certainly _can_ forbid doing certain things _if_ you want
to claim compliance with the specification we write.

   Granted, I'm generally happier when we don't forbid very much...

> Given the real delta ConEx offers is in the policing area,

   (I don't accept that as "given".)

> I imagine any experiments will be focused on policing, so it would be
> futile to disallow them.

   "Forbid" may be the wrong word -- we might rather define them as
out-of-scope of the ConEx specification.

> much higher than that every day and we just consider that normal.

   I'm not sure the two are related -- per-packet overhead tends to
dominate designs in the backbone, while per-bit overhead (in the
transmission medium) tends to dominate near the edges.

> the space where signals are created in the first place (AQM etc).

   This, I think, is Bob's fundamental argument. I am not convinced
that "congestion signals" are necessarily "byte-based". The particular
case we're designing for in Active Queue Management, where IMHO the
packets are dropped or marked regardless of their size.

   Furthermore, I'd claim that AQM kicks in more based on average
size of all packets in the queue than the particular size of the
packet dropped or marked.

> about it.

   (I've never tried that!)

> So it's best to read that draft if you want to do this. If you
> remember doing probability questions in college, you'll remember 

   I _do_ remember that!

>>> ... If you can get through an audit function by doing
>>> packet-for-packet signalling, when audit is checking byte-for-byte
>>> balance, then good luck to you...
>>

   (Except for the part about "match drops byte-for-byte", I agree.)

> enough to mandate that TCP ConEx code has to match signals

   It's hard for me to imagine a situation where that mandate would be
appropriate...

> thing. More correct to say they can probably get away with doing the 

   Why do we need to do either?

   (And I'm not sure we'd be justified in calling it "the wrong thing.")

> bytes (based on "sound technical reasoning", at least IMHO).

   This is perhaps the crux of the issue: to Bob, "sound technical
reasoning" is paramount; to me, it is merely unlikely. I agree that
Bob has done his academic work based on "congestion volume" which
counts (some) bytes: I don't agree that "congestion volume" is a
necessary construct for ConEx -- least of all for early experiments.
It's probably time for someone else to weigh in on this...

> doing ConEx right now, and we're already behind with milestones.

   Exactly!

   The more time we spend on details which implementors are likely
to ignore, the harder it is to meet deadlines!

---

From: Bob Briscoe <[email-masked]>
Date: Tue, 11 Oct 2011 11:24:18 +0100

John,

>    Granted, I'm generally happier when we don't forbid very much...

Also, we're not standardising policing. We're standardising 
information that might be used for policing.

>out-of-scope of the ConEx specification.

Why would experiments involving policing be out of scope of the ConEx spec?

Anyway, we were talking about audit.

>transmission medium) tends to dominate near the edges.

If the designers of backbone gear have to limit line rate because 
packet processing would otherwise not keep up, that doesn't mean the 
packet processor will get congested at run-time. It more likely means 
they have designed it so the line limit will protect the packet 
processor from getting congested for typical and perhaps worst-case 
packet size distributions.

>packets are dropped or marked regardless of their size.

The last sentence is indeed correct.

>packet dropped or marked.

There's a distinction between the basis on which a machine should 
measure the queue of all packets, and the basis on which it decides 
to drop a particular arriving packet. This is explained in 
draft-ietf-tsvwg-byte-pkt-congestion

The draft gives the benefit of all the thoughts and arguments 
recorded over the years. Let's not continue a conversation based on 
musings that don't have the benefit of having read the output of an 
IETF w-g that is working on this.

>    (Except for the part about "match drops byte-for-byte", I agree.)

On what grounds is your opinion based? You haven't given any 
reasoning. Otherwise we're just going round in circles.

>    (And I'm not sure we'd be justified in calling it "the wrong thing.")

Again, at the moment, this is an opinion with no reasoning given.

>to ignore, the harder it is to meet deadlines!

Mirja is an implementer. She is asking, exactly because this question 
comes up when you write code.

Bob

---

From: Mirja Kuehlewind <[email-masked]>
Date: Tue, 11 Oct 2011 13:42:43 +0200

Hi Bob,

thanks for this separate answer on the second question but actually your first 
answer help me already. When I was reading your mail, I've just though, that 
we can at the endsystem just count the TCP payload. Under the assumption that 
the endsystem is mostly sending equally sized packets, there will be 
automatically as many header with ConEx marks as there have been causing the 
original congestion. If the sender is doing some wired things, like sending 
smaller retransmit packets, the sender knows about it and can take exactly 
this knowledge into account. 

I talked to Richard about this and he was bring up the case, when there is ACK 
congestion (control packets without payload get marked). Not sure what to do 
with this case. If an ACK get lost, we actually don't really know. We could 
assume that the receiver should ack as least every second packet but we never 
know what the receiver did. I believe usually control packets without payload 
are as well ECN-capable. So with ECN we  would have a congestion signal. But 
is this a common case that pure ACKs get ECN markings?

Mirja 

On Monday 10 October 2011 18:46:06 Bob Briscoe wrote:
> >lost. We do not know the number of packets/headers that were used to
> > transmit this data. If we would loose one big packet and then retransmit
> > a large number of small packets instead (which are all ConEx marked) that
> > might give quite a different ConEx signal. On the other hand, what causes
> > the congestion is the whole packet incl header(s). What is the right
> > thing to do?
>

-- 
-------------------------------------------------------------------
Dipl.-Ing. Mirja Kühlewind
Institute of Communication Networks and Computer Engineering (IKR)
University of Stuttgart, Germany
Pfaffenwaldring 47, D-70569 Stuttgart

tel: +49(0)711/685-67973
email: [email-masked]
web: www.ikr.uni-stuttgart.de
-------------------------------------------------------------------

---

From: Bob Briscoe <[email-masked]>
Date: Tue, 11 Oct 2011 13:35:04 +0100

Mirja,

If pure acks were ECN-capable, it would be just 
as likely that they would get an ECN mark as any 
data packet. However, RFC3168 says pure ACKS must 
be sent as Not-ECT. We should keep to this in 
ConEx unless something better has been done that 
I'm not aware of - ConEx isn't chartered to work on loss/marking of pure acks.

There is RFC5690, which is informational (rather than experimental):
"Adding Acknowledgement Congestion Control to TCP"

The relevant section in the re-ECN spec about 
pure acks, and byte vs packet ConEx signalling is 
here, which covers nearly everything in the present thread:
<http://tools.ietf.org/html/draft-briscoe-tsvwg-re-ecn-tcp-09#section-6.1.5>

This leaves the outstanding question of which 
ConEx doc should write about this stuff.
* abstract-mech ought to cover things like which packet headers to include.
* conex-tcp-modifications obviously ought to talk about TCP-specifics
* that leaves where to put design principles 
applicable to all transports. 2 options:
   a) abstract-mech?
   b) a section in conex-tcp-modifications 'guidelines for other transports'?

I prefer a) but we don't really want extremely detailed stuff in abstract-mech.

Bob

---

From: "Scheffenegger, Richard" <[email-masked]>
Date: Tue, 11 Oct 2011 19:05:25 +0100

Bob, Mirja,

Currently, there are probably two relevant RFCs in this space - RFC5562 (experimental) and RFC5590 (informational), which Bob already mentioned.

RFC5562 adds one specific case of control packets [SYN,ACK] to the ECT-capable packets.

So there exists at least one case, where a pure control packet in TCP may carry ECN information. RFC5562 is actually implemented and available in IBM AIX5.3 and later.

I believe it should be clear that Conex Marks should only be carried on those packets, that the transport layer would mark ECN-capable.

Further, Section 6.1.5 in RFC3168 also states, that retransmitted packets should not be sent with ECT. That probably settles the discussion, if a Conex L mark should be sent right with the retransmission of the loss. If I'm not mistaken that Conex "requires" a ECN capable transport (and therefore a qualifying transport segment).

However, there are some obvious potential issue with such an approach; for example, a CE-marked SYN-ACK may never be followed by a data segement which could carry the Conex E mark for an extensive period of time. Same could hold for a retransmission which may be the last data to be sent to the receiver, again for some (human scale) time periods - as tcp keepalive / window probe windows by definition won't carry ECT / Conex, correct?

Alternatively, Conex marks can be completely decoupled from the transport (or rather, sent with every packet of that ECN-capable transport, regardless if the transport actually sends a ECT packet or not). That would alleviate some of the potential implication a long delayed conex signal may cause...

(Note: reECN implicitly coupled re-ECN marks with ECT, always; but with the current IPv6 signaling scheme, this is no longer absolutely required).

Best regards,

Richard Scheffenegger

> -----Original Message-----
> From: Bob Briscoe [mailto:[email-masked]]
> Sent: Dienstag, 11. Oktober 2011 14:35
> To: Mirja Kuehlewind
> Cc: [email-masked]; Scheffenegger, Richard
> Subject: Re: [conex] Accounting of ConEx signals
> 
> Mirja,
> 
> > > The thread largely overlooked your second question, which is also a
> good
> > > one.
> > >
> the IP
> payload got

---

From: John Leslie <[email-masked]>
Date: Wed, 12 Oct 2011 06:30:05 -0400

> draft-ietf-tsvwg-byte-pkt-congestion

   I do recommend reading this draft...

> IETF w-g that is working on this.

   Though it is generally safe to assume an average IETF participant
hasn't read any particular draft ;^) Bob should be careful about
assuming _I_ haven't read it!

   I don't have any time today to compose anything useful, but I am
willing to get a round tuit if folks find this entertaining...

   But I wonder if it's worth doing if Bob intends to belittle folks
who haven't read this (expired) draft. I can't carry the discussion
alone...

---

From: Mirja Kuehlewind <[email-masked]>
Date: Sun, 30 Oct 2011 21:26:22 +0100

Hi Richard,

after reading the relevant parts of the RFCs stated below again, I would say 
the answer is more or less obvious.

A sender does not have any feedback information (no ECN nor loss detection) 
about control packets not carrying any user data. Of course those packets can 
cause congestion as well but neither the sender nor the audit will be able to 
detect this congestion. That means a sender can only give conex feedback 
about the all other packets and this conex information must be seen with 
respect to only those packets we have congestion information about.
Suddenly an IP packet with ConEx header but X bit no set makes sense. I added 
this to the most current version of the IP Dest Opt draft.

SYN and SYN/ACK might be a different case for ECN. But for ConEx as this is 
the first packet sent, for sure we do not have any congestion feedback 
information yet and taking the SYN or SYN/ACK into account when calculating 
the congestion level should usually not really make a big difference as you 
need anyway a certain amount of packets before you can get a significant 
result.

Regarding the retransmitted packets, we, kind of, have information about, even 
though they are not ECN-capable, because the RTO might cause a sender to 
retransmit a restransmitted packet again. The reason for ECN to make 
retransmitted as not-ECN-capable is because the receiver might not see a 
unnecessarily-retransmitted packet and thus might not report an CE mark on an 
unnecessarily-retransmitted packet to the sender. In our case the ConEx is 
meant to be information for network nodes and not for the receiver, thus it 
is not a problem. I would recommend to make restransmitted packet 
ConEx-capable.

For the TCP modifications draft, we can simply account the TCP payload, when 
we assume equal sized packets, as the conex marked packet as well as the 
original packets causing the congestion will both have a header of the same 
size. 
In case of not equal sized packets we have to account the headers as well but 
we cannot provide a generic way how to do this. A sender which sends 
different sized packets probably knows about the pattern/reason to do so and 
thus can reconstruct the exact number of headers based on this information.

Any thoughts?
Mirja

On Tuesday 11 October 2011 20:05:25 Scheffenegger, Richard wrote:
> Bob, Mirja,
>
> Currently, there are probably two relevant RFCs in this space - RFC5562
> (experimental) and RFC5590 (informational), which Bob already mentioned.
>
> So there exists at least one case, where a pure control packet in TCP may
> carry ECN information. RFC5562 is actually implemented and available in IBM
> AIX5.3 and later.
>
>
> I believe it should be clear that Conex Marks should only be carried on
> those packets, that the transport layer would mark ECN-capable.
>
> Further, Section 6.1.5 in RFC3168 also states, that retransmitted packets
> should not be sent with ECT. That probably settles the discussion, if a
> Conex L mark should be sent right with the retransmission of the loss. If
> I'm not mistaken that Conex "requires" a ECN capable transport (and
> therefore a qualifying transport segment).
>
> However, there are some obvious potential issue with such an approach; for
> example, a CE-marked SYN-ACK may never be followed by a data segement which
> could carry the Conex E mark for an extensive period of time. Same could
> hold for a retransmission which may be the last data to be sent to the
> receiver, again for some (human scale) time periods - as tcp keepalive /
> window probe windows by definition won't carry ECT / Conex, correct?
>
>
> Alternatively, Conex marks can be completely decoupled from the transport
> (or rather, sent with every packet of that ECN-capable transport,
> regardless if the transport actually sends a ECT packet or not). That would
> alleviate some of the potential implication a long delayed conex signal may
> cause...
>
> (Note: reECN implicitly coupled re-ECN marks with ECT, always; but with the
> current IPv6 signaling scheme, this is no longer absolutely required).
>
> Best regards,
>
>
> Richard Scheffenegger
>
> > >answer help me already. When I was reading your mail, I've just
> >
> > though, that
> >
> > >we can at the endsystem just count the TCP payload. Under the
> >
> > assumption that
> >
> > >automatically as many header with ConEx marks as there have been
> >
> > causing the
> >
> > >original congestion. If the sender is doing some wired things, like
> >
> > sending
> >
> > >smaller retransmit packets, the sender knows about it and can take
> >
> > exactly
> >
> > >congestion (control packets without payload get marked). Not sure what
> >
> > to do
> >
> > >with this case. If an ACK get lost, we actually don't really know. We
> >
> > could
> >
> > >assume that the receiver should ack as least every second packet but
> >
> > we never
> >
> > >know what the receiver did. I believe usually control packets without
> >
> > payload
> >
> > >are as well ECN-capable. So with ECN we  would have a congestion
> >
> > signal. But
> >
> > > > IMO the size should include the network layer header, but not link
> >
> > layer.
> >
> > > > case, the transport just adds a pseudo-size for the IP and TCP
> >
> > headers.
> >
> > > > >lost. We do not know the number of packets/headers that were used
> >
> > to
> >
> > > > > transmit this data. If we would loose one big packet and then
> >
> > retransmit
> >
> > > > > a large number of small packets instead (which are all ConEx
> >
> > marked) that
> >
> > > > > might give quite a different ConEx signal. On the other hand,
> >
> > what causes
> >
> > > > > the congestion is the whole packet incl header(s). What is the
> >
> > right
> >
> > > > > thing to do?
> > > >

tel: +49(0)711/685-67973