Subject: Re: [conex] Crediting [was: Re: Review of draft-ietf-conex-abstract-mech-06]

From: Bob Briscoe <[email-masked]>
Date: Mon, 15 Jul 2013 12:19:08 +0100

Mirja,

Sorry for being unresponsive on this thread at the time. Inline...

(I'll reply separately where nec. to earlier postings from others too.)

At 12:37 26/06/2013, Mirja Kuehlewind wrote:
>Hi Bob,
>
>another definition could be:
>
>A credits gets invalid/is used when a new congestion signal (CE or loss)
>arrives at the audit and the number of congestion-marked or lost packets
>(red) is currently larger than ConEx-marked ones (black/purple).
>
>This would mean you potentially have to resend credits when congestion occurs.

I think what you're saying is that when a credit 
needs to be used, it is used for ever, even if 
later a re-echo makes it up. So the re-echoes gradually replace the credit.

IMO that doesn't work either. During a 
long-running flow, the sender will always re-echo 
after the associated congestion. So it always 
needs a round-trip worth of credit to save its 
flow from being hit by the auditor. If all credit 
is invalidated whenever it is used, then, in 
steady state, every time the sender re-echoes it 
will also have to send an equal amount of credit. 
THat doubles the 'cost'. Actually, all the sender 
will do is send credit and not re-echo.

This is a hard dilemma isn't it? I will try to state it here:

There are two opposing requirements:
1) We want the sender to have to send sufficient 
credit to cover the large amount of harm it might 
cause due to a slow-start overshoot. But we don't 
want the sender to know that it can just get that 
credit back again if it ends up not being needed. 
Otherwise, after a short flow, it might as well 
keep sending even dummy traffic just to use up 
the spare credit. And if it has traffic to send, 
it can cause congestion without any more 'cost'. 
That would be like getting your insurance premium 
back at the end of the year if you hadn't claimed 
- it would not cover the risks with other 
people's QoE that you took by doing slow-start.
This implies credit should expire over time.
2) However, during a long-running flow, we don't 
want credit to expire, because the sender needs 
it to hold back any penalty for one round trip, 
for every little congestion event.

Bob

>Mirja
>
>
>On Sunday 16 June 2013 08:34:35 Bob Briscoe wrote:
> > Mirja,
> >
> > At 16:03 09/06/2013, Mirja Kühlewind wrote:
> > >Hi,
> > >
> > >so back on this one:
> > > > >9) Section 5.5.1 introdues the credit concept. Not sure if the meaning
> > > > > of credits is well enough specified here. My personal option is that
> > > > > credits should somehow get invalid (at some point in time). This is
> > > > > left open in the current text.
> > > > >
> > > > >
> > > > >I think we need to agree before we can talk
> > > > >about writing down what we agree...
> > > > >
> > > > >
> > > > >I think that abstract-mech needs to embrace
> > > > >*both*, explicitly if not implicitly.  I need to
> > > > >think about this some more, but I suspect that
> > > > >it means we have unnecessarily over constrained audit here.
> > > >
> > > > [BB]: We need to allow multiple experiments at
> > > > this experimental stage. But ultimately, sources
> > > > will need to unambiuously know what Credit means,
> > > > so they know how much to introduce and when.
> > >
> > >Yes, but we need to propose a mechanism when to send credits for the TCP
> > > mod draft and this means we need to have a common understanding how to
> > > handle credits in the endsystem and the audit. I guess that's what
> > > standards are good for. We might need a separate document for this. Not
> > > sure we are able to agree on this right now. As an alternative, I could
> > > also add some text in the TCP mod draft that the crediting is an open
> > > issue for experiments...?
> >
> > I would rather we supported diversity by stating
> > one concrete definition, and allowed experiments
> > with other definitions. But I would (naturally)
> > prefer my definition (ie credit lasts for as long
> > as a flow is active, and an auditor MAY time out
> > a flow after no less than 1 second of inactivity).
> >
> > Actually, I think this is the only concrete
> > definition on the table, unless you have a
> > proposal for exactly how to define your idea of credit gradually expiring.
> >
> > > > >Rather than thinking of Credit expiring after a
> > > > >time, one can think of the combination of recent
> > > > >Re-Echo signals and earlier Credit signals
> > > > >keeping the Credit state fresh within a flow. As
> > > > >long as you've sent Credit before a round of
> > > > >congestion, then if you send Re-Echo afterwards
> > > > >the Auditor can switch it round as if you sent
> > > > >the Re-Echo before and the Credit after.
> > >
> > >I don't think this would change anything. Maybe make it even worse. As you
> > >could also just send credit instead of ConEx marks
> >
> > That's not such a big problem, although I admit
> > it devalues the distinction between Credit &
> > Re-Echo, and I admit too that I don't have a solution to that.
> >
> > >and moreover there is
> > >still no incentive to send further marks when you have build up a large
> > >number of credits during Slow Start.
> >
> > My proposed definition of credit means that the sender wouldn't need to.
> >
> > It's not ideal from a network monitoring point of
> > view for the network to see such an anomalous
> > amount of credit and nothing to be able to
> > monitor recent actual congestion. But at least it
> > still provides full incentives against cheating.
> >
> > And, much more importantly, it also provides
> > incentives to slow-start more carefully (e.g.
> > watching the ACK timing to detect onset of
> > overshoot a round trip earlier), so that so much
> > credit isn't wasted. The Linux patch to do this
> > was too gentle, but the developers gave up
> > because at the time there was no way to reason
> > about how much less gentle it should be - ConEx provides that.
> >
> > > > >So, when the Auditor holds Credit, it allows up
> > > > >to that amount of Re-Echo to be considered as
> > > > >having been sent before the congestion, rather
> > > > >than after. Then, as it switches the Re-Echoes
> > > > >back in time, it switches the Credits forward, so they always stay
> > > > > recent.
> > > > >
> > > > >Credit is primarily a mechanism to ensure the
> > > > >sender has to suffer some cost before it is
> > > > >trusted to pay back some cost. Credit doesn't
> > > > >need to degrade over time if the cost to the
> > > > >sender of introducing credit doesn't degrade over time.
> > > > >
> > > > >Does this move us forward, or do you still
> > > > >disagree? If so, I suggest a new thread would be useful.
> > >
> > >I have two concerns:
> > >1) As mentioned above if a sender has sent a large number of credits
> > > during Slow Start and causes only few congestion during the rest of the
> > > transmission (as today's TCP usually does), there is no incentive to send
> > > further ConEx marks at all (neither credits nor loss/ECN ConEx marks).
> >
> > See above.
> >
> > >2) When sufficient markings has been sent during Slow Start, no further
> > >credits should be needed. But if the audit for any reason will loose state
> > >(e.g. because of memory restriction or a new audit is used due to
> > > rerouting), the sender will still not send new credits as will and thus
> > > will cause the audit penalize the flow eventhough the sender did do
> > > nothing wrong.
> >
> > Audit would only lose state due to a re-route.*
> > Then, admittedly it would lose a load of credit,
> > but if the sender's flow was in progress, it
> > wouldn't need a huge repeat of credit again. Just
> > one, or perhaps two, ConEx re-echoes in response
> > to the ensuing audit losses would be sufficient to re-instate the audit
> > state.
> >
> > * = (If audit has mem restrictions it would not
> > have set the state in the first place.)
> >
> > > > >This is probably correct, but I really don't think it belongs in A-M.
> > >
> > >We might need an own document but there might also be some additional
> > >requirements that should be added to this document.
> >
> > Going back on what I say below, I think it would
> > be better to state one full definition of credit
> > in abstr mech, just to be concrete, but then say
> > that if anyone prefers another definition, they
> > are encouraged to experiment with it and write up the outcome.
> >
> > > > [BB]: I don't think it should either. This is a
> > > > discussion with Mirja, rather than a proposal for text.
> >
> > Tx again for you review.
> >
> >
> > Bob
> >
> >
> > ________________________________________________________________
> > Bob Briscoe,                                                  BT
>
>
>
>--
>-------------------------------------------------------------------
>Dipl.-Ing. Mirja Kühlewind
>Institute of Communication Networks and Computer Engineering (IKR)
>University of Stuttgart, Germany
>Pfaffenwaldring 47, D-70569 Stuttgart
>
>tel: +49(0)711/685-67973
>email: [email-masked]
>web: www.ikr.uni-stuttgart.de
>-------------------------------------------------------------------

________________________________________________________________
Bob Briscoe,                                                  BT

---

From: Bob Briscoe <[email-masked]>
Date: Mon, 15 Jul 2013 12:32:25 +0100

David,

I ought to read your credit draft first, but I'll just quickly 
respond here (altho nothing much additional worth saying I'm afraid), ...

At 18:44 10/06/2013, David Wagner wrote:
>Hi,
>
>I think there are some strong arguments for "aging" credits:
>1) they avoid problems with restarted audits / rerouting: if credits 
>are valid for e.g. 1 hour, any newly started auditer should not 
>indicate reliable information for that time.
>This can be extended to running auditers seeing new (rerouted?) IP 
>addresses, but of course there is a trade-off for coverage.

This doesn't really "avoid problems". It just means the auditor knows 
it has a problem for a certain period.

>2) the value for credit is likely to degrade over time for the 
>sender, typically already sent credits will count less than credits 
>to be sent now.

That's not an argument for time-degrading credit, you're just saying 
it's likely. It is likely if there's an argument for it, and it's not 
likely if there isn't.

>This applies for rate-based credit / ConEx allowance mechanisms 
>since, if using non-vanishing credits, the starting phase is much 
>more costly than maintaining a flow. In other words: non-vanishing 
>credit is an incentive to keep connections open. Do we want that?

We want the amount of credit to reflect the risk of cost to others. 
(See my point in the email to Mirja about the perverse incentives 
that would arise if you knew you would get your insurance premiums 
back if you hadn't made a claim by the end of the year. Then it would 
no longer be insurance, it would be an accident damage pre-payment 
account.) But we also have the dilemma I outlined in the mail to Mirja.

>Anyway, I don't yet have a good credit concept.

Yes, this is a problem.

>Which also needs to address handling loss of ConEx-marked packets, 
>at the sender and at the audit.

I don't think of that as a problem. I may not have covered it at the 
IETF, but I think I did in my PhD thesis.

Bob

>David
>
> > Hi,
> >
> > so back on this one:
> >
> > > >9) Section 5.5.1 introdues the credit concept. Not sure if the 
> meaning of
> > > >credits is well enough specified here. My personal option is 
> that credits
> > > >should somehow get invalid (at some point in time). This is left open in
> > > > the current text.
> > > >
> > > >
> > Yes, but we need to propose a mechanism when to send credits for 
> the TCP mod
> > draft and this means we need to have a common understanding how to handle
> > credits in the endsystem and the audit. I guess that's what standards are
> > good for. We might need a separate document for this. Not sure we 
> are able to
> > agree on this right now. As an alternative, I could also add some 
> text in the
> > TCP mod draft that the crediting is an open issue for experiments...?
> >
> > >
> > could also just send credit instead of ConEx marks and moreover there is
> > > >back in time, it switches the Credits forward, so they always 
> stay recent.
> > > >
> > 1) As mentioned above if a sender has sent a large number of 
> credits during
> > Slow Start and causes only few congestion during the rest of the 
> transmission
> > (as today's TCP usually does), there is no incentive to send further ConEx
> > marks at all (neither credits nor loss/ECN ConEx marks).
> rerouting),
> > the sender will still not send new credits as will and thus will cause the
> > audit penalize the flow eventhough the sender did do nothing wrong.
> >
> > > >
> > > >
> > _______________________________________________
> > conex mailing list
> > [email-masked]
> > https://www.ietf.org/mailman/listinfo/conex
> >
>
>
>--
>Dipl.-Inf. David Wagner
>University of Stuttgart
>Pfaffenwaldring 47, D-70569 Stuttgart, Germany
>
>web: www.ikr.uni-stuttgart.de   email: [email-masked]
>phone: +49 711 685-67965        fax: +49 711 685-57965

---

From: David Wagner <[email-masked]>
Date: Mon, 15 Jul 2013 16:57:02 +0200

Hi Bob, 

> it has a problem for a certain period.
Well, that might be enough anyway. If the auditor knows it shouldn't penalize flows for a certain destination network for the time x, auditing could still cover most of the traffic and thus still be effective. 
> it's likely. It is likely if there's an argument for it, and it's not 
Agreed. But there are other & stronger arguments in favor of time-degrading / expiring credits. 
> >Anyway, I don't yet have a good credit concept.
I think this is a fundamental one since it questions the credibility of ConEx info and thus the incentive to deploy it. 
> IETF, but I think I did in my PhD thesis.
oops, I didn't check that. 
I wrote some sentences on it in the discussion draft, mainly coming to the conclusion that an auditor could estimate average loss of connection, thus providing an upper bound for loss of Conex-marked packets. 

Anyway, I'd really like to discuss ConEx crediting further. 

David

---

From: Bob Briscoe <[email-masked]>
Date: Mon, 15 Jul 2013 18:23:39 +0100

David,

At 15:57 15/07/2013, David Wagner wrote:
>I think this is a fundamental one since it questions the credibility 
>of ConEx info and thus the incentive to deploy it.

Yes, in as much as every part of a security system is fundamental, 
just as every one of the four walls around a castle is fundamental.

>oops, I didn't check that.

S.7.4.4 & 7.4.5
<http://www.bobbriscoe.net/projects/refb/#refb-dis>

>I wrote some sentences on it in the discussion draft, mainly coming 
>to the conclusion that an auditor could estimate average loss of 
>connection, thus providing an upper bound for loss of Conex-marked packets.

I made it the responsibility of the sender to repair (it can know if 
a packet it marked as re-echoed was lost).

>Anyway, I'd really like to discuss ConEx crediting further.

Yes, I'm sure the chairs will be making this a subject for discussion 
in Berlin. And I'll try to comment on your draft on the list if I get 
to it before then.

Cheers

Bob

---

From: Bob Briscoe <[email-masked]>
Date: Tue, 30 Jul 2013 21:05:40 +0100

David, Mirja,

I've been worrying that the surcharge scheme doesn't work. Then I 
re-read the definition of it in your draft, and the definition seems 
incomplete. It doesn't say anything about a penalty for having a 
negative balance of ECN or loss.

The ECN or loss balances will go negative every congestion event 
until ConEx markings balance them. Now I give a few options, because 
I have to guess how you meant to define the scheme:
* If negative loss or ECN balance is penalised
   o If a sufficient credit balance covers negativity of either loss 
or ECN, then there is no need to re-balance loss or ECN with ConEx 
re-echo marks, the sender can just send credit, which is the same 
problem as subsitution.
   o If credit does not cover negativity of loss or ECN, then what's it for?
* And if negative loss or ECN balance is not penalised, what is the 
incentive to make them balance?

As I said offlist before the ConEx meeting, I think the surcharge 
scheme just conceals the same problem as the substitute scheme. 
Without the definition of the scheme written down, I don't know 
whether I'm being stupid and missing something obvious, or it's just broken.

Bob

At 18:23 15/07/2013, Bob Briscoe wrote:
>>I think this is a fundamental one since it questions the 
>>credibility of ConEx info and thus the incentive to deploy it.
>
>Yes, I'm sure the chairs will be making this a subject for 
>discussion in Berlin. And I'll try to comment on your draft on the 
>list if I get to it before then.
>
>Cheers
>
>
>
>Bob
>
>
>>David
>> >
>> >
>> >
>> > Bob
>> >
>> >
>> > >David
>> > >
>> > > > Hi,
>> > > >
>> > > > so back on this one:
>> > > >

---

From: David Wagner <[email-masked]>
Date: Wed, 31 Jul 2013 02:33:56 +0200

Hi  Bob, 

the surcharge approach does solve part of / shift the problem: 

of course the ecn-/loss-balance will get negative but the sender is supposed to have sent credit sufficient credit _in advance_ to cover that congestion event, thus making the credit counter remain non-negative after the congestion event. 
The point is, if there is no limit to replacing L-/E-marks by credit, _then_ there is not benefit / difference compared to the substitute approach (can also be seen at the penalty criteria). _If_ there is anything that verifies the observed loss and ECN-CE marks against the seen L- & E-marks, _then_ there is a benefit. E.g. the criterion#2. Or there also is a benefit if credits expire... 

So the resume remains, surcharge is not worse than substitute, and either the rtt-criterion or the expiration-approach should be applied in order improve it compared to substitute to ensure that credit really does not replace ConEx-marks. 
Or do _I_ miss something?

David

---

From: Bob Briscoe <[email-masked]>
Date: Wed, 31 Jul 2013 08:57:01 +0100

David,

What I'm missing before I can even understand anything you're saying 
is a full definition of surcharge. As I said,...

 > the definition seems

Bob

At 01:33 31/07/2013, David Wagner wrote:
>of course the ecn-/loss-balance will get negative but the sender is 
>supposed to have sent credit sufficient credit _in advance_ to cover 
>that congestion event, thus making the credit counter remain 
>non-negative after the congestion event.
>The point is, if there is no limit to replacing L-/E-marks by 
>credit, _then_ there is not benefit / difference compared to the 
>substitute approach (can also be seen at the penalty criteria). _If_ 
>there is anything that verifies the observed loss and ECN-CE marks 
>against the seen L- & E-marks, _then_ there is a benefit. E.g. the 
>criterion#2. Or there also is a benefit if credits expire...
>
>So the resume remains, surcharge is not worse than substitute, and 
>either the rtt-criterion or the expiration-approach should be 
>applied in order improve it compared to substitute to ensure that 
>credit really does not replace ConEx-marks.