Subject: Re: [ledbat] [Iccrg] Looking for reviewer for draft-ietf-ledbat-congestion-06

From: Michael Welzl <[email-masked]>
Date: Sun, 12 Jun 2011 09:41:24 +0200

thank you!

I'm cc'ing ledbat in my answer.

Cheers,
Michael

On Jun 12, 2011, at 8:53 AM, Lisong Xu wrote:

> Hi Michael,
>
> Below are my comments after reading the draft.
> Thanks
> Lisong
>
>
> **** Section 3.1 ****
> First sentence: "a loss occurs [RFC5681], which, in the absence of  
> any Active Queue Management (AQM) in the network, occurs only when  
> the queue at the bottleneck link on the end-to-end path overflows "
>
> Since a loss may occur due to link errors, such as fiber errors and  
> wireless errors, we may change the first sentence to
> "... (AQM) and link errors in the network,...."
>
> **** Section 3.3 ****
> The data type of each variable is not specified. Since some  
> variables must be signed variables, it is important to specify the  
> type of each variable in order to correctly implement the algorithm.
>
> For example, tcp_time_stamp is an unsigned int, so local_timestamp  
> and remote_timestamp are also unsigned int.  But variable  
> acknowledgement.delay should be a signed int, since local_timestamp- 
> remote_timestamp may be negative.
>
> **** Section 3.4.2 ****
> "a LEDBAT sender stores BASE_HISTORY+1 separate minima- one each for  
> the last BASE_HISTORY minutes, and one for the running current  
> minute."
>
> But the update_current_delay(delay) code maintains only BASE_HISTORY  
> minima. So the above sentence should be changed to
> "a LEDBAT sender stores BASE_HISTORY separate minima- one each for  
> the last BASE_HISTORY-1 minutes, and one for the running current  
> minute."
>
> **** Section 3.5 ****
> "we recommend that the sender SHOULD use at least 4 samples in each  
> RTT. Thus, CURRENT_FILTER SHOULD be at least 4, and limited such  
> that no samples in list are older than an RTT in the
> past."
>
> But what if cwnd is less than 4? In this case, there are less than 4  
> samples in each RTT.
>
>
> **** General *****
>
> It is not clear what the relation between cwnd and flightsize is. Is  
> flightsize always the same as cwnd, or could be less than cwnd? When  
> is flightsize less than cwnd?
>
>
> On 6/5/2011 12:38 AM, Michael Welzl wrote:
>> Hi,
>>
>> We're urgently looking for a volunteer to review
>> draft-ietf-ledbat-congestion-06:
>> http://tools.ietf.org/html/draft-ietf-ledbat-congestion-06
>> which is currently in Last Call, ending in less than two weeks (15  
>> June).
>>
>> If anyone here is willing to do a review, please drop Murari and me a
>> note ASAP!
>>
>> Thanks,
>> Michael
>>
>>
>> _______________________________________________
>> Iccrg mailing list
>> [email-masked]
>> http://oakham.cs.ucl.ac.uk/mailman/listinfo/iccrg
>>
>
> -- 
> Lisong Xu, Associate Professor
> Computer Science & Engineering
> University of Nebraska-Lincoln
> http://cse.unl.edu/~xu

---

From: Mirja Kühlewind <[email-masked]>
Date: Wed, 15 Jun 2011 15:07:46 +0200

Hi Lisong

thank you very much for your review! Please see comments inline.

Mirja

On Sunday 12 June 2011 08:53:10 Lisong Xu wrote:
> First sentence: "a loss occurs [RFC5681], which, in the absence of any
> Active Queue Management (AQM) in the network, occurs only when the queue
> at the bottleneck link on the end-to-end path overflows "
>
Done.

> **** Section 3.3 ****
> The data type of each variable is not specified. Since some variables
> must be signed variables, it is important to specify the type of each
> variable in order to correctly implement the algorithm.
>
> For example, tcp_time_stamp is an unsigned int, so local_timestamp and
> remote_timestamp are also unsigned int.  But variable
> acknowledgement.delay should be a signed int, since
> local_timestamp-remote_timestamp may be negative.
I don't want to specify any data type here because that's only pseudo code of 
the algorithm and the range and resolution of a variable are implementation 
specific and might depend on the framing protocol used and of course as well 
on the programming language. 

Your example is a quite special case as one would assume that delay is always 
positive but you might get negative values because of the clock offset. 
Issues like clock offset and skew, which can come up in a real system, are 
not regarded as part of the algorithm. Anyway there is some discussion in the 
appendix. Moreover, in fact in this special example it doesn't matter if the 
delay is positive or negative because its canceled out in the calculation of 
the queuing delay and even an overflow would work here.

The only case where it is important that a value can/must get negative is the 
off_target and this is mentioned explicit in 3.4.1.

>
> **** Section 3.4.2 ****
> "a LEDBAT sender stores BASE_HISTORY+1 separate minima- one each for the
> last BASE_HISTORY minutes, and one for the running current minute."
>
> "a LEDBAT sender stores BASE_HISTORY separate minima- one each for the
> last BASE_HISTORY-1 minutes, and one for the running current minute."
Thanks. You are right! Done.

> **** Section 3.5 ****
> "we recommend that the sender SHOULD use at least 4 samples in each RTT.
> Thus, CURRENT_FILTER SHOULD be at least 4, and limited such that no
> samples in list are older than an RTT in the
> past."
>
That's why there is a SHOULD and not a MUST. If you have less samples, you 
can, of course, only use whatever you get. But the results might be very 
noisy and if you permanently have less then 4 samples, LEDBAT might not work 
correctly (because of noise in real systems). 

Should we make this more explicit?

> flightsize always the same as cwnd, or could be less than cwnd? When is
> flightsize less than cwnd?
flightsize can be less if your transfer is application-limited (and 
ALLOWED_INCREASE is large enough). Thus you don't have enough data to fill 
the cwnd.

In the code you find
" on acknowledgment:
    # flightsize is the amount of data outstanding before this ack 
    #    was received and is updated later by update_flightsize();
    # bytes_newly_acked is the number of bytes that this ack 
    #    newly acknowledges, and it MAY be set to MSS; and
    # cwnd is in bytes."

I've added the following to 'on initialization:' (eventhough you find the same 
explanation in 3.2.)
"   # cwnd is the amount of data that is allowed to send in one RTT and 
    # is defined in bytes."

Does this help?

> > which is currently in Last Call, ending in less than two weeks (15 June).
> >

---

From: Mirja Kuehlewind <[email-masked]>
Date: Fri, 17 Jun 2011 11:33:09 +0200

Hi Lisong,

you are right, if it is an unsigned this might cause errors but its actually 
not a problem. If local_timestamp-remote_timestamp is negative but 
acknowledgement.delay is signed you will get a wrong result (but this doesn't 
matter as this delay does not show a real delay value anyway) but you will 
get a result because there simply will be an overflow.
If you now calculate queuing_delay = current_delay - base_delay. Also this 
error will cancel out and you get an valid result.

Anyway, its in the responsibility of the implementor to get the type and range 
right; its not part of the algorithm itself, from my point of view. Do you 
have a different opinion here? 

Mirja

On Thursday 16 June 2011 18:36:24 Lisong Xu wrote:
> Mirja,
>
> Thanks for your reply. I have only one question now.
>
> "it doesn't matter if the delay is positive or negative because its
> canceled out in the calculation of the queuing delay and even an
> overflow would work here."
>
> You are right that it does not matter whether acknowledgement.delay is
> positive or negative. But variable acknowledgement.delay MUST be a
> signed int. If it is implemented as an unsigned int, then sometimes
> there will be a problem.
>
> Thanks
> Lisong
>
> On 6/15/2011 6:07 AM, Mirja Kühlewind wrote:
> > Hi Lisong
> >
> > I don't want to specify any data type here because that's only pseudo
> > code of the algorithm and the range and resolution of a variable are
> > implementation specific and might depend on the framing protocol used and
> > of course as well on the programming language.
> >
> > Your example is a quite special case as one would assume that delay is
> > always positive but you might get negative values because of the clock
> > offset. Issues like clock offset and skew, which can come up in a real
> > system, are not regarded as part of the algorithm. Anyway there is some
> > discussion in the appendix. Moreover, in fact in this special example it
> > doesn't matter if the delay is positive or negative because its canceled
> > out in the calculation of the queuing delay and even an overflow would
> > work here.
> >
> > The only case where it is important that a value can/must get negative is
> > the off_target and this is mentioned explicit in 3.4.1.
> >
> >> **** Section 3.4.2 ****
> > That's why there is a SHOULD and not a MUST. If you have less samples,
> > you can, of course, only use whatever you get. But the results might be
> > very noisy and if you permanently have less then 4 samples, LEDBAT might
> > not work correctly (because of noise in real systems).
> >
> > ALLOWED_INCREASE is large enough). Thus you don't have enough data to
> > fill the cwnd.
> >
> > In the code you find
> > " on acknowledgment:
> > I've added the following to 'on initialization:' (eventhough you find the
> > same explanation in 3.2.)

-- 
-------------------------------------------------------------------
Dipl.-Ing. Mirja Kühlewind
Institute of Communication Networks and Computer Engineering (IKR)
University of Stuttgart, Germany
Pfaffenwaldring 47, D-70569 Stuttgart

tel: +49(0)711/685-67973
email: [email-masked]
web: www.ikr.uni-stuttgart.de
-------------------------------------------------------------------

---

From: Mirja Kuehlewind <[email-masked]>
Date: Sat, 18 Jun 2011 10:28:56 +0200

Hi Lisong,

> A scientific paper is meant for a researcher, but an RFC is for a
> programmer who wants to implement the protocol. This is why I want to
> specify the data type of acknowledgement.delay (possible others too).
Other than many RFCs this document is NOT specifying a protocol. It 'only' 
documents the algorithm. There is anyway some work for the implementor  to 
transfer the algorithm based on the used framing protocol and programming 
language. E.g. if you want to implement LEDBAT on a small low-energy device 
with limited memory and you only have unsigned short, you have to scope with 
it...
And even protocol specifications, which are even more implementor guides, try 
to specify a protocol deliberately without any implementation specify 
assumptions such that a protocol can be implemented in many different 
devices.
We are actually still thinking about a document to define LEDBAT for TCP 
(which would be more like a implementation guide then).

>
> OK, here is an example to illustrate the problem of unsigned variables.
>
> assume all variables are unsigned short.
>
> at some time (low queueing delay):
>    local_timestamp = 3, and remote_timestamp = 5
>    then acknowledgement.delay = 65534
>    then base_delay = 65534
>
> at some time later (high queueing delay):
>    local_timestamp = 7, and remote_timestamp = 6
>    then acknowledgement.delay = 1
>    then base_delay = 1
>    then queueing_delay = 0
In this example there is a very strong clock skew. You would need some clock 
correction, otherwise the result are incorrect anyway.

Mirja

>
> Thanks
> Lisong
>
> On 6/17/2011 2:33 AM, Mirja Kuehlewind wrote:
> > Hi Lisong,
> >
> > you are right, if it is an unsigned this might cause errors but its
> > actually not a problem. If local_timestamp-remote_timestamp is negative
> > but acknowledgement.delay is signed you will get a wrong result (but this
> > doesn't matter as this delay does not show a real delay value anyway) but
> > you will get a result because there simply will be an overflow.
> > If you now calculate queuing_delay = current_delay - base_delay. Also
> > this error will cancel out and you get an valid result.
> >
> > Anyway, its in the responsibility of the implementor to get the type and
> > range right; its not part of the algorithm itself, from my point of view.
> > Do you have a different opinion here?
> >
> > Mirja
> >
> >>>> Active Queue Management (AQM) in the network, occurs only when the
> >>>> queue at the bottleneck link on the end-to-end path overflows "
> >>>>
> >>> implementation specific and might depend on the framing protocol used
> >>> and of course as well on the programming language.
> >>>
> >>> discussion in the appendix. Moreover, in fact in this special example
> >>> it doesn't matter if the delay is positive or negative because its
> >>> overflow would work here.
> >>>
> >>> The only case where it is important that a value can/must get negative
> >>> is the off_target and this is mentioned explicit in 3.4.1.
> >>>
> >>>> **** Section 3.4.2 ****
> >>>> RTT. Thus, CURRENT_FILTER SHOULD be at least 4, and limited such that
> >>>> no samples in list are older than an RTT in the
> >>>> past."
> >>>>
> >>> very noisy and if you permanently have less then 4 samples, LEDBAT
> >>> might not work correctly (because of noise in real systems).
> >>>
> >>> I've added the following to 'on initialization:' (eventhough you find
> >>> the same explanation in 3.2.)

tel: +49(0)711/685-67973