Subject: [tcpm] Fujitsu and Saratoga

From: Anthony Sabatini <[email-masked]>
Date: Mon, 4 Feb 2013 17:11:07 -0500

L. Wood, W. Eddy -

1) TDLC is a true message based transmission layer control protocol (unlike TCP which mixes layers).  As such it is concerned only with the secure and timely delivery of messages.  It always has had event based transmission management, not timer based and without timers there is no congestion collapse.

2) I chose a bit mapped representation for missing messages because number pair schemes, such as SACK, falls apart if the number of outstanding holes exceeds what can be carried in the options area of one message.  For large holes in the stream this is fine, for "popcorn" noise and loss situations it is a disaster.  Additionally since I number messages, not bytes, I need 10 bit smaller numbers to express the same capabilities that TCP does.  So comparing apples to apples, if three sack pairs are in a message three holes can be expressed (in some cases 4 holes) where as with a bitmap I can span 192 messages uniquely, the equivalent of 192KB.  Using CCITT Group 3 One dimensional bit compression encoding I could express between 4 and 20 times that many messages in the same space.

3) Since TDLC is state/event driven as opposed to timer driven, many congestion situations will even themselves out by transmission rates slowing waiting for holes to be filled.  TDLC was designed to work with ECN, not to try to second guess what the router is doing or what it needs, it was implemeted with a token based ECN system where a nessage of less tahn 512 bytes took one token and and additional token for each 512 bytes or fraction thereof.  These numbers were chosen after consultation with the router manufacturers of the time (lower speed links used 256 bytes instead of 512).

4) The reason we are seeing so many alternatives to TCP/IP is that TCP/IP is failing badly in the real world.  As long as link quality kept getting better TCP/IP was fine, but with the random drop situations in mobile phones and WiFi and the increase in congestion on cellular, WiFi and backbone links, TCP/IP is breaking down (I live in New York, I can document at least 1 TCP/IP failure per hour, minimum, due to noise, packet loss and congestion that happens to me personally).

Preliminary comments on Saratoga -

1) Before I give comments please understand I like and support where this protocol is going, I just want people to think about how to make it better.

2) The strength of the IETF protocols are simplicity and solving the general case at the expense of the edge cases.  Saratoga is starting to suffer from way to many options, you need to define a core implementation and a path to expansion in the base document not all possible variants otherwise it will be a bear to implement since all features must be included and treated equally.  I know you have put in some thought on this and there are many places you say what is mandatory but as a whole document can be overwhelming in it's implications.

3) This protocol suffers from an extreme case of OSI layer violation (worse than TCP/IP), the underlying transport should be separated into it's own document and redone for more general use.  The file control should ride atop that protocol.  A selection from the TDLC standards document is below for your review.

 From the TDLC standard document -

Transmission Control Byte
76543210    
xx          Type of packet
00          User Data
01          User Command
10          Supervisory Data
11          Supervisory Command

  xx        Record Control
  11        Record is complete in this message
  10        Record starts with first byte of message
  01        Record ends with last byte of this message
  00        Record continues with this message
  
    xx      Record Group Control
    11      Record Group is complete in this message
    10      Record Group starts with first byte of this message
    01      Record Group ends with last byte of this message
    00      Record group continues with this message

Notes -

Type of packet - User data is the norm.  User command is used for things like FTP commands, the file is transferred as user data. Supervisory command and data perform similar functions but handle situations such as interrupt.    

Record Control - Record control is shared between user and network control.  Only complete records are delivered to user.  If network needs to split packet then first packet will contain first bit and last packet will contain second bit, other bits here would be zero.

Record Group Control - Obeys same rules as Record control as far as packet splitting but signals to user when first message of group is read and when last message of group is read.

Numeric Compression

It is only necessary to have one absolute number in the header, the "next in sequence" (The TCP Acknowledge field), all other numbers in the message should be realtive to this number.

TIFF defines a very light weight compression scheme, a number consists of multiple bytes, high to low order, the first bit of the byte is a "final" indicator and the low 7 bits are the value.  As bytes are read the current value is shifted left 7 bits and the low order 7 bits of the incoming byte are added or ored in.  If the high order bit is a one the proccess is complete, otherwise it keeps repeating until a byte with the high order bit set to one is encountered. Thus values are 7, 14, 21, 28 et seq. bits in length.

To be truly efficient numbers should be realtive to the immediately precceding number thus requiring the smallest number of bits to express the difference.

Bit Map Compression

CCITT Group 3 one dimensional coding standard is a run length limited code for binary compression which works well in compressing a long bitmap of outstanding messages.

Tony

Anthony Sabatini
200 West 20th Street
Apt. 1216
New York, NY 10011

Phone: (212) 867-7179
Mobile: (917) 224-8388

---

From: Michael Welzl <[email-masked]>
Date: Tue, 5 Feb 2013 09:42:19 +0100

Dear Tony, all,

> 2) I chose a bit mapped representation for missing messages because number pair schemes, such as SACK, falls apart if the number of outstanding holes exceeds what can be carried in the options area…

That makes sense to me. What I really wonder is: do you have any data showing that with an **up-to-date** TCP SACK, these situations really do become catastrophic? I guess such an investigation would have to answer questions like "in such extreme heavy loss situations, if TCP runs into a timeout, wasn't that the better thing to do anyway?" and "does such 'popcorn' noise really appear at large enough scales to cause problems for today's TCP SACK"?

One point of concern that I see is that bandwidth, and with it the BDP, is growing, but the usual MTU isn't. Hence, if the space-wasted-by-SACK-problem that Tony describes really isn't an issue today, is it possible that this becomes a true problem tomorrow?

> 4) The reason we are seeing so many alternatives to TCP/IP is that TCP/IP is failing badly in the real world.  As long as link quality kept getting better TCP/IP was fine, but with the random drop…

Do you have any data showing that these failures are SACK related, i.e. would be fixed with better SACK design as you suggest?

> 2) The strength of the IETF protocols are simplicity and solving the general case at the expense of the edge cases.  Saratoga is starting to suffer from way to

"The strength of the IETF protocols are simplicity" => this is a joke, right??

> many options, you need to define a core implementation and a path to expansion in the base document not all possible variants otherwise it will be a bear to implement since all features must be included and treated equally.  I know you have put in some thought on this and there are many places you say what is mandatory but as a whole document can be overwhelming in it's implications.
> 
> 3) This protocol suffers from an extreme case of OSI layer violation (worse than TCP/IP), the underlying transport should be separated into it's own document
> and redone for more general use.  The file control should ride atop that protocol.  A selection from the TDLC standards document is below for your review.
> 
> _______________________________________________
> tcpm mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/tcpm

Cheers,
Michael

---

From: Michael Welzl <[email-masked]>
Date: Tue, 5 Feb 2013 10:28:28 +0100

Oops. I just noticed that I've been thinking of SCTP too much lately, letting me forget about a TCP limitation here - so here's me trying to fix my own mistake before others do it, below:

> One point of concern that I see is that bandwidth, and with it the BDP, is growing, but the usual MTU isn't. Hence, if the space-wasted-by-SACK-problem that Tony describes really isn't an issue tod…

... of course the MTU isn't the limit, it's the option size. I think this is fixed in SCTP (although not with a bit-map, hence presumably still wasting more space than TDLC). So, I guess that my questions above could already be (partially) answered by some SCTP papers that show whether encoding more SACK information makes SCTP significantly outperform TCP under realistic conditions.

---

From: <[email-masked]>
Date: Wed, 6 Feb 2013 11:43:08 +0000

ON various things:

We encounter popcorn noise when wireless devices are coming into range, e.g. a satellite low on the horizon when a pass is starting and link budget is at its worst (and we can aggregate popcorn into large holes, trading off a littlle retransmission on the fast downlink for space on the constrained uplink.) But this isn't a scenario where we would consider using TCP.

Did TDLC's bitmap and fixed packet size ever have to contend with changing MTU size? Bitmaps strike me as coniderable computational effort for a scenario where the links are as fast as possible, but the processors are slow, and fun for interop testing.

Yes, Saratoga has optional parts; it's clear no-one cares about carrying bundles, and bundle support is in a separate draft. UDP Lite is nice to have so that time is minimised on checksums, but implementations live without it - but thar's not enough spec to fill a separate draft. Streaming is needed by some.

Saratoga's SACK information can fill a packet, with a bit to indicate if this is all SACK info when sent, or more is needed in other packets. But implementations don't need to hold SACK state between the state supplied in acks, and can work with what comes in in a fresh packet, so not having complete information is not a problem.

Can you explain the OSI layer violation comment? If you want layering in a transfer protocol, you can run ftp over TCP. If you want OSI, well, use OSI; layering over UDP is pretty clean. tftp (an internet standard) is not dissimilar in layering, though rather slower in performance.

Lloyd Wood
http://saratoga.sf.net/

________________________________________
From: [email-masked] [[email-masked]] On Behalf Of Michael Welzl [[email-masked]]
Sent: 05 February 2013 09:28
To: tcpm
Subject: Re: [tcpm] Fujitsu and Saratoga

Dear Tony, all,

L. Wood, W. Eddy -

  xx        Record Control

    xx      Record Group Control

Notes -

Numeric Compression

Bit Map Compression

Tony

Anthony Sabatini
200 West 20th Street
Apt. 1216
New York, NY 10011

Phone: (212) 867-7179
Mobile: (917) 224-8388

_______________________________________________
tcpm mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/tcpm

Cheers,
Michael