Subject: [tcpm] Re: [Tsvwg] Re: [e2e] Are you interested in TOEs and related issues

From: Sunay Tripathi <[email-masked]>
Date: Thu, 4 Mar 2004 04:56:52 -0800 (PST)

Leonid,

<SNIP>

> > most modern NICs (including our 10GbE Adapter) do LSO. Also, for normal
> > frames LSO has quite dramatic effect at 10GbE rates, but for Jumbo
> > frames performance gain diminishes to ~7%.

> Do you have any insight on why you would see a degradation with Jumbo
> frames? As long as you are stateless, the jumbograms should show
> no degradation. 

I apologize. Jeremy Harris pointed out that I misread '~' for '-'. So
what you say makes sense now :)

Thanks,
Sunay

-- 
Sunay Tripathi
Senior Staff Engineer,
Solaris Kernel Networking,
Sun MicroSystems Inc.

email: [email-masked]		 Phone:	650-786-6007 (W)

----- End of forwarded message (env-from sunay) -----

-- 
Sunay Tripathi
Solaris Kernel Networking,
Sun MicroSystems Inc.

email: [email-masked]		 Phone:	650-786-6007 (W)

_______________________________________________
tcpm mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/tcpm

---

From: "Tal Anker" <[email-masked]>
Date: Tue, 9 Mar 2004 10:35:08 +0200

I think that what Marco meant was that when a congestion occurs, i.e., the incoming traffic rate is higher then the service rate (the interface speed) the potential for congestion also depends on the packet sizes. This is so since the queue is not comprised from "bytes" only, but rather of buffer descriptors... The queue length in many cases is the number of descriptors, and the queue length in bytes is indeed depending on the packet sizes enqueued. 

Thus, for Ethernet interfaces if an interface with speed L (b/s) is transmitting a 1500 bytes frame, and several other interfaces (say N interfaces) are sending traffic to that the same interface, then at the time L/(1500*8) seconds,  there can be either N packets - descriptors that needs to be queued, or N*(1500/64) in the worst case... the worst case is quite hard on some HW implementations as the number of descriptors in many cases is the number of max size packet buffers (packets are usually buffered in a fixed size buffers equal to the max frame size).

- Tal

-----Original Message-----
From: Craig Partridge [mailto:[email-masked]]
Sent: Monday, March 08, 2004 6:15 PM
To: Marko Zec
Cc: [email-masked]; [email-masked]; [email-masked]
Subject: Re: [Tsvwg] Re: [e2e] Are you interested in TOEs and related issues 

In message <[email-masked]>, Marko Zec writes:

>On Sunday 07 March 2004 23:06, Alexandre L. Grojsgold wrote:

>> Small packets get more buffer frames, but are also more rapidly sent.
>> The gueue grows fats, but shrinks fast too. It means that a burst of
>> small packets will not increase the packet loss, no mather the length
>> of the buffer (in packets).
>
>
>This theory is unfortunately wrong, because it ignores the fact that 
>when congestions do occur, traffic flows consisting of small packets 
>(regardless whether they are bursty or smooth) will use more buffer 
>slots then the flows carrying the larger ones, leaving less remaining 
>buffer slots available for queuing further packets / bursts.

I must be missing something here.  I think Alexandre's generally right.
If the buffers load and empty according to the amount of data in them,
then packet size isn't an issue.

The only situations in which this isn't true, that I can think off the
top of my head, are:

    * If the router has places where it moves all the bytes in the packet
      buffer, regardless of how much data the buffer actually contains.
      Since this would cause extraordinary internal bandwidth challenges,
      I doubt anyone does it for any but the lowest speed routers.

    * If the router has a deep pipeline and the time at each stage in the
      pipeline is set to equal the average packet time rather than the
      minimum packet time.  But most folks carefully design router pipelines
      for minimum packet times precisely because of this risk.

So clue me in -- what detail of router design am I missing?

Craig

_______________________________________________
tsvwg mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/tsvwg

---

From: [email-masked]
Date: Fri, 26 Mar 2004 14:57:48 -0700

Hmm, that's kind of funny. My thoughts are just the opposite: jumbo frames may be a necessary evil until TOEs become universal.

Pat

-----Original Message-----
From: [email-masked] [mailto:[email-masked]]On Behalf Of
<SNIP>

Bursty traffic may be ok for LAN, but is considered evil for WAN.
TOE doesn't suffer this problem (since it is stateful), and may be
a necessary evil until jumbo frames become universal.

Jerry

---

From: "Michael Christian" <[email-masked]>
Date: Thu, 4 Mar 2004 23:53:34 -0800

>One would hope TCP design would not be guided by Oracle licensing
quirks.

>> 2) On low end (1-2 CPU) x86 based machines, cost of adding a
processor
>>    is much higher than adding a TOE (I personally haven't verified
this).

>It's not obvious why this should necessarily be the case, given that
>it is likely that there will continue to be quite a bit more
>general-purpose CPUs made than TCP offload engines.

Well, current pricing, with a *huge* negotiated discount, when going
from a 2-proc to 4-proc box is on the order of $20+ grand.  Yes, you do
need to include the hardware, the OS (you need Linux AS vs ES which is
substantially more $$$), and potentially the Oracle licenses (which I
haven't included in the number).

Or, I could buy a $400 TOE. 

Yeah, CPU's are cheap in the 1-2 range, but when you are TOEing the line
between a 2-proc and a 4-proc, the difference is huge.

-F