Subject: Re: TCP's handling of IP precedence

From: Vernon Schryver <[email-masked]>
Date: Wed, 15 Sep 1999 07:57:12 -0600 (MDT)

> From: Vern Paxson <[email-masked]>
>
> > I would agree with Allyn that this sets a terrible precedent.
> > In the past, whenever change to TCP was proposed, the answer has been that 
> > there are millions of existing TCP instances out there so we could not 
> > change it.
>
> I'm sorry to say it, but the time to have voiced concerns over this change
> was during the IETF Last Call on the diffserv documents.  It is too late now
> to rescind the fact that the diffserv architecture allows for modification
> of the TOS byte in flight.

If you think about it for a momenet, that reasoning is obviously incorrect.
It assumes that everyone is required to read all IETF documents.  Since
getting anything changed during Last Call is at best difficult, it requires
that everyone participate in all working groups.  It also assumes that no
recent standards track document is ever wrong or ever needs significant
revision.  It assumes that DiffServ is more special and unchangable than
TCP as defined in RFC 793, which can be freely changed, despite the fact
that there are far more RFC 793 implementations and installations than
RFC 2474.  It provides a procedure for a small part of the IETF to make
arbitrary changes to protocols that most of the IETF cares about, simply
publish an RFC, sneak it through Last Call, and then make the claim of
fait accompli.

However, all of that observed, I'm not sure this problem requires 
major changes to TCP or DiffServ.

> We just sent in the I-D on the proposed change to TCP.  When it pops
> out, I'll send a note to tcp-impl (not end2end-interest, since I think
> this change is more closely related to TCP implementation) and we can
> continue discussion there.  (That said, those wanting an advance peek can
> fetch a copy from http://www.aciri.org/vern/draft-xiao-tcp-prec-00.txt.)

Instead of requiring changes to an unknown but apparently significant
number of existing TCP implementations and literally uncountable TCP
installations, why not strengthen the DiffServ words about upward
compatibility?  If the RFC 1349/RFC 1812 values of the IP TOS byte were
required to be get through unchanged (after possibly having been mapped
to other values and back again), would any existing combinations of
applications and host break?  For example, if the most common value, 0,
were required to be handed to the receiver if 0 were sent by the sender,
wouldn't things be a lot healthier?

Vernon Schryver    [email-masked]

---

From: "Smith, Allyn D" <[email-masked]>
Date: Wed, 15 Sep 1999 11:51:53 -0400

>publish an RFC, sneak it through Last Call, and then make the claim of

Nicely put, Vernon.  My sentiments precisely.  I don't think the precedence
change to TCP is really a big deal, but the philosophy and the reasoning
that is causing TCP to change bothers me: 
1. Emerging DiffServ technology redefines a protocol header field (dangerous
proposition at best).
2. It's discovered that header field redefinition in the DiffServ emerging
technology (not a standard yet) causes TCPs (a well established, long
existing standard) to send resets when they previously did not. 
3. DiffServ bug causes undesirable TCP behavior.
4. Fix to DiffServ: change all existing TCPs.

Very illogical.  DiffServ should have formalized the TCP change before
proceeding with the DiffServ implementation.  They are very naughty network
citizens.

_______________________________
Al Smith
Unisys
1.651.635.5769    NET 524.5769
[email-masked]
_______________________________

---

From: Vern Paxson <[email-masked]>
Date: Wed, 15 Sep 1999 09:16:17 PDT

Vernon Schryver writes:

> It assumes that everyone is required to read all IETF documents.

That's the point of Last Call - if you care about where the IETF is going,
you are supposed to read [email-masked] and see what is being
proposed.  You don't have to read all the documents, just the ones that
appear relevant.  The abstract of RFC 2474 is clear that the document is
about redefining the TOS byte.

> Since getting anything changed during Last Call is at best difficult

The WG raised the issue of what depends on the current structure of the TOS
byte at length, had considerable discussion, and no one brought up the TCP
incompatibility.  It's not as though it wasn't high profile - 400+ people
attend each diffserv meeting.  But even so, it was on the radar as a
significant issue, and I think it would have been treated very seriously
as a last call comment.

> compatibility?

One of the key questions is: is it really a significant number of TCP
implementations?  That's why this discussion should happen on tcp-impl.

> If the RFC 1349/RFC 1812 values of the IP TOS byte were
> applications and host break?

How do you know what to map them back to?  It's not a one-to-one mapping.

		Vern

---

From: Vern Paxson <[email-masked]>
Date: Wed, 15 Sep 1999 09:22:51 PDT

> proceeding with the DiffServ implementation.  They are very naughty network

That's not the sequence.  It's:

	1.  Emerging DiffServ technology redefines a protocol header field
	    (dangerous proposition at best).

	2.  This change is widely publicized and debated.

	3.  A formal IETF last call occurs saying "here's your last chance
	    to comment on this before it becomes a Proposed Standard."
	    No one raises the TCP problem.

	4.  Diffserv becomes a Proposed Standard.

	5.  6 months later it's discvered that the redefinition causes one
	    TCP to send resets.  The email describing this says

		  > As it turns out, this is an archaically (sp) known
		  > problem, with a pre-existing patch.
	    
	    and elicits a reply:

		  > MacTCP is *old*, and has many more problems than
		  > this.  Please tell your help desks that if they run
		  > across users still using MacTCP, that they advise
		  > them to upgrade to at least MacOS 7.5.3 (preferably
		  > 7.5.5), which is free, and includes Open
		  > Transport.

	6.  The question now is how many significant TCPs are actually
	    effected, and, accordingly, how to fix the problem.

- Vern

---

From: "Smith, Allyn D" <[email-masked]>
Date: Wed, 15 Sep 1999 10:48:50 -0600

Sorry for the sarcastic tone in my last message.  I don't, and can't, read
and debate everything that is published.  I was blind-sided by the
redefinition of the TOS byte, causing me to be defensive.  The sequence of
events I stated was as I saw them from my blind-sided perspective, however,
I am aware of the official standards track.  It appears that the TCP change
to ignore the precedence is probably the compromise required to resolve this
problem.

Al Smith

-----Original Message-----
From: Vern Paxson [mailto:[email-masked]]
Sent: Wednesday, September 15, 1999 11:23 AM
To: Smith, Allyn D
Cc: [email-masked]
Subject: Re: TCP's handling of IP precedence

> 1. Emerging DiffServ technology redefines a protocol header field
(dangerous

	    

- Vern

---

From: Joe Touch <[email-masked]>
Date: Wed, 15 Sep 1999 13:50:33 -0700 (PDT)

> Date: Wed, 15 Sep 1999 13:33:07 -0600 (MDT)
> From: Vernon Schryver <[email-masked]>
> To: [email-masked]
> > ...
> > 		  > this.  

I wouldn't describe implementing the existing standards correctly, 
as known at the time, and presently, as a problem.

> Agreed.
> However, on general grounds the fait accompli argument is completely
> inadmissable.  400 people participating in DiffServ meetings are not a
> majority of the IETF, even if majorities mattered, which they don't. 
> Even if you accept the history as you have stated it (which I don't quite),
> the onus for bearing fixing the problem lies with DiffServe, *NOT* TCP.
> If a painless change can be made to TCP, then that would be great, and
> would remove the burden from DiffServ.  If there is no painless TCP fix,
> then the pain *MUST* be borne by DiffServ.

I agree heartily. There are way too many groups in the IETF 
to follow all of what all groups are doing, even if they go
to last call.

It _must_ be the onus of the group proposing the change to
inform and seek input from other communities that the change
would affect.

If this document proposes to change IP TOS and doesn't address
how it affects existing standards, it is _they_ who should have
read, understood, and addressed these problems _before_ going to
proposed draft.

Sounds like time to unravel the standards track chain a little
and back up here.

Joe

---

From: Ross Harvey <[email-masked]>
Date: Wed, 15 Sep 1999 13:45:00 -0700 (PDT)

> of the TOS byte in flight.

Speaking unofficially from an embedded-SW supplier with a network stack,
I have to say "resolve this like engineers and scientists, not lawyers".

I mean, others could respond with "the time to have voiced concerns over
TCP TOS was in 1981". To attempt to squelch the technical complaints and
interoperation concerns with a lawyer-type "sorry, too late" response is
not, I hope, the way IETF debates will go in the future.

The "just upgrade" argument is dangerous. A relatively new GHS product
element is a TCP exclusively deployed in embedded systems. It doesn't have
the MacTCP problem, but it could have. It's also annoying to hear "just
upgrade" after all the hype about internet protocol appliances and networked
embedded systems.  If the IETF doesn't really mean that, come out and say
it: "It's only for desktops and servers running supported and upgradable SW."

Now, perhaps the diffserv steamroller is all worth it. My point isn't that
diffserv should change and not TCP, it's that this argument needs to be
decided on technical merits and not by hiding behind administrative
procedure, and that the "just upgrade" argument should be used with care.

How, exactly, do you propose to test compatibility with embedded systems?
It's going to be hard, which is why we usually just make compatible
extensions.

	Ross Harvey <[email-masked]>
	Green Hills Software

---

From: "Smith, Allyn D" <[email-masked]>
Date: Wed, 15 Sep 1999 17:07:45 -0400

All of the arguments presented on how DiffServ got to this point are good
ones, yet, through all of the formal processes, postings, and debates, this
one issue of TCPs handling of IP precedence still remains.  The result of
the formal process is that the DiffServ design
overlooked/ignored/didn't-fully-comprehend the impact of something and now
potentially many applications and TCP implementations may pay the penalty.
I'm not saying the formal process was wrong, but maybe there was something
missing from it.  I understand that it's probably too late to change the
DiffServ spec now, but perhaps we can learn something from this.

From what I can tell, several red flags went up that should have caused a
closer scrutiny on the impact of the DiffServ DS field:
1. Redefining a long existing protocol header byte.  This one is a big red
flag.  This should rarely be allowed.  Because the proper use of the
precedence bits was somewhat weakly defined, you are bound to get many
variations in the enforcement and handling of those bits.
2. Redefining a long existing protocol header byte defined by the DoD.
Bigger red flag.
3. When it was discovered that even one TCP checked the precedence, it
should have triggered the thought: "Hey, here's a TCP that enforces the
precedence validation so, I wonder if there might be others?"

I also monitor the end2end-interest and tcp-impl lists and don't recall
seeing any mention of the redefinition of the TOS byte.

Oh well.  Sigh...  I'm off to making sure my TCP's comply with the new
definition.

Al Smith

-----Original Message-----
From: Vern Paxson [mailto:[email-masked]]
Sent: Wednesday, September 15, 1999 11:16 AM
To: [email-masked]
Subject: Re: TCP's handling of IP precedence

		Vern

---

From: Scott Bradner <[email-masked]>
Date: Wed, 15 Sep 1999 19:07:16 -0400 (EDT)

> I think the only commonly used RFC 1349 values are 0 (extremely common),
> 0x10 (minimize delay), and 0x80 (maximize throughput). 

what about the other side of the coin - what level of support is
there in the current Internet infrastructure for these?

Scott

---

From: Vernon Schryver <[email-masked]>
Date: Wed, 15 Sep 1999 17:53:26 -0600 (MDT)

> From: Scott Bradner <[email-masked]>

> there in the current Internet infrastructure for these?

What do you mean by "current Internet infrastructure"?  While what the
500-1000 biggest routers in the Internet might do is interesting, there
are other important areas.  I count at least the following:
  1. corporate WANs
  2. LANs ranging from a handful to a gross of 802.3, FDDI, HIPPI, etc. nets
  3. embedded systems
  4. paths between hosts on The Intenret.

Surveying all of those sounds hard.  However, if you look at the first
4.4BSD-Lite CDROM, you'll find that the SLIP driver looks for 0x10 (low
delay).  I think my PPP code is far from the only PPP to support TOS
queuing.  I think I've seen requirements documents for Tbit/sec routers
that mentioned support for TOS queuing (but maybe they'll end up doing
DiffServ).  Cisco was slow to support TOS queuing, but I think it's been
in the IOS for at least 2 and maybe 4 years.

Note that where TOS queuing is most important in The Internet is the
opposite of where (I think) DiffServ is important.  People keep talking
about charging different $$$ rates based on DiffServ bits, and that matters
where streams from many customers meet.  TOS queuing matters most where
the queues are longest, and that happens where the link speed differences
are highest, at the edges in the remote access servers.  The owner of the
canonical Ascend PPP ISDN or modem box won't care how the customer's bits
are marked, but the customer does care about seeing telnet and NTP queued
ahead of FTP data.

---

From: Vernon Schryver <[email-masked]>
Date: Wed, 15 Sep 1999 18:15:32 -0600 (MDT)

> From: "Smith, Allyn D" <[email-masked]>

>            ....   I understand that it's probably too late to change the

If it is not too late to change RFC 793, then how can it be too late to
change RFC 2474?  Does the fact that RFC 793 is about 10 times older
and at least 10,000,000 times more widely implemented really mean that
RFC 2474 is the one that cannot be changed?   SHEESH!

Let's agree that there is a problem, and set out to solve it with the
minimum overall pain, but with the onus for bearing most of any unavoidable
pain on DiffServ.

] From: Grenville Armitage <[email-masked]>

] ...
] Seems to me that the 'failure' is a TCP-based application believing
] it is operating over a path that uses Precedence values e2e to
] provide service differentiation, when in fact the path does not.

That confuses the new notion of DiffServ with the old TOS definition. 
Under the old rules, it was perfectly reasonable for some routers in the
path to pay attention to the TOS bits while other routers ignored them.

] There are also three 'solutions' that dont require TCP mods:
]
] 	- Apps dont get smart - leave precedence as 000, and DS
] 	  regions default to re-mapping the lower 3 bits
] 	  of the DSCP to 000 on egress.

That solution is impossible, because there are already "smart,"
very widely deployed applications.  Please note again what you will
find if you search the nearest BSD source tree for "TOS_"

That solution also does not help purely dumb applications whose TCP
connections break because newfangled DiffServe routers in the path don't
all agree on how to map 0x00, and the path varies as a result of dynamic
routing.

] 	- Apps that want to be 'smart' take their chances.

That also doesn't help those dumb applications whose TCP connections
break.  

It also seems unfair to smart applications that have been deployed in
the last seven (7) years to use TOS queuing as suggested by RFC 1349.

] 	- Apps request 'precedence bits' that happen to be the same
] 	  as lower 3 bits of the DSCP the DS region will assign on
] 	  ingress.
]
] Although the third option begs some obvious questions, all in
] all it might be easier to start with the above 'usage rules' than
] to insist all TCP stacks are upgraded.

I'm not sure I understand what is meant by that.  The talk of PHB's and
router TOS bits in RFC 2474 also makes little sense to me.  The words in
RFC 2474 about mapping TOS values seems to be based on the obviously false
notion that only 0 and the "router" TOS values were ever used.

If you mean something that includes the idea that 0x00, 0x10, and 0x08
would be somehow carried unchange from host to host (but with possible
intermediate changes to other values), then that's what I've been trying
to say.

] There is also the ugly fourth option:
]
] 	- If a DS region sees packets arriving with non-zero
] 	  precedence values, the packets are tunneled to the
] 	  likely egress from the DS region, with only the outer
] 	  header having its TOS/DS field twiddled.

Let's not, if only because that doesn't help dumb applications using
0 whose TCP connections are broken when 0 is mapped to non-0.

---

From: Scott Bradner <[email-masked]>
Date: Thu, 16 Sep 1999 00:09:59 -0400 (EDT)

>         - intends that they change after a connection is established?

if the packet stream were going through a policing gateway that
remarks the field if the trafic load is above a limit the bits could
change during the session

see draft-heinanen-diffserv-trtcm-01.txt

Scott

---

From: Steve Alexander <[email-masked]>
Date: Wed, 15 Sep 1999 21:06:39 -0700

Changing the 5 TOS bits on the fly may work fine in implementations
that are picky about precedence, since precedence is defined as the
upper 3 bits; my old sendmail changed the TOS from LOW-DELAY to 
THROUGHPUT when switching from header mode to data mode. I think that came
from Dave Borman's UNICOS /etc/iptos (which I blatantly ripped
off after Dave posted the API and file format to the tcpip list).  Changing
the 3 precedence bits was the big no-no, at least if 
you're a strict constructionist of RFC 793.

-- Steve

> From: Joe Touch [mailto:[email-masked]]
> Sent: Wednesday, September 15, 1999 6:06 PM
> To: [email-masked]; [email-masked]
> Cc: [email-masked]; [email-masked]
> The idea behind TCP aborting the connection when
> the TOS bits change is that a property of the 
> connection has changed after being negotiated.

---

From: "Smith, Allyn D" <[email-masked]>
Date: Thu, 16 Sep 1999 10:08:58 -0600

My mail delivery has been rather untimely (I also think I may not have
received all my messages) so I'm not quite sure where the discussion is,
but, here are another couple of comments I hope are relative.

>minimum overall pain, but with the onus for bearing most of any unavoidable

Well excuuuuuusssssse me.  DiffServ made a potentially catastrophic blunder
here and should be changed if it makes sense.  What I really meant to say
was that I singularly do not have the power to retract the current
pre-standard existing DiffServ implementations that are causing problems nor
the power to change the existing spec.  In the mean time, I have to fight
the fires caused by those existing DiffServ implementations and succumb to
the redefinition of the TOS byte to ensure that clients TCP connections are
not abnormally terminated, thanks to DiffServ.  By conforming to the
redefinition, I also have to pray that I don't break any of their
applications that may be dependent on the precedence bits. 

Here's what I think should formally happen:

- The IETF should retract the current standing of the DiffServ spec.
- The proposed ID, "TCP Processing of the Precedence Field", should be
bandied about and debated.
- The outcome of the debate should determine the standing of the proposed ID
and current DiffServ spec.

Let's try to "put the horse before the cart".

Al Smith

---

From: "Smith, Allyn D" <[email-masked]>
Date: Fri, 17 Sep 1999 10:45:10 -0500

Here's some fuel for the fire.  As evidence that DiffServs twiddling of the
TOS byte bits may potentially be a big problem, Microsoft has a bug where
they fail to initialize the TOS byte causing them to send random TOS byte
values.  The net effect of this is the same as the DiffServ bit twiddling:
reset TCP connections.

If you're interested, you may view the Microsoft knowledge base article
describing this at:

   http://support.microsoft.com/support/kb/articles/q217/2/07.asp

Regards,

Al Smith