Subject: Re: [apps-discuss] Status of draft-ietf-appsawg-http-forwarded

From: Alexey Melnikov <[email-masked]>
Date: Sat, 01 Sep 2012 21:28:43 +0100

On 31/08/2012 06:42, Barry Leiba wrote:
> The IESG discussed draft-ietf-appsawg-http-forwarded-07 on this week's
> teleconference (Thursday).  The IESG evaluation record can be found
> here: http://datatracker.ietf.org/doc/draft-ietf-appsawg-http-forwarded/ballot/
>
> Stephen Farrell holds the main objections, but several other ADs agree
> with him or with variants of his position.  The main two points are
> these:
> (1) This is NOT a good thing to standardize, and we shouldn't.
> (2) This causes a serious privacy exposure, so *if* we do standardize
> it, we have to address that.
>
> After the telechat, Stephen switched his ballot from DISCUSS to
> ABSTAIN.  There are two other ABSTAIN positions.  There also remain
> three DISCUSS positions.  The document cannot progress unless all
> three of those DISCUSS positions are cleared *and* at least one of
> them moves to NO OBJECTION (well, *and* that assumes that both ADs who
> are on vacation come back and ballot NO OBJECTION as well).
Just to explain to people less familiar with IETF process: the ABSTAIN 
position is basically "I have a philosophical disagreement with the 
editors/WG about this document and nothing you can do can change my 
mind. However I am not blocking the document from being published". 
ABSTAINs are somewhat better than DISCUSSes (because they don't need to 
be addressed, although editors should try to improve text in order to 
possible address expressed concerns), unless there are too many of them.
> In short, this document is in serious trouble.
>
> To continue discussion, the IESG intends to have a teleconference with
> the document authors, and perhaps with other proponents.  If anyone
> would like to actively participate in that discussion, please contact
> me off list, and we'll see if we can add you -- we do need to keep it
> small.  Then we'll see if we have a way forward.

---

From: Mark Nottingham <[email-masked]>
Date: Sun, 2 Sep 2012 20:10:22 +1000

So, are we going to discourage use of the Received header in SMTP now?

On 31/08/2012, at 3:42 PM, Barry Leiba <[email-masked]> wrote:

> Barry
> _______________________________________________
> apps-discuss mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/apps-discuss

--
Mark Nottingham   http://www.mnot.net/

---

From: Ned Freed <[email-masked]>
Date: Sun, 02 Sep 2012 09:18:36 -0700 (PDT)

> So, are we going to discourage use of the Received header in SMTP now?

I have to agree with Mark here - the semantics of this field are similar to
Received: in email, and as such have similar utility as well as privacy
concerns (or perhaps non-concerns).

However, there's actually a closer analogy in the case of email submission.
When this is done through a proxy, it is in many cases essential that client
identity information be passed through so that appropriate security checks cna
be made. There is presently no standarized mechanism for this, and what happens
in practice is that RFPs are written requiring one of the various ad-hoc
mechanisms out there, e.g., Postfix's  XCLIENT command. It's pretty much the
same situation as exists with X-Forwarded-For, X-Forwarded-By, and
X-Forwarded-Proto. (And yes, you'd be correct in thinking that I beieve that
standards action is needed in this area.)

And this in turn points to a particularly pernicious form of magical thinking
the IETF often engages in that I have to say I find to be particularly
annoying: The notion that both the presence and absence of a standard for
something are imbued with special powers. In this particular case it's the
apparent belief that someone is going to say, "The IETF hasn't standardized
this so it must be a bad idea so let's not do it."

The reality, of course, is that all the lack of a standard for this has done is
created an interoperability mess. And by not having a clear set of rules for
doing it as well as a specification that details the security implications,
the present situation is far more likely to have caused security problems than
prevented them.

As for the IESG comments themselves, most these folks seem to have missed the
OPTIONAL nature of this field. Of course a proxy that is being operated in
whole or in part for purposes of providing client anonymity MUST NOT insert it!

If we really need to make such an obvious statement, then let's please just
make it: Put something like this in the security considerations:

    The entire purpose of a proxy inserting "Forwarded" field is to expose
    client identification to the server that would otherwise be unavailable.
    As such, proxies that wish to provide client anonymity MUST NOT
    insert a "Forwarded" field.

				Ned

---

From: Paul Hoffman <[email-masked]>
Date: Sun, 2 Sep 2012 11:06:05 -0700

> whole or in part for purposes of providing client anonymity MUST NOT insert it!

There are three outstanding DISCUSS comments, and at least one seems predicated not on that but on the lack of description for the value of this field. Adrian Farrel says:
=====
I think it is unfortunate that this document does not give any
explanation of why it is desirable to expose the information that is
lost during proxying and which the proposed extensions make available.

It is normal for work in the IETF to be presented along with some
motivations based on utility. While it is quite clear that the proposed
mechanism would work to provide the function described, it is a
mystery why any proxy would implement it or consider the extension
valuable.
=====

One or two paragraphs in the Introduction should suffice.

>    insert a "Forwarded" field.

+1

--Paul Hoffman

---

From: Ned Freed <[email-masked]>
Date: Sun, 02 Sep 2012 11:52:12 -0700 (PDT)

> One or two paragraphs in the Introduction should suffice.

OK, if Captain Obvious needs to fly to the rescue again, so be it. How about
something like:

    Uses of the information provided by "Forwarded" fields include, but are 
    not limited to: Logging, access control, and rate limits.

I'm sure there are more, but these all seem like obvious uses to me. (They 
definitely are how XCLIENT and similar mechanisms are regularly used in email.)

Of course once if we head down this path we may be risking arguments about
the appropriateness of using IP addresses or whatever for these purposes,
but the reality is they are used that way.

Come to think of it, this also provides a response to one of the other IESG
comments, which was a request to provide a means of providing only an address
prefix rather than an entire address. Consideration of only part of an address
is of course entirely appropriate in many cases, but that's really a job for
the agent consuming the information. Doing it at the proxy spreads network
configuration around unnecessarily and complicates management.

Whereas the only real benefit to partial addresses that I see would be in some
sort of situation where the proxy and server are separately administered and
the proxy trusts the server to the extent of being willing to provide part of
an address but not to the extent of providing the whole thing. I'm having
difficulty envisioning a real use case of sufficient utility to outweigh the
"bad idea" aspects.

				Ned

---

From: Paul Hoffman <[email-masked]>
Date: Sun, 2 Sep 2012 13:42:41 -0700

>    not limited to: Logging, access control, and rate limits.

Playing the good captain's sidekick, I would propose adding: "The uses of the "Forwarded" fields described here for HTTP are similar to the many uses that the "Received" fields in SMTP messages have provided for decades."

--Paul Hoffman

---

From: John C Klensin <[email-masked]>
Date: Sun, 02 Sep 2012 17:02:12 -0400

--On Sunday, September 02, 2012 13:42 -0700 Paul Hoffman
<[email-masked]> wrote:

>>    Uses of the information provided by "Forwarded" fields
>>    include, but are  not limited to: Logging, access control,
>>    and rate limits.
>> 
> 
> Playing the good captain's sidekick, I would propose adding:
> "The uses of the "Forwarded" fields described here for HTTP
> are similar to the many uses that the "Received" fields in
> SMTP messages have provided for decades."

Paul,

Because of some locus of control differences in the way proxies
are configured versus the way mail relays are, I think that,
beyond a certain point, the analogy is actually debatable.
Unless we want to have that debate for some reason, I'd suggest
leaving the lily ungilded.

   john

---

From: Stephen Farrell <[email-masked]>
Date: Sun, 02 Sep 2012 22:28:52 +0100

> So, are we going to discourage use of the Received header in SMTP now?

Actually I don't agree that XFF and Received are such a close analogy.
But I don't think we need to go there.

On 09/02/2012 07:52 PM, Ned Freed wrote:
> ... How about

Other uses that have been mentioned in discussion with the IESG
include:

- dealing with wrongly localized pages
- requests you get for end user identity from law enforcement
- IP-based lockout along the lines Wikipedia use

If we were really designing a solution for all that then I
don't think we'd end up with XFF. But we're being asked to
standardise essentially a fixed-up XFF when IMO we'd be better
off not doing that but figuring out what is needed (that
could be more privacy friendly) and then specifying that.
Or else just document what's already in use and how its
used.

I also reckon that different people have very different opinions
as to whether there are privacy issues with this spec. For
me that makes it unlikely that we should ask Captain obvious
to handle this one:-)

S.

PS: Since I'm abstaining on this, I'm not blocking it, but
I also didn't take a vow of silence:-)

---

From: Hannes Tschofenig <[email-masked]>
Date: Mon, 3 Sep 2012 09:41:56 +0300

Hi Willy, 

On Aug 31, 2012, at 10:54 AM, Willy Tarreau wrote:

>  It's just like saying that we should
> forbid eating with forks and knifes in restaurants because some people
> might use the knife to kill their noisy neighbour. Everything which is
> created has good and bad usages. 

This is a common response: engineers tell you that they are just producing tools -- they are not responsible for how they are used (even though they very well know how these tools are used). Those who deploy then argue that this is what has been given to them - the industry best current practice. Nothing can be done to improve the privacy (or security) of the system. 

I am trying to figure out what your views are. 

Are you saying that 
* there are no privacy requirements that need to be addressed? 
* you do not understand what those privacy requirements are? 
* you believe that these privacy requirements are not applicable? 

Ciao
Hannes

---

From: Hannes Tschofenig <[email-masked]>
Date: Mon, 3 Sep 2012 09:46:49 +0300

Hi Andreas, 

On Aug 31, 2012, at 1:52 PM, Andreas Petersson wrote:

> On Fri, 31 Aug 2012 10:39:54 +0300
> "Tschofenig, Hannes (NSN - FI/Espoo)" <[email-masked]> wrote:
> 
>> Eliot, 
>> 
>> The reverse proxy story is a way to make this look good and we know it
>> will be used otherwise. 
>> 
>> I don't know why aren't acknowledging that there are privacy concerns as
>> well even if they are inconvenient for an engineer. Just adding fluffy
>> and warm text does not make these concerns go away. I think we are
>> cheating here. 
>> 
> 
> Have you read the section "Privacy Considerations"?
> 
The privacy consideration section to me reads as follows: "Yes, we have heard about privacy concerns. This technology has been deployed already. As such, nothing can be done anymore. Sorry."

Ciao
Hannes

> /Andreas Petersson

---

From: Hannes Tschofenig <[email-masked]>
Date: Mon, 3 Sep 2012 09:52:26 +0300

Hi Eliot, 

On Aug 31, 2012, at 11:19 AM, Eliot Lear wrote:

> 
> On 8/31/12 9:39 AM, Tschofenig, Hannes (NSN - FI/Espoo) wrote:
>> Eliot, 
>> 
> I think you are concerned that an ill informed national policy might
> require the use of this header, such that it exposes PII to UNAUTHORIZED
> people.  It's not that you're thinking that enterprise administrators
> will have removed their brains and simply would allow this sort of
> exposure.  Is that correct?  If so, I understand and share this concern,
> and it should be documented that this is the wrong lever and can lead to
> serious security problems, if used on an outbound proxy.

Yes, that's one concern that I have. It is not that providers do these types of stuff because they have their brains removed (as you call it) but there are in their views very valid reasons to do so. Some of these reasons are driven by the desire to make business. In any case the lack of transparency, missing consent and missing ability for users to state their preferences is the problem. 

Btw, I also like the writeup in http://www.ietf.org/rfc/rfc3238.txt since it captures some of the concerns quite well. 

In general, there seems to be the excitement by some group of people (particularly those who work for network equipment manufacturers) to develop solutions that do not involve the end point (and consequently the user) in exchange. Those tend to raise privacy concerns, regardless of whether it is media recording, media transcoding, "security features", etc. 

> 
> As I understand it, you and Alissa just released a brand new related RFC
> on this topic relating to GEOPRIV, by the way.  Congratulations!

Thanks. 

Ciao
Hannes

> Eliot

---

From: Hannes Tschofenig <[email-masked]>
Date: Mon, 3 Sep 2012 09:55:31 +0300

Julian, this is the approach the IETF had done with lawful intercept work. 
See http://tools.ietf.org/html/rfc2804

On Aug 31, 2012, at 4:43 PM, Julian Reschke wrote:

> On 2012-08-31 15:32, Willy Tarreau wrote:
>> ...
>>> I did. I meant that just because something is deployed is not
>>> by itself sufficient reason to standardise (a less buggy version
>>> of) that thing.
>> 
>> It's not because it's deployed, It's because there is a real need and
>> without any standard, interoperability issues are created.
>> 
>> In turn, I don't see why addressing issues in something which already
>> exists should be wrong and discouraged.
>> ...
> 
> +1
> 
> The alternative is to take the draft, publish it elsewhere, and have the header field name registered anyway.
> 
> Best regards, Julian

---

From: Hannes Tschofenig <[email-masked]>
Date: Mon, 3 Sep 2012 10:12:28 +0300

Willy, 

see the entire discussion from an abstract point of view. When you design a solution there are different assumptions, preferences and requirements that get thrown together. The challenge is then to navigate through this solution space. 

From what I can tell your approach is as follows: 

"
If solution is deployed 
then 
     skip other requirements 
else 
     look at the requirements.
" 

Wouldn't it be useful to also look at some of these requirements that others are raising? You refer to security laws and those seem to matter for you but for some reason privacy laws do not count. That's an interesting way to pick requirements. (I personally prefer to generalize a bit and not to refer to specific laws.)

To illustrate my point let me pick a completely different example. You work on security and you know that credit cards aren't the world best authentication mechanism. Following your approach of standardization we would have to standardize credit cards. They are widely deployed and users seem to like them as well (because they are easy to use). Still, wouldn't it be useful to come up with a different mechanism that fulfills additional requirements (for security and privacy)? One could, for example, argue that the choice of selecting the "username" to be also the password wasn't such a great idea. Of course, if you work as a PCI DSS auditor, then that was clearly the greatest thing ever. 

Ciao
Hannes

On Aug 31, 2012, at 4:32 PM, Willy Tarreau wrote:

> On Fri, Aug 31, 2012 at 02:16:00PM +0100, Stephen Farrell wrote:
>>> All people working on the server side are already convinced about that,
>>> they're all facing the same interoperability issues and they need it
>>> for logging purposes (at least).
>> 
>> That last is kinda vague though isn't it?
> 
> How is the need for server-side logging something vague ? It's mandatory
> by law in a number of places (maybe everywhere ?) and is one of the
> basic security requirements. Most servers require this also to protect
> against abuse. I won't make the insult of presenting what purposes a
> server has of its visitor's IP address, we all have such requirements.
> 
>>>> I suspect its really a different document entirely that
>>>> documents current XFF implementation and deployment
>>>> practices. I reckon that text might be a good bit longer
>>>> than the entire current draft.
>>> 
>>> It would not be wise to document wrong practices of non-standard header
>>> fields which do not respect the standardized header field format. HTTP
>>> is crappy enough to avoid adding another exception on top of it.
>> 
>> I don't get how that squares with what you said earlier:
>> 
>>   "And I see this draft as an opportunity
>>    to remind implementors about what to do and what not to do.
>>    Right now, they're acting blindly, deploying their proxies
>>    without even knowing that they're sending xff to the whole
>>    world."
> 
> Right now this is something needed but not documented since not
> standardized. Some people even point me to wikipedia to explain
> to me what they're trying to achieve! So they tend to use default
> settings from the products they use, and there are not two similar
> behaviours across products so there is no good nor bad practice.
> I have already seen XFF being emitted by outgoing proxies and I
> had to explain the admin what to do to avoid this.
> 
>>>> I do get the point but I don't agree that just because a
>>>> buggy <<foo>> has been deployed means we ought to standardise
>>>> <<foo>> without those bugs.
>>> 
>>> I don't get your point here, sorry. I read it as "we failed somewhere,
>>> let's not fix it because it already exists" so I assume you meant
>>> something else :-/
>> 
> Regards,
> Willy

---

From: Tony Finch <[email-masked]>
Date: Mon, 3 Sep 2012 13:11:36 +0100

Paul Hoffman <[email-masked]> wrote:
> Playing the good captain's sidekick, I would propose adding: "The uses
> of the "Forwarded" fields described here for HTTP are similar to the
> many uses that the "Received" fields in SMTP messages have provided for
> decades."

I think that's going too far. Received: lines are mosly designed for human
consumption, whereas X-Forwarded-For: is for automated use. So spam
filters that parse Received: headers have a horrible time of it, and (as
Ned implied) MTAs usually use private SMTP extensions for the kind of
thing he listed above, not Received: headers.

Tony.
-- 
f.anthony.n.finch  <[email-masked]>  http://dotat.at/
Forties, Cromarty: East, veering southeast, 4 or 5, occasionally 6 at first.
Rough, becoming slight or moderate. Showers, rain at first. Moderate or good,
occasionally poor at first.

---

From: Barry Leiba <[email-masked]>
Date: Mon, 3 Sep 2012 09:22:12 -0400

> Those who deploy then argue that this is what has been given to them - the
> industry best current practice. Nothing can be done to improve the privacy
> (or security) of the system.

OK, so let's start by getting rid of that strawman: This spec does say
that this is an optional header field, and does say something about
when you might not use it.  If someone wants it to say more about
that, I'm sure that can happen, but see below.

> * you believe that these privacy requirements are not applicable?

I think, as I have said before, that we have two information flows
involved here:

1. client -> server
2. client -> *proxy -> server

In flow 1, the server already gets all the information that this
header field provides, without any use of this header field.

In flow 2, the first proxy gets all the information, but subsequent
proxies and the server get this same information about the previous
proxy.  This header field allows (doesn't require, but allows) the
proxies to relay the information about the client, along with the
client's request, *which the server would have anyway if the proxies
were not involved*.

Therefore, I think this *does not cause a privacy exposure* in the
general case of the proxy that's meant to simply relay the client's
request.  The fact that the proxies have caused some of that
information to be lost has been accidental (and often undesirable).
It has not been a *design* of the proxies, not a *reason* that the
proxies are there.

The *only* case where the situation is otherwise is when the proxy is
there *for the purpose* of hiding this information.  Where the user
*expects* the proxy to provide a privacy shield.

I believe that all this document needs to do about this whole situation is

a. Describe the privacy aspects of the information passed in this header field.
b. Describe the use cases for the protection of that information
(privacy shield use).
c. Make it clear that when those use cases are applicable, this
feature shouldn't be used (or MUST NOT, or whatever).
d. Make it clear that because the information can be passed on in
other ways, including through this feature, it's important to
establish a trust relationship with the proxy whenever a proxy is
being used for those use cases.

Barry

---

From: "Adrian Farrel" <[email-masked]>
Date: Sun, 2 Sep 2012 20:45:21 +0100

Hi Ned,

I appreciate the Captain's attempts to fly to the rescue, and for me (i.e. my
Discuss) this has started the process of getting what I want to see added to the
document.

I hope you won't think me too petty (just petty enough) when I say that telling
me what the feature will be used for is not the same as telling me that the
feature is needed. Hopefully, if this is so very obvious, it will not be hard or
painful to explain the use cases a little more fully and note that there are no
existing mechanisms that provide the desired function.

Cheers,
Adrian

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]]
> On Behalf Of Ned Freed
> Sent: 02 September 2012 19:52
> To: Paul Hoffman
> Cc: Barry Leiba; Apps Discuss
> Subject: Re: [apps-discuss] Status of draft-ietf-appsawg-http-forwarded
> 
> > > whole or in part for purposes of providing client anonymity MUST NOT
insert
> it!
> 
> > There are three outstanding DISCUSS comments, and at least one seems
> predicated not on that but on the lack of description for the value of this
field.
> Adrian Farrel says:

---

From: SM <[email-masked]>
Date: Mon, 03 Sep 2012 14:15:43 -0700

>It has not been a *design* of the proxies, not a *reason* that the

Some design decisions are intentional.  There can be side-effects; 
I'll call them unintentional decisions.  The topic under discussion 
is somewhat similar to what has been argued about for some DNS 
questions.  Although one can question the design, it is easier not 
to.  The "being there" is what I would call "fait accompli".  The 
IETF version is "can of worms".  The "fait accompli" is a subject of 
debate in non-IETF venues.  In I* venues one should know better than 
to ask about that.

The simplest form of the discussion is that:

   (a) There is an X- header field in wide use

   (b) The X- should be removed and the header field standardized
       to make the Internet work better.

I read the message from Hannes Tschofenig ( 
http://www.ietf.org/mail-archive/web/apps-discuss/current/msg06954.html 
) as what's the impact of doing that.  Once the information has been 
lost, it is a "fait accompli".  The header field could be added to 
the relevant registry and the specification published as an 
Informational RFC.  Alternatively, the WG could figure out how to 
make the proposal palatable for the Standards Track.

Regards,
-sm

---

From: Stephen Farrell <[email-masked]>
Date: Mon, 03 Sep 2012 23:07:11 +0100

So just for completeness...

> *expects* the proxy to provide a privacy shield.

As Barry knows, he and I disagree about this.

I don't think that the user's expectation or lack thereof
is relevant to the argument. For almost all users for almost
all the time, the user is unaware of what might or might
not be going on and we should not design our protocols
for only those users who know this header field exists.
(For users that do know about such things, the protocol
should also do the right thing, but the numbers there
are in the noise.)

Secondly, we also disagree as to whether there can be a
privacy "exposure" here. My argument is that without this
header field the proxies closer to the web server and the
web server would not know the user agent's IP, and with
this header field, they do know that.

If knowing the user agent's IP can ever be a privacy issue,
and it can, then that's a potential privacy "exposure"
for me.

S.

PS: "exposure" is in quotes because I don't think the
terminology there is quite right, but I don't have a
better term.

---

From: Ned Freed <[email-masked]>
Date: Mon, 03 Sep 2012 14:54:07 -0700 (PDT)

> leaving the lily ungilded.

It really depends on what analogy you're trying to draw. I don't think anyone
here is saying the semantics of Received: in email and Forwarded: in HTTP are
identical. But there is substantial overlap, and it is *absolutely* the case
that the privacy unicorn people seem to be hunting here, if real, would also
exist with Received:. Which I believe was Mark's point.

I'm a little unclear what this says about using Received: as an example though.
If there is a need to justify this field by appealing to something that has
been standardized and, far from being a security problem in actual deployment,
is in fact a significant asset, then so be it. Otherwise I guess I agree with
you that the debate is best avoided.

				Ned

---

From: John C Klensin <[email-masked]>
Date: Mon, 03 Sep 2012 21:34:48 -0400

--On Monday, September 03, 2012 14:54 -0700 Ned Freed
<[email-masked]> wrote:

>...
>> Because of some locus of control differences in the way
>> proxies are configured versus the way mail relays are, I
>> think that, beyond a certain point, the analogy is actually
>> debatable. Unless we want to have that debate for some
>> reason, I'd suggest leaving the lily ungilded.
> 
> It really depends on what analogy you're trying to draw. I
> don't think anyone here is saying the semantics of Received:
> in email and Forwarded: in HTTP are identical. But there is
> substantial overlap, and it is *absolutely* the case that the
> privacy unicorn people seem to be hunting here, if real, would
> also exist with Received:. Which I believe was Mark's point.

Ned,

Really no disagreement.  However, if your suggested text is
acceptable, then the document stands perfectly well with that
text and without any discussion of "Received:" or any other
analogies whether good, bad, or indifferent.  So, from my point
of view, there is no need to do whatever hairsplitting might be
necessary to figure out (and try to reach consensus on) what
analogy people are trying to draw.

Perhaps more to the point...

> I'm a little unclear what this says about using Received: as
> an example though. If there is a need to justify this field by
> appealing to something that has been standardized and, far
> from being a security problem in actual deployment, is in fact
> a significant asset, then so be it. Otherwise I guess I agree
> with you that the debate is best avoided.

So suppose we make the analogy and argue that, if "Received:" is
ok, then "Forwarded: is ok too.  If I were trying to ride around
on a privacy high horse while hunting that unicorn, I'd just say
"what was good enough thirty years ago isn't any longer, so the
analogy is absolutely useless".

And that takes me back to the comments I made last week.  It
seems to me that there are two possible cases here:

(1) "Forwarded" has significant value but needs to be used with
care and we have a good idea and rough consensus (independent of
the privacy issues) about how it should be defined.  

    or

(2) "Forwarded" is positively evil, whether because of privacy
issues or otherwise, and we should do nothing to encourage its
use.

Now, if the first case applies --as I suspect it does-- then,
IMO, your text (or something very much like it) is the right
fix, with or without appeals to Captain Obvious.  For the
reasons above, comparisons to "Received:" don't help much, even
if we can agree on the nature of the analogy and similarities
and differences in trust relationships.

If the second case applies but the field is in wide use and
likely to remain in wide use no matter what curses the IETF
passes around, then it is appropriate to publish as
Informational with explanations of the issues included and
whatever "don't use this but, if you must, be aware of these
considerations" text seems appropriate.

    john

---

From: Ned Freed <[email-masked]>
Date: Mon, 03 Sep 2012 19:10:44 -0700 (PDT)

> analogy is absolutely useless".

Fair point.

> considerations" text seems appropriate.

I don't have enough experience with web proxies at scale to assess the
situation there directly, but as I noted in my original response, we
have a very similar situation in email that has nothing to do with
Received: fields and it is causing real problems.

I would prefer (1) personally, but I can live with (2). That said, having
been through similar processes in the past, I'm worried that if approach
(2) is taken we'll just go full circle and end up back where we started
once the informational document is at this point.

				Ned

---

From: Eliot Lear <[email-masked]>
Date: Tue, 04 Sep 2012 06:51:52 +0200

Ned,

> exist with Received:. Which I believe was Mark's point.

As you state, there's a major difference between Received: headers and
THIS, so far as I can tell.  Received: headers actually provide a
security BENEFIT that is necessary to deal with a vast swathe of spam. 
We can't do without them because each SMTP server makes use of them (for
better or worse) to decide whether a message is authentic.  And so
there's a tradeoff, and it has been documented since at least RFC-2822
and known about long before then.  What's the real benefit here?  Not at
all clear.  First, it's not stated in this draft and 2nd even in this
discussion there has been disagreement.  I remind you that Willy
responded to Hannes stating that he thought the primary use of this
draft would be for REVERSE proxies, while your pro forma justification
was for regular proxies (I think there are examples of both in the doc). 

And predictably, I take issue with your "obvious" justification.  HTTP
already provides other capabilities to do this, and they provide far
more useful trace information, like User-agent and cookies, which
uniquely identify a client ACROSS IP addresses.  So maybe the first
thing to do is to make clear what the ACTUAL intended benefit, according
to the authors (if no one else) of revealing the IP address is.  [By the
way, the sort of leakage with cookies is far FAR worse than anything
we're talking about - insert animal/barn door analogy, here].

The "privacy unicorn" people know one thing: if an interface is
standardized it will be used for purposes that we don't intend or
desire, and you really may not like who mandates its use.  At the very
least that's worth a discussion – in the draft.

Eliot

---

From: Willy Tarreau <[email-masked]>
Date: Tue, 4 Sep 2012 08:31:06 +0200

Hi Hannes,

On Mon, Sep 03, 2012 at 09:41:56AM +0300, Hannes Tschofenig wrote:
> This is a common response: engineers tell you that they are just producing
> tools -- they are not responsible for how they are used (even though they
> very well know how these tools are used). Those who deploy then argue that
> this is what has been given to them - the industry best current practice.

But we're not reinventing anything here, that's the difference. Basically a
name was set on something which already exists, and is widely deployed.

> Nothing can be done to improve the privacy (or security) of the system. 

I don't get your point here.

> * you believe that these privacy requirements are not applicable? 

It's a mix of those. Barry has summarized this very well. What the draft
explains is who a device can inform components about information it could
not preserve due to its nature. This is the problem with reverse proxies.
Most of them cannot work in transparent mode due to operating system
limitations, so by nature they are forced to transform the traffic (at
least IP:port, sometimes protocol for v6->v4). For whatever is before the
device, nothing is changed. The user asked to access service X and it went
to the device which is presented to offer this service. This device relies
on other servers and has to pass the connection there. But since it cannot
pass it unmodified, it adds the missing data as HTTP header fields so that
for the server it's just as if the device in question was not there or was
more transparent.

You see, I'm maintaining a load balancer. The first thing *every* user asks
is "what should I do so that I don't lose the client's IP in my requests ?".
The device in front of the server clearly is the problem, and the draft aims
at solving this exact problem.

The problem you're seeing with privacy is something more related to outgoing
proxies and a bit different orthogonal. The problem exists only for users who :
  - know they are using a proxy, and
  - know how a proxy works, and
  - expect that the proxy will hide them

>From my experience, users 1) don't know what a proxy is, 2) don't know that
a proxy is in use, and 3) know for sure that "they leave traces" of their
activities when they're going to remote sites. You'll even notice that for
most users, their connectivity to the internet is very simple :
  - their PC
  - the company's firewall
  - the remote site.

I know some well informed users who deliberately use anonymizing proxies to
hide their activities.

This clearly means that when users don't know, they know that their activities
are visible, and when they care about this, they know how to hide them.

The real issue that came years ago with the default setting of X-Forwarded-For
on many proxies is that it used to reveal internal network addresses to the
outside world at a period where firewalls were starting to be deployed and
to hide such information. This is why nowadays it's strongly recommended to
disable this header on outgoing proxies. But even if it's provided, at no
point the user is betrayed, because no more information than what he provided
With its client is seen along the whole chain.

Regards,
Willy

---

From: Alissa Cooper <[email-masked]>
Date: Tue, 4 Sep 2012 10:26:29 -0400

Hi Willy,

On Sep 4, 2012, at 2:31 AM, Willy Tarreau wrote:
> activities when they're going to remote sites.

It would be great to see some data in support of claim #3. I think if you asked most users whether sites to which they give no affirmative information about themselves nonetheless retain data that can be used to subpoena information about their identities, they would say no.

I'll note that we had a discussion during WGLC of how user expectation factors in to the privacy impact of this header, and the draft now contains language in 8.3 that is pretty clear about where it ended up:

"Proxies using this extension will preserve the information of a
   direct connection.  This has an end-user privacy impact regardless of
   whether the end-user or deployer knows or expects that this is the
   case."

Alissa

---

From: Willy Tarreau <[email-masked]>
Date: Tue, 4 Sep 2012 19:16:36 +0200

Hi Alissa,

On Tue, Sep 04, 2012 at 10:26:29AM -0400, Alissa Cooper wrote:
> It would be great to see some data in support of claim #3. I think if you
> asked most users whether sites to which they give no affirmative information
> about themselves nonetheless retain data that can be used to subpoena
> information about their identities, they would say no.

Maybe it's different in other countries, but at least here in France, due
to various news reports of people getting caught due to their IP address
(or similarly their payment card), people are well aware that they "leave
traces" over the net. This is the term I often use from them and it's common
knowledge around my relatives, especially non-technies. Years ago, some ISPs 
even presented their dynamic address attribution as a way to limit address
based tracking. So I'm observing that this is something commonly known, but
I cannot provide data on this, I could as well ask for data showing the
reverse and it would not make more sense.

Regards,
Willy

---

From: Ned Freed <[email-masked]>
Date: Tue, 04 Sep 2012 10:05:50 -0700 (PDT)

> security BENEFIT that is necessary to deal with a vast swathe of spam.

Actually, Received: headers perform this function quite poorly due to widely
varying syntax. (In hindsight I'm baffled as to why we didn't introduce a
specific clause for this information instead of relying on comments.) As a
result most high end antispam systems that make use of IP addresses get this
information through other means, and as I noted in my original response when
proxies are involved the solutions that are used are quite close to what is
being proposed here for HTTP.

> better or worse) to decide whether a message is authentic.

Actually, in my experience that's incorrect. Some antispam systems don't make
use of client IP addresses at all and SMTP/SUBMIT servers often use IP address
information for other purposes than antispam, e.g., rate limiting,
internal/external classification, and auditing.

> And so
> and known about long before then.  What's the real benefit here?

The real benefit is pretty much the same as in SMTP/SUBMIT, which is why the 
analogy is a hell of a lot better than you seem to think. IP addresses are
routinely used for rate limiting, identifying sources of problematic material,
auditing, and so on

> Not at all clear.

I have to say I find this statement nothing short of flabbergasting. Surely you
realize that being able to track where stuff comes from is of *critical*
inportance is a vast number of applications? Surely you realize that not
everything on the web can or sould use some stronger form of authentication
and/or tracking?

> First, it's not stated in this draft and 2nd even in this

And the relevance of this is .... what, exactly?

> uniquely identify a client ACROSS IP addresses.

OK... So you want every web site out there requiring the use of cookies and/or
basing its decisions on a field that at best identifies a class of
browser/platform and which in my browser at least I can change at will with a
pulldown menu.

This argument is nothing short of surreal.

> So maybe the first
> to the authors (if no one else) of revealing the IP address is.

I have already suggested text to do this which, curiously, you haven't seen fit
to comment on. I suggest that rather than providing unconstructive criticism
like this you instead engage on getting the text into the document that
clarifies or addresses the issues that concern you.

> [By the

Indeed. So why are you arguing in favor of using this instead of standardizing
a less intrusive mechanism that is more than adequate in many cases?

> least that's worth a discussion – in the draft.

But they have apparently forgotten that the Internet routes around obstacles.

There is a serious problem here that people are going to solve one way or
another.

The IETF's choices are to either standardize something along with appropriate
guidelines as to when it should and should not be used and hope people take
note, publish an informational document that's less likely to have the desired
impact, or do nothing and just hope it all works out.

Of course the third course has consistently proved to work quite well in the
past. Oh wait...

				Ned

---

From: Alissa Cooper <[email-masked]>
Date: Tue, 4 Sep 2012 13:46:17 -0400

Perhaps there is data on both sides (e.g., [1]), which may be a good reason not to base design decisions on our collectively vague/intuition-based assumptions about users' expectations and awareness. 

[1] http://www.annenbergpublicpolicycenter.org/Downloads/Information_And_Society/20030701_America_and_Online_Privacy/20030701_online_privacy_report.pdf

---

From: Willy Tarreau <[email-masked]>
Date: Tue, 4 Sep 2012 20:13:03 +0200

On Tue, Sep 04, 2012 at 01:46:17PM -0400, Alissa Cooper wrote:
> Perhaps there is data on both sides (e.g., [1]), which may be a good reason
> not to base design decisions on our collectively vague/intuition-based
> assumptions about users' expectations and awareness. 

Please note that nobody was designing based on intuition, but only on
technical grounds based on an engineering approach which consists in
addressing issues caused by a category of devices which are used on
almost every web site nowadays. The issues is just that the presence
of these devices cause the client's address to be lost when the absence of
the device would have let it pass. Once again, nothing is done to report
more information than what was initially provided.

The assumption seems to be that such information might be used for wrong
purposes once the device is installed, while there is nothing apparently
wrong with it being already present in another form before the device is
installed.

Regards,
Willy

---

From: Eliot Lear <[email-masked]>
Date: Tue, 04 Sep 2012 20:38:25 +0200

Ned,

First, we disagree in our experiences and on facts, and my point is
*not* to kill the draft, but rather, as I wrote at the beginning, to
find the few sentences at the beginning so that the purpose of the
feature is clear– and it's *not*, and your logic isn't their logic.  And
then a clear discussion on privacy implications.  And there are privacy
implications.  Also, there should be some statements about the
limitations the approach (like the fact that sessions can and do outlast
IP addresses.

How hard could this be?

Eliot

---

From: Stephen Farrell <[email-masked]>
Date: Tue, 04 Sep 2012 19:38:37 +0100

> wrong with it being already present in another form before the device is

So even I don't see any issue with the reverse proxy case
where the same operator runs both the web server and
front-end.

But that's not all this draft supports. Can you comment on
the engineering reasons for supporting this in outbound
proxies?

S

---

From: Willy Tarreau <[email-masked]>
Date: Tue, 4 Sep 2012 20:45:34 +0200

Hi Stephen,

> the engineering reasons for supporting this in outbound

It's simple, sometimes you chain multiple proxies together. So you need for
instance that the first proxy (eg: the cache) forwards the IP information to
the second one (eg: url filter) so that it can be decided whether the guest
network has access to the whole internet or just the company's, or whether 
the boss' address is filtered or not.

Once again, any admin is very careful about disabling XFF on the border
proxy. The admin feels dumb even before his users when he sees his internal
addresses leak on the net.

Probably that one proposal could be to recommend that the feature SHOULD
not be enabled by default in implementations so that we're pretty sure
that whenever it's done, it's on purpose. This would probably solve most
of the issues expressed here with this header.

Regards,
Willy

---

From: Carsten Bormann <[email-masked]>
Date: Tue, 4 Sep 2012 20:46:03 +0200

On Sep 4, 2012, at 20:38, Stephen Farrell <[email-masked]> wrote:

> Can you comment on

I didn't have time to read the whole thread, but isn't a great discussion for one such reason given in 

[1] http://en.wikipedia.org/wiki/X-Forwarded-For
[2] http://meta.wikimedia.org/wiki/XFF_project

which you yourself cited in a previous message in this thread?
(I.e., differentiating reputation that may be assigned to the outgoing IP address of the proxy based on the previous origin of the request?)

Grüße, Carsten

---

From: Eliot Lear <[email-masked]>
Date: Tue, 04 Sep 2012 20:47:45 +0200

Stephen,

> the engineering reasons for supporting this in outbound

The issue here is this:

                                                 / ------(Q)
Host <A>    <PROXY> ---------------------->    <Reverse Proxy> /  ------(R)
                                                                       
                                                \ -------(S)

You want to provide state stickiness for a pair <A,S>, for instance, you
want something to identify <A>, which is what this header provides.  <A>
doesn't need to be an IP address, by the way.  The draft handles that part.

Eliot

---

From: Stephen Farrell <[email-masked]>
Date: Tue, 04 Sep 2012 19:50:05 +0100

Hi Willy,

> of the issues expressed here with this header.

That would be a definite improvement, I agree.

I'm not sure I agree it solves most issues, but nor am I sure
there's any way to solve most issues, given XFF is out there
already.

S.

---

From: Stephen Farrell <[email-masked]>
Date: Tue, 04 Sep 2012 19:54:49 +0100

> doesn't need to be an IP address, by the way.  The draft handles that part.

Its not clear to me why it ever needs to be an IP address
for (A,S) stickiness.

E.g. Encr(k,A-addr||stuff) for some k known only to
PROXY (or similar structures) would seem to me to be far
better. (Various forms of <<stuff>> might make sense.)

S.

---

From: Stephen Farrell <[email-masked]>
Date: Tue, 04 Sep 2012 20:09:50 +0100

> which you yourself cited in a previous message in this thread?

I did. I've not thought through whether I like that or
not but its clearly used. (Is that only by wikipedia or
more broadly?)

> (I.e., differentiating reputation that may be assigned to the outgoing IP address of the proxy based on the previous origin of the request?)

Yep. But why put the UA's IP address in there? (See my response
to Eliot.)

S.

---

From: Carsten Bormann <[email-masked]>
Date: Tue, 4 Sep 2012 22:42:10 +0200

On Sep 4, 2012, at 21:09, Stephen Farrell <[email-masked]> wrote:

> why put the UA's IP address in there

If the user has a public address and multiple proxies to choose from, you need coordination to arrive at the same token for the same user.
(If these are private addresses -- and, for multiple proxies, if coordination is achievable -- I certainly agree that there is no need to disclose that information.
But your scheme loses correlation between "close" addresses, which may be useful for assigning reputation to entire blocks of addresses.  
There are well-known schemes for "anonymizing" addresses that keep that correlation available, say, CryptoPan.)

Grüße, Carsten

---

From: Mark Nottingham <[email-masked]>
Date: Wed, 5 Sep 2012 12:02:53 +1000

> exist with Received:. Which I believe was Mark's point.

Indeed.

> you that the debate is best avoided.

One further point; I saw it asserted somewhere in this voluminous thread that Received is used for spam control, whereas X-Forwarded-For (and presumably Forwarded) are not. 

In fact, many sites use XFF as one input into their abuse protection systems; I've worked on a few. This includes data from headers generated by "reverse" proxies as well as "normal" ones.

Cheers,

---

From: Alexey Melnikov <[email-masked]>
Date: Wed, 5 Sep 2012 16:47:53 +0100

Hi Stephen,

> better. (Various forms of <<stuff>> might make sense.)

I am pretty sure that obfuscated-IP and obfuscated-port are exactly what you ask for and they were in the document for ages.

---

From: Alissa Cooper <[email-masked]>
Date: Wed, 5 Sep 2012 12:12:51 -0400

Hi Alexey,

> I am pretty sure that obfuscated-IP and obfuscated-port are exactly what you ask for and they were in the document for ages.

Except that there isn't any normative suggestion that the obfuscated identifiers should be preferred or used by default, or that client addresses should only be used if there is some address-related functionality required by the endpoint. 

Alissa

---

From: Stephen Farrell <[email-masked]>
Date: Wed, 05 Sep 2012 17:17:53 +0100

> I am pretty sure that obfuscated-IP and obfuscated-port are exactly what you ask for and they were in the document for ages.

Yes, there is an option for that. But it doesn't say when to use
it that I recall, nor why.

And if that'd work, then as I said above I don't see why the
UA's IP address is ever needed for the (A,S) stickiness that Eliot
asked about.

Which brings me back to not seeing a way in which this document
can be fixed without radical surgery, which folks don't seem to
be up for doing.

(I don't know if Carsten's point about adjacent addresses is a
real issue or not btw.)

S

---

From: Willy Tarreau <[email-masked]>
Date: Wed, 5 Sep 2012 18:18:01 +0200

Hi Alissa,

On Wed, Sep 05, 2012 at 12:12:51PM -0400, Alissa Cooper wrote:
> > I am pretty sure that obfuscated-IP and obfuscated-port are exactly what
> > you ask for and they were in the document for ages.
> > 
> 
> Except that there isn't any normative suggestion that the obfuscated
> identifiers should be preferred or used by default, or that client addresses
> should only be used if there is some address-related functionality required
> by the endpoint. 

OK, so now it looks clear that what the draft is missing is indication about
what the default behaviour should be in situations where it's not desired to
let internal addresses leak outside.

Possibly the draft authors can amend it now so that this issue comes to an end ?

Regards,
Willy

---

From: Alexey Melnikov <[email-masked]>
Date: Wed, 5 Sep 2012 17:36:20 +0100

Hi,

> Except that there isn't any normative suggestion that the obfuscated identifiers should be preferred or used by default, or that client addresses should only be used if there is some address-relate…

Ok, fair point. Text can be added to discuss this.

---

From: Andreas Petersson <[email-masked]>
Date: Wed, 5 Sep 2012 18:42:46 +0200

On Wed, 5 Sep 2012 18:18:01 +0200

> Possibly the draft authors can amend it now so that this issue comes to an end ?

It currently states:
  "A proxy that needs the ability to trace the source of requests, but
   does not want to leak the information further, can obfuscate the
   client address."

It is also mentioned in "Information Leak".

Should we change that "can" to an "is recommended to", or "should"?
Or have anyone further suggestion of text that should be added?

I'm currently writing an updated version for tomorrows meeting with
the IESG.
Not sure I will manage to fix this suggestion in this draft.
My intention is to submit this new version tonight (CEST).

 /Andreas

---

From: Willy Tarreau <[email-masked]>
Date: Wed, 5 Sep 2012 18:55:37 +0200

Hi Andreas,

> Or have anyone further suggestion of text that should be added?

Since Alissa talked about "normative suggestion", I think a SHOULD would be
more appropriate to tell the recommended behaviour depending on the use case.
Especially I think based on previous discussion with Stephen on this thread
that it should be added that forward proxies SHOULD not pass such information
in by default unless explicitly configured for this.

> I'm currently writing an updated version for tomorrows meeting with

Nice.

> Not sure I will manage to fix this suggestion in this draft.

Maybe not much is needed in fact. If everyone is happy with a SHOULD to
disable the feature by default on forward proxies, maybe just a few lines
are enough, I don't know.

Regards,
Willy

---

From: Martin Thomson <[email-masked]>
Date: Wed, 5 Sep 2012 10:25:58 -0700

>    does not want to leak the information further, can obfuscate the

What process do you recommend for this obfuscation?  rot13?

It really comes down to what the goal of the obfuscation is.  Is it
unlinkability?  I'd guess that you want the identifier be stable over
time, but structured so that the obfuscation is not reversible.
That's a tricky thing to get right.

---

From: Andreas Petersson <[email-masked]>
Date: Wed, 5 Sep 2012 22:48:13 +0200

On Wed, 5 Sep 2012 10:25:58 -0700

> That's a tricky thing to get right.

It is tricky indeed.

(the scope in this mail is forwarding proxies)

I'm no crypto expert, so I don't want to recommend any thing at all. :-)
If you REALLY want to hide your users, don't
use this and deal with the other problems that comes up. :-)

If the proxy have some sort of concept of sessions, maybe random() is
enough, and store the mapping between the client IP address and the
token somewhere.
That way users will both get a new token relatively often and should not
be traceable.
If that doesn't work, a static mapping between client IP address<->token
(token can still be just random) may be used. The draft however
recommends that the token should not be sticky for too long time.

You could also use some complete different scheme that happens to fit
your environment. I believe that when this is used, the type of the
token is in some way negotiated with the receiver of the request.

Maybe this draft should hint proxy servers to only forward the
client IP address to destinations that the proxy server actively choose?

 /Andreas

---

From: Carsten Bormann <[email-masked]>
Date: Wed, 5 Sep 2012 22:55:49 +0200

On Sep 5, 2012, at 19:25, Martin Thomson <[email-masked]> wrote:

> I'd guess that you want the identifier be stable over

If that is your only requirement, there are rather obvious solutions.
Apply a keyed MAC (e.g., HMAC) to the address, or use Stephen's scheme.
To achieve stability, you just have to keep the key constant (and secret).

If you also want to maintain the distance between different IP addresses a user might have available (e.g., keep addresses within a subnet related), there is a body of work on IP address anonymization that shares this goal.
A convenient entry point to that (from the point of view of anonymizing packet traces) is:
http://www.caida.org/projects/predict/anonymization/
"Summary of Anonymization Best Practice Techniques"
This also points to places where you can download working, debugged code.

Grüße, Carsten

---

From: Andreas Petersson <[email-masked]>
Date: Wed, 5 Sep 2012 23:19:43 +0200

On Tue, 04 Sep 2012 20:09:50 +0100
Stephen Farrell <[email-masked]> wrote:
> (Is that only by wikipedia or
> more broadly?)

From the top of my head, a (incomplete) list of software that supports
XFF:
squid, apache httpd, apache traffic server, varnish,
nginx, haproxy, cisco cache engine, lighttpd.

Google, uses it, lots of phone operators, for sure a lot of sites use
it in reverse proxies.
I am also pretty sure that lots of sites logs XFF and use it in
statistics.

Best regards,
 Andreas

---

From: Stephen Farrell <[email-masked]>
Date: Wed, 05 Sep 2012 22:24:16 +0100

> I am also pretty sure that lots of sites logs XFF and use it in

Sorry, I mean the wikipedia use-case, which I understand as
using XFF to separate the reputation of different end-users
behind the same proxy. I was wondering if its only wikipedia
that does that or if its more generally done.

I wasn't asking who implements XFF (which I think you answered
for me before anyway:-).

S.

---

From: Ned Freed <[email-masked]>
Date: Wed, 05 Sep 2012 15:38:57 -0700 (PDT)

> Ok, fair point. Text can be added to discuss this.

Agreed. How about adding something like this:

    Use of obfuscation techniques are appropriate when a need exists to be
    able to track connections back to the client only in some cases and only
    with the cooperation of the proxy operator.

It also might be appropriate to say something about this being more of a
forward proxy thing than a reverse proxy thing, but I'm not sure how to put it.

The security considerations also need to say something about information
leak from obfuscated identifiers themselves. For example, correlations are
possible if the same obfuscated id is always used for a particular IP address.

THe security considerations might also want to note the issues inherent
in the appearance of RFC 1918 addresses, which are definitely allowed. (Or
not - we certain don't make an issue of this in numerous othere places where
this happens all the time.)

And FWIW, having already explored this space a little, the main problem with
obfuscation techniques isn't knowing when to use them but rather figuring out
how they will actually be employed in an operational environment. That would be
a lot harder to address with text and I suggest not going there.

				Ned

---

From: Ben Niven-Jenkins <[email-masked]>
Date: Thu, 6 Sep 2012 01:12:20 +0100

Stephen,

> that does that or if its more generally done.

I don't know about "more generally" but customers of mine have certainly raised this use case but I've never dug into it to really understand what they're using it for because our product enables XFF by default so once I mention that they go off and implement their use case without needing anything more from me.

The difference with the wikipedia use case is that we build CDN products which in the context of this thread equates to a high performance reverse proxy which sits in front of our customers entire website, so without XFF our customers lose total visibility of all client IPs, compared to the forward proxy case where an origin server would only lose visibility of the subset of IPs sat behind a particular forward proxy.

More generally and not specifically in response to your mail, FWIW there are interoperability issues with XFF in the wild and I think publishing this document as a standards track RFC is a good thing. Not publishing it will not improve client privacy as proliferation of XFF will just continue so we're better off having a document specifying the header and discussing appropriate use(s) of it and how inappropriate use can lead to unintended consequences such as privacy leakage etc. If the current document does not do that adequately then lets wordsmith it so it does.

Ben

---

From: Alexey Melnikov <[email-masked]>
Date: Wed, 05 Sep 2012 16:59:33 +0100

> The privacy consideration section to me reads as follows: "Yes, we have heard about privacy concerns. This technology has been deployed already. As such, nothing can be done anymore. Sorry."

I think you just volunteered to help.

---

From: Carsten Bormann <[email-masked]>
Date: Thu, 6 Sep 2012 12:45:17 +0200

>    with the cooperation of the proxy operator.

In the wikipedia use case, obfuscation (in the sense of IP address anonymization) would also be appropriate, but neither leg of the predicate behind the "when" evaluates to true.

Grüße, Carsten

---

From: Alissa Cooper <[email-masked]>
Date: Thu, 6 Sep 2012 10:10:38 -0400

I'm looking forward to Andreas' draft update, but wanted to share some thoughts in the meantime.

There have been claims of interoperability problems with XFF in the wild. If there is indeed an interoperability problem to be solved, attempting to do so in a standards track RFC is often a good option. However, if such a standard were to be produced, successfully overcoming the interoperability challenge will require changes to deployed code. Therefore, there aren't strong reasons (or at least I haven't seen them articulated) for having the RFC track as closely as possible to the way XFF already works -- otherwise there would be no point in standardizing, because no one would have to change anything that's already out there.

This means that, assuming work towards an RFC continues, there is an opportunity to design a solution (or several) that attempts to meet the requirements of current consumers of XFF, but in a better and more privacy-friendly way that XFF does. Assuming that there is an actual interoperability problem and that XFF use will continue regardless of what happens with Forwarded, the standardization process seems like an opportunity to replace what is already out there with something better.

The use cases for XFF that have been mentioned on the list thus far (as summarized by Stephen) are:  

- logging
- access control
- rate limits

Each of these uses has different requirements for what kind of information needs to be present in the header, how unique it needs to be, and how permanent it needs to be. A privacy-oriented design process would start by asking which uses can be fulfilled with the least identifiable, unique, and permanent information made available, and work backward from there to begin making exceptions for sharing more identifiable, unique, or permanent identifiers. Probably some uses will require client IP address, but that's not a good reason to standardize a header that will always be used to share client IP even when it's not necessary. The draft would be a good place to recommend against such behavior.

The above analysis is likely much more difficult to conceptualize in the forward proxy case than in the reverse proxy case, but it's at least worth trying.

Alissa

---

From: Scott Kitterman <[email-masked]>
Date: Thu, 06 Sep 2012 10:14:44 -0400

On Thursday, September 06, 2012 10:10:38 AM Alissa Cooper wrote:
...
> There have been claims of interoperability problems with XFF in the wild. If
> there is indeed an interoperability problem to be solved, attempting to do
> so in a standards track RFC is often a good option. However, if such a
> standard were to be produced, successfully overcoming the interoperability
> challenge will require changes to deployed code. Therefore, there aren't
> strong reasons (or at least I haven't seen them articulated) for having the
> RFC track as closely as possible to the way XFF already works -- otherwise
> there would be no point in standardizing, because no one would have to
> change anything that's already out there.
...

To flip that around, it sounds like you are saying the only point in 
standardizing things is to cause changes in deployed code.  Surely that's not 
right?

Scott K

---

From: Alissa Cooper <[email-masked]>
Date: Thu, 6 Sep 2012 10:55:50 -0400

> standardizing things is to cause changes in deployed code.  Surely that's not 

No, in the abstract there can be plenty of reasons to standardize something. The reason given in this thread seems to be that different implementations of XFF are not interoperable. If people want to solve that problem, some of those implementations will have to change. If they don't, the interoperability problems will not go away.

To be clear, I haven't seen the interoperability problems with XFF specifically described and I don't know what they are or if they actually exist.

Alissa

---

From: Ned Freed <[email-masked]>
Date: Thu, 06 Sep 2012 08:21:34 -0700 (PDT)

> In the wikipedia use case, obfuscation (in the sense of IP address anonymization) would also be appropriate, but neither leg of the predicate behind the "when" evaluates to true.

Feel free to suggest text.

				Ned

---

From: John C Klensin <[email-masked]>
Date: Thu, 06 Sep 2012 12:10:34 -0400

Alissa,

One observation on a very specific part of your comment, which I
think may be critical...

--On Thursday, September 06, 2012 10:10 -0400 Alissa Cooper
<[email-masked]> wrote:

>...
> Assuming that there is an actual interoperability problem and
> that XFF use will continue regardless of what happens with
> Forwarded, the standardization process seems like an
> opportunity to replace what is already out there with
> something better.

If "XFF use will continue regardless of what happens with
Forwarded"

   then

the standardization process is largely irrelevant unless there
is a clear commitment to change a significant fraction of those
non-interoperable implementations to match the standard.   That
is true whether we make one of the existing implementations the
standard (presumably the "least bad" one on whatever scale that
is evaluated) or whether we do something new and improved.  

Note that a "something better" replacement is substantially
guaranteed to be be non-interoperable with the installed base.
I think it is a fair summary of our general experience that such
replacement is likely to be deployed in the marketplace and
widely accepted only if it hugely better and/or it fills a need
that almost all users of existing methods are feeling and feel
is very pressing.

That could change if there were general consensus that one of
the versions is better than the others (i.e., that there was
reason to believe the others would adapt to it if it were
standardized) or that the interoperability problems were causing
so much trauma that there was demand for a replacement with
"something IETF-approved" or "something new".  I haven't read
every work of every message on this thread, but I haven't seen a
strong case made for either of those positions or strong
commitments to them.

Standardization may still be worthwhile if it provides a target
for new implementations and an excuse to get the facilities, the
issues, and the points of non-interoperability documented.   The
subset of that role which is just to document what is going on
and what the issues are could be served as well by an
Informational document.  But let's be careful about assuming
that "something better" will take off merely because there are
interoperability problems with existing (and likely continuing)
practices (if, in fact, that is the case).

best,
   john

---

From: Andreas Petersson <[email-masked]>
Date: Thu, 6 Sep 2012 18:30:34 +0200

On Thu, 6 Sep 2012 10:10:38 -0400

> I'm looking forward to Andreas' draft update, but wanted to share some thoughts in the meantime.

It is already submitted. One thing I forgot to add was to
encouraging this feature to be off by default, that will be added in
the next revision.

> The above analysis is likely much more difficult to conceptualize in the forward proxy case than in the reverse proxy case, but it's at least worth trying.

We believe that if the new header field is too far away from XFF it
will not be used to solve the problems that arise when the client source
IP address is unavailable.

A change in semantics will require a redesign of all involved
components, while a change in syntax requires a parser code patch.

Best regards,
 /Andreas

---

From: Andreas Petersson <[email-masked]>
Date: Thu, 6 Sep 2012 18:40:20 +0200

On Thu, 6 Sep 2012 10:55:50 -0400

> To be clear, I haven't seen the interoperability problems with XFF specifically described and I don't know what they are or if they actually exist.

Some examples of interoperability issues would be:
 * IPv6 format
 * Wrong order, or simply not being aware that the
   XFF can contain a list of tokens
 * Synchronization between different X-Forwarded-* header fields.

I do believe that this draft addresses these issues. I also think
that the initial discussions on the http-mailing list involved enough
proxy- and http-people to ensure that it solves these issues in a
reasonable way.

Best regards,
 Andreas

---

From: Ned Freed <[email-masked]>
Date: Thu, 06 Sep 2012 14:05:44 -0700 (PDT)

> is evaluated) or whether we do something new and improved.

I really don't think you mean this because by this rule, MIME would have been
standardized. (At the time there were at least five deployed systems for
multimedia mail and not only was there no commitment to change from any of
them, there was significant opposition to it. At least two of those are still
in use today, as a matter of fact.)

> that almost all users of existing methods are feeling and feel

Forwarded: definitely offers capabilities XFF does not have. Are those plus
the promise of better interoperability enough to cause adoption? I have no
idea.

I personally would have preferred a specification that just standardized XFF.
But the consensus went the other way, and one of the things I try to avoid is
second guessing such decisions.

> strong case made for either of those positions or strong

Well, if we're going to go by past experience, I'll just point out that the
predictive capability of IETFers engaging in such discussions is highly
questionable. So if you're looking for assurances as to the future, I think tea
leaves provide a better alternative: At least with them you get a nice cup of
tea out of the deal.

> practices (if, in fact, that is the case).

On this we agree. But I'll also note that another thing having a standard
will do is instruct people as to both the utility and the risks of such
mechanisms.

				Ned

---

From: John C Klensin <[email-masked]>
Date: Thu, 06 Sep 2012 18:13:48 -0400

--On Thursday, September 06, 2012 14:05 -0700 Ned Freed
<[email-masked]> wrote:

>> is a clear commitment to change a significant fraction of
>> those non-interoperable implementations to match the
>> standard.   That is true whether we make one of the existing
>> implementations the standard (presumably the "least bad" one
>> on whatever scale that is evaluated) or whether we do
>> something new and improved.
> 
> I really don't think you mean this because by this rule, MIME
> would have been standardized. (At the time there were at least
> five deployed systems for multimedia mail and not only was
> there no commitment to change from any of them, there was
> significant opposition to it. At least two of those are still

I assume you meant "not been standardized".  The difference is
that, at the time MIME was done, there was significant
perception of a need for an improved multimedia mail (and
non-ASCII mail) model and a similarly strong perception that the
existing/ deployed systems weren't up to the job as others saw
it.  That perception certainly wasn't universal, nor was there
universal agreement that MIME was the right approach.  As you
say, some of the predecessor systems are still in use.  But
there was a sufficient perception of need to result in careful
consideration of the MIME approach and its fairly broad adoption.

I haven't seen that level of perceived need with the proposals
that are now coming out.  Perhaps it is there and I've just lost
it in the noise.  What I have seen (and I'm not accusing Alissa
of this) is a lot of concerns and complaints about functionality
that is not present (especially with regard to privacy
protection) without either a clear explanation of why what
people are trying to use "forwarded" for isn't necessary.  That
would be ok except that it is not clear to me that we have a way
to provide solid privacy protection other than "if you are
worried about privacy, or need to consider this capability in a
privacy-sensitive environment, don't use this".   That may be a
fine answer, but, if it is the acceptable one, we should get a
statement about it into the document and move on.

What I'm objecting to -- and I may still not be saying this very
well-- is any belief that, if there are interoperability
problems now, either

	 -- An IETF choice of one set of options over another
	without a really strong rationale as to why that is
	superior to other choices   or
	
	 -- A completely new IETF solution that is still
	recognizably similar to the existing approaches

can be counted on to automagically be adopted and take off.  I'd
expect a good deal of reactions along the lines of "why should
we bother with that when it requires changes to our installed
base and doesn't provide significant additional functionality".
For MIME, we had an answer to that question (actually, several
separate ones -- there was significant additional
functionality).   A better example, sadly, has been IPv6 and the
long uphill struggle to get it adopted and widely deployed.

>> I think it is a fair summary of our general experience that
>> such replacement is likely to be deployed in the marketplace
>> and widely accepted only if it hugely better and/or it fills
>> a need that almost all users of existing methods are feeling
>> and feel is very pressing.
> 
> Forwarded: definitely offers capabilities XFF does not have.
> Are those plus the promise of better interoperability enough
> to cause adoption? I have no idea.

I don't either.  All I am trying to do is to encourage people to
ask that question clearly and with a maximum of objectivity and
minimum of passion.  And also I'm questioning whether the IETF
has heard demand for this beyond "we know how to make it better,
therefore we should".  I definitely do not know the answer to
that question either, but I'm hoping that there are some clear
answers out there.

> I personally would have preferred a specification that just
> standardized XFF. But the consensus went the other way, and
> one of the things I try to avoid is second guessing such
> decisions.

Agreed.  But I think we should be clear-headed about what we are
getting into.

>> standardized) or that the interoperability problems were
>> causing so much trauma that there was demand for a
>> replacement with "something IETF-approved" or "something
>> new".  I haven't read every work of every message on this
>> thread, but I haven't seen a strong case made for either of
>> those positions or strong commitments to them.
> 
> Well, if we're going to go by past experience, I'll just point
> out that the predictive capability of IETFers engaging in such
> discussions is highly questionable. So if you're looking for
> assurances as to the future, I think tea leaves provide a
> better alternative: At least with them you get a nice cup of
> tea out of the deal.

No disagreement other than to again point out that we've had
much better success deploying protocols that fill a gap (whether
recognized at the time or not) than we have with protocols that
try to replace existing ones by making a small incremental but
incompatible change.   That is _not_ a prediction about any
particular case and there are certainly exceptions to the
general pattern.  But it is a pattern of which I think we should
be aware.
 
>> Standardization may still be worthwhile if it provides a
>> target for new implementations and an excuse to get the
>> facilities, the issues, and the points of
>> non-interoperability documented.   The subset of that role
>> which is just to document what is going on and what the
>> issues are could be served as well by an Informational
>> document.  But let's be careful about assuming that
>> "something better" will take off merely because there are
>> interoperability problems with existing (and likely
>> continuing) practices (if, in fact, that is the case).
> 
> On this we agree. But I'll also note that another thing having
> a standard will do is instruct people as to both the utility
> and the risks of such mechanisms.

And that is why I definitely favor publishing something in this
area, something that addresses those topics in a clear and fair
way.  I prefer that it be done with a standard and would prefer
that even if we somehow knew in advance that the standard would
not deploy.  But if we can't get agreement on a standard, I
strongly favor an Informational document that describes the
status quo, utility, and risks so that the knowledge doesn't get
lost.

   john

---

From: Ned Freed <[email-masked]>
Date: Thu, 06 Sep 2012 16:27:27 -0700 (PDT)

> I assume you meant "not been standardized".

My bad.

> The difference is

But "perception of need" wasn't part of your proposed criteria. Your
criteria was "commitment to change", and that was what I was responding to.

> people are trying to use "forwarded" for isn't necessary.

Once again we are in agreement.

> That

I think that's exactly what is needed, and I have attempted to propose
text to that effect.

	

> can be counted on to automagically be adopted and take off.

I agree with the sentiment, but what I'm not seeing are all the people that are
in fact making these assumptions. Standardizing something that replaces
existing ad-hoc mechanism is always risky. And you can argue till the cows come
home about whether or not you've chosen the right tradeoff.

I've already stated that I have no idea if this is the correct tradeoff or not.
If someone has evidence that it is or isn't the correct one, I'd love to hear
it. (The only thing I've seen that comes close is one position posted by
someone who builds product in this area saying this is a good thing.)

>  I'd

Very different kettle of fish, I think. Extrapolating from what it costs me to
add support for a new field to our product, this should amount to litle more
than an option setting plus a few lines of code on either end. Testing is  also
pretty easy. In contrast, supporting IPv6 easily required several options and
took 20X as much code as that in our application alone, never mind the small
matter of network stack support. (You don't want to get me started on the topic
of what's involved in IPv6 testing...)

> status quo, utility, and risks so that the knowledge doesn't get

Complete agreement on all points. What worries me is that the outcome of this
exercise will be nothing, that's far worse than almost any other result.

				Ned

---

From: Andreas Petersson <[email-masked]>
Date: Fri, 7 Sep 2012 12:39:02 +0200

On Thu, 06 Sep 2012 18:13:48 -0400
John C Klensin <[email-masked]> wrote:
> it in the noise.  

> What I have seen (and I'm not accusing Alissa

I think the non present functionality is requested by the people that
are loudly fighting for privacy, while the implementors are quite
satisfied.

> that question either, but I'm hoping that there are some clear

We at Opera see the need for it, and will implement it. Willy
from HAProxy seemed interested, so does Apache traffic server. PHK from
varnish was much involved in the technical discussions in the
beginning, and says it can easily be implemented in varnish.
(Note, I don't speak for them, it is what I've perceived).

I also got the impression from the discussions on the http-list that
it was a welcome effort to clean up in the X-Forwarded-*-land. The last,
months however, the discussions have been mostly about how to split the
ABNF into different sections, and the privacy implications.

I believe that the interest for such discussions is pretty low from
those who are interested in implementing this.

Note: I don't say these discussions are wrong or unnecessary, just that
everyone doesn't have the interest to participate in these discussions
over and over again. :-)

> Agreed.  But I think we should be clear-headed about what we are

The initial version, draft-petersson-forwarded-for-00 did just that.
Interest for extending it came up immediately.

Best regards,
 Andreas Petersson

---

From: Andreas Petersson <[email-masked]>
Date: Fri, 7 Sep 2012 12:39:36 +0200

On Thu, 06 Sep 2012 16:27:27 -0700 (PDT)
> I think that's exactly what is needed, and I have attempted to propose

I think the text is pretty clear on that point. But, I have got the
perception from some people that this is not enough.
If you think the text do not clearly say "if you are worried about
privacy, or need to consider this capability in a privacy-sensitive
environment, don't use this", please suggest concrete changes to the
document.

I have used some of you suggested text, Ned, thanks. :)
Is there further changes you suggest?

Best regards,
 Andreas Petersson

---

From: Barry Leiba <[email-masked]>
Date: Fri, 7 Sep 2012 10:08:39 -0400

...
> environment, don't use this", please suggest concrete changes to the

This is really the key point we're in right now, so let me be clear
about it to everyone:

1. Andreas is working with Adrian Farrel to resolve his DISCUSS, which
involves sufficient text in the introduction to explain what's being
done and why.  Adrian thinks the most recent version is going in the
right direction, and he is preparing some proposed text to try to
finish it up.

2. Andreas has an outstanding change that will go in at the same time
as (1), which recommends that implementations have the use of
"Forwarded:" turned off by default, requiring explicit configuration
to enable it.  This should resolve Stewart Bryant's DISCUSS.

3. If there are any further concerns about how much the text covers
the privacy issues, we need to address those, and we will.  But they
must come with specific proposed text.  It's certainly fair for some
of that text to include something like "[...insert explanation
here...]" if you don't feel you're the right person to craft that
section of text, but you have to give Andreas something clear that he
can work with so he knows specifically *what* you want added or
changed *where*.

Alyssa, can you make a specific suggestion for Andreas to take?
Hannes, do you have anything to add here?  John?  Others?

I'd like to get this wrapped up very soon into a version that we can
at least agree addresses the privacy issues adequately, even though
some would wish that we were not publishing this as a proposed
standard.

Barry, responsible AD

---

From: Ned Freed <[email-masked]>
Date: Fri, 07 Sep 2012 08:42:37 -0700 (PDT)

> feature is clear– and it's *not*, and your logic isn't their logic.  

Which is why I have suggested text. Which you still haven't commented on,
although apparently events have passed us both by and text has been added
to the document to address this.

> And then a clear discusison on privacy implications.

I suggested some text for that as well.

> Also, there should be some statements about the

> How hard could this be?

Pretty darned hard when you refuse to engage on actual text.

I freely acknowledge that the text I've suggested may be completely and total
shit but it's hard to improve it without feedback.

				Ned

P.S. You will notice I did not respond as to your assertions about facts and
experiences. This is not because I have no respone, but rather because I now
deem such a response to be irrelevant.