Subject: [v6ops] draft-palet-v6ops-he-reporting-00

From: JORDI PALET MARTINEZ <[email-masked]>
Date: Thu, 16 Nov 2017 13:58:45 +0800

And last one … I’ve already talked with Carlos, the co-author, in order to work on the path forward and how to make sure that security/privacy are correctly taken into account in the document.

Also, I think we got the clear message that the reporting should be done with both IPv6 and IPv4 (as fallback by means of happy eyeballs).

Does anyone have a suggestion about how to “secure” syslog, or an alternative way for doing the reporting?

Anything else that we are missing in the document so we can improve it?

Regards,
Jordi

**********************************************
IPv4 is over
Are you ready for the new Internet ?
http://www.consulintel.es
The IPv6 Company

This electronic message contains information which may be privileged or confidential. The information is intended to be for the exclusive use of the individual(s) named above and further non-explicilty authorized disclosure, copying, distribution or use of the contents of this information, even if partially, including attached files, is strictly prohibited and will be considered a criminal offense. If you are not the intended recipient be aware that any disclosure, copying, distribution or use of the contents of this information, even if partially, including attached files, is strictly prohibited, will be considered a criminal offense, so you must reply to the original sender to inform about this communication and delete it.

---

From: Jen Linkova <[email-masked]>
Date: Fri, 17 Nov 2017 17:36:13 +1100

> Also, I think we got the clear message that the reporting should be done with both IPv6 and IPv4 (as fallback by means of happy eyeballs).

If it's syslog, it's UDP. How are you going to implement HE??
Using IPv6 does not sound very helpful as IPv6 might be broken. Using
IPv4 introduces a significant risk of those logs to be sent to anyone
who advertises that
/24.

I personally do not see much value in all that logging in general but
it might be just me. However the solution currently proposed in the
draft sounds very scary.

-- 
SY, Jen Linkova aka Furry

---

From: Fred Baker <[email-masked]>
Date: Fri, 17 Nov 2017 14:50:43 +0800

SYSLOG was also identified in the session as a privacy risk (the system is logging its own identity and the identity or address of its destination). The more I think of it, I suspect it is also an effective DDOS attack if there are any number of systems reporting.

> On Nov 17, 2017, at 2:36 PM, Jen Linkova <[email-masked]> wrote:
> 
> _______________________________________________
> v6ops mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/v6ops

---

From: JORDI PALET MARTINEZ <[email-masked]>
Date: Fri, 17 Nov 2017 14:54:01 +0800

I believe most of the problem, according to my experience is beyond the access link, so that will ensure the reporting.

IF the access-link is broken, definitively that ISP has a bigger issue.

Furthermore, when considering, initially this draft, I was looking to a long term, when the access will be IPv6-only, that’s why for me make sense to look into IPv6 reporting only.

However, if we still believe it make sense to do the reporting with dual-stack, and that syslog is not the best way to report, then we can either find another solution (any other commonly used reporting way that is common in networks), or if we believe syslog is the way, we can change it to TCP to allow dual-stack fall back with HE, right?

Regards,
Jordi

---

From: Jen Linkova <[email-masked]>
Date: Fri, 17 Nov 2017 19:19:39 +1100

> I believe most of the problem, according to my experience is beyond the access link, so that will ensure the reporting.

Our experience may vary. I've been seeing exactly the opposite.
If the issue is within the operator's domain then that operator (if
they care) should be easily able to monitor it (actually the hardest
part is to monitor
the access, especially for ISPs but even for enterprises...)
If the issue is beyond their controlled domain, it will be totally
non-actionable alert with almost zero value.

> IF the access-link is broken, definitively that ISP has a bigger issue.

which you are not trying to solve, correct?

> Furthermore, when considering, initially this draft, I was looking to a long term, when the access will be IPv6-only, that’s why for me make sense to look into IPv6 reporting only.

I need more coffee...Why are we talking about happy eyeballs if the
access is Ipv6-only? Or what do you mean by 'access; in this case?

> However, if we still believe it make sense to do the reporting with dual-stack, and that syslog is not the best way to report, then we can either find another solution (any other commonly used repo…

I feel like we need to stop asking. 'how?' before 'should?' ;)

---

From: JORDI PALET MARTINEZ <[email-masked]>
Date: Sat, 18 Nov 2017 04:00:12 +0800

Hi Jen,

Below in-line …

Saludos,
Jordi

-----Mensaje original-----
De: Jen Linkova <[email-masked]>
Responder a: <[email-masked]>
Fecha: viernes, 17 de noviembre de 2017, 16:20
Para: Jordi Palet Martinez <[email-masked]>
CC: "[email-masked]" <[email-masked]>
Asunto: Re: [v6ops] draft-palet-v6ops-he-reporting-00

[Jordi] The presenter from Cisco also agreed with me that the problem is more often at the destination (sometimes at the transit). In general, at least the cases and customers I’ve, monitoring all the internal network is easy (even the access), the problem is to know if the path from any part in the internal network to a give destination is broken with IPv6, but still works with IPv4. My guess: 1) IPv6 transit becomes slower than IPv4 -> happy eyeballs fallback with may be few ms among them, typically is a temporary problem. 2) Wrong configuration of some box (or setting up AAAA when IPv6 is broken), IPv6 almost never come back, or the extra delay in ms is too high, and this is not-so-temporary problem.

[Jordi] Exactly, no need to correct that, because in that case, typically neither IPv4 or IPv6 will work … and as said, ISPs typically notice if the (customer) access-link is broken. For example, in GPON, they know even if there are losses in packets, or dB, etc., and they get immediate alerts for that. For the “overall” IPv4 and IPv6 transit, they also get immediate alerts if something is wrong. The problem is when there is a problem to a specific destination or set of them (for example a given content provider) – and in this case the problem may be only from the destination or (partially) the transit.

[Jordi] IPv6-only access doesn’t mean that you don’t have dual-stack at the customer LANs (DS-LITE, lw4o6, MAP-T/E, 464XLAT …).

[Jordi]    Agree, however, I’m clear that we should ;-) because it is a real problem, and I’m not the only one has reports on it. I was commenting with several folks (that you know), exactly the same issue when I quickly presented it at the last IPv6 WG in the Dubai RIPE meeting – they come to me to tell “finally” we do something about this …
Of course, we can’t report “directly” to the destination if that's the problem, but the ISP that is suffering from frequent HE fallback in its network, typically is interested in knowing 1) that this is happening 2) if happens only to few destinations or if they are related which case, may mean same content provider or transit 3) if happens only from specific customers or many (which may indicate a more local failure, or internal to the customers networks, or CE configurations, etc.).

---

From: Mikael Abrahamsson <[email-masked]>
Date: Sat, 18 Nov 2017 14:59:47 +0100 (CET)

> alternative way for doing the reporting?

There are multiple initiatives that try to have end devices talk to a 
"network information server" of some kind. It would be good if this work 
regarding HE failures could boot-strap off of their work.

CAPPORT has their RA/DHCP options and a format to talk to something that 
regulates network access.

ALTO has a network information server to get information about what 
networks that might be "better" or "worse" to send traffic to.

The provisioning domain work in INTAREA (previsouly in MIF) has a JSON 
blob that will be downloaded, that could have information about what 
server HE failures should be reported to.

All of these would be good if they were coordinated somehow, and clients 
would have a way to both get and send information to the ISP.

The whole thing about blindly sending UDP packets to a well-known address 
is a complete no-starter for me. I like the idea of ISPs getting 
preformance reports from devices, but this is not the way to do it 
(UDP/SYSLOG). So we need to figure out how to send the reports, and what 
should go in the reports. Probably we also need different kinds of reports 
depending on privacy settings. A report that states "in the past day, I 
have had X HE failures for v4 and fell back to v6" etc would perhaps be ok 
even for the highly privacy minded, because it says very little about what 
sites were visited, just a generic failure metric.

Also, as an ISP I would like to have a way for the user to report in more 
detailed information when they're for instance is in contact with customer 
support, to help diagnosing their problems. Would be beneficial if we had 
a standard way of doing that as well.

-- 
Mikael Abrahamsson    email: [email-masked]

---

From: Fernando Gont <[email-masked]>
Date: Sat, 18 Nov 2017 20:28:05 +0800

> <[email-masked]> wrote:
[....]
> access is Ipv6-only? Or what do you mean by 'access; in this case?

I haven't been following this thread closely, but... I'd note that the
underlying issue that HE tries to address is not specific of mixed
IPv6/IPv4 environments, but also present in IPv4-only, IPv6-only, etc:

A server that is available on multiple addresses, some of which are
unavailable, poses the same problem -- no matter whether the
aforementioned addresses are IPv4-only, IPv6-ly, or mixed IPv6/IPv4.

If a list of available addresses is tried in sequence, the potential for
long details in "connection establishment" exists.

Thanks,
-- 
Fernando Gont
SI6 Networks
e-mail: [email-masked]
PGP Fingerprint: 6666 31C6 D484 63B2 8FB1 E3C4 AE25 0D55 1D4E 7492

---

From: Brian E Carpenter <[email-masked]>
Date: Sun, 19 Nov 2017 08:47:50 +1300

> long details in "connection establishment" exists.

Sure. And these issues crop up for MPTCP and SHIM6 too.
https://tools.ietf.org/html/draft-naderi-ipv6-probing-01

     Brian

---

From: JORDI PALET MARTINEZ <[email-masked]>
Date: Sun, 19 Nov 2017 07:03:45 +0000

Hi Mikael,

Thanks a lot for all this info. I will definitively take a look.

My reason to choose, as an initial approximation, syslog, is because this is often present in even the smaller service provider network, so it is a simple way for all them to deploy it, as simply requires having the correct address setup in the actual syslog server and you’re done.

My fear with any other alternatives is that they aren’t deployed. So, we will need to investigate first which one has a global deployment or if there is a way to secure syslog/TCP.

Someone suggested me to use Netflow, but I’m not really sure it can be used to detect those failures including the failed destination.

I think not reporting the source address is fine, but not reporting the destination becomes useless, because then the ISP can’t see where is the source of the problem.

>From a privacy perspective, I still believe that if the ISP needs to log the “failed” destinations, this should not be a privacy issue. I believe regulations allow that if this is needed for O&M. Of course, what law doesn’t allow is any further usage of that data or even less disclosing it to third parties.

Regards,
Jordi

---

From: Mikael Abrahamsson <[email-masked]>
Date: Sun, 19 Nov 2017 12:12:13 +0100 (CET)

On Sun, 19 Nov 2017, JORDI PALET MARTINEZ wrote:

>> From a privacy perspective, I still believe that if the ISP needs to 
>> log the “failed” destinations, this should not be a privacy issue. I

It is a privacy issue. Huge one. Yes, it can be argued that the ISP can do 
the same thing by looking at DNS query logs, but it's still a privacy 
issue.

As an ISP I would more like to understand if I have a problem in my own 
network. So getting hourly or daily statistics about HE failures tied to a 
certain customer connection would be nice (not reporting destinations, 
only source). I don't think this should be a privacy issue in practice, 
even though there might be some theoretical ones.

---

From: Mikael Abrahamsson <[email-masked]>
Date: Sun, 19 Nov 2017 12:14:02 +0100 (CET)

On Sun, 19 Nov 2017, JORDI PALET MARTINEZ wrote:

> I think not reporting the source address is fine, but not reporting the 
> destination becomes useless, because then the ISP can’t see where is the 
> source of the problem.

Btw, it would probably be better if there was an HTTP standard to report 
HE statistics to the web server in question. Then I imagine there are no 
privacy issues (because you're already speaking to this web server).

So a new header type, perhaps? I am no HTTP expect...

---

From: Sander Steffann <[email-masked]>
Date: Sun, 19 Nov 2017 13:40:33 +0100

Hi,

> Op 19 nov. 2017, om 12:14 heeft Mikael Abrahamsson <[email-masked]> het volgende geschreven:
> 
> Btw, it would probably be better if there was an HTTP standard to report HE statistics to the web server in question. Then I imagine there are no privacy issues (because you're already speaking to this web server).
> 

That sounds useful. Content hosters/owners have an interest in looking into issues/delays/etc that their users might have. It's a different target group than in Jordi's draft though, so the ideas are not mutually exclusive.

I think reporting HE information to the webserver would be useful and I'd happily help with writing a draft if more people are interested.

Cheers,
Sander

---

From: JORDI PALET MARTINEZ <[email-masked]>
Date: Sun, 19 Nov 2017 13:53:53 +0100

Hi Sander,

I will be happy to help if this is the “way to go”, however, my concern … do you think content providers will make use of this?

What happens if the HE fallback is not a webserver (http/s) but some other service?

Basically, those questions were the ones, in addition to make it simple to deploy, that I initially considered when started with this document.

Regards,
Jordi

-----Mensaje original-----
De: Sander Steffann <[email-masked]>
Responder a: <[email-masked]>
Fecha: domingo, 19 de noviembre de 2017, 13:40
Para: Mikael Abrahamsson <[email-masked]>
CC: JORDI PALET MARTINEZ <[email-masked]>, <[email-masked]>
Asunto: Re: [v6ops] draft-palet-v6ops-he-reporting-00

    Hi,

    
    
    Cheers,
    Sander

---

From: Mikael Abrahamsson <[email-masked]>
Date: Sun, 19 Nov 2017 14:43:11 +0100 (CET)

> Basically, those questions were the ones, in addition to make it simple to deploy, that I initially considered when started with this document.

I think both ISPs and the people hosting the information are interested in 
this information. I fully support it being reported to both, I just don't 
think sending syslog blindly using UDP is the way to do it.

So the idea is great behind this draft is great, I just think the current 
suggested way of sending the reports isn't going to get enough support.

I am also not a stranger to having aggregate information being sent to 
some kind of neutral third party, regularily. Right now I think some 
device vendors collect this kind of statistics, but I don't know exactly 
what is being collected and how it's used.