Subject: [TLS] Malware (was Re: draft-green-tls-static-dh-in-tls13-01)

From: Martin Thomson <[email-masked]>
Date: Fri, 14 Jul 2017 16:02:21 +1000

On 14 July 2017 at 01:08, Kathleen Moriarty
<[email-masked]> wrote:
> It sounds like for malware, we could do something to better document
> your security options as well as monitoring.  While the documentation
> is there for key pinning and trust anchors, this might not be obvious
> to network managers - what RFC to look at and how they fit together.

Just an aside, though I think Kathleen already made this point:  If I
were writing malware, I would use TLS.  It's pretty good at what it
does and it's hard to distinguish from legitimate uses (there are some
trick's suggested by McGrew's research on this point).

At the point that I have sufficient control over a host that I can run
my software, then I would pin certificates and the best you could do
is block me.  None of the advice about configuration of trust anchors
(pinning, overrides, etc...) helps at that point.

Most discussions regarding breaking TLS focus on the breaking of TLS
to *prevent* malware from infesting machines.  There at least the
defender has a reason to attack end-to-end security.  But then we're
talking about a very different deployment model than the gazillion
emails recently have contemplated.

---

From: "Blumenthal, Uri - 0553 - MITLL" <[email-masked]>
Date: Fri, 14 Jul 2017 02:24:51 -0400

TLS is a tool. Good guys want to use it to defend against the bad guys. Bad guys may want to use it against the good guys. (No surprise here, right?)

You cannot “sabotage” the second use case without sabotaging the first one at the same time. 

Two decades ago Jeff Schiller said something that remains perfectly valid today: (paraphrasing) “The price of having a secure infrastructure is our adversaries having a secure infrastructure”.

Understandably, you don’t want bad guys to use TLS “shield" against your “weapons”. Breaking this “shield” would accomplish this goal - but at a cost I (for one) don’t want to pay.
—
Regards,
Uri

> On Jul 14, 2017, at 02:02, Martin Thomson <[email-masked]> wrote:
> 
> _______________________________________________
> TLS mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/tls

---

From: Kathleen Moriarty <[email-masked]>
Date: Fri, 14 Jul 2017 12:17:52 +0200

Sent from my iPhone

> On Jul 14, 2017, at 8:02 AM, Martin Thomson <[email-masked]> wrote:
> 

I do think we could help tie together this advice for the many operators using TLS so they know and understand options better.

Yes, agreed, detection at that point relies on anomalous behavior on the managed end point, connection information (unusual destination, size of packets, timing of streams, etc.), or known indicators of compromise.  These all work with encryption and TLS should not be a hinderance to incident detection at that point
> emails recently have contemplated.

I think the points on why the solution doesn't help with malware have been clear, but just in case, preventing malware from infecting endpoints receiving content over HTTP/TLS can also be done using indicators of compromise.  Breaking TLS only works if the malware has been identified and can be detected by the analysis box. This does work when it's been deployed to newly infected servers when the malware hasn't been altered enough to avoid detection.  This is useful to enterprises, but you'd need the server key or more realistically going forward, would need a TLS proxy. Otherwise, with the proposed solution, your still relying on indicators of compromise that can be detected using the encrypted traffic.

Best regards,
Kathleen

---

From: "Roland Dobbins" <[email-masked]>
Date: Mon, 17 Jul 2017 12:59:00 +0200

> trick's suggested by McGrew's research on this point).

The bad guys make use of TLS all the time, including lateral movement 
within intranets.

> (pinning, overrides, etc...) helps at that point.

Correct.  Which is why it's critical in the intranet context, within a 
single span of administrative control, to have visibility into the 
actual cryptostream.

> to *prevent* malware from infesting machines.

Correct - and that's the wrong emphasis.  The emphasis should be on 
having the ability to monitor the crypto stream within the intranet in 
order to detect and classify compromised machines, because that ability 
is used very heavily for that purpose on intranets, and has been for 
many years.

-----------------------------------
Roland Dobbins <[email-masked]>

---

From: "Roland Dobbins" <[email-masked]>
Date: Mon, 17 Jul 2017 13:01:10 +0200

On 14 Jul 2017, at 12:17, Kathleen Moriarty wrote:

> Otherwise, with the proposed solution, your still relying on 
> indicators of compromise that can be detected using the encrypted 
> traffic.

Actually, it's often important to have visibility into the intranet 
cryptostream in order to detect and classify aberrant behavior which 
can't otherwise be detected/classified standing outside the tunnel.

Organizations do this to identify compromised/abusive machines on 
intranet networks all the time.

---

From: Martin Thomson <[email-masked]>
Date: Mon, 17 Jul 2017 14:01:17 +0200

On 17 July 2017 at 12:59, Roland Dobbins <[email-masked]> wrote:
> single span of administrative control, to have visibility into the actual
> cryptostream.

Roland, I think that you missed my point here.  My point was that you
don't get that visibility when it is malware at both ends of the
connection (assuming a modest amount of competency from the authors).

---

From: "Dobbins, Roland" <[email-masked]>
Date: Mon, 17 Jul 2017 12:07:50 +0000

On Jul 17, 2017, at 14:01, Martin Thomson <[email-masked]<mailto:[email-masked]>> wrote:

 My point was that you

Seeing it when it's only at one end is quite useful.

And, yes, one does often see it at both ends.   Assuming competence on the part of the authors is unwarranted.

Why do you persist in treating this technology as theoretical & ineffective, when it's been in use quite effectively for many years?

Network operators have been doing this for a long time - because it works.

-----------------------------------
Roland Dobbins <[email-masked]<mailto:[email-masked]>>

---

From: Russ Housley <[email-masked]>
Date: Mon, 17 Jul 2017 08:13:56 -0400

> connection (assuming a modest amount of competency from the authors).

I think that the IDS is trying to detect the an infected server trying to migrate to another server.  Malware often includes a series of exploits that are tried in sequence to infect a neighbor, and this activity provides a detectable signature.

Russ

---

From: "Dobbins, Roland" <[email-masked]>
Date: Mon, 17 Jul 2017 13:02:15 +0000

On Jul 17, 2017, at 14:14, Russ Housley <[email-masked]<mailto:[email-masked]>> wrote:

Correct. And not just between servers.

---

From: Carl Mehner <[email-masked]>
Date: Mon, 17 Jul 2017 08:15:10 -0500

On Mon, Jul 17, 2017 at 8:02 AM, Dobbins, Roland <[email-masked]> wrote:
>
>
> On Jul 17, 2017, at 14:14, Russ Housley <[email-masked]> wrote:
>
> I think that the IDS is trying to detect the an infected server trying to
> migrate to another server.  Malware often includes a series of exploits that
> are tried in sequence to infect a neighbor, and this activity provides a
> detectable signature.
>
>

I think the point that Martin was making (and if he wasn't, I will):
that malware is becoming increasingly aware that IDS/IPS and TLS proxy
boxes are looking into TLS traffic, and they're beginning to encrypt
traffic inside the TLS tunnel. That pushes the problem back into the
application layer, and on the endpoint to be dealt with by antivirus
tools.

---

From: "Dobbins, Roland" <[email-masked]>
Date: Mon, 17 Jul 2017 13:35:23 +0000

On Jul 17, 2017, at 15:15, Carl Mehner <[email-masked]<mailto:[email-masked]>> wrote:

beginning to encrypt traffic inside the TLS tunnel.

Yes, some (but by no means all) are - which means that in such cases, the ability to look inside the TLS tunnel so as to be able to detect the presence of an additional level of encryption as a possible indicator of compromise is extremely important.

---

From: Carl Mehner <[email-masked]>
Date: Mon, 17 Jul 2017 08:40:16 -0500

On Mon, Jul 17, 2017 at 8:35 AM, Dobbins, Roland <[email-masked]> wrote:
>> On Jul 17, 2017, at 15:15, Carl Mehner <[email-masked]> wrote:
> Yes, some (but by no means all) are - which means that in such cases, the
> ability to look inside the TLS tunnel so as to be able to detect the
> presence of an additional level of encryption as a possible indicator of
> compromise is extremely important.

Are you worried about malware encrypting traffic between nodes in an
intranet communicating with servers on that intranet you control which
would use this draft? that seems very unlikely. Why would malware use
this draft? Malware would use either it's own server, or basic
utilities provided by the system (i.e. wannacry's use of SMB).

---

From: "Dobbins, Roland" <[email-masked]>
Date: Mon, 17 Jul 2017 13:54:17 +0000

On Jul 17, 2017, at 15:40, Carl Mehner <[email-masked]<mailto:[email-masked]>> wrote:

Why would malware use this draft?

Nobody said anything about malware using this draft.

What I'm saying is that the ability to look inside the TLS tunnel & infer the presence of an additional, unexpected cryptostream - even without the ability to decrypt it - is quite valuable.

---

From: Carl Mehner <[email-masked]>
Date: Mon, 17 Jul 2017 08:59:51 -0500

I have not heard any assertions that looking at unencrypted tls
traffic is not valuable. I agree that there are cases that it is. What
I and others have disagreed with is that the examples provided on the
list and in the draft of where it is necessary are either not
applicable, or simply 'easier' rather than necessary.
In the email below, I was trying to find out which case malware would
fall into. do you have an example of where malware would be on your
intranet using this draft (the only way that this draft would help you
with malware analyzing), if you do not, let's remove malware analysis
from this list of arguments for this draft.

On Mon, Jul 17, 2017 at 8:54 AM, Dobbins, Roland <[email-masked]> wrote:
>
>
> On Jul 17, 2017, at 15:40, Carl Mehner <[email-masked]> wrote:
>
> What I'm saying is that the ability to look inside the TLS tunnel & infer
> the presence of an additional, unexpected cryptostream - even without the
> ability to decrypt it - is quite valuable.
>

---

From: "Dobbins, Roland" <[email-masked]>
Date: Mon, 17 Jul 2017 14:11:25 +0000

On Jul 17, 2017, at 15:59, Carl Mehner <[email-masked]<mailto:[email-masked]>> wrote:

the only way that this draft would help you
with malware analyzing)

This statement is factually incorrect.  It’s not the only way, as I've just explained.

Again, why are you trying to pretend that the use of this technique is not prevalent nor important in the security context, when it is in fact quite prevalent & important, & has been for many years?

And why are you unable to understand that that in the case of an additional layer of attacker-generated crypto nestled within a TLS tunnel, as you posited, that the ability to simply detect the presence of such an additional layer of unexpected crypto, even without the ability to immediately decrypt it, has substantial value in a security context?

Are you unfamiliar with the concept of traffic analysis, in the crypto sense of the term?

---

From: "Blumenthal, Uri - 0553 - MITLL" <[email-masked]>
Date: Mon, 17 Jul 2017 14:23:22 +0000

And why are you unable to understand that that in the case of an additional layer of 
attacker-generated crypto nestled within a TLS tunnel, as you posited, that the ability
to simply detect the presence of such an additional layer of unexpected crypto, even 
without the ability to immediately decrypt it, has substantial value in a security context?

It may, or it may not – depending on the sophistication of your adversary. It is not given that you’d be able to “simply detect the presence of an additional crypto layer”, particularly if measures are taken to hide it.

The standard definition of “traffic analysis” is deducing information from the metadata and the patterns of communications. It explicitly does NOT rely on knowing the content of the traffic (which is assumed to be opaque). You may learn more about it here https://en.wikipedia.org/wiki/Traffic_analysis :)

---

From: Jeffrey Walton <[email-masked]>
Date: Mon, 17 Jul 2017 10:30:49 -0400

> It may, or it may not – depending on the sophistication of your adversary. It is not given that you’d be able to “simply detect the presence of an additional crypto layer”, particularly if measures…

+1.

For example, Rule of 2 in Fishbowl encryption:
https://en.wikipedia.org/wiki/Multiple_encryption.

And since WebCrypto is [mostly] standardized, sometimes it will happen
as JavaScript is encryption applied to the protected stream that
inadvertently gets TLS encryption applied. Its just a matter of time
before it trickles into components like RabbitMQ.

Jeff

---

From: Carl Mehner <[email-masked]>
Date: Mon, 17 Jul 2017 09:52:29 -0500

On Mon, Jul 17, 2017 at 9:11 AM, Dobbins, Roland <[email-masked]> wrote:
>
>
> On Jul 17, 2017, at 15:59, Carl Mehner <[email-masked]> wrote:
>

Ok, sure, sorry for being imprecise. it's the only way that would help
you if you were trying to use this draft. If you're not using this
draft, then why are we talking about malware here? Do you have an
example of where malware would be on your intranet where using this
draft would help you?

> Again, why are you trying to pretend that the use of this technique is not
> prevalent nor important in the security context, when it is in fact quite
> prevalent & important, & has been for many years?

First, I was not the person you asked this to the first time, unless
you were asking the list, if you were I think there has been a
considerable effort to try and understand. Second, now that you have
asked me... I have, that's why i'm asking questions, (that you haven't
answered) Thirdly, just because something is prevalent and important
and has been used for many years, doesn't mean that it should
continue.

Looking at HTTP-only traffic for debugging was prevalent and sometimes
was even considered "OK" because, "we're in an intranet, not going
over the Internet", now we understand that it is not longer acceptable
to use plaintext traffic. Because of that tools changed, network
designs changed. A similar thing is happening now. Non-FS was ok,
because it's "intranet", now, it is not tools and designs will have to
change. (And since we are throwing around PCI hypotheticals.. maybe
PCI will mandate forward secrecy without static keys because it's more
secure.. we just don't know.)
The bar, as Steven pointed out earlier, is for you to prove. You
should be proving that it is necessary, not just that it is prevalent
or easier.

> And why are you unable to understand that that in the case of an additional
> layer of attacker-generated crypto nestled within a TLS tunnel, as you
> posited, that the ability to simply detect the presence of such an
> additional layer of unexpected crypto, even without the ability to
> immediately decrypt it, has substantial value in a security context?

I didn't say that I didn't understand that knowing nested crypto was
occurring was beneficial. I do. I agree it is beneficial. I'm trying
to get you to explain how this draft helps with that.

> Are you unfamiliar with the concept of traffic analysis, in the crypto sense
I recently analyzed some malware that was sending encrypted traffic
back and forth nested inside TLS. But we didn't need to open up the
TLS stream to know that it was malware. Also, this draft did not apply
to that strain of malware, it wouldn't have even helped determine that
the crypto was doubled.

---

From: "Roland Dobbins" <[email-masked]>
Date: Mon, 17 Jul 2017 17:11:41 +0200

On 17 Jul 2017, at 16:23, Blumenthal, Uri - 0553 - MITLL wrote:

> It may, or it may not – depending on the sophistication of your 
> adversary. It is not given that you’d be able to “simply detect 
> the presence of an additional crypto layer”, particularly if 
> measures are taken to hide it.

Sure.  I'm familiar with those counter-detection techniques, as I'm sure 
many (most?) of those involved in this discussion are.  And of course 
there are counters to those counters . . . it's counters all the way 
down!

;>

> The standard definition of “traffic analysis” is deducing 
> information from the metadata and the patterns of communications. It 
> explicitly does NOT rely on knowing the content of the traffic (which 
> is assumed to be opaque).

That's what I was trying to get across - that uncovering an unexpected 
layer of encryption, even without the ability to decrypt it, is very 
useful in a security context.

Sorry for being unclear!

---

From: "Roland Dobbins" <[email-masked]>
Date: Mon, 17 Jul 2017 17:32:13 +0200

On 17 Jul 2017, at 16:52, Carl Mehner wrote:

>  Do you have an example of where malware would be on your intranet 
> where using this
> draft would help you?

Sure - detecting attempted additional compromise and lateral movement 
utilizing exploits within TLS-encrypted traffic.

Another is detecting (and subsequent blocking of) the download of 
malware by intranet users.

Detecting data exfiltration is also a common use of this technique in 
intranet environments.

> Thirdly, just because something is prevalent and important

Sure - but it's also important to understand the mechanisms and 
techniques which are important to network operators, and to provide 
non-burdensome, non-iatrogenic ways for them to continue fulfilling 
critical functions within their own intranet environments.

>
> Non-FS was ok, because it's "intranet", now, it is not tools and 
> designs will have to
> change.

I understand what you're saying; but, conversely, isn't it in the 
interests of all of us working within this WG to help define practicable 
solutions that network operators can use on their own intranets to 
troubleshoot and ensure the security of those networks without requiring 
iatrogenic measures?

> (And since we are throwing around PCI hypotheticals.. maybe

Agree with you 100%.  That being said, the PCI/DSS board are quite 
well-versed in the monitoring techniques utilized by relevant 
organizations, so (hopefully!) they'll take this into account, when the 
time comes.

But you're absolutely right - we just don't know.

> should be proving that it is necessary, not just that it is prevalent

The problem is that in many cases, the changes required to accommodate 
pervasive PFS within the intranet have the potential to be both 
impractical as well as iatrogenic in nature.  I completely agree with 
you that PFS is very important whenever sending/receiving traffic across 
multiple spans of administrative control, like the public Internet.

> I'm trying to get you to explain how this draft helps with that.

It helps because if the intranet network operator has visibility within 
the TLS tunnel on his own intranet, he has the opportunity to infer the 
existence of unknown and unexpected traffic types, including possible 
superencryption.

Just being able to something that looks out of place and warrants 
further investigation, without being able to instantly classify it, is 
very valuable, in a security context.

Does that make more sense?

> TLS stream to know that it was malware.

Understood - you were able to user some other mechanism, like traffic 
analysis or an endpoint mechanism, to determine the the presence of this 
particular malware, yes?

Unfortunately, this isn't always going to be the case, in many 
circumstances and contexts.

> it wouldn't have even helped determine that the crypto was doubled.

Was the malware in question using countersurveillance/obfuscation 
techniques that made it more difficult to infer the presence of the 
additional layer of encryption?

---

From: Carl Mehner <[email-masked]>
Date: Mon, 17 Jul 2017 11:32:41 -0500

On Mon, Jul 17, 2017 at 10:32 AM, Roland Dobbins <[email-masked]> wrote:
>>  Do you have an example of where malware would be on your intranet where
>> using this
>> draft would help you?
>
>
Why is the malware setting up servers that have been explicitly
configured to use this draft and communicate with your enterprise
static key server? An common used example is metasploit, this sets up
its own keys and servers for communication and exploitation, another
is powershell empire, which sets up its own listeners rather than
using existing servers for intranet cross node communication.

> Another is detecting (and subsequent blocking of) the download of malware by
> intranet users.
Why not just use an endpoint solution and delete the malware from your
servers? Endpoint agents seem the best place architecturally for this
issue.

> Detecting data exfiltration is also a common use of this technique in
Exfiltration sounds like something done on the path towards the
Internet, where a proxy would site terminating TLS. DLP solutions
currently exist that take care of this requirement without needing
this draft.

> I understand what you're saying; but, conversely, isn't it in the interests
> of all of us working within this WG to help define practicable solutions
> that network operators can use on their own intranets to troubleshoot and
> ensure the security of those networks without requiring iatrogenic measures?
I don't think it's the WG's job to define network architecture. There
are ways to change the network that allow the necessary
troubleshooting. This goes back to Rich's question of is TLS 1.3 going
to be mandated within 5 years. If not, then you have 5 years to pivot
on architectural designs and deployed software to gain the same ends
by different means.

> Does that make more sense?
I agree that seeing the presence of superencryption is valuable.

> analysis or an endpoint mechanism, to determine the the presence of this
yes
> Unfortunately, this isn't always going to be the case, in many circumstances
> and contexts.
>
> Was the malware in question using countersurveillance/obfuscation techniques
> that made it more difficult to infer the presence of the additional layer of
> encryption?
nope, it was reaching out to a malicious site. That's why it's a bad example.

---

From: Simon Friedberger <[email-masked]>
Date: Mon, 17 Jul 2017 18:40:41 +0200

> utilizing exploits within TLS-encrypted traffic.
This is about decrypting traffic inside your network.

> Detecting data exfiltration is also a common use of this technique in
This is about decrypting traffic that goes outside of your network.

I'm not sure the same considerations should apply to both those situations.

---

From: "Roland Dobbins" <[email-masked]>
Date: Mon, 17 Jul 2017 18:48:22 +0200

> I'm not sure the same considerations should apply to both those 

Actually, they do, when you're on your network prior to the egress point 
- apologies for being unclear about that.

Many enterprises force all outbound user-generated traffic through 
proxies, which then inspect TLS-wrapped traffic, blocking bad traffic 
(like data exfiltration) while then opening up proxy connections for 
legitimate traffic, FYI.

Conversely, they do the same with inbound traffic in response to said 
user-generated traffic, and block things like malware downloads.

---

From: Watson Ladd <[email-masked]>
Date: Mon, 17 Jul 2017 09:55:16 -0700

On Jul 17, 2017 9:48 AM, "Roland Dobbins" <[email-masked]> wrote:

Actually, they do, when you're on your network prior to the egress point -
apologies for being unclear about that.

Many enterprises force all outbound user-generated traffic through proxies,
which then inspect TLS-wrapped traffic, blocking bad traffic (like data
exfiltration) while then opening up proxy connections for legitimate
traffic, FYI.

So FS has no impact on this, correct?

---

From: "Roland Dobbins" <[email-masked]>
Date: Mon, 17 Jul 2017 19:01:18 +0200

> So FS has no impact on this, correct?

It's often desirable to be able to inspect closer to the internal user 
traffic sources/destinations, as well as at the proxy.  It can greatly 
reduce the scope of traffic which is to be analyzed in any given 
investigative context (and possibly groveled through), which is a 
genuine operational concern.  So, having the ability to look at this 
traffic prior to it reaching the proxy can be valuable.

Many organizations do this, today.

---

From: "Roland Dobbins" <[email-masked]>
Date: Mon, 17 Jul 2017 19:06:21 +0200

> Many organizations do this, today.

And some also go the passive-only route - I forgot to mention that.  
They'll use commercial IDS/IPS, or Snort with viewssld, et. al.

---

From: "Blumenthal, Uri - 0553 - MITLL" <[email-masked]>
Date: Mon, 17 Jul 2017 17:28:03 +0000

That's what I was trying to get across - that uncovering an unexpected 
    layer of encryption, even without the ability to decrypt it, is very 
    useful in a security context.    Sorry for being unclear!

You were perfectly clear. Apparently I was not clear enough explaining that the likelihood of being able to determine the presence of an unexpected layer of encryption is becoming increasingly slim, as all the bars (no pun intended :) keep rising. 

Organized crime capabilities are reaching the level of nation states, ankle biters reach up to where the organized crime was yesterday… Betting on malefactors to stay silly (send their traffic over TLS that complies with your monitoring, doing the extra work to add super-encryption but forgetting to obfuscate it, etc.) is not a safe or reasonable bet. Certainly not worth it, considering the risks that all the legitimate users will be subjected to by this feature.

---

From: "Dobbins, Roland" <[email-masked]>
Date: Mon, 17 Jul 2017 18:11:22 +0000

On Jul 17, 2017, at 19:28, Blumenthal, Uri - 0553 - MITLL <[email-masked]<mailto:[email-masked]>> wrote:

Organized crime capabilities are reaching the level of nation states, ankle biters reach up to where the organized crime was yesterday…

I understand all this - I have to deal with it every day, so I understand where you're coming from.

But it's also important for understand that security is additive in nature, not all the criminals are bright or sophisticated, & so the emergence of a few smarter ones doesn't make those less so disappear.

In reality, most of them are awful blunderers - they succeed because the defenders are worse blunderers.

Consequently, there hasn't been an alarming (heh) dropoff in the need for TLS visibility on the intranet - quite the opposite.

And the need for it isn't limited to the security space.  It'd extremely important for troubleshooting, as well.

---

From: "Blumenthal, Uri - 0553 - MITLL" <[email-masked]>
Date: Mon, 17 Jul 2017 18:35:21 +0000

But it's also important for understand that security is additive in nature,

not all the criminals are bright or sophisticated, & so the emergence of a

few smarter ones doesn't make those less so disappear. 

:-) It’s the Law Enforcement job to make the dumber ones disappear. 

The question is whether the risk all of the legit users would be subjected to is justified by the preserved ability to detect the dumber criminals using an outdated method, instead of evolving with times.

In reality, most of them are awful blunderers - they succeed because the defenders

are worse blunderers.  Consequently, there hasn't been an alarming (heh) dropoff in

the need for TLS visibility on the intranet - quite the opposite. 

Let’s exchange our criminals – I’d much rather deal with yours. ;-)

My point though is – this dropoff in visibility *will* come, like it or not. 

Based on my experience troubleshooting – I disagree. If I control at least one end of the communications – I have all the visibility into the traffic that I need.

---

From: "Roland Dobbins" <[email-masked]>
Date: Mon, 17 Jul 2017 20:49:08 +0200

> :-) It’s the Law Enforcement job to make the dumber ones disappear.

We aren't talking about law enforcement, per se - and it's law 
enforcement's job to make them *all* disappear.

> Let’s exchange our criminals – I’d much rather deal with yours. 

Sadly, I have to deal with all kinds.  I don't have the luxury of being 
selective, worse luck.

;>

> My point though is – this dropoff in visibility *will* come, like it 

When that begins to occur in perceptible increments, then it's time to 
revisit.  But avoiding breaking existing, useful mechanisms is something 
we should strive for, IMHO.

> Based on my experience troubleshooting – I disagree. If I control at 
> least one end of the communications – I have all the visibility into 
> the traffic that I need.

Based on my experience troubleshooting, I disagree with your 
disagreement; while in many circumstances a great deal can be inferred 
from one end, it's sometimes vitally necessary to gain visibility from 
multiple points in the relevant topological scope.  Perhaps the scope 
and complexity of what we respectively troubleshoot differs 
considerably.

---

From: "Roland Dobbins" <[email-masked]>
Date: Mon, 17 Jul 2017 20:53:35 +0200

On 17 Jul 2017, at 20:49, Roland Dobbins wrote:

> multiple points in the relevant topological scope.

Also, it's quite common for the party responsible for the 
troubleshooting to *not* have ownership/admin privileges of/on either of 
the relevant endpoints.