Subject: Re: [Netconf] [netmod] draft-ietf-netconf-netconf-client-server – TCP keepalives

From: Kent Watsen <[email-masked]>
Date: Tue, 12 Jun 2018 01:22:40 +0000

Looking into this just a little more, I know that Heartbeat was supported by OpenSSL before (recall Heartbleed bug?), so I grepped the 1.1.0g source code (which has the Heartbleed fix) and found evidence that the support might still be in the code.  That said, I can't tell if the code is specific to DTLS or works on TLS as well…

/kw

===== original message =====

[+netconf, -netmod]

The issue appears to be with current TLS libraries not implementing TLS keepalives, the HeartbeatRequest messages defined by [RFC6520].   I have not myself validated this yet, does anyone have any experience?

If it is true that HeartbeatRequest messages is not supported today, do we:
  a) encourage the TLS library maintainers to implement it
  b) or introduce an ability to configure TCP-level keepalives
  c) or both?

Any other ideas?

Thanks,
Kent

On 6/11/18, 12:32 PM, "netmod on behalf of NICK HANCOCK" <[email-masked] on behalf of [email-masked]> wrote:

Hi All, 
 
A couple of companies are working on a solutions to implement devices, such as DPUs, based on the requirements of the Broadband Forum Technical Report TR-301 issue 2 “Architecture and Requirements for Fiber to the Distribution Point”, which requires TLS for the persistent NETCONF connection, for which the configuration of call home is to be by means of the ‘ietf-netconf-server’ module. 
 
TLS heartbeat cannot be supported to keep the call home connection alive, because TLS heartbeat is not or no longer supported by many TLS libraries, such as OpenSSL in the wake of the Heartbleed security bug. Although TCP keep-alives are not secure, we will nevertheless be required to support TCP keepalives to ensure that the connection remains persistent and these keepalives would also need to be configurable. Unfortunately, the keepalive configuration implemented in ‘ietf-netconf-server’, although not bound to the ‘transport’ choice, is bound to the secure layer textually in the description of the data nodes (references to “SSH/TLS client” and “SSH/TLS-level message”), which makes its use for configuring TCP keepalives for specific implementations possible, but obviously problematic. RFC 8071, Section 4.1, S7, also heavily implies that it is intended to be used for the designated transport layer (e.g., SSH, TLS).
 
Since this issue affects the industry as a whole, we believe it would be better to provide support for the configuration of TCP keepalives within the ‘ietf-netconf-server’ module from the beginning, rather than wait for other SDOs or vendors to augment the module after publication as an RFC, which they will be practicably forced to do.
 
Would supporting TCP keepalives in the IETF-defined module be something the WG would agree to discuss? A possible solution, shown below, could be to add a new container parallel to the existing ‘keep-alives’ container to explicitly support the configuration for TCP keepalives. In addition, a feature statement (e.g. "keep-alives") could be added to the existing ‘keep-alives’ container, as RFC 8071 S7 says SHOULD (not MUST). 
                   container tcp-keep-alives {
                     if-feature tcp-keep-alives;
                     description
                       "Configures the keep-alive policy, to
                        proactively test the aliveness of the TCP
                        peer.  An unresponsive TCP peer will
                        be dropped after approximately max-attempts *
                        max-wait seconds.";
                     reference
                       "RFC 1122: Requirements for Internet Hosts -- 
                        Communication Layers, section 4.2.3.6.";
                     leaf max-wait {
                       type uint16 {
                         range "1..32767";
                       }
                       units seconds;
                       default 30;
                       description
                        "Sets the amount of time in seconds after
                         which if no data has been received from
                         the TCP peer, a TCP-level message
                         will be sent to test the aliveness of the
                         TCP peer.";
                     }
                     leaf max-attempts {
                       type uint8 {
                         range "1..127";
                       }
                       default 3;
                       description
                        "Sets the maximum number of sequential keep-
                        alive messages that can fail to obtain a
                        response from the TCP peer before
                        assuming the TCP peer is no longer
                        alive.";
                     }
                     leaf interval-between-attempts {
                       type uint16  {
                         range "1..32767";
                       }
                       units seconds;
                       default 30;
                       description
                        "Sets the amount of time in seconds after
                         which, if no reply to a keep-alive message
                         has been received from the TCP peer, the
                         next keep-alive message will be sent.";
                     }
                   }
                 }

What is the opinion of the list? Would this solution work?
 
Best regards
Nick & Yves

_______________________________________________
Netconf mailing list
[email-masked]
https://urldefense.proofpoint.com/v2/url?u=https-3A__www.ietf.org_mailman_listinfo_netconf&d=DwIGaQ&c=HAkYuh63rsuhr6Scbfh0UjBXeMK-ndb3voDTXcWzoCI&r=9zkP0xnJUvZGJ9EPoOH7Yhqn2gsBYaGTvjISlaJdcZo&m=n1Ew69P_92NcpKfb6HiepQwhe21v4fTuNEa-YZ_vs6s&s=CVqduXP2RuuZY7nPF0drm5h9oFCMIMGg0ux6shk88OI&e=

---

From: Mikael Abrahamsson <[email-masked]>
Date: Tue, 12 Jun 2018 10:56:35 +0200 (CEST)

I support enabling possibility of TCP keepalives, and relevant 
configuration. I encourage this to be used everywhere. Operationally, I 
have seen lots of problems caused by not having keepalives enabled on 
sessions, and code not handling them stalling. TCP keepalives means the 
programmers do not really have to do anything, the TCP stack solves this 
for them.

On Tue, 12 Jun 2018, Kent Watsen wrote:

> https://www.ietf.org/mailman/listinfo/netconf

-- 
Mikael Abrahamsson    email: [email-masked]

---

From: "Beauville, Yves (Nokia - BE/Antwerp)" <[email-masked]>
Date: Tue, 12 Jun 2018 13:36:29 +0200

Hi Kent,

 From the change log of OpenSSL 
(https://www.openssl.org/news/changelog.txt), I can see the following 
change being logged between 1.1.0h and 1.1.1:

   *) Heartbeat support has been removed; the ABI is changed for now.
      [Richard Levitte, Rich Salz]

Thanks,
Yves

---

From: Kent Watsen <[email-masked]>
Date: Tue, 12 Jun 2018 14:12:45 +0000

Yes, it seems that they're in the process:

https://github.com/openssl/openssl/issues/4856

Kent

Hi Kent,

 From the change log of OpenSSL 
(https://urldefense.proofpoint.com/v2/url?u=https-3A__www.openssl.org_news_changelog.txt&d=DwIDaQ&c=HAkYuh63rsuhr6Scbfh0UjBXeMK-ndb3voDTXcWzoCI&r=9zkP0xnJUvZGJ9EPoOH7Yhqn2gsBYaGTvjISlaJdcZo&m=zzFq9Fp2lEWaEupRxa7mXtOgQHfoylXJshq8HQwfUnA&s=b2Ckx4ZL4J51XaBFNl95mcQaSVBtJUvrmCHAWFK6mUU&e=), I can see the following 
change being logged between 1.1.0h and 1.1.1:

Thanks,
Yves

On 12-06-18 03:22, Kent Watsen wrote:
> https://urldefense.proofpoint.com/v2/url?u=https-3A__www.ietf.org_mailman_listinfo_netconf&d=DwIDaQ&c=HAkYuh63rsuhr6Scbfh0UjBXeMK-ndb3voDTXcWzoCI&r=9zkP0xnJUvZGJ9EPoOH7Yhqn2gsBYaGTvjISlaJdcZo&m=zzFq9Fp2lEWaEupRxa7mXtOgQHfoylXJshq8HQwfUnA&s=gxTeCP_OaETTpPPkfQ7cgU-ELC_B8b_vVF0XCONqtVE&e=

---

From: "Beauville, Yves (Nokia - BE/Antwerp)" <[email-masked]>
Date: Thu, 21 Jun 2018 17:28:22 +0200

Hi Kent,

I understand that OpenSSL folks confirmed that they are not supporting  TLS heartbeat.

We want to move forward with using the ietf-netconf-server defined in draft-ietf-netconf-netconf-client-server and to do this we minimally need to support configuration of TCP keepalives. Having a TCP keep alive container, controlled by a feature flag, will enable us to achieve this target.

Would this be a valid and acceptable path for ietf-netconf-server to follow?

Thanks,
Yves

---

From: NICK HANCOCK <[email-masked]>
Date: Thu, 21 Jun 2018 16:04:17 +0000

Hi Kent,

At this time, we are being required to support the configuration of TCP keepalives for the on the connection between the NETCONF client and server and specifically to avoid multiple proprietary solutions within the industry, encourage the support directly within ietf-netconf-server. Adding a separate container to configure TCP keepalives, does not exclude the support of keepalives to test the aliveness of the SSH/TLS client. Through the feature flags, implementations can advertise exactly what they support.

Regards
Nick
This message has been classified General Business by NICK HANCOCK on Thursday, 21 June 2018 at 18:04:16.

From: Beauville, Yves (Nokia - BE/Antwerp) [mailto:[email-masked]]
Sent: Thursday, June 21, 2018 5:28 PM
To: Kent Watsen; NICK HANCOCK
Cc: [email-masked]
Subject: Re: [Netconf] [netmod] draft-ietf-netconf-netconf-client-server – TCP keepalives

Hi Kent,

Thanks,
Yves

On 12-06-18 16:12, Kent Watsen wrote:
> https://github.com/openssl/openssl/issues/4856<https://github.com/openssl/openssl/issues/4856>
>
> Kent
>
> (https://urldefense.proofpoint.com/v2/url?u=https-3A__www.openssl.org_news_changelog.txt&d=DwIDaQ&c=HAkYuh63rsuhr6Scbfh0UjBXeMK-ndb3voDTXcWzoCI&r=9zkP0xnJUvZGJ9EPoOH7Yhqn2gsBYaGTvjISlaJdcZo&m=zzFq9Fp2lEWaEupRxa7mXtOgQHfoylXJshq8HQwfUnA&s=b2Ckx4ZL4J51XaBFNl95mcQaSVBtJUvrmCHAWFK6mUU&e=<https://urldefense.proofpoint.com/v2/url?u=https-3A__www.openssl.org_news_changelog.txt&d=DwIDaQ&c=HAkYuh63rsuhr6Scbfh0UjBXeMK-ndb3voDTXcWzoCI&r=9zkP0xnJUvZGJ9EPoOH7Yhqn2gsBYaGTvjISlaJdcZo&m=zzFq9Fp2lEWaEupRxa7mXtOgQHfoylXJshq8HQwfUnA&s=b2Ckx4ZL4J51XaBFNl95mcQaSVBtJUvrmCHAWFK6mUU&e=>), I can see the following
>> Communication Layers, section 4.2.3.6<http://4.2.3.6>.";
>> leaf max-wait {
>> type uint16 {
>> range "1..32767";
>> }
>> units seconds;
>> default 30;
>> description
>> https://urldefense.proofpoint.com/v2/url?u=https-3A__www.ietf.org_mailman_listinfo_netconf&d=DwIGaQ&c=HAkYuh63rsuhr6Scbfh0UjBXeMK-ndb3voDTXcWzoCI&r=9zkP0xnJUvZGJ9EPoOH7Yhqn2gsBYaGTvjISlaJdcZo&m=n1Ew69P_92NcpKfb6HiepQwhe21v4fTuNEa-YZ_vs6s&s=CVqduXP2RuuZY7nPF0drm5h9oFCMIMGg0ux6shk88OI&e=<https://urldefense.proofpoint.com/v2/url?u=https-3A__www.ietf.org_mailman_listinfo_netconf&d=DwIGaQ&c=HAkYuh63rsuhr6Scbfh0UjBXeMK-ndb3voDTXcWzoCI&r=9zkP0xnJUvZGJ9EPoOH7Yhqn2gsBYaGTvjISlaJdcZo&m=n1Ew69P_92NcpKfb6HiepQwhe21v4fTuNEa-YZ_vs6s&s=CVqduXP2RuuZY7nPF0drm5h9oFCMIMGg0ux6shk88OI&e=>
>>
>>
>> https://urldefense.proofpoint.com/v2/url?u=https-3A__www.ietf.org_mailman_listinfo_netconf&d=DwIDaQ&c=HAkYuh63rsuhr6Scbfh0UjBXeMK-ndb3voDTXcWzoCI&r=9zkP0xnJUvZGJ9EPoOH7Yhqn2gsBYaGTvjISlaJdcZo&m=zzFq9Fp2lEWaEupRxa7mXtOgQHfoylXJshq8HQwfUnA&s=gxTeCP_OaETTpPPkfQ7cgU-ELC_B8b_vVF0XCONqtVE&e=<https://urldefense.proofpoint.com/v2/url?u=https-3A__www.ietf.org_mailman_listinfo_netconf&d=DwIDaQ&c=HAkYuh63rsuhr6Scbfh0UjBXeMK-ndb3voDTXcWzoCI&r=9zkP0xnJUvZGJ9EPoOH7Yhqn2gsBYaGTvjISlaJdcZo&m=zzFq9Fp2lEWaEupRxa7mXtOgQHfoylXJshq8HQwfUnA&s=gxTeCP_OaETTpPPkfQ7cgU-ELC_B8b_vVF0XCONqtVE&e=>
>
>

---

From: Kent Watsen <[email-masked]>
Date: Thu, 21 Jun 2018 16:54:48 +0000

Hi Yves,

Here's an update:

1. The OpenSSL folks confirm that 1) they're in the process of removing DTLS heartbeat, 2) they never implemented TLS heartbeat (even though the standard defines support for both), 3) they might be willing to reconsider support for RFC 6520, but would like the IETF to make some statements around why that might be important first.

2. The NETCONF chairs (per OpenSSL request) have raised this issue to the TLS chairs, TLS ADs, and TSV ADs.  Currently we're trying to determine if the IETF needs to issue a statement, maybe a BCP, discouraging the use of cleartext keepalives on a lower-level transport used to carry a higher-level secure transport.  There is no doubt that TCP keepalives are incredibly useful in some protocols, it's just the interaction of TCP-keepalives for a TLS (or SSH) session that is in question.  FWIW, no decision on the need for a statement has been made yet.

3. It seems that NETCONF WG needs to wait for this outcome, but I'm open opinions on this. If it turns out that the statement is a SHOULD NOT, instead of a MUST NOT, which is likely (I think), then NETCONF WG can do whatever we want and, assuming we decide to also support TCP-keepalives, then the Security Considerations section in those two drafts would just have to explain the concerns around using the TCP keepalives.

4. My personal opinion is that there isn't a need to move quickly to define a solution now, as this issue will surely resolve faster than the crypto-types/trust-anchors/keystore parts.

Thanks,
Kent

Hi Kent,

Thanks,
Yves

On 12-06-18 16:12, Kent Watsen wrote:
> https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_openssl_openssl_issues_4856&d=DwIDaQ&c=HAkYuh63rsuhr6Scbfh0UjBXeMK-ndb3voDTXcWzoCI&r=9zkP0xnJUvZGJ9EPoOH7Yhqn2gsBYaGTvjISlaJdcZo&m=wy_mSGhilcJt-WgbikwojJsvlopQrB3pORCRq1uNhpM&s=N9fg8k6x8L18fQDTUzhAqBKhohkoMzeaDHxmbwaJd8I&e=
>
> Kent
>

---

From: Kent Watsen <[email-masked]>
Date: Thu, 21 Jun 2018 17:43:54 +0000

Just a couple comments from my

1) TCP-keepalives are not always implemented.  From Spencer Dawkins: "we've been recommending that people not rely on TCP keepalives for so long that implementation support for them is still a MAY in https://tools.ietf.org/html/draft-ietf-tcpm-rfc793bis-09#section-3.8.4<https://urldefense.proofpoint.com/v2/url?u=https-3A__tools.ietf.org_html_draft-2Dietf-2Dtcpm-2Drfc793bis-2D09-23section-2D3.8.4&d=DwMFaQ&c=HAkYuh63rsuhr6Scbfh0UjBXeMK-ndb3voDTXcWzoCI&r=9zkP0xnJUvZGJ9EPoOH7Yhqn2gsBYaGTvjISlaJdcZo&m=C6ZMJa-eXYZ8LEFBzCeFiZHsjvW0ncjeAY_Qzl35rMQ&s=oxAmYSIqAeQ9YdJIh6YwgqasacCu8Rvta9vz6P_Oalw&e=>".  I don't view this as a blocker from a modelling perspective as, if we support configuring TCP-keepalives at all, we would use a "feature" statement so the support could be advertised on a per-device basis.  However, from a BBF solution perspective, this might be something to consider.

2) TCP-keepalives may not always support second-level polling rates.  Apparently, one of the motivations for TLS-keepalive was that they could be sent more often: https://tools.ietf.org/html/rfc6520#section-5.2.   I don't understand why TCP-keepalives are any slower.  Sure, I see the TCP-keepalive interval MUST default be no less than two hours, but I don't yet see why it couldn't be configured to as low as one second (e.g. http://tldp.org/HOWTO/TCP-Keepalive-HOWTO/usingkeepalive.html), and second-level granularity is all the ietf-*conf-[client|server] modules are using.  Anyway, someone might want to look into this also, since RFC 6520 goes through the effort of calling it out.

Kent // contributor

On 6/21/18, 12:04 PM, "NICK HANCOCK" <[email-masked]<mailto:[email-masked]>> wrote:

Hi Kent,

Regards
Nick

Hi Kent,

Thanks,
Yves

On 12-06-18 16:12, Kent Watsen wrote:
> https://github.com/openssl/openssl/issues/4856<https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_openssl_openssl_issues_4856&d=DwMGaQ&c=HAkYuh63rsuhr6Scbfh0UjBXeMK-ndb3voDTXcWzoCI&r=9zkP0xnJUvZGJ9EPoOH7Yhqn2gsBYaGTvjISlaJdcZo&m=f1mQZB52BQHRxF7T0NNUYOMmlBSI42GDpC_NX1VCdzw&s=to5OiGRFh8ClW2BY9m21BFNQR7CESg-Dgh-lwBQp4C4&e=>
>
> Kent
>
>> Communication Layers, section 4.2.3.6<https://urldefense.proofpoint.com/v2/url?u=http-3A__4.2.3.6&d=DwMGaQ&c=HAkYuh63rsuhr6Scbfh0UjBXeMK-ndb3voDTXcWzoCI&r=9zkP0xnJUvZGJ9EPoOH7Yhqn2gsBYaGTvjISlaJdcZo&m=f1mQZB52BQHRxF7T0NNUYOMmlBSI42GDpC_NX1VCdzw&s=tOy-XYY63vW0GIzmo_whpDffv-YEaiPy-lDshUOyx-o&e=>.";
>> leaf max-wait {
>> type uint16 {
>> range "1..32767";
>> }
>> units seconds;
>> default 30;
>> description

---

From: "Beauville, Yves (Nokia - BE/Antwerp)" <[email-masked]>
Date: Fri, 22 Jun 2018 11:02:43 +0200

Hi Kent,

Thank you for the clear update, and the actions you have initiated in 
order to clarify the current status and come up to a resolution.

It all makes perfect sense to me.

Yves