Subject: Re: [Netconf] WG Last Call Comments ondraft-ietf-netconf-reverse-ssh-03.txt

From: t.petch <[email-masked]>
Date: Thu, 1 May 2014 10:58:20 +0100

Kent

I commented earlier that I was uncertain about s.5 of reverse-ssh.  I
think that it has not got public key use (SSH host key) quite right.

If I were asked to describe the use of public keys, then I would
identify three steps.

1) get hold of a public key
2) verify that the key is tied to the identity of the other party
3) get the other party to demonstrate possession of the private key

At that level, it is irrelevant where the key comes from, X.509
certificate, pre-configuration, Trust On First Use, etc.  What varies is
step 2).

I can see no difference between the use of certificates with ssh and
with TLS.  And I think that certificates are naturally associated with
TLS, and so the place to describe that is in 5539bis.  In which case, I
would omit any description of it from the ssh I-D and just put in a
reference to 5539bis I-D.  Currently, reverse-ssh does have some
suggestions that are not in 5539bis but then they seem to me equally
appropriate to the use of certificates with TLS and so I think that they
should be in 5539bis.

By contrast, other ways of verifying the key are more associated with
ssh, in which case, they belong in reverse-ssh and 5539bis may then want
to reference this I-D.

I would be willing to expand this into replacement text if this
suggestion meets with approval.

Tom Petch

----- Original Message -----
From: "t.petch" <[email-masked]>
To: "Bert Wijnen (IETF)" <[email-masked]>; "Kent Watsen"
<[email-masked]>
Cc: <[email-masked]>
Sent: Tuesday, April 08, 2014 2:44 PM
Subject: Re: [Netconf] WG Last Call Comments
ondraft-ietf-netconf-reverse-ssh-03.txt

> I do not think that this is ready to advance.  I had not intended to
> comment but having started on netconf-server, and found I had to read
> 5539bis to understand where it was going, I then found that
reverse-ssh
> was another piece of the jigsaw.
>
> In particular, I think that this I-D is more or less updating RFC4742
in
> its considerations of security.  And I think it may be confused about
> public keys and certificates and their role in authentication.
>
> Taken together, and I do not think they can be considered separately,
I
> do not think that the three I-Ds give a coherent view of how to secure
a
> NETCONF session.
>
> I would point to the queries raised by Tadek and Bajpai which, to me
on
> the one hand seem spot on and on the other, seem to regard the subject
> material of these three I-Ds as of a one - which I obviously agree
with.
>
> I think that netconf-server is the one in most need of progress and
see
> it as a Normative, not Informative, prerequisite.
>
> Tom Petch
>
>
> ----- Original Message -----
> From: "Bert Wijnen (IETF)" <[email-masked]>
> To: "Kent Watsen" <[email-masked]>
> Cc: <[email-masked]>
> Sent: Thursday, April 03, 2014 12:36 PM
>
> > WGLC period ended last Tuesday (April 1st).
> >
> > Kent, where are we with this one?
> >
> > Do you expect to post a rev 04 that addresses all comments we have
> received?
> > Either WGLC comments or other comments? Once that is done, I would
> like to ask:
> > - Kent to post which comments have not been addressed and why (if
any)
> > - If possible, Kent to post a summary of changes (although that may
> already be
> >    in the document, and that is fine too).
> > - All those that made comments to check if they are OK with the way
> that
> >    they have been addressed or responded to.
> >
> > Bert
> >
> > On 27/03/14 17:02, Kent Watsen wrote:
> > >
> > > Hi Alan,
> > >
> > >
> > > Awesome comments - thanks!
> > >
> > >
> > > I applied all fixes mentioned below to my working copy of what
will
> be
> > > -04, so you'll have to wait until then to see the diff.  The
reason
> I'm
> > > not posting -04 now is because I'm still trying to work out an
issue
> with
> > > Applicability Statement and with the port's name.
> > >
> > > Below are my detailed responses.
> > >
> > > Cheers,
> > > Kent
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >> Typos
> > >> -----
> > >>
> > >
> > > I fixed all the issues you found.
> > >
> > >
> > >
> > >
> > >
> > >
> > >> Section 2.1, Page 3, Second Paragraph:  (wording)
> > >> -------------------------------------------------
> > >>
> > >> The second paragraph seems vague.  Would something along the
lines
> of
> > >> the following text be clearer and/or more specific?
> > >
> > > I forwarded this comment to Steve Hanna, who wrote the
Applicability
> > > Statement.  At first I was just going to ask if he was OK with the
> > > proposed text, but then I started thinking that I didn't agree
with
> it.
> > > That is, I think that the SSH protocol does require mutual
> authentication
> > > and that NETCONF's requirement that it be possible to derive a
> "username"
> > > from the transport doesn't change that.  I'm still OK limiting
> Reverse SSH
> > > to just NETCONF, but I want the reason to be accurate.  I'm now
> waiting
> > > for a response from Steve on this.
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >> Section 4, Page 4:  (wording)
> > >> -----------------------------
> > >>
> > >> The current text reads:
> > >>
> > >>    o  The NETCONF server initiates a TCP connection to the
NETCONF
> > >>       client on the IANA-assigned Reverse SSH port YYYY.
> > >>
> > >>    o  The TCP connection is accepted and a TCP session is
> established.
> > >
> > >
> > > I'm not sure about this change since these bullet-points are
> preceded by
> > > the line "From the NETCONF server's perspective:".  The intent is
to
> only
> > > explain the NETCONF-server's perspective, and to let the next
> section
> > > provide the client's perspective.   Currently, neither section
> references
> > > the remote peer, so that its text remains squarely focused on its
> side of
> > > the connection.  What do you think?
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >> Section 5, Page 5, First Paragraph:  (wording)
> > >> ----------------------------------------------
> > >>
> > >> The current text reads:
> > >>
> > >>    "When the management system accepts a new incoming connection,
> it
> > >>     needs to authenticate the remote peer.  Ultimately, this
> entails
> > >>     identifying the peer and verifying its SSH host key.
> > >>
> > >>     Due to Reverse SSH having the network element initiate the
TCP
> > >>     connection,"
> > >
> > >
> > > I changed the wording to be more clear, but did it another way,
> please let
> > > me know if you think it's OK!
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >> Section 5, Page 6, First Paragraph:  (clarification)
> > >> ----------------------------------------------------
> > >>
> > >> The current text reads:
> > >>
> > >>    "However, configuring distinct host keys on the management
> system
> > >>    doesn't scale well, which is an important consideration to a
> network
> > >>    management system.  A more scalable strategy is to have the
> network
> > >>    element's host key signed by a common trusted key, such as a
> > >>    certificate authority.  Thus, the mangement system only needs
to
> > >>    trust a single public key, which vouches for the authenticity
of
> the
> > >>    various network element public keys."
> > >>
> > >> Please help me understand this.  The way this paragraph is
written,
> it
> > >> is not clear to me why "signing each network element's host key
> with a
> > >> common trusted key" scales better than "configuring distinct host
> keys
> > >> on the management system".
> > >>
> > >> In either of these two situations, it seems like an administrator
> must
> > >> do work for each network element.  In one case, it is configuring
> host
> > >> keys, in another, it is signing each network element's host key.
> If
> > >> anything, it seems like signing a host key for each network
element
> > >> requires _more_ work for each network element, so it seems like
> this
> > >> would scale less well.
> > >>
> > >> What am I missing here?
> > >
> > >
> > > Very good question.  Of course it comes down to how the device's
> "entity
> > > certificates", as they are called, is distributed.  In the best
> case, as
> > > described in the zero-touch draft, the device would ship from
> factory with
> > > a built-in entity-certificate, signed by a trust-chain to its
> vendor's
> > > well-known trust anchor.  In this case, there is no additional
> effort
> > > needed, the management system only needs to trust the vendor's
> certificate
> > > for its well-known trust anchor.  For cases where the device
doesn't
> ship
> > > from factory with an entity-certificate, introducing PKI is still
> > > advantageous as it decouples the management-system from direct
> > > involvement, such that it could be outsourced to some 3rd-party.
> Makes
> > > sense?  What update would you like to see in the draft?
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >> Section 7, Page 7, Third Paragraph:  (wording)
> > >> ----------------------------------------------
> > >>
> > >> In a few places in the third paragraph, and possibly in other
> places
> > >> in the document, would changing "needs to" to lowercase "must",
or
> > >> lowercase "should", make the document a little more concise?
> Example:
> > >>
> > >>    "Note that since the SSH server would have to be configured to
> know
> > >>     which IP address it needs to connect to,"
> > >>                         ^^^^^^^^
> > >
> > >
> > > I changed "needs to" to "is to" for this cited location and
another
> I
> > > found.
> > >
> > >
> > >
> > >
> > >
> > > Thanks again,
> > > Kent
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > > _______________________________________________
> > > Netconf mailing list
> > > [email-masked]
> > > https://www.ietf.org/mailman/listinfo/netconf
> > >
> >
> > _______________________________________________
> > Netconf mailing list
> > [email-masked]
> > https://www.ietf.org/mailman/listinfo/netconf
>
> _______________________________________________
> Netconf mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/netconf

---

From: "Bert Wijnen (IETF)" <[email-masked]>
Date: Tue, 06 May 2014 13:11:34 +0200

om, is this still an issue with rev 5 of the reverse-ssh document?

Bert

---

From: t.petch <[email-masked]>
Date: Wed, 7 May 2014 10:20:50 +0100

----- Original Message -----
From: "Bert Wijnen (IETF)" <[email-masked]>
To: "t.petch" <[email-masked]>; "Kent Watsen" <[email-masked]>
Cc: <[email-masked]>
Sent: Tuesday, May 06, 2014 12:11 PM

> om, is this still an issue with rev 5 of the reverse-ssh document?

Yes

The changes I made for -05 were only the one change of terminology, so
that if it were to be unacceptable for any reason, then that change
could be easily reversed.

So, my still outstanding points are

- s.5, re-arrange to dovetail better with 5539bis

- wordsmith the Abstract/Introduction (as first suggested last
November:-) where I think the first reference to 'SSH Connection' is
wrong, so make it something like

"This memo presents a technique for a NETCONF server to request that a
NETCONF client initiates a SSH connection to the NETCONF server,
a technique referred to as 'call home'."

Tom Petch

---

From: Kent Watsen <[email-masked]>
Date: Wed, 7 May 2014 17:54:57 +0000

Hi Tom,

>- s.5, re-arrange to dovetail better with 5539bis

My understanding is that you believe that both drafts share the issue of
the northbound management application being able to identify and verify
the [SSH/TLS] server that uses call-home to connect to it.   I agree.

And that the text in the reverse-ssh draft, while ostensibly about SSH
host keys, could similarly apply to TLS and its use of X.509 certificates.
  I agree again, there is an overlap.

Thus you think that much of the text should be moved to 5539bis and for
the reverse-ssh draft to reference it there.  I don¹t agree, for two
reasons:

1) if there is a need to define common call-home behavior, we should have
a ³call-home² draft that covers both TLS and SSH call-home together.  I
recall this being one of the options discussed before, but the WG decided
to move ahead with this document structure.  In lieu of that, I think that
the reverse-ssh draft is closer to being a ³call-home² draft than 5539bis,
and so suggest putting common call-home information into it, perhaps
pulled out into a section called ³common call-home behavious² - what do
you think?

2) The text in the reverse-ssh draft is also much about the use of legacy
host-keys versus the new X.509 based keys with SSH.  Saying that use of
legacy keys is possible and allowed, but fraught with issues that are
resolved when using X.509 keys.  Maybe this needs to be may clearer, but I
don¹t think the information should be lost.

>a technique referred to as 'call home'."

I like this text, especially since we switched everything else to
"call-home" in -05.   I just updated my local copy this this change, but
will wait for resolution of the above before putting out -06

Thanks,
Kent

---

From: Kent Watsen <[email-masked]>
Date: Wed, 7 May 2014 22:15:46 +0000

Hi Tom, 

No need to wait for -06, here's the new text for the Abstract:

   This document presents a technique for a NETCONF server to request
   that a NETCONF client initiates a SSH connection to the NETCONF
   server, a technique referred to as 'call home'.  Call home is needed
   to support deployments where the NETCONF client is otherwise unable
   to initiate a SSH connection to the NETCONF server directly.

As you can see, I rewrote the rest of the paragraph as well, simplifying
it and focusing it more on motivation than solution.  What do you think?

Thanks,
Kent

---

From: t.petch <[email-masked]>
Date: Thu, 8 May 2014 09:59:31 +0100

----- Original Message -----
From: "Kent Watsen" <[email-masked]>
To: "t.petch" <[email-masked]>; "Bert Wijnen (IETF)"
<[email-masked]>
Cc: <[email-masked]>
Sent: Wednesday, May 07, 2014 6:54 PM

Hi Tom,

And that the text in the reverse-ssh draft, while ostensibly about SSH
host keys, could similarly apply to TLS and its use of X.509
certificates.
  I agree again, there is an overlap.

1) if there is a need to define common call-home behavior, we should
have
a ³call-home² draft that covers both TLS and SSH call-home together.  I
recall this being one of the options discussed before, but the WG
decided
to move ahead with this document structure.  In lieu of that, I think
that
the reverse-ssh draft is closer to being a ³call-home² draft than
5539bis,
and so suggest putting common call-home information into it, perhaps
pulled out into a section called ³common call-home behavious² - what do
you think?

2) The text in the reverse-ssh draft is also much about the use of
legacy
host-keys versus the new X.509 based keys with SSH.  Saying that use of
legacy keys is possible and allowed, but fraught with issues that are
resolved when using X.509 keys.  Maybe this needs to be may clearer, but
I
don¹t think the information should be lost.

<tp>
Kent

No, I think I am not making myself clear.

I am not saying that we need a common call home section.

I am saying that having obtained a public key somehow, then it needs
verifying, that it is tied to the party that we are trying to
communicate with, and that that process is largely the same whether this
is call home or not.

One form of verification is using X.509 certs, and that is the usual way
for TLS.  My logic then is put everything we have to say about the use
of X.509 certs for verification in the TLS I-D and reference it from the
(reverse-)ssh I-D.  The approach is the same for SSH and TLS and call
home and not call home IMHO so put it in the one place.

Currently, reverse-ssh lacks some of the points that 5539bis makes about
the use of certs, and the base ssh RFC says nothing, but reverse-ssh has
points that 5539bis does not such as the paragraph
" Since both the identification and verification issues are addressed
   using certificates, this draft RECOMMENDS network elements use a host
   key that can encode a unique identifier (e.g., its serial number) and
   be signed by a common trust anchor (e.g., a certificate authority).
   Examples of suitable public host keys are the X.509v3 keys defined in
   defined in [RFC6187]."
5539bis would be better for having that information in it alongside what
it currently has so I would move that to 5539bis leaving behind
something like

" Since both the identification and verification issues are addressed
   using certificates, this draft RECOMMENDS network elements use them -
   more details can be found in [5539bis]."

By contrast, the use of raw public keys is rare in TLS and commonplace
in SSH, so I would keep everything about that that is currently there in
reverse-ssh - mostly it is nothing to do with call home but the base ssh
RFC does not cover it so we might as well do the job properly now.

So between them, reverse-ssh and 5539bis currently have all we need, it
is just that you have to read both in tandem to learn what you should
know.

Tom Petch

Thanks,
Kent

---

From: t.petch <[email-masked]>
Date: Thu, 8 May 2014 10:02:32 +0100

----- Original Message -----
From: "Kent Watsen" <[email-masked]>
To: "t.petch" <[email-masked]>; "Bert Wijnen (IETF)"
<[email-masked]>
Cc: <[email-masked]>
Sent: Wednesday, May 07, 2014 11:15 PM
> it and focusing it more on motivation than solution.  What do you
think?

Looks good - go with it.

And the Introduction needs a comparable change

Tom Petch

certificates.
have
decided
that
5539bis,
legacy

---

From: "Liubing (Leo)" <[email-masked]>
Date: Fri, 9 May 2014 01:20:28 +0000

Hi all,

This mail is regarding to the Netconf keep-alive issue raised in several weeks ago, sorry for my late intervene to the discussion.
My comment/proposal is inline.

> -----Original Message-----
> From: Netconf [mailto:[email-masked]] On Behalf Of Kent
> Watsen
> Sent: Tuesday, April 15, 2014 3:27 AM
> To: t.petch; Bert Wijnen (IETF)
> Cc: [email-masked]
> Subject: Re: [Netconf] periodic connections, heartbeats, reconnection
...
> >Reading the details, I struggle to see why these functions should be
> >tied to
> > - SSH
> > - TLS
> > - listen
> > - call home
> > - NETCONF client
> > - NETCONF server
> >That is, why shouldn't any NETCONF engine send a heartbeat, timeout if
> >no response in a suitable period, drop a connection and reconnect when
> >it wants to?  These functions seem useful in any setting, rather than
> >specific to a particular scenario.
> 
> Any NETCONF engine?  Are you suggesting an <are-you-alive/> RPC?

[Bing] As explained in the follow-ups mails, Tom didn't mean to suggest such an RPC. But I just support to have such a kind of extension to Netconf.

I agree with Tom that we need a heartbeat/keep-alive mechanism, not only in some a specific scenario like reverse-SSH/call home/server/client.etc, but a quite generic requirement. 
One typical scenarios is, the NMS requiring a heavy search on the router, which might cost the router a long time to process or even make the router suspend. Then it would be good for the NMS to learn the status of the router through a periodic keep-alive, then it could close the session without waiting until the timeout in SSH/TLS if the router suspend.

If we agree it is a generic requirement, then I think it's better to have the keep-alive function in Netconf-level rather than in SSH/TLS-level. 
The main reason in my mind is that using SSH/TLS messages to inform Netconf status would invoke a layering violation. SSH/TLS heartbeat messages should only represent the status of the transport channel, and should not represent the status of the Netconf processing. 

What are your thoughts on this issue?

Best regards,
Bing

---

From: Andy Bierman <[email-masked]>
Date: Thu, 8 May 2014 19:00:12 -0700

On Thu, May 8, 2014 at 6:20 PM, Liubing (Leo) <[email-masked]>wrote:

> Hi all,
>
> This mail is regarding to the Netconf keep-alive issue raised in several
> weeks ago, sorry for my late intervene to the discussion.
> [Bing] As explained in the follow-ups mails, Tom didn't mean to suggest
> such an RPC. But I just support to have such a kind of extension to Netconf.
>
> I agree with Tom that we need a heartbeat/keep-alive mechanism, not only
> in some a specific scenario like reverse-SSH/call home/server/client.etc,
> but a quite generic requirement.
> One typical scenarios is, the NMS requiring a heavy search on the router,
> which might cost the router a long time to process or even make the router
> suspend. Then it would be good for the NMS to learn the status of the
> router through a periodic keep-alive, then it could close the session
> without waiting until the timeout in SSH/TLS if the router suspend.
>
> If we agree it is a generic requirement, then I think it's better to have
> the keep-alive function in Netconf-level rather than in SSH/TLS-level.
> The main reason in my mind is that using SSH/TLS messages to inform
> Netconf status would invoke a layering violation. SSH/TLS heartbeat
> messages should only represent the status of the transport channel, and
> should not represent the status of the Netconf processing.
>
I do not agree a complex protocol extension is needed.
Many implementations do not want to keep idle sessions alive.
That's why they have --idle-timeout CLI parameters.

SSH (OpenSSH anyway) has ClientAliveInterval and ClientAliveCountMax
CLI parameters to deal with this issue (in addition to TCPKeepAlive).

I want an idle session to get dropped by the server.
It is trivial to implement a no-op RPC as a client keep-alive:

    rpc no-op;

Clients can deal with keep-alive activity on their own.
I am not against standardizing a reasonable solution, but servers
generally support a limited number of concurrent sessions and
dropping idle sessions is a feature.

Best regards,
> Bing

Andy

---

From: "Liubing (Leo)" <[email-masked]>
Date: Fri, 9 May 2014 05:05:10 +0000

Hi Andy,

Thanks for your quick response. Please see replies inline.

From: Andy Bierman [mailto:[email-masked]]
Sent: Friday, May 09, 2014 10:00 AM
To: Liubing (Leo)
Cc: [email-masked]
Subject: Re: [Netconf] Netconf keep-alive (was periodic connections, heartbeats, reconnection, timeout)

On Thu, May 8, 2014 at 6:20 PM, Liubing (Leo) <[email-masked]<mailto:[email-masked]>> wrote:
Hi all,

> From: Netconf [mailto:[email-masked]<mailto:[email-masked]>] On Behalf Of Kent
> Watsen
> Cc: [email-masked]<mailto:[email-masked]>
...

I do not agree a complex protocol extension is needed.
[Bing] As in my mind, it doesn't need to be complex. Just a simple "keep-alive" indication might be sufficient.

I want an idle session to get dropped by the server.
It is trivial to implement a no-op RPC as a client keep-alive:
[Bing] The point is, the keep-alive is just NOT for idle session, it is for an active session.
If no Netconf keep-alives are received in a reasonable period, it means something wrong happened in the Netconf server , then the client could terminate the session without waiting until SSH ClientAliveCountMax. And as I said above, it might be better to de-couple the Netconf-alive and the SSH/TLS-alive, so that the status could be more precisely identified.

Best regards,
Bing

    rpc no-op;

Best regards,
Bing

Andy

---

From: t.petch <[email-masked]>
Date: Fri, 9 May 2014 10:16:22 +0100

----- Original Message -----
From: "Liubing (Leo)" <[email-masked]>
To: "Andy Bierman" <[email-masked]>
Cc: <[email-masked]>
Sent: Friday, May 09, 2014 6:05 AM

Hi Andy,

From: Andy Bierman [mailto:[email-masked]]
Sent: Friday, May 09, 2014 10:00 AM
To: Liubing (Leo)
Cc: [email-masked]
Subject: Re: [Netconf] Netconf keep-alive (was periodic connections,
heartbeats, reconnection, timeout)

On Thu, May 8, 2014 at 6:20 PM, Liubing (Leo)
<[email-masked]<mailto:[email-masked]>> wrote:
Hi all,

> From: Netconf
[mailto:[email-masked]<mailto:[email-masked]>] On
Behalf Of Kent
...

[Bing] As explained in the follow-ups mails, Tom didn't mean to suggest
such an RPC. But I just support to have such a kind of extension to
Netconf.

I agree with Tom that we need a heartbeat/keep-alive mechanism, not only
in some a specific scenario like reverse-SSH/call
home/server/client.etc, but a quite generic requirement.
One typical scenarios is, the NMS requiring a heavy search on the router
, which might cost the router a long time to process or even make the
router suspend. Then it would be good for the NMS to learn the status of
the router through a periodic keep-alive, then it could close the
session without waiting until the timeout in SSH/TLS if the router
suspend.

If we agree it is a generic requirement, then I think it's better to
have the keep-alive function in Netconf-level rather than in
SSH/TLS-level.
The main reason in my mind is that using SSH/TLS messages to inform
Netconf status would invoke a layering violation. SSH/TLS heartbeat
messages should only represent the status of the transport channel, and
should not represent the status of the Netconf processing.

I do not agree a complex protocol extension is needed.
[Bing] As in my mind, it doesn't need to be complex. Just a simple
"keep-alive" indication might be sufficient.

[Bing] The point is, the keep-alive is just NOT for idle session, it is
for an active session.
If no Netconf keep-alives are received in a reasonable period, it means
something wrong happened in the Netconf server , then the client could
terminate the session without waiting until SSH ClientAliveCountMax. And
as I said above, it might be better to de-couple the Netconf-alive and
the SSH/TLS-alive, so that the status could be more precisely
identified.

<tp>
It came as a suprise to those on the syslog list that a TCP connection
could go down, be down for hours and noone noticed so that when it was
needed to convey a critical fault, syslog was not working.

By analogy, there will be those using NETCONF who think that the
reliable transport means that when a critical alert needs to get
through, they can rely on the connection being there.

So I see keep-alives as something that the IETF should recommend for
anything involving alarms, alerts, traps, notifications etc.  The SNMP
NMS that I am familiar with poll at regular intervals - yes, over UDP,
but I would hope they did the same over TCP, TLS, etc

Ideally, NETCONF would do this at its layer, but we are not updating
that and we are updating SSH and TLS transports thereof, so,
compromising architectural purity, I would include it at that layer pro
tem.

As before, I do not see any disctinction is this regard between call
home and regular sessions.

Tom Petch

Best regards,
Bing

    rpc no-op;

Best regards,
Bing

Andy

---

From: Andy Bierman <[email-masked]>
Date: Fri, 9 May 2014 06:14:29 -0700

> compromising architectural purity, I would include it at that layer pro

Why?
NETCONF does not run directly over TCP.
Why is it NETCONF's job to make sure the TCP connection stays up?
It seems like that should be SSH or TLS doing that.
I have never seen this problem in real deployments because OpenSSH
is supposed to be configured correctly (TCPKeepAlive yes is the default).

Andy

---

From: Kent Watsen <[email-masked]>
Date: Fri, 9 May 2014 18:58:09 +0000

Before debating how we do keep-alives, did we agree that we want to support persistent connections?  Keep-alives are only needed for persistent connections, and I recall Martin questioning if persistent connections needed to be supported.  My answer at the time and now is that I think persistent connections are important for call-home scenario where it's the NMS wants to send commands to the device immediately, without waiting for the device's next "periodic" connection.  On the flip-side, we could take out persistent connections, and the NMS could configure the device to connect often (e.g., every 30 seconds) so that the wait isn't unnecessarily long.  This may or may not fly.  I don't know, but perhaps we should be conservative and define it anyways, leaving it as an option for deployments to decide if they want to use it or not.

Also, Tom wrote before that we should have symmetry, not a device-side only solution.   I think that the current solution is symmetric because, in the normal case where the NMS initiates the TCP connection, it already has the ability to keep the connection open as long as it likes and even send TCP/SSH/TLS-level keep-alives as it sees fit.   We don't need to define a configuration model for the NC client (i.e. the NMS).   We only need to define a config model for the NC server (i.e. the device).   Maybe I'm misunderstanding the symmetry concern, another way of looking at it is that the current netconf-server-model only configures keep-alives under the "call-home" node (not the "listen") node.   Tom, is this what you meant?

Thanks,
Kent

From: Andy Bierman <[email-masked]<mailto:[email-masked]>>
Date: Friday, May 9, 2014 at 9:14 AM
To: "t.petch" <[email-masked]<mailto:[email-masked]>>
Cc: NetConf <[email-masked]<mailto:[email-masked]>>
Subject: Re: [Netconf] Netconf keep-alive (was periodic connections, heartbeats, reconnection, timeout)

On Fri, May 9, 2014 at 2:16 AM, t.petch <[email-masked]<mailto:[email-masked]>> wrote:
----- Original Message -----
From: "Liubing (Leo)" <[email-masked]<mailto:[email-masked]>>
To: "Andy Bierman" <[email-masked]<mailto:[email-masked]>>
Cc: <[email-masked]<mailto:[email-masked]>>
Sent: Friday, May 09, 2014 6:05 AM

Hi Andy,

From: Andy Bierman [mailto:[email-masked]<mailto:[email-masked]>]

On Thu, May 8, 2014 at 6:20 PM, Liubing (Leo)
<[email-masked]<mailto:[email-masked]><mailto:[email-masked]<mailto:[email-masked]>>> wrote:
Hi all,

> -----Original Message-----
[mailto:[email-masked]<mailto:[email-masked]><mailto:[email-masked]<mailto:[email-masked]>>] On
Behalf Of Kent
> Cc: [email-masked]<mailto:[email-masked]><mailto:[email-masked]<mailto:[email-masked]>>
...

<tp>

Why?

Tom Petch

Best regards,
Bing

Andy

    rpc no-op;

Best regards,
Bing

Andy

---

From: Phil Shafer <[email-masked]>
Date: Fri, 9 May 2014 15:35:14 -0400

Kent Watsen writes:
>Before debating how we do keep-alives, did we agree that we want to support
>persistent connections?

Aren't keep-alives a transport issue?  For example SSH should use
the existing ssh keep-alives.

Thanks,
 Phil

---

From: Andy Bierman <[email-masked]>
Date: Fri, 9 May 2014 12:39:29 -0700

On Fri, May 9, 2014 at 11:58 AM, Kent Watsen <[email-masked]> wrote:

>
>  Before debating how we do keep-alives, did we agree that we want to
> support persistent connections?  Keep-alives are only needed for persistent
> connections, and I recall Martin questioning if persistent connections
> needed to be supported.  My answer at the time and now is that I think
> persistent connections are important for call-home scenario where it's the
> NMS wants to send commands to the device immediately, without waiting for
> the device's next "periodic" connection.  On the flip-side, we could take
> out persistent connections, and the NMS could configure the device to
> connect often (e.g., every 30 seconds) so that the wait isn't unnecessarily
> long.  This may or may not fly.  I don't know, but perhaps we should be
> conservative and define it anyways, leaving it as an option for deployments
> to decide if they want to use it or not.
>
>  Also, Tom wrote before that we should have symmetry, not a device-side
> only solution.   I think that the current solution is symmetric because, in
> the normal case where the NMS initiates the TCP connection, it already has
> the ability to keep the connection open as long as it likes and even send
> TCP/SSH/TLS-level keep-alives as it sees fit.   We don't need to define a
> configuration model for the NC client (i.e. the NMS).   We only need to
> define a config model for the NC server (i.e. the device).   Maybe I'm
> misunderstanding the symmetry concern, another way of looking at it is that
> the current netconf-server-model only configures keep-alives under the
> "call-home" node (not the "listen") node.   Tom, is this what you meant?
>
>

I also said I prefer not to have persistent connections,
at least for client-initiated sessions.

Side issue: a leaf in the /netconf-state tree indicating the idle-timeout
(if any) used by the server will help clients know how the server will
treat their session.

Andy

>
>
>   From: Andy Bierman <[email-masked]>
> To: "t.petch" <[email-masked]>
> Cc: NetConf <[email-masked]>

---

From: Andy Bierman <[email-masked]>
Date: Fri, 9 May 2014 12:39:50 -0700

On Fri, May 9, 2014 at 12:35 PM, Phil Shafer <[email-masked]> wrote:

> support
+1

---

From: Kent Watsen <[email-masked]>
Date: Fri, 9 May 2014 21:08:36 +0000

Hi Tom,

Regarding the paragraph you quoted below, note that it's about more than
just X.509.  Specifically, there are other SSH host-key formats that can
encode a unique identifier and be signed by a common trust anchor (e.g.,
the PGP keys from RFC 4253).   I admit the text could be made better - how
about this?

   Examples of suitable public host keys are the X.509v3 keys defined in
   defined in [RFC6187] and the PGP keys defined in [RFC5253].

                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

If this doesn't resolve the issue, can you also provide the text you hope
to now show up in 5539bis?  I see below what text is proposed to be
removed from draft-ietf-netconf-reverse-ssh, but it doesn't make sense to
just put that into 5539bis, what more do you hope to see in 5539bis?

Thanks,
Kent

---

From: t.petch <[email-masked]>
Date: Sat, 10 May 2014 12:23:57 +0100

----- Original Message -----
From: "Phil Shafer" <[email-masked]>
To: "Kent Watsen" <[email-masked]>
Cc: "Andy Bierman" <[email-masked]>; "t.petch"
<[email-masked]>; "Netconf" <[email-masked]>
Sent: Friday, May 09, 2014 8:35 PM
support
> the existing ssh keep-alives.

Isn't Transport Level Security a Transport issue?  yet the IETF takes it
upon itself to put a lot of effort into specifying how applications
could and should use it; and do not do so in the Transport Area (or the
Security Area)!

As I have said before, there are plenty of clued-up engineers who think
that a reliable TCP connection will tell them when it fails, as in the
syslog WG, and were surprised to learn that that was not the case.

So, knowing that, I think that we have a responsibility to document that
in a reasonably obvious place.  I am not suggesting that we need to
create a new protocol or protocol variant if there is already one
available but I do think, particuarly in the context of call home, that
this is a signficant issue that we would be irresponsible to omit.

And yes, I think that the rationale for call home makes persistent
connections an essential option, not something that can be ignored.

Tom Petch

---

From: t.petch <[email-masked]>
Date: Sat, 10 May 2014 12:38:14 +0100

----- Original Message -----
From: "Kent Watsen" <[email-masked]>
To: "Andy Bierman" <[email-masked]>; "t.petch" <[email-masked]>
Cc: "Netconf" <[email-masked]>
Sent: Friday, May 09, 2014 7:58 PM

Before debating how we do keep-alives, did we agree that we want to
support persistent connections?  Keep-alives are only needed for
persistent connections, and I recall Martin questioning if persistent
connections needed to be supported.  My answer at the time and now is
that I think persistent connections are important for call-home scenario
where it's the NMS wants to send commands to the device immediately,
without waiting for the device's next "periodic" connection.  On the
flip-side, we could take out persistent connections, and the NMS could
configure the device to connect often (e.g., every 30 seconds) so that
the wait isn't unnecessarily long.  This may or may not fly.  I don't
know, but perhaps we should be conservative and define it anyways,
leaving it as an option for deployments to decide if they want to use it
or not.

Also, Tom wrote before that we should have symmetry, not a device-side
only solution.   I think that the current solution is symmetric because,
in the normal case where the NMS initiates the TCP connection, it
already has the ability to keep the connection open as long as it likes
and even send TCP/SSH/TLS-level keep-alives as it sees fit.   We don't
need to define a configuration model for the NC client (i.e. the NMS).
We only need to define a config model for the NC server (i.e. the
device).   Maybe I'm misunderstanding the symmetry concern, another way
of looking at it is that the current netconf-server-model only
configures keep-alives under the "call-home" node (not the "listen")
node.   Tom, is this what you meant?

<tp>
Kent

My experience with SNMP is that every NMS polls; if something is down,
then likely, when you need the Notification most, you will not be able
to get it.

Some people think that this is a UDP issue. Switch to TCP and the NMS
will know when things are down - err, no!

So I think that NMS need keepalives, or the option of using keepalives -
they may know better and dispense with them but the option should be
there, and we SHOULD recommend their use - i.e. use them unless you know
what you are doing and know better.

The device I think is not so concerned and I would not mention it in
this context, except with call home, when I think that the device SHOULD
take responsibility for ensuring that a connection is there when the NMS
needs it - only it can do something about it.

In practice, the keep-alive protocols I am familiar with are mostly
symmetric, they take it for granted that both ends have an interest in
knowing what is going on and so the protocol is defined such that either
or both ends can detect an outage and decide what to do.  So while we
are at it, that is what I would recommend; it is what I think people
would expect of a keep-alive (and reusing an existing protocol if a
suitable one is available which is just what SNMP does).

Tom Petch

Thanks,
Kent

On Fri, May 9, 2014 at 2:16 AM, t.petch
<[email-masked]<mailto:[email-masked]>> wrote:
----- Original Message -----
From: "Liubing (Leo)"
<[email-masked]<mailto:[email-masked]>>
To: "Andy Bierman" <[email-masked]<mailto:[email-masked]>>
Cc: <[email-masked]<mailto:[email-masked]>>
Sent: Friday, May 09, 2014 6:05 AM

Hi Andy,

From: Andy Bierman
[mailto:[email-masked]<mailto:[email-masked]>]

On Thu, May 8, 2014 at 6:20 PM, Liubing (Leo)
<[email-masked]<mailto:[email-masked]><mailto:leo.liubin
[email-masked]<mailto:[email-masked]>>> wrote:
Hi all,

> -----Original Message-----
[mailto:[email-masked]<mailto:[email-masked]><mailto
:[email-masked]<mailto:[email-masked]>>] On
Behalf Of Kent
> Cc:
[email-masked]<mailto:[email-masked]><mailto:[email-masked]<mailto
:[email-masked]>>
...

<tp>

Why?
is supposed to be configured correctly (TCPKeepAlive yes is the
default).

Tom Petch

Best regards,
Bing

Andy

    rpc no-op;

Best regards,
Bing

Andy

---

From: Andy Bierman <[email-masked]>
Date: Sat, 10 May 2014 08:38:34 -0700

> connections an essential option, not something that can be ignored.

SSH has its own keep-alive. NETCONF uses that.
In implementation, the NETCONF layer has access to
an SSH channel, not directly to a TCP socket.  If the
TCP connection is lost, then NETCONF relies on the SSH session
and channels getting closed.  If SSH cannot detect the loss of TCP,
then how would NETCONF be able to do so?

The SSH keep-alives are essentially the same thing as if NETCONF
used the channel periodically to send a keep-alive message, so why
bother replicating that functionality at the application layer?

Tom Petch
Andy

---

From: t.petch <[email-masked]>
Date: Mon, 12 May 2014 10:40:49 +0100

---- Original Message -----
From: "Andy Bierman" <[email-masked]>
To: "t.petch" <[email-masked]>
Cc: "Kent Watsen" <[email-masked]>; "Phil Shafer"
<[email-masked]>; "Netconf" <[email-masked]>
Sent: Saturday, May 10, 2014 4:38 PM
> bother replicating that functionality at the application layer?

If the functionality is there and meets the requirements, then use it!

Kent was stepping back a little and asking as a matter of principle, do
we need keep-alives?

Yes.  Looking at the four ends of a connection, NMS and device, call
home and normal, IMHO, the most important case is 'device call home' but
I think that 'NMS normal' is not far behind.  'NMS call home' is
interesting because it can do nothing about it, but is again signficant.
'device normal' would be the lowest priority of the four - I would
expect such a device to log the failure and wait for the connection to
be resumed but I would still want that to happen.  But my starting point
would be to RECOMMEND all four, unless there are onerous technical
difficulties in doing so, in which case I would apply the priorities
above to revisit that recommendation.

I think that persistence is a subjective concept.  Keeping a connection
up for an hour might be persistent for some, for me persistence would
cover more like a shift, a day or a month, and it is these sorts of
timescales that I am used to seeing with NMS.

Tom Petch

---

From: t.petch <[email-masked]>
Date: Mon, 12 May 2014 12:29:52 +0100

----- Original Message -----
From: "Kent Watsen" <[email-masked]>
To: "t.petch" <[email-masked]>; "Bert Wijnen (IETF)"
<[email-masked]>
Cc: <[email-masked]>
Sent: Friday, May 09, 2014 10:08 PM

Hi Tom,

Regarding the paragraph you quoted below, note that it's about more than
just X.509.  Specifically, there are other SSH host-key formats that can
encode a unique identifier and be signed by a common trust anchor (e.g.,
the PGP keys from RFC 4253).   I admit the text could be made better -
how
about this?

<tp>

Kent

This is where we start to part company.  A host key is a host key, it is
public, it is generated for me as part of a key pair for use in public
key cryptography.  Of itself, it conveys nothing, no verification, no
validation, no identification.

It only acquires value when it can be tied into an identity or
identifier, which X.509 certificates do.  Or it can preconfigured into
the NMS (the key or a fingerprint thereof) but is the act of configuring
that gives the key some value, there is nothing inherent in the key.

You say in the I-D that this does not scale, but that is true of
(almost) everything.  If you use X.509 certificates, then they bind the
key to an identity; guess what, that identity has to be configured into
the NMS, so if configuring raw keys does not scale then neither do X.509
certificates:-(

The only thing that scales is Trust On First Use, which I think too
flaky for us to recommend.  Manual acceptance when a key is first used
is more secure, but, arguably, scales no better.

We have been here before - look at RFC5592, which basically says
'preconfigure host keys' to which I would now add 'or else use
certificates', RFC6125 etc.

So my section 5 (skating over PGP use) would be more like

"When the management system accepts a new incoming TCP connection on the
NETCONF over SSH Call Home port, it starts the SSH client protocol. As
the SSH client, it MUST authenticate the SSH server, by both identifying
the network element and verifying its SSH host key.

When not using call home, the management system initiates the TCP
connection and so may place some reliance on the fact that the device
that responds is the one that it is configured to communicate with.
With call home, the network element initiates the TCP connection and no
reliance can be placed on the source IP address of the TCP connection.
Identifying the remote peer by its source IP address, as recommended in
[RFC6187] is NOT RECOMMENDED in the case of call home, unless the
network takes steps to validate IP source addresses.

The management system may be configured with the host keys of the
network elements, or fingerprints thereof, and these MAY be used as an
identifier.

The management system MAY request manually verification of a host key or
a fingerprint thereof the first time the network element connects, and
any time the host key changes thereafter.

Digital certificates, such as those in X.509 version 3 (X.509v3) bind a
public key to a digital identity [RFC5280] and are used in many
environments for identity management.  This approach is discussed in
more detail in [5539bis]; this approach is RECOMMENDED."

Tom Petch

If this doesn't resolve the issue, can you also provide the text you
hope
to now show up in 5539bis?  I see below what text is proposed to be
removed from draft-ietf-netconf-reverse-ssh, but it doesn't make sense
to
just put that into 5539bis, what more do you hope to see in 5539bis?

Thanks,
Kent

---

From: Kent Watsen <[email-masked]>
Date: Mon, 12 May 2014 16:37:38 +0000

Hi Tom,

>that gives the key some value, there is nothing inherent in the key.

RFC 4251 section 4.1 on "Host Keys" extends the definition to include
certificates too. 

>the NMS, so if configuring raw keys does not scale then neither do X.509

Yes, the NMS needs to be programed with the identity of the CA, but that
one CA can vouch for any number of devices.   This *is* scalable, from the
NMS's POV.   What is potentially not scalable, is the need to provision
each device with a signed certificate, as the naïve solution would again
be per-device, though it has the up-side that the certificate-provisioning
could be delegated to a 3rd-party, leveraging PKI to its fullest.
However, any solution that needs the device to be "touched" before being
put into commission is not ideal.  The  only no-touch solution I know of
is for the device to already have a signed-certificate when released from
its manufacturer's factory.  This is the strategy is codified in the
zero-touch draft.

>is more secure, but, arguably, scales no better.

Agreed, TOFU is not ideal.

>more detail in [5539bis]; this approach is RECOMMENDED."

This text is almost the same as what is there now.  One thing different is
I don't see the word "scale" anymore.   So you want 5539bis to say PKI is
scalable and hence particularly suitable for NMS and then have this draft
reference it for that fact?   You didn't provide text for 5539bis, so I'm
just guessing, is there anything else?

Thanks,
Kent

---

From: Kent Watsen <[email-masked]>
Date: Tue, 13 May 2014 22:33:44 +0000

Hi Tom, 

I think we should close this discussion with resolution of not moving any
text to 5539bis.   

I would be OK moving text to a generic "call home" draft, but the WG
already decided against that approach.  Too bad, as there is quite a bit
in addition to S.5; for instance, look at S.3 "Benefits to Device
Management" and parts of S.7 "Security Considerations".  All of it seems
to apply to nearly any call-home solution.  Oh well.

If you recall, this thread began by your saying reverse-ssh wasn't ready
for LC.  Since then, we've:
 - changed all the "reverse ssh" references to "call home"
 - updated the abstract and introduction sections
 - added that PGP keys also satisfy the RECOMMENDED key algorithm criteria
 - clarified why a device config model is out of scope (e.g., Informative)
 
But this last bit about moving text to 5539bis seems questionable and not
worth it discussing further.  So, can we agree to close this particular
discussion now?   If so, then I'll post -06 so the draft can enter LC
again.

Thanks,
Kent

---

From: t.petch <[email-masked]>
Date: Wed, 14 May 2014 10:10:43 +0100

----- Original Message -----
From: "Kent Watsen" <[email-masked]>
To: "t.petch" <[email-masked]>; "Bert Wijnen (IETF)"
<[email-masked]>
Cc: <[email-masked]>
Sent: Monday, May 12, 2014 5:37 PM

Hi Tom,

Yes, the NMS needs to be programed with the identity of the CA, but that
one CA can vouch for any number of devices.   This *is* scalable, from
the
NMS's POV.   What is potentially not scalable, is the need to provision
each device with a signed certificate, as the naïve solution would again
be per-device, though it has the up-side that the
certificate-provisioning
could be delegated to a 3rd-party, leveraging PKI to its fullest.
However, any solution that needs the device to be "touched" before being
put into commission is not ideal.  The  only no-touch solution I know of
is for the device to already have a signed-certificate when released
from
its manufacturer's factory.  This is the strategy is codified in the
zero-touch draft.

<tp>
Kent,

No, we are still not meeting anywhere in the middle.

It seems to me that you are proposing that all devices share the same
private key and so the same public key (in which case the choice of CA
seems irrelevant).

AFAICT, anything else requires that the NMS is configured with some
identifier for each and every device, which could be the public key,
could be something that appears in the subjectAltName, could be the IP
address (although I think not for 'call home') etc.

I see you describing this per device configuration as 'does not scale'
but I see no other way.

As
http://www.rfc-editor.org/rfc/rfc5592.txt
says "this requires the client
   to know the host's public key a priori and to verify that the correct
   key is being used."
which is the sort of thing I would expect to see, in the absence of
X.509 certificates.  With certificates, then you know the subjectAltName
a priori.  Either way, you configure the NMS for every device.

Tom Petch

>more detail in [5539bis]; this approach is RECOMMENDED."

This text is almost the same as what is there now.  One thing different
is
I don't see the word "scale" anymore.   So you want 5539bis to say PKI
is
scalable and hence particularly suitable for NMS and then have this
draft
reference it for that fact?   You didn't provide text for 5539bis, so
I'm
just guessing, is there anything else?

Thanks,
Kent

---

From: Kent Watsen <[email-masked]>
Date: Wed, 14 May 2014 21:30:31 +0000

Hi Tom,

>No, we are still not meeting anywhere in the middle.

On this we can agree, but hopefully we'll get there soon  ;)

>private key and so the same public key (in which case the choice of CA

Of course not, each device must have unique private keys.  The zero-touch
draft goes into more specifics regarding certificate construction and use
of PKI.

>a priori.  Either way, you configure the NMS for every device.

The zero-touch draft touches on this as well:

Section 3.5:

   The NMS SHOULD also validate that the unique identifier (e.g., serial
   number), within the "Common Name" field of the device's X.509
   certificate, is an identity that the NMS is expecting, and not
   another device having the same device type.  In order for the NMS to
   know the unique identifiers for devices shipped directly to their
   destinations, it may be necessary for the device manufacturer to
   provide the unique identifiers along with other shipping or billing
   information.  This draft not specify a format for this information
   exchange.

Section 4.1:

   The unique identifier MUST be something that is both known to the
   device and easily tracked through labels affixed to the device as
   well as the box it is packaged in.  A device's serial number is
   commonly treated this way and would be suitable for this purpose.

I guess my point is that we agree on how the solution can scale.  The
reverse-ssh draft leaves it up to deployments to decide what to do, while
the zero-touch draft specifies a specific solution that satisfies all its
particular use-cases.

Thanks,
Kent

---

From: Kent Watsen <[email-masked]>
Date: Wed, 14 May 2014 23:37:20 +0000

[Sorry for the delay - busy busy]

>If the functionality is there and meets the requirements, then use it!

That's our working assumption until shot down  ;)

>Kent was stepping back a little and asking as a matter of principle, do

Yes, I wanted to reaffirm that we needed to discuss keep-alives at all.  I
agree with Tom that we should support persistent connections.  I believe
that deployments should be able to decide for themselves if they want to
use persistent connections or not, but not due to an artificial limitation
we introduced to the model...

>above to revisit that recommendation.

I like your looking at the four ends.  Presenting them another way, we
have:

For listen (normal) case:
  - device:
     - keep-alives can be configured (e.g., via sshd_config)
     - YANG for configuring this could be an augmentation to the
system-mgmt draft
     - this is because it effects the system-wide sshd listening on port 22
  - NMS:
     - keep-alives can be configured (e.g., via ssh_config)
     - no need to define a YANG model since we do that for NC clients
     - note: it's most likely the NMS is linking to a SSH client library

For call home case:
  - device
     - keep-alives can be configured (e.g., via sshd_config)
     - YANG for configuring this is in the netconf-server-model draft

     - this is because it's a distinct `sshd` process per call-home
  - NMS
     - keep-alives can be configured (e.g., via ssh_config)
     - no need to define a YANG model since we do that for NC clients

     - note: it's most likely the NMS is linking to a SSH client library

So, at least for SSH, each of the four ends can have keep-alives - this is
what you wanted, right?  I don't know yet if TLS keep-alives are
symmetric, but I imagine so.  As you say, it's normal for a protocol to
offer it at both ends...

>timescales that I am used to seeing with NMS.

The intent is that it is "forever" until the device is configured
otherwise.  Should I clarify the text? - currently it says this:

   Applications may vary greatly on how frequently they need to interact
   with a device, how responsive interactions with devices need to be,
   and how many simultaneous connections they can support.  Some
   applications may need a persistent connection to devices to optimize
   real-time interactions, while others are satisfied with periodic
   interactions and reduced resources required.  Therefore, when it is
   necessary for devices to initiate connections, the type of connection
   desired should be configured.

Thanks,
Kent

---

From: Martin Bjorklund <[email-masked]>
Date: Thu, 15 May 2014 08:54:59 +0200 (CEST)

> That's our working assumption until shot down  ;)

This is regarding SSH keepalives, right?  As I've said before, I agree
it can be used, but you need to provide a reference to what you mean
when you say "SSH keepalive".  AFAIK, this is not defined in any RFC,
but implementations use a trick (hopefully the same) where they send a
bogus packet and expect a failure reply back.

/martin

---

From: t.petch <[email-masked]>
Date: Thu, 15 May 2014 15:11:20 +0100

----- Original Message -----
From: "Martin Bjorklund" <[email-masked]>
To: <[email-masked]>
Cc: <[email-masked]>; <[email-masked]>; <[email-masked]>
Sent: Thursday, May 15, 2014 7:54 AM

> bogus packet and expect a failure reply back.

Martin,

I think not, that this is a NETCONF issue or at least a call home issue,
that a call home device should be using keepalives regardless of the
transport in use (and, not quite so important, I think that a normal NMS
should be using them as well).  So while we may have started talking
about it in the context of SSH call home, I think that Kent is right to
widen the discussion.

If we have it for call home over one (unreliable:-) transport, then I
think we need it for all.

Whether or not the mechanics are the same is, for me, a separate
question.  Neither SSH nor TLS saw fit to include keepalives initially,
TLS has added them later (RFC6520), ssh can fudge it.

Tom Petch.

---

From: t.petch <[email-masked]>
Date: Thu, 15 May 2014 15:22:29 +0100

----- Original Message -----
From: "Kent Watsen" <[email-masked]>
To: "t.petch" <[email-masked]>; "Andy Bierman" <[email-masked]>
Cc: "Phil Shafer" <[email-masked]>; "Netconf" <[email-masked]>
Sent: Thursday, May 15, 2014 12:37 AM

>Kent was stepping back a little and asking as a matter of principle, do

Yes, I wanted to reaffirm that we needed to discuss keep-alives at all.
I
agree with Tom that we should support persistent connections.  I believe
that deployments should be able to decide for themselves if they want to
use persistent connections or not, but not due to an artificial
limitation
we introduced to the model...

     - this is because it effects the system-wide sshd listening on port
22

For call home case:
  - device

So, at least for SSH, each of the four ends can have keep-alives - this
is
what you wanted, right?  I don't know yet if TLS keep-alives are
symmetric, but I imagine so.  As you say, it's normal for a protocol to
offer it at both ends...

<tp>
TLS defines keepalives in RFC6520 as  a separate protocol running on top
of the Record layer, requested by a TLS Hello extension.  The extension
says that the sender (TLS client) wants to be able to send
HeartbeatRequest and get back HeartbeatResponse and it also allows or
disallows the other end to send a HeartbeatRequest.  So symmetric
operation is configurable.

Tom Petch

Thanks,
Kent

---

From: t.petch <[email-masked]>
Date: Fri, 16 May 2014 17:38:38 +0100

----- Original Message -----
From: "Kent Watsen" <[email-masked]>
To: "t.petch" <[email-masked]>; "Bert Wijnen (IETF)"
<[email-masked]>
Cc: <[email-masked]>
Sent: Wednesday, May 14, 2014 10:30 PM

Hi Tom,

>private key and so the same public key (in which case the choice of CA

Of course not, each device must have unique private keys.  The
zero-touch
draft goes into more specifics regarding certificate construction and
use
of PKI.

<tp>

So what:-)  Perhaps we are not agreeing because you have zero touch in
mind, whereas I expect certificates to be used without zero touch (as
they long have been with TLS).  I expect this I-D to make sense when
taken with its Normative references and, for me, at present, it does
not.

If you have 1000 devices, then the NMS needs configuring with 1000
identifiers, and if that does not scale for host keys, then I do not see
it scaling for any other identifier, be it the subjectAlternateName in a
certificate or anything else, so for me, the references to 'does not
scale' do not make sense.  There is no magic short cut to configuring
the NMS once for 1000 devices, unless they share some property such as a
private key.

Likewise, if the identifier is embedded in the host key by some means,
you still have 1000 identifiers to configure in the NMS

I do not think it enough to leave it up to implementations.  We should
recommend something that is secure, and do it not just because a
Security AD says so.  My experience is that Security ADs usually ask for
too much but equally, most WGs start out with too little and for me at
present, we are too close to the latter.

Tom Petch

Section 3.5:

Section 4.1:

I guess my point is that we agree on how the solution can scale.  The
reverse-ssh draft leaves it up to deployments to decide what to do,
while
the zero-touch draft specifies a specific solution that satisfies all
its
particular use-cases.

Thanks,
Kent

---

From: "Liubing (Leo)" <[email-masked]>
Date: Fri, 23 May 2014 04:02:16 +0000

Hi all,

This mail is regarding to the keep-alive discussion recently.
Following the discussion, I noticed there might be two different things that I mixed them together. Let me try to distinguish them as the following.

1)      To keep an idle session alive, so that a persistent connection could be retained. This should be the common understanding of keep-alive. In this case, basically I agree with what Andy said in the below mail, that re-using SSH keep-alive might be sufficient for Netconf.

2)      To make an active session indicating the Netconf running state instantly. For example, as the case I described in the previous mail, the NMS requests a search operation which might cost the router a long time to process or even make the router suspend. Then it would be good for the NMS to learn the state of the router through a periodic message. In the simplest case, the state is just about whether the router is still alive on the operation.

The second one is just the actual topic I intended to discuss. I called such message as "keep-alive", which might not be very proper. Right or wrong, the point is about the running state indication, just similar as the real-time feedbacks in CLI when the operation time is long.
If there is such a message mechanism, the Netconf server could indicate the client something like:

-        it is still processing the operation ("keep-alive")

-        the rate of progress

-        some key processing information

-        ...etc.

Do you think it is good to have such a feature in Netconf?

Regards,
Bing

From: Netconf [mailto:[email-masked]] On Behalf Of Andy Bierman
Sent: Saturday, May 10, 2014 11:39 PM
To: t.petch
Cc: Netconf
Subject: Re: [Netconf] Netconf keep-alive (was periodic connections, heartbeats, reconnection, timeout)
...
SSH has its own keep-alive. NETCONF uses that.
In implementation, the NETCONF layer has access to
an SSH channel, not directly to a TCP socket.  If the
TCP connection is lost, then NETCONF relies on the SSH session
and channels getting closed.  If SSH cannot detect the loss of TCP,
then how would NETCONF be able to do so?

---

From: t.petch <[email-masked]>
Date: Fri, 23 May 2014 10:52:56 +0100

---- Original Message -----
From: "Liubing (Leo)" <[email-masked]>
To: "Netconf" <[email-masked]>
Cc: "Andy Bierman" <[email-masked]>; "t.petch"
<[email-masked]>; "Yangang" <[email-masked]>; "Zhengguangying"
<[email-masked]>
Sent: Friday, May 23, 2014 5:02 AM

This mail is regarding to the keep-alive discussion recently.
Following the discussion, I noticed there might be two different things
that I mixed them together. Let me try to distinguish them as the
following.

1)      To keep an idle session alive, so that a persistent connection
could be retained. This should be the common understanding of
keep-alive. In this case, basically I agree with what Andy said in the
below mail, that re-using SSH keep-alive might be sufficient for
Netconf.

2)      To make an active session indicating the Netconf running state
instantly. For example, as the case I described in the previous mail,
the NMS requests a search operation which might cost the router a long
time to process or even make the router suspend. Then it would be good
for the NMS to learn the state of the router through a periodic message.
In the simplest case, the state is just about whether the router is
still alive on the operation.

The second one is just the actual topic I intended to discuss. I called
such message as "keep-alive", which might not be very proper. Right or
wrong, the point is about the running state indication, just similar as
the real-time feedbacks in CLI when the operation time is long.
If there is such a message mechanism, the Netconf server could indicate
the client something like:

-        the rate of progress

-        ...etc.

<tp>
No:-(  I think that such progress reports are lovely in theory but
practically meaningless; they are just too hard to implement reliably.
I see them for file downloads and similar operations where the estimated
time to completion goes up and down like a yoyo, or an e-mail download
yesterday, from an (antisocial:-) participant on an IETF list who posted
a megagbyte of mostly nothingness  and I got  a progress report up to
the point where it alleged that about half the e-mail had been
downloaded and then - the download report vanished.  Had the download
succeeded, had it been terminated by the server (a common occurrence) or
what?  I am still wondering.

I see the underlying issue that any long running task is going to be a
background task, running as and when, so forecasts of how long it will
take are forecasts of what higher priority task will preempt it, and
that is unpredictable, so the forecasts are bad.  Is a bad forecast
better than no forecast?  Judging by the questions and comments from my
non-technical friends and colleagues, I say no forecast.

So a keepalive to say that the link is up is good news, because it tends
to be small and reliable and a reliable indicator.  If the keepalive
comes from the application layer, then that is better because it is
reporting more of the stack's liveliness, but if the keepalive is tied
into a background task in any way, then no, it defeats the concept of a
keepalive and becomes as meaningless as my file download progress
reports.

Tom Petch

Regards,
Bing

From: Netconf [mailto:[email-masked]] On Behalf Of Andy
Bierman

---

From: "Liubing (Leo)" <[email-masked]>
Date: Fri, 23 May 2014 10:26:02 +0000

Hi Tom,

Thanks for your comments.

The cases listed below are just some rough divergent thinking. Our basic idea is just to inform the Netconf client that the processing is going on, please wait. And if there's no indication messages, the client could judge the server are down or suspend. 
However, if we have such a messaging mechanism, then we could indicate more things than just Netconf-level keep-alive. But the "more things" might need further discussion.

May I ask are you opposing the whole idea of running state indication, or just the specific "progress of rate" use case?

Regards,
Bing

> From: t.petch [mailto:[email-masked]]
> Sent: Friday, May 23, 2014 5:53 PM
> To: Liubing (Leo); Netconf
> Cc: Andy Bierman; Yangang; Zhengguangying
> Subject: Re: Netconf running state indication-//RE: [Netconf] Netconf
> keep-alive (was periodic connections, heartbeats, reconnection, timeout)
> 
> Following the discussion, I noticed there might be two different things that I
> mixed them together. Let me try to distinguish them as the following.
> 
> could be retained. This should be the common understanding of keep-alive.
> In this case, basically I agree with what Andy said in the below mail, that
> re-using SSH keep-alive might be sufficient for Netconf.
> 
> instantly. For example, as the case I described in the previous mail, the NMS
> requests a search operation which might cost the router a long time to
> process or even make the router suspend. Then it would be good for the
> NMS to learn the state of the router through a periodic message.
> In the simplest case, the state is just about whether the router is still alive on
> the operation.
> 
> The second one is just the actual topic I intended to discuss. I called such
> message as "keep-alive", which might not be very proper. Right or wrong,
> the point is about the running state indication, just similar as the real-time
> feedbacks in CLI when the operation time is long.
> No:-(  I think that such progress reports are lovely in theory but practically
> meaningless; they are just too hard to implement reliably.
> yesterday, from an (antisocial:-) participant on an IETF list who posted a
> megagbyte of mostly nothingness  and I got  a progress report up to the
> point where it alleged that about half the e-mail had been downloaded and
> then - the download report vanished.  Had the download succeeded, had it
> been terminated by the server (a common occurrence) or what?  I am still
> wondering.
> 
> background task, running as and when, so forecasts of how long it will take
> are forecasts of what higher priority task will preempt it, and that is
> unpredictable, so the forecasts are bad.  Is a bad forecast better than no
> forecast?  Judging by the questions and comments from my non-technical
> friends and colleagues, I say no forecast.
> 
> So a keepalive to say that the link is up is good news, because it tends to be
> small and reliable and a reliable indicator.  If the keepalive comes from the
> application layer, then that is better because it is reporting more of the
> stack's liveliness, but if the keepalive is tied into a background task in any
> way, then no, it defeats the concept of a keepalive and becomes as
> meaningless as my file download progress reports.
> 
> Tom Petch
> 
> 
> 
> 
> 
> Regards,
> Bing
> 
> heartbeats, reconnection, timeout) ...
> In implementation, the NETCONF layer has access to an SSH channel, not
> directly to a TCP socket.  If the TCP connection is lost, then NETCONF relies
> on the SSH session and channels getting closed.  If SSH cannot detect the
> loss of TCP, then how would NETCONF be able to do so?
> 
> The SSH keep-alives are essentially the same thing as if NETCONF used the
> channel periodically to send a keep-alive message, so why bother replicating
> that functionality at the application layer?
> 
>

---

From: t.petch <[email-masked]>
Date: Fri, 23 May 2014 15:25:00 +0100

----- Original Message -----
From: "Liubing (Leo)" <[email-masked]>
To: "t.petch" <[email-masked]>; "Netconf" <[email-masked]>
Cc: "Andy Bierman" <[email-masked]>; "Yangang" <[email-masked]>;
"Zhengguangying" <[email-masked]>
Sent: Friday, May 23, 2014 11:26 AM
Subject: RE: Netconf running state indication-//RE: [Netconf] Netconf
keep-alive (was periodic connections, heartbeats, reconnection, timeout)

Hi Tom,

The cases listed below are just some rough divergent thinking. Our basic
idea is just to inform the Netconf client that the processing is going
on, please wait. And if there's no indication messages, the client could
judge the server are down or suspend.
However, if we have such a messaging mechanism, then we could indicate
more things than just Netconf-level keep-alive. But the "more things"
might need further discussion.

May I ask are you opposing the whole idea of running state indication,
or just the specific "progress of rate" use case?

<tp>
I am against progress indicators unless they can be accurate and I doubt
that they can be accurate for long running (and hence background) tasks.

I do like liveness indicators and notice how they have gone from nowhere
to widespread in the time I have been involved with the IETF, so with
almost any protocol, I ask 'where is the keepalive?'.  And the higher up
the stack it is the better, as long as it does not get tied into a low
priority activity!

So if you wanted to tell the client that this request had not been
lost - I don't know how common an occurrence this is at this time - then
how would you identify the request or requests - like an interim
response, perhaps, a well-formed rpc reply with the right ID in it?  but
then how do you ensure that that does not get tied up with the
outstanding request that may or may not still be running?

You seem to be positing a server processing one request, whereas I think
of servers doing many things, so that, at least with other protocols, I
perform my own liveness check by asking something very simple - e.g.
SNMP get sysUpTime - over a parallel session.  Which my client might
already be doing for me.

There is a background philosophy which SNMP articulates, that you need
this most when things are going badly, so make it such that it is most
likely to succeed, ie Simple.

Tom Petch

Regards,
Bing

> could be retained. This should be the common understanding of
keep-alive.
> instantly. For example, as the case I described in the previous mail,
the NMS
> In the simplest case, the state is just about whether the router is
still alive on
> If there is such a message mechanism, the Netconf server could
indicate the
> client something like:
practically
> background task, running as and when, so forecasts of how long it will
take

---

From: Radek Krejčí <[email-masked]>
Date: Mon, 26 May 2014 10:23:09 +0200

Hi,

IMO the NETCONF Event Notifications mechanism can already work as
keep-alive/heartbeat - what if a server has an event stream named
keepalive or heartbeat (or whatever) and in this stream the server sends
the keep-alive notification to the interested clients. keep-alive period
can be configurable. Of course, there is no sense for replay feature in
this event stream.

I don't like operation keep alive (possibly with a progress info). Tom
already stated some reasons. From my point of view, it needlessly
complicates the keep-alive mechanism in NETCONF. I don't think that it
is a good idea to send progress information or even operation processing
info to _all_ clients - it should be received only by the client that
invoked the operation. In that case we have different keep-alive
messages sent to different clients. And, as I said, it needlessly
complicates implementation.

I think that (if we need NETCONF keep-alive) we just need it to say
"NETCONF server is alive". I believe, that, as Kent stated, SSH
keep-alives are good enough for saying "NETCONF session is alive" -
just, please, Keep It Simple, Stupid.

Regards,
Radek

Dne 23.5.2014 06:02, Liubing (Leo) napsal(a):
> Following the discussion, I noticed there might be two different
> things that I mixed them together. Let me try to distinguish them as
> the following.
>
>  
>
> below mail, that re-using SSH keep-alive might be sufficient for Netconf.
>
> for the NMS to learn the state of the router through a periodic
> message. In the simplest case, the state is just about whether the
> router is still alive on the operation.
>
>  
>
> The second one is just the actual topic I intended to discuss. I
> called such message as “keep-alive”, which might not be very proper.
> Right or wrong, the point is about the running state indication, just
> similar as the real-time feedbacks in CLI when the operation time is
> long.  
>
> indicate the client something like:
>
> -        it is still processing the operation (“keep-alive”)
>
> -        the rate of progress
>
> *From:*Netconf [mailto:[email-masked]] *On Behalf Of *Andy
> Bierman
> *Sent:* Saturday, May 10, 2014 11:39 PM
> *To:* t.petch
> *Cc:* Netconf
> *Subject:* Re: [Netconf] Netconf keep-alive (was periodic connections,

-- 
Radek Krejci
mobile: +420 732 212 714
office: +420 234 680 256
e-mail: [email-masked]

CESNET
Association of Legal Entities
160 00 Praha 6, Zikova 4
Czech Republic

---

From: "Liubing (Leo)" <[email-masked]>
Date: Mon, 26 May 2014 10:01:08 +0000

Hi Tom,

> <tp>
> I am against progress indicators unless they can be accurate and I doubt that
> they can be accurate for long running (and hence background) tasks.
> 
> I do like liveness indicators and notice how they have gone from nowhere to
> widespread in the time I have been involved with the IETF, so with almost
> any protocol, I ask 'where is the keepalive?'.  And the higher up the stack it
> is the better, as long as it does not get tied into a low priority activity!
> 
> So if you wanted to tell the client that this request had not been lost - I don't
> know how common an occurrence this is at this time - then how would you
> identify the request or requests - like an interim response, perhaps, a
> well-formed rpc reply with the right ID in it?  but then how do you ensure
> that that does not get tied up with the outstanding request that may or may
> not still be running?
[Bing] I haven't started the specific design, but the rough idea in my mind is kind of <rpc-reply>, which is just in the band of the specific Netconf session. 
 
> You seem to be positing a server processing one request, whereas I think of
> servers doing many things, so that, at least with other protocols, I perform
> my own liveness check by asking something very simple - e.g.
[Bing] I agree sometimes trick methods in implementation are simple and effective. But after all these methods cannot be guaranteed to be available. A standardize way would be much reliable.

However, maybe it is a lit earlier to discuss the solution. It seems we haven't agreed on the requirement of such running state indication. We might need more work on the use cases.  

Regards,
Bing

> 
> There is a background philosophy which SNMP articulates, that you need this
> most when things are going badly, so make it such that it is most likely to
> succeed, ie Simple.
> 
> Tom Petch
> 
> Regards,
> Bing
> 
> things that I
> called such

---

From: "Liubing (Leo)" <[email-masked]>
Date: Mon, 26 May 2014 12:08:22 +0000

Hi Radek,

Thanks for your comment, please see inline.

From: Radek Krejčí [mailto:[email-masked]]
Sent: Monday, May 26, 2014 4:23 PM
To: Liubing (Leo); Netconf
Cc: Zhengguangying; Yangang
Subject: Re: [Netconf] Netconf running state indication-//RE: Netconf keep-alive (was periodic connections, heartbeats, reconnection, timeout)

Hi,

IMO the NETCONF Event Notifications mechanism can already work as keep-alive/heartbeat - what if a server has an event stream named keepalive or heartbeat (or whatever) and in this stream the server sends the keep-alive notification to the interested clients. keep-alive period can be configurable. Of course, there is no sense for replay feature in this event stream.
[Bing] Event Notification might be too heavy to do the running state indication. My rough idea is that a messaging mechanism within the operation session would be good.

I don't like operation keep alive (possibly with a progress info). Tom already stated some reasons. From my point of view, it needlessly complicates the keep-alive mechanism in NETCONF. I don't think that it is a good idea to send progress information or even operation processing info to _all_ clients - it should be received only by the client that invoked the operation.
[Bing] My proposal is NOT to send the information to _all-clients. It only indicate the client who sent the <rpc> reqest, it is only within the relevant operation. I think we actually share the same opinion on this issue.
In that case we have different keep-alive messages sent to different clients. And, as I said, it needlessly complicates implementation.

I think that (if we need NETCONF keep-alive) we just need it to say "NETCONF server is alive". I believe, that, as Kent stated, SSH keep-alives are good enough for saying "NETCONF session is alive" - just, please, Keep It Simple, Stupid.
[Bing] As replied to Tom, it might depend on the use cases. We need to work more on use cases.

Best regards,
Bing

Regards,
Radek

The second one is just the actual topic I intended to discuss. I called such message as “keep-alive”, which might not be very proper. Right or wrong, the point is about the running state indication, just similar as the real-time feedbacks in CLI when the operation time is long.
If there is such a message mechanism, the Netconf server could indicate the client something like:

-        the rate of progress

-        ...etc.

Regards,
Bing

Netconf mailing list

[email-masked]<mailto:[email-masked]>

--

Radek Krejci

mobile: +420 732 212 714

office: +420 234 680 256

e-mail: [email-masked]<mailto:[email-masked]>

CESNET

160 00 Praha 6, Zikova 4

Czech Republic