Subject: [Netconf] NETCONF and interactive CLI's

From: Mahesh Jethanandani <[email-masked]>
Date: Mon, 8 Sep 2014 14:47:25 -0700

I was asked to fashion a problem statement around how NETCONF deals with
what I call "interactive CLI" models.

That model includes commands that are today issued on a CLI and involve a
command being issued, and a continuous response coming back till such time
it is interrupted either by the CLI or because there is a terminating
condition. In the period that the command is running, the CLI may be
blocked from issuing any other commands.

Commands issued like these are fairly common debug commands issued as part
of testing the configuration on the box and are unlike getting statistics
which are not interactive in nature.

A very common example of this is the ping command. Other examples include
traceroute command. A successful ping would have a response like this.

 > ping google.com
PING google.com (74.125.239.98) 56(84) bytes of data.
64 bytes from nuq05s01-in-f2.1e100.net (74.125.239.98): icmp_req=1 ttl=55
time=4.38 ms
64 bytes from nuq05s01-in-f2.1e100.net (74.125.239.98): icmp_req=2 ttl=55
time=4.26 ms
64 bytes from nuq05s01-in-f2.1e100.net (74.125.239.98): icmp_req=3 ttl=55
time=4.38 ms

--- google.com ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2002ms
rtt min/avg/max/mdev = 4.260/4.343/4.389/0.096 ms

Here also the response is split in two parts, one that is interactive
response which spits out one line at a time as the ping reply comes back
and the second which for the lack of a better term a "terminating response"
- a cumulative response of the complete interaction, prompted by the
issuance of a terminating condition i.e. CTRL-C or TTL expiry.

In the case ping fails, there is no interactive response, but there is a
terminating response like in this case.

 > ping pyramid.com
PING pyramid.com (109.228.17.0) 56(84) bytes of data.

--- pyramid.com ping statistics ---
7 packets transmitted, 0 received, 100% packet loss, time 5999ms

Without getting into specific implementation details, these commands today
would follow a pattern of

<rpc>
...
<rpc/>

<rpc-reply>
....
<rpc-reply/>

<rpc-reply>
....
<rpc-reply/>

<rpc-reply>
....
<rpc-reply/>

<rpc>
CTRL-C for the above NETCONF session
<rpc/>

<rpc-reply>
....
<rpc-reply/>

How does NETCONF deal with such a command? I would not confuse this with
another thread that was talking about "bulk response" because this issue is
more than how to deal with fragments of large responses. It has to do with
the interactive nature that the current NETCONF protocol does not seem to
support. If it does not, what are semantics needed to support these kind of
commands.

-- 
Mahesh Jethanandani
[email-masked]

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Mon, 8 Sep 2014 23:58:55 +0200

On Mon, Sep 08, 2014 at 02:47:25PM -0700, Mahesh Jethanandani wrote:

[...]
 
> support. If it does not, what are semantics needed to support these kind of

I believe a proper solution should address both your requirement as
well as the fragmentation requirement discussed in the other thread
and I also think a proper solution needs support from the messages
layer so that we do not have to hand-craft solutions for an increasing
number of RPCs.

/js

-- 
Juergen Schoenwaelder           Jacobs University Bremen gGmbH
Phone: +49 421 200 3587         Campus Ring 1, 28759 Bremen, Germany
Fax:   +49 421 200 3103         <http://www.jacobs-university.de/>

---

From: Andy Bierman <[email-masked]>
Date: Mon, 8 Sep 2014 15:03:35 -0700

On Mon, Sep 8, 2014 at 2:47 PM, Mahesh Jethanandani <[email-masked]
> wrote:

> support. If it does not, what are semantics needed to support these kind of
NETCONF does not support multiple <rpc-reply> messages for the same <rpc>.
Once the client receives the first reply with a matching message-id, then
it does
not expect to receive more messages for the <rpc>.

NETCONF servers do not maintain state for an <rpc>.
There is no concept of "Control-C for the above NETCONF session".
I think there are proposals already to do ping and traceroute in some
module.

IMO, updating the NETCONF protocol so it behaves more like a CLI is
not a high priority.  A YANG module to do ping or traceroute is good enough.
It should use the available protocol mechanisms and not require a new
protocol revision.

Andy

> --
> Mahesh Jethanandani
> [email-masked]
>
> _______________________________________________
> Netconf mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/netconf
>
>

---

From: Kent Watsen <[email-masked]>
Date: Mon, 8 Sep 2014 22:30:34 +0000

I'm sure others have their opinions, but to me this follows the pattern of
a long-running request, whereby I'd expect a general mechanism to
query/cancel currently running tasks, something like this:

	<rpc>
		<something like ping or traceroute/>
	</rpc>
	<rpc-reply>
		<task-id>23424</task-id>
	</rpc-reply>

	<rpc>
		<get-active-tasks/>	// listing of all currently running tasks
	</rpc>
	<rpc-reply>
		<active-task>
			<task-id>23424</task-id>
			<wait-secs>1</wait-secs>
			<run-secs>23</run-secs>
			<percent>55</percent>
			<user>admin</user>
			<command>ping foobar.com</command>
		</active-task>
		...
	</rpc-reply>

	<rpc>
		<get-active-task-details>
			<task-id>23424</task-id>	// details on this one task
		</get-active-task-details>
	</rpc>
	<rpc-reply>
		<active-task-details>
			<wait-seconds>1</wait-seconds>
			<run-seconds>23</run-seconds>
			<percentage--complete>55</percentage-complete>
			<user>admin</user>
			<command>ping foobar.com</command>
			<execution-status></execution-status>
			<completion-status></completion-status>
			<last-update>
			64 bytes from foobar.com (11.222.333.44): icmp_req=1 ttl=55 time=4.38 ms
			</last-update>
 		</active-task-details>
	</rpc-reply>

	<rpc>
		<cancel-active-task>
			<task-id>23424</task-id>
		</cancel-active-task>
	</rpc>
	<rpc-reply>
		<ok/>
	<rpc-reply>

	<rpc>
		<get-active-task-details>
			<task-id>23424</task-id>	// details on this one task
		</get-active-task-details>
	</rpc>
	<rpc-reply>
		<rpc-error>
			<error-message>task id does not exist</error-message>
		</rpc-error> 
	<rpc-reply>

Just an idea, what do you think?

Kent

From:  Mahesh Jethanandani <[email-masked]>
Date:  Monday, September 8, 2014 at 5:47 PM
To:  NetConf <[email-masked]>
Subject:  [Netconf] NETCONF and interactive CLI's

That model includes commands that are today issued on a CLI and involve a
command being issued, and a continuous response coming back till such time
it is interrupted either by the CLI or because there is a terminating
condition. In the period that the
 command is running, the CLI may be blocked from issuing any other
commands.

 > ping google.com <http://google.com>
PING google.com <http://google.com> (74.125.239.98) 56(84) bytes of data.
64 bytes from nuq05s01-in-f2.1e100.net <http://nuq05s01-in-f2.1e100.net>
(74.125.239.98): icmp_req=1 ttl=55 time=4.38 ms
64 bytes from nuq05s01-in-f2.1e100.net <http://nuq05s01-in-f2.1e100.net>
(74.125.239.98): icmp_req=2 ttl=55 time=4.26 ms
64 bytes from nuq05s01-in-f2.1e100.net <http://nuq05s01-in-f2.1e100.net>
(74.125.239.98): icmp_req=3 ttl=55 time=4.38 ms

--- google.com <http://google.com> ping statistics ---

Here also the response is split in two parts, one that is interactive
response which spits out one line at a time as the ping reply comes back
and the second which for the lack of a better term a "terminating
response" - a cumulative response of the complete
 interaction, prompted by the issuance of a terminating condition i.e.
CTRL-C or TTL expiry.

 > ping pyramid.com <http://pyramid.com>
PING pyramid.com <http://pyramid.com> (109.228.17.0) 56(84) bytes of data.

--- pyramid.com <http://pyramid.com> ping statistics ---

<rpc>
...
<rpc/>

<rpc-reply>
.... 
<rpc-reply/>

<rpc-reply>
....
<rpc-reply/>

<rpc-reply>
....
<rpc-reply/>

<rpc>

<rpc-reply>
....
<rpc-reply/>

How does NETCONF deal with such a command? I would not confuse this with
another thread that was talking about "bulk response" because this issue
is more than how to deal with fragments of large responses. It has to do
with the interactive nature that
 the current NETCONF protocol does not seem to support. If it does not,
what are semantics needed to support these kind of commands.

-- 
Mahesh Jethanandani

[email-masked]

---

From: Andy Bierman <[email-masked]>
Date: Mon, 8 Sep 2014 15:41:38 -0700

On Mon, Sep 8, 2014 at 2:58 PM, Juergen Schoenwaelder <
[email-masked]> wrote:

> is

I think a generic approach could be done with existing mechanisms.
The YANG would need 'anyxml' but we are used to that.

  rpc start-command {
     input {
         anyxml request;
     }
     output {
         leaf handle {
            mandatory true;
            type string;
         }
     }
  }

  rpc terminate-command {
     input {
        leaf handle {
            mandatory true;
            type string;
        }
     }
  }

  notification command-data {
       leaf handle {
          mandatory true;
          type string;
       }
       leaf is-last {
          type boolean;
          default false;
       }
       leaf data-id {
          type string;
       }
       anyxml data;
  }

Example:

  <start-command>
      <request>
        <ping>
           ....
         </ping>
       </request>
   </start-command>

returns handle XJE43

notifications follow:

  <command-data>
      <handle>XJE43</handle>
      <data-id>1</data-id>
      <data>
          ....
      </data>
   </command-data>

  <command-data>
      <handle>XJE43</handle>
      <is-last>true</is-last>
      <data-id>2</data-id>
      <data>
          ....
      </data>
   </command-data>

Termination via handle:

 <terminate-command >
      <handle>XJE43</handle>
   </terminate-command>

Andy

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Tue, 9 Sep 2014 01:53:52 +0200

>   notification command-data {

How do you define the 'commands' in YANG? Why are they not simply
RPCs? You left out the detail that notifications require a
notification subscription. Do you require :interleave? I guess you
have to.

Changing the message layer to support multiple responses to an RPC
invocation and providing a cancel mechanism at the RPC layer is
relatively simple and fixes the problem where it is rather than adding
a new protocol layer on top of RPCs and notifications.

/js

---

From: Andy Bierman <[email-masked]>
Date: Mon, 8 Sep 2014 18:11:37 -0700

> a new protocol layer on top of RPCs and notifications.
That solution requires a new version of NETCONF.
Next time we update the protocol, then changes to the <rpc> layer can be
made.
I was trying to suggest a solution that uses the existing protocol.
Maybe this can be done with an optional capability,
but if even 1 sentence in RFC 6241 says <rpc> is handled a specific way
that does not include the proposed changes, then a capability will not work.

Andy

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Tue, 9 Sep 2014 07:41:35 +0200

> that does not include the proposed changes, then a capability will not work.

Yes. I also always thought a proper solution requires a revision of
RFC 6241. I just now tried to search in RFC 6241 where it actually
says that multiple <rpc-reply> messages with the same message-id is an
error. I did not find anything that makes a clear statements. So
perhaps an update of RFC 6241 could even work.

/js

---

From: Martin Bjorklund <[email-masked]>
Date: Tue, 09 Sep 2014 08:31:05 +0200 (CEST)

> perhaps an update of RFC 6241 could even work.

Even if this was doable, it is still not possible to make the server
process a new rpc like "<cancel-job>" while it is sending the
rpc-replies.

/martin

---

From: Martin Bjorklund <[email-masked]>
Date: Tue, 09 Sep 2014 08:38:53 +0200 (CEST)

[...]

On the call yesterday I suggested something similiar, but more direct:

  rpc start-ping-job {
    input {
      // ping parameters here...
    }
    output {
      leaf ping-job-handle { ... }
    }
  }

  rpc get-one-ping-result {
    input {
      leaf ping-job-handle { ... }
    }
    output {
      // ping result here
    }
  }  

  rpc cancel-ping-job {
    input {
      leaf ping-job-handle { ... }
    }
  }

The call sequence would be:

   start-ping-job         ---> <handle>
   get-one-ping-result    ---> <data>
   get-one-ping-result    ---> <data>
   ...
   [cancel-ping-job        ---> ok]

Yes, this is not as elegant as built-in support, but it works today.

Built-in support needs an updated protocol *and* updatad data
modelling language.

/martin

---

From: "Bert Wijnen (IETF)" <[email-masked]>
Date: Tue, 09 Sep 2014 09:43:12 +0200

Interesting discussion.

In my head I have his question:

    NETCONF was for Network CONFIGURATION

    So what do traceroute, ping and possibly other long running commands
    have to do with network configuration??

In my mind they seem to be debugging or monitoring tools.

Bert (speaking as a contributor)

---

From: Mikael Abrahamsson <[email-masked]>
Date: Tue, 9 Sep 2014 09:59:53 +0200 (CEST)

> In my mind they seem to be debugging or monitoring tools.

I want to stop using SNMP and CLI for network monitoring. What should I 
use?

-- 
Mikael Abrahamsson    email: [email-masked]

---

From: David Bannister <[email-masked]>
Date: Tue, 9 Sep 2014 07:46:20 -0400

> Yes, this is not as elegant as built-in support, but it works today.

That would kind of work.  What you are suggesting is highly transactional in nature which means one of two things.  There is a human at the other end watching dots & bangs form on the screen in a blocking state until he/she sends the cancel request.  Or, a process at the other end sending a large number of micro-rpc messages waiting for some event or group of events to happen before sending the cancel rpc.  My goal is to eliminate the first and hopefully avoid the second.    I would lean towards an approach that uses a model to define the monitoring or testing action bounding it by time, failure conditions, etc.  Also contained in that model is an identifier, a single use crypto key or nonce, a transport method and a destination host(s) etc.  Once the network element completes the test it forwards the results to host(s) responsible for analysis.  This approach also allows me to constantly send out monitoring and performance tests in an asynchronous fashion and farm the results out to free compute in the datacenter.  I would think Netconf could handle this RJE type action, we just need to develop the models.

---

From: "ietfdbh" <[email-masked]>
Date: Tue, 9 Sep 2014 11:34:16 -0400

+1

Are we now trying to build a full CLI replacement, rather than just a
protocol to standardize configuration?

David Harrington
[email-masked]
+1-603-828-1401

> -----Original Message-----
> From: Netconf [mailto:[email-masked]] On Behalf Of Bert Wijnen
> (IETF)
> Sent: Tuesday, September 09, 2014 3:43 AM
> To: Martin Bjorklund; [email-masked]
> Cc: [email-masked]
> Subject: Re: [Netconf] NETCONF and interactive CLI's
> 
> Interesting discussion.
>

---

From: Andy Bierman <[email-masked]>
Date: Tue, 9 Sep 2014 08:47:34 -0700

> In my mind they seem to be debugging or monitoring tools.
What if the operator needs to measure some behavior
of the network in order to adjust the configuration somehow?

Remote ping and traceroute have been needed and available in the CLI for
a long time, so using a Remote Procedure Call in NETCONF to manage it
is not such a stretch.

Andy

---

From: Balazs Lengyel <[email-masked]>
Date: Tue, 9 Sep 2014 18:12:20 +0200

Hello,

We have similar long running operations in configuration management
as well. E.g. upgrade, restore-backup, create-backup, restart-card.

IMHO this could be solved by using an operation-handle to query the
state of the operation, to cancel it,  and notifications for interim
results. As I remember Andy proposed something similar.

regards Balazs

On 2014-09-09 17:47, Andy Bierman
wrote:

On Tue, Sep 9, 2014 at 12:43 AM, Bert
Wijnen (IETF) <[email-masked]>
wrote:

Interesting
discussion.

So what do traceroute, ping and possibly other long
running commands

of the network in order to adjust the configuration
somehow?

Remote ping and traceroute have been needed and
available in the CLI for

a long time, so using a Remote Procedure Call in
NETCONF to manage it

is not such a stretch.

Andy

Kent Watsen <[email-masked]>
wrote:

I'm sure others have their opinions, but to me this
follows the pattern of

a long-running request, whereby I'd expect a general
mechanism to

query/cancel currently running tasks, something like
this:

[...]

On the call yesterday I suggested something similiar,
but more direct:

rpc start-ping-job {

input {

}

output {

}

}

input {

}

output {

// ping result here

}

}

rpc cancel-ping-job {

input {

}

}

...

Yes, this is not as elegant as built-in support, but it
works today.

Built-in support needs an updated protocol *and* updatad
data

modelling language.

/martin

Netconf mailing list

[email-masked]

Netconf mailing list

[email-masked]

--
Balazs Lengyel Ericsson Hungary Ltd.
Senior Specialist
ECN: 831 7320 Tel: +36-1-437-7320
Mobile: +36-70-330-7909 email: [email-masked]

---

From: Juergen Schoenwaelder <[email-masked]>
Date: Tue, 9 Sep 2014 18:34:01 +0200

> In my mind they seem to be debugging or monitoring tools.

Successful protocols are often used beyond their original scope.

/js

---

From: "Romascanu, Dan (Dan)" <[email-masked]>
Date: Wed, 10 Sep 2014 10:19:00 +0000

+1 

What we and other had in mind in 2004 when the NETCONF work started is important for historians. 
What is important now is what is useful and works for the operators of 2014 and beyond. 

Regards,

Dan

> From: Netconf [mailto:[email-masked]] On Behalf Of Juergen
> Schoenwaelder
> Sent: Tuesday, September 09, 2014 7:34 PM
> To: Bert Wijnen (IETF)
> Cc: [email-masked]