Subject: Re: [netconf] New Version Notification for draft-zhou-netconf-multi-stream-originators-06.txt

From: Qin Wu <[email-masked]>
Date: Tue, 9 Jul 2019 06:02:59 +0000

I think your idea in this draft is to eliminate broker for internal consolidation at the sending side, but you introduce the new attribute message ID to multiplex multi-stream into one stream at the receiving side.
Two quick questions:
1. In dynamic subscription case, how does Publisher agent behind Publisher master know where a particular receiver is when pushing subscribed data to the receiver in a separate channel?
2. How multi-stream originator works for configured subscription case? How does receive know the number of publisher agents or message originators? Can receiver negotiate with publisher agent directly? What's the role of publisher master in between?

-Qin
-----邮件原件-----
发件人: netconf [mailto:[email-masked]] 代表 Tianran Zhou
发送时间: 2019年7月8日 11:07
收件人: [email-masked]
抄送: [email-masked]
主题: [netconf] FW: New Version Notification for draft-zhou-netconf-multi-stream-originators-06.txt

Hi WG,

Thanks Kent for the remind of adding examples to illustrate the model usage.
1. This revision mainly work on the examples. 
2. We also add a grouping on the resource-access-list. So that some transport, e.g., RESTCONF, is able to access multiple message generators.

I think the document is ready for the working group adoption for further improvement.

There is an open discussion, whether to include the transport configuration in this document?
Existing subscribed notifications does not include this part. So this is for example defined in the https notif with a new "receivers" container.
https://datatracker.ietf.org/doc/draft-mahesh-netconf-https-notif/

In multiple message originators scenario, I prefer to leaving the transport configuration to specific transport just as the subscribed notifications draft.
So I add an example on how to configure the https notif in multiple message originators scenario.

Thanks,
Tianran
-----Original Message-----
From: [email-masked] [mailto:[email-masked]] 
Sent: Monday, July 08, 2019 10:39 AM
To: Andy Bierman <[email-masked]>; Eric Voit <[email-masked]>; Zhengguangying (Walker) <[email-masked]>; Alexander Clemm <[email-masked]>; Tianran Zhou <[email-masked]>
Subject: New Version Notification for draft-zhou-netconf-multi-stream-originators-06.txt

A new version of I-D, draft-zhou-netconf-multi-stream-originators-06.txt
has been successfully submitted by Tianran Zhou and posted to the IETF repository.

Name:		draft-zhou-netconf-multi-stream-originators
Revision:	06
Title:		Subscription to Multiple Stream Originators
Document date:	2019-07-07
Group:		Individual Submission
Pages:		21
URL:            https://www.ietf.org/internet-drafts/draft-zhou-netconf-multi-stream-originators-06.txt
Status:         https://datatracker.ietf.org/doc/draft-zhou-netconf-multi-stream-originators/
Htmlized:       https://tools.ietf.org/html/draft-zhou-netconf-multi-stream-originators-06
Htmlized:       https://datatracker.ietf.org/doc/html/draft-zhou-netconf-multi-stream-originators
Diff:           https://www.ietf.org/rfcdiff?url2=draft-zhou-netconf-multi-stream-originators-06

Abstract:
   This document describes the distributed data export mechanism that
   allows multiple data streams to be managed using a single
   subscription.  Specifically, multiple data streams are pushed
   directly to the collector without passing through a broker for
   internal consolidation.

Please note that it may take a couple of minutes from the time of submission until the htmlized version and diff are available at tools.ietf.org.

The IETF Secretariat

_______________________________________________
netconf mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/netconf

---

From: Tianran Zhou <[email-masked]>
Date: Tue, 9 Jul 2019 06:38:36 +0000

Hi Qin,

Thank you very much for your interest. Please see my reply in line.
Hope it can address your question.

Tianran
> From: Qin Wu
> Sent: Tuesday, July 09, 2019 2:03 PM
> To: Tianran Zhou <[email-masked]>; [email-masked]
> Subject: RE: New Version Notification for
> draft-zhou-netconf-multi-stream-originators-06.txt
> 
> I think your idea in this draft is to eliminate broker for internal
> consolidation at the sending side, but you introduce the new attribute message
> ID to multiplex multi-stream into one stream at the receiving side.
> Two quick questions:
> 1. In dynamic subscription case, how does Publisher agent behind Publisher
> master know where a particular receiver is when pushing subscribed data to
> the receiver in a separate channel?
I think Appendix A.1 gives one example to show how this works in RESTCONF case.
Firstly, the reply to the dynamic subscription will return the resource access URIs.
Then, the receiver can use the URIs to GET the data.
This is similar to draft-ietf-netconf-restconf-notif-15.

> 2. How multi-stream originator works for configured subscription case? 
In appendix A.2, I use the similar example as in draft-mahesh-netconf-https-notif-00 to illustrate.
In brief, the originators run on client mode, and connect to the receiver actively.

> How does receive know the number of publisher agents or message originators? 
> Can receiver negotiate with publisher agent directly? What's the role of
> publisher master in between?
Before the receiver actually start to receive data, the subscription-started message is sent to the receiver by the publisher master.
We extend existing sub-notif model with a list of message-originator-ids in the subscription-started message.
So that the receive know the number of publisher agents.
In this sense, the receiver does not negotiate with publisher agent directly, but with the master.

> 主题: [netconf] FW: New Version Notification for
> draft-zhou-netconf-multi-stream-originators-06.txt
> 
> Hi WG,
> 
> 2. We also add a grouping on the resource-access-list. So that some transport,
> e.g., RESTCONF, is able to access multiple message generators.
> 
> Existing subscribed notifications does not include this part. So this is for
> example defined in the https notif with a new "receivers" container.
> In multiple message originators scenario, I prefer to leaving the transport
> configuration to specific transport just as the subscribed notifications
> draft.
> To: Andy Bierman <[email-masked]>; Eric Voit <[email-masked]>;
> Zhengguangying (Walker) <[email-masked]>; Alexander Clemm
> <[email-masked]>; Tianran Zhou <[email-masked]>
> Subject: New Version Notification for
> draft-zhou-netconf-multi-stream-originators-06.txt
> 
> 
> URL:
> https://www.ietf.org/internet-drafts/draft-zhou-netconf-multi-stream-ori
> ginators-06.txt
> Status:
> https://datatracker.ietf.org/doc/draft-zhou-netconf-multi-stream-origina
> tors/
> Htmlized:
> https://tools.ietf.org/html/draft-zhou-netconf-multi-stream-originators-
> 06
> Htmlized:
> https://datatracker.ietf.org/doc/html/draft-zhou-netconf-multi-stream-or
> iginators
> Diff:
> https://www.ietf.org/rfcdiff?url2=draft-zhou-netconf-multi-stream-origin
> ators-06
> 
> Abstract:
> Please note that it may take a couple of minutes from the time of submission
> until the htmlized version and diff are available at tools.ietf.org.
> 
> The IETF Secretariat
>

---

From: Kent Watsen <[email-masked]>
Date: Wed, 10 Jul 2019 02:24:13 +0000

Hi Tianran,

> This is similar to draft-ietf-netconf-restconf-notif-15.

I was also unsure (the draft needs to be more clear with regards to expected behavior).  That said, I have a fews concerns:

1) will the client (a.k.a. the receiver) establish a full RESTCONF protocol connection to each line-card?  What YANG-library response might the line-cards return?
2) are we sure the client connections will always be routable to the line cards?
3) are there any security issues with line-cards having open ports?

Given the above, and the potential symmetry to the configured case, did you consider letting the server establish the connection to the client?  From an operator's perspective, perhaps both options need to be supported?   

> In brief, the originators run on client mode, and connect to the receiver actively.

This example appears to be an exact copy/paste from draft-mahesh-netconf-https-notif, at least.  I didn't notice any differences and, that being the case, am unsure (again) what the expected behavior was intended to be.  Could you add a sequence diagram illustrating internal interactions?

Kent // contributor

---

From: Tianran Zhou <[email-masked]>
Date: Wed, 10 Jul 2019 09:19:44 +0000

Hi Kent，
Thanks for your comments. Please see inline.

Tianran

From: Kent Watsen [mailto:[email-masked]]
Sent: Wednesday, July 10, 2019 10:24 AM
To: Tianran Zhou <[email-masked]>
Cc: Qin Wu <[email-masked]>; [email-masked]
Subject: Re: [netconf] New Version Notification for draft-zhou-netconf-multi-stream-originators-06.txt

Hi Tianran,

[Tianran]: I am not sure if I am doing the right thing on the examples. Maybe you can give some more suggestions.
This draft intent to provide a transport independent solution. Actual transport design, say RESTCONF, NETCONF, HTTPS, may have several options and need some more considerations. I am not sure if we should consider them all in this draft. So the question is what kind of example is helpful? I did not see a similar example in the draft-ietf-netconf-subscribed-notifications.

The following questions are respect to one potential RESTCONF transport solution I am considering. I give only my considerations.
1) will the client (a.k.a. the receiver) establish a full RESTCONF protocol connection to each line-card?  What YANG-library response might the line-cards return?
[Tianran]: I do not think it’s necessary to establish a full RESTCONF connection to each line-card. May be the http connection with server push capability is OK.

2) are we sure the client connections will always be routable to the line cards?
[Tianran]: I assume the address is routable.

3) are there any security issues with line-cards having open ports?
[Tianran]: The master returns the URIs containing the line-card port to the receiver within an security channel.

Given the above, and the potential symmetry to the configured case, did you consider letting the server establish the connection to the client?  From an operator's perspective, perhaps both options need to be supported?
[Tianran]: Good question. It’s impossible to include all the transports in this draft. This example seek to be compatible with draft-ietf-netconf-restconf-notif.
On the other hand, maybe we can consider only to use the https client as in draft-mahesh-netconf-https-notif as the only tcp transport for all the scenario.

[Tianran]Again, I want to be compatible with existing transport proposals as in draft-mahesh-netconf-https-notif.  So I suggested to include a list of channels in the receiver configuration. On one hand, it can support the multiple message originators scenario. On the other hand, one channel configuration can be compatible with the one message originator scenario.

Only this modification :)
       <receiver>
         <name>foo</name>
         <channel>
           <tcp-params>

I can add a sequence diagram to show how it works for this transport/example.

Kent // contributor

---

From: Kent Watsen <[email-masked]>
Date: Sun, 14 Jul 2019 19:14:40 +0000

=== Re: dynamic subscription case ===

> This draft intent to provide a transport independent solution. Actual transport design, say RESTCONF, NETCONF, HTTPS, may have several options and need some more considerations. I am not sure if we…

Do not take draft-ietf-netconf-subscribed-notifications as an example to follow when it comes to examples.  The lack of examples in that draft continues to cause confusion.  As RFC 8407 says: "Each specification that defines one or more modules SHOULD contain usage examples, either throughout the document or in an appendix.".

It is understood that, in addition to variations in encoding (e.g., XML, JSON), this document (and SN) have variations in transports (RC, HTTP, etc.).  This additional dimension is easily addressed, just preface your examples with something like "For instance, assuming the use of the transport defined by [draft-foo-bar] and using the XML (or JSON) encoding, the following example illustrates yada yada..."

> [Tianran]: I do not think it’s necessary to establish a full RESTCONF connection to each line-card. May be the http connection with server push capability is OK.

Fine, but my point is that your draft needs to explicitly say it and, more importantly, state exactly what is or is not allowed over such less than full connections.

> [Tianran]: I assume the address is routable.

Again, the draft needs to state this.

> [Tianran]: The master returns the URIs containing the line-card port to the receiver within an security channel.

True, but the line-card has an open port that could be detected via a port sweep and thereafter attacked (DoS?).  The Security Considerations section needs to mention this.

> On the other hand, maybe we can consider only to use the https client as in draft-mahesh-netconf-https-notif as the only tcp transport for all the scenario.

Hmmm, my guess is that pretty much all transports supporting configured subscriptions will have the server (i.e., line cards) proactively initiate the connections to the notifications-receivers (and not have the receivers connecting to the servers).   This is more a macro-observation than a draft-mahesh-netconf-https-notif specific thing, though that draft follows this pattern too. 

=== Re: configured subscription case ===
 
> Only this modification J

Ah, a "channel" container was added, I didn't see that before.  The draft should've defined YANG module to add a "channel" container.  That said, I don't think what you're hoping to achieve here is legal/valid YANG (please ensure that all examples are validated using e.g., `yanglint`).

Kent // contributor

---

From: Tianran Zhou <[email-masked]>
Date: Mon, 15 Jul 2019 03:25:55 +0000

Hi Kent,

Thank you very much for your suggestions. I will update the draft and discuss them during the meeting.
However, I am still not clear about the following discussions.

[Tianran2]: I am sorry I not quite clear. Here we are talking about the dynamic subscription. I conclude there are four potential modes.
1. The linecard runs on server mode. The connections are dynamically set up, when the receiver know the subscription decomposition result at run time.
2. The linecard runs on server mode. All the potential connections are set up. The receiver only get data dynamically.
3. The linecard runs on client mode. The connections are dynamically set up, when the linecard know the subscription decomposition result at run time.
4. The linecard runs on client mode. All the potential connections are set up. The linecards send data dynamically.

[Tianran2]: I did not define a YANG module for this, because this may make the this draft depend on draft-mahesh-netconf-https-notif. But I sent the comment to the mailing list to suggest draft-mahesh-netconf-https-notif to apply this modification.
https://mailarchive.ietf.org/arch/msg/netconf/YmIh2l1xGJUq5gYwvrzcwnRphPg

Or do you think this draft can define a configuration framework, not include the details? So that transport like draft-mahesh-netconf-https-notif can augment.

Thanks,
Tianran

Sent: Monday, July 15, 2019 3:15 AM

Only this modification ☺
       <receiver>
         <name>foo</name>
         <channel>
           <tcp-params>

Kent // contributor

---

From: Kent Watsen <[email-masked]>
Date: Mon, 15 Jul 2019 16:33:48 +0000

[CC-ing Eric due to urgent SN issue]

Hi Tianran,

> 4. The linecard runs on client mode. All the potential connections are set up. The linecards send data dynamically.

Yes, this section regards dynamic subscriptions, but the macro-observation is that it would be good if dynamic-subscriptions behaved like configured-subscriptions and, since configured-subscriptions will most likely always have the server initiating the connection, it would be consistent if that were true for dynamic-subscriptions too (as opposed to having open ports).   Makes sense?

> https://mailarchive.ietf.org/arch/msg/netconf/YmIh2l1xGJUq5gYwvrzcwnRphPg <https://mailarchive.ietf.org/arch/msg/netconf/YmIh2l1xGJUq5gYwvrzcwnRphPg>
>  

Ah, I recall seeing that message fly by before.  Seems like no one replied, too bad.   Really, this is not a something any transport draft (e.g., draft-mahesh-netconf-https-notif) should have to specify.  It would have to be the SN draft (currently in AUTH48) and likely published before Montreal.  This creates quite a problem.  I strongly advise raising this to the SN draft authors (specifically Eric Voit, CC-ed) ASAP! 

Kent // contributor

---

From: "Eric Voit (evoit)" <[email-masked]>
Date: Mon, 15 Jul 2019 17:59:55 +0000

From: Kent Watsen, July 15, 2019 12:34 PM

Hi Tianran,

Hmmm, my guess is that pretty much all transports supporting configured 
subscriptions will have the server (i.e., line cards) proactively initiate the 
connections to the notifications-receivers (and not have the receivers 
connecting to the servers).   This is more a macro-observation than a 
draft-mahesh-netconf-https-notif specific thing, though that draft follows 
this pattern too.

[Tianran2]: I am sorry I not quite clear. Here we are talking about the 
dynamic subscription. I conclude there are four potential modes.

1. The linecard runs on server mode. The connections are dynamically set up, 
when the receiver know the subscription decomposition result at run time.

2. The linecard runs on server mode. All the potential connections are set up. 
The receiver only get data dynamically.

3. The linecard runs on client mode. The connections are dynamically set up, 
when the linecard know the subscription decomposition result at run time.

4. The linecard runs on client mode. All the potential connections are set up. 
The linecards send data dynamically.

Yes, this section regards dynamic subscriptions, but the macro-observation is 
that it would be good if dynamic-subscriptions behaved like 
configured-subscriptions and, since configured-subscriptions will most likely 
always have the server initiating the connection, it would be consistent if 
that were true for dynamic-subscriptions too (as opposed to having open 
ports).   Makes sense?

<eric> I think the dynamic subscription case is what several WG members have 
indicated has the most market demand.  With these dynamic subscriptions, there 
has certainly been a desire to match these to the existing transport paradigms 
of NETCONF / RESTCONF.   So I wouldn't suggest changing the overall 
connectivity paradigm of single-stream dynamic subscriptions based on the 
needs multi-line-cards.

 Ah, a "channel" container was added, I didn't see that before.  The draft 
should've defined YANG module to add a "channel" container.  That said, I 
don't think what you're hoping to achieve here is legal/valid YANG (please 
ensure that all examples are validated using e.g., `yanglint`).

[Tianran2]: I did not define a YANG module for this, because this may make the 
this draft depend on draft-mahesh-netconf-https-notif. But I sent the comment 
to the mailing list to suggest draft-mahesh-netconf-https-notif to apply this 
modification.

 <https://mailarchive.ietf.org/arch/msg/netconf/YmIh2l1xGJUq5gYwvrzcwnRphPg> 
https://mailarchive.ietf.org/arch/msg/netconf/YmIh2l1xGJUq5gYwvrzcwnRphPg

Or do you think this draft can define a configuration framework, not include 
the details? So that transport like draft-mahesh-netconf-https-notif can 
augment.

Ah, I recall seeing that message fly by before.  Seems like no one replied, 
too bad.   Really, this is not a something any transport draft (e.g., 
draft-mahesh-netconf-https-notif) should have to specify.  It would have to be 
the SN draft (currently in AUTH48) and likely published before Montreal.  This 
creates quite a problem.  I strongly advise raising this to the SN draft 
authors (specifically Eric Voit, CC-ed) ASAP!

<eric> There have been discussions on whether to add multi-line type card 
augmentations for the basic YANG model.  But to-date, these have been seen as 
future augmentations to be worked through the WG.  (Which we are exploring 
now.)

Based on leaving the AUTH48 work as-is, it seems like Tianran's suggestion 
seems a reasonable approach...  Anything specific to multi-line card 
augmentation should be in draft-zhou-netconf-multi-stream-originator.   And 
then any transports which want to support these multi-line card models can 
refer to the appropriate augmentations.

Eric

Kent // contributor