Subject: [Roll] Comments on the Industrial Requirements Draft

From: "David E. Culler" <[email-masked]>
Date: Thu, 17 Apr 2008 15:37:01 -0700

Now that we have our first WG document, let's keep things rolling.  The 
following are comments on the Industrial Routing Requirements draft from 
November 9.  Hope they are helpful.

1. Terminology
 - defines access point, but not router.  In the ROLL WG meeting there 
was a good deal of terminology discussion because the term gateway was 
used in places where router was intended.  Here that confusion has been 
avoided, but it may make sense to be clear on where the R in ROLL is 
involved.

 - "host application" may be a little confusing.  In IP terms, the field 
devices is also a host. 

- Superframe - a very specific link layer mechanism.  Unclear that it 
belongs in a routing requirements doc.

- HART - belongs in Informative Reference section (where it does 
appear).  Probably not in Terminology.
- ISA - same

Slotted-link - specific choice of link layer media access control 
mechanism.  Unclear in belongs in routing requirements doc.
Timeslot - same

2. Introduction

page 4: can access points be wireless too?  Presumably not low power or 
lossy

"wired HART" - should other wired industrial options be mentioned, like 
foundation field bus?

capacity analysis is for a localized region.  It does not account for 
spatial reuse, which is expected to be common place in geographically 
large deployments - where Routing is most critical.

2.1 - class are out of sp100.11a RFP.  Should it be referenced?

Critical function... add "those that"

Page 5 - customers => users

discussion of typical data rates is important.  Should that paragraph 
draw implications for route update rates?

Page 6

Nine-9s discussion seems a bit misleading.  Trues that there are a 
trillion seconds in 30 years, but that is hardly the point.  This seems 
to suggest that there is no mechanism for error detection or recovery 
anywhere other than plant shutdown.

First MUST relates to multi-topology routing.  It seems like it would be 
good to develop more fully what we mean by this in the industrial setting.

Second MUST seems entirely out of place.  "color slotted-links" is a 
particular form of a particular link layer mechanism.  It may be a great 
thing, but it hardly seems like it belongs in a routing requirements 
doc.  There is probably a requirement having to do with isolation or 
predictability or something that this particular solution was standing 
as a proxy for.  We should try to state the requirement, rather than 
dictate a particular solution at this stage.

Third MUST is indeed a very important point that should bubble up to a 
requirement, but it isn't developed here.  It wants supporting evidence, 
which fundamentally relates to the L's.  Note that this is the reverse 
of previous MUST.  This one takes general characteristics of the 
underlying link and the needs of applications to yield a requirement on 
routing.  That is what these docs should do. 

QoS.
 -3. Is 200 ms serving as a bound or an expected?
 -4. Can we bound limited period?

QoS parameters

Transmission time seems to be calling for time correlated sampling, 
i.e., NTP, rather than coordinated transmissions.  Are we establishing a 
requirement or advocating a solution?

Reliability - 99.9% - Should probably clarify that this is end-to-end.

Transmission priority - this seems to address forwarding, rather than 
routing.

Page 7-8.  We should be careful not to confuse the presence of a 
particular approach or solution with a requirement.

Page 8.  "MUST be able to set up routes that are directional".  Is this 
to mean that A ----> B need not follow the reverse of the route B 
-----A.  Or is this dealing with the asymmetry of the flows?

3.1 two additional MUSTs.  First seems like it needs to be clarified.  I 
think it is trying to get at adjusting to different flow rates - 
sampling may be very different from maintenance.  This primarily a 
forwarding issue.  Even with adaptation, it doesn't mean that it MUST be 
changing.

Second MUST confuses dynamic path determination with a particular link 
layer mechanism.  First is probably the intended requirement.  Send is a 
particular solution.

Section 4 para 2.  Appears to define MAC requirement, rather than 
routing req.  Perhaps is intended to define end-to-end application 
requirement.  Also a piece of a service requirement (NTP?).

Throughout analysis is for a particular MAC.

Graph - Access points or routers?  Both?

Page 10.  Section 7.  Seems to want to deal with transient loss.  Is it 
necessary to put it in terms of a particular link mechanism?

section 8.  Demographic discussion is probably accurate, but is it 
relevant for routing?  "Should support the wireless worker" is probably 
a bit too vague a requirement.

section 9 - so is this calling for Autoconf?

Section 10 - Claims of the relative tolerance to failure in these 
industries should probably be removed.

_______________________________________________
Roll mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/roll

---

From: "Pascal Thubert (pthubert)" <[email-masked]>
Date: Thu, 24 Apr 2008 11:29:28 +0200

Hi David:

I'm trying to summarize here what we did or did not do (yet) about your comments here. BTW thanks a huge deal for your help here. Also this response somùetimes reflects my opinion and not necessarily a consensus amonst the authors.

I note that there's still work to do on 3.1 and 7 for the chicken and an egg problem between routing and dynamic radio links instantiations, in section 10 about security and provisionning, and per your comments on rev 1.

Also: to avoid nested >>> I kept your text without a > and added mine prefixed with [Pascal]

---------------------------------------------------------

Now that we have our first WG document, let's keep things rolling.  The following are comments on the Industrial Routing Requirements draft from November 9.  Hope they are helpful.

1. Terminology
 - defines access point, but not router.  In the ROLL WG meeting there was a good deal of terminology discussion because the term gateway was used in places where router was intended.  Here that confusion has been avoided, but it may make sense to be clear on where the R in ROLL is involved.

 [Pascal] we need to find the right term. Right now L2N AP

 - "host application" may be a little confusing.  In IP terms, the field devices is also a host. 
 
[Pascal] now plant application

- Superframe - a very specific link layer mechanism.  Unclear that it belongs in a routing requirements doc.

[Pascal] gone

- HART - belongs in Informative Reference section (where it does appear).  Probably not in Terminology.
- ISA - same

[Pascal] Not done yet. I would agree though; terminology mostly defines new terms and refers to docs for existing terms

Slotted-link - specific choice of link layer media access control mechanism.  Unclear in belongs in routing requirements doc.
Timeslot - same

[Pascal] still a ref in section 5 that we could remove

2. Introduction

page 4: can access points be wireless too?  Presumably not low power or lossy

[Pascal] sure but then we need to express the same reliability and security as in the WSN. At this point the term "wired" was removed.

"wired HART" - should other wired industrial options be mentioned, like foundation field bus?

[Pascal] still there. No big del is it? and yes we could name the alternate

capacity analysis is for a localized region.  It does not account for spatial reuse, which is expected to be common place in geographically large deployments - where Routing is most critical.

[Pascal] Should routing know about spatial reuse? Is there text that this specifically aims at changing or is it entirely new?

[Pascal] yes I think so.

[Pascal] fixed, removed 'are'

[Pascal] done

discussion of typical data rates is important.  Should that paragraph draw implications for route update rates?

[Pascal] there is now a section on route maintenance

Page 6

Nine-9s discussion seems a bit misleading.  Trues that there are a trillion seconds in 30 years, but that is hardly the point.  This seems to suggest that there is no mechanism for error detection or recovery anywhere other than plant shutdown.

[Pascal] I also fail to understand this text:
  "In critical control, tens of milliseconds of latency is typical.  In
   many of these systems, if a packet does not arrive within the
   specified interval, the system enters an emergency shutdown state,
   often with substantial financial repercussions.  For a one-second
   control loop in a system with a mean-time between shutdowns target of
   30 years, the latency requirement implies nine 9s of reliability."

First MUST relates to multi-topology routing.  It seems like it would be good to develop more fully what we mean by this in the industrial setting.

[Pascal] that text is gone and the development was done in section 2.2

Second MUST seems entirely out of place.  "color slotted-links" is a particular form of a particular link layer mechanism.  It may be a great thing, but it hardly seems like it belongs in a routing requirements doc.  There is probably a requirement having to do with isolation or predictability or something that this particular solution was standing as a proxy for.  We should try to state the requirement, rather than dictate a particular solution at this stage.

[Pascal] isolation is missing in that version but we are working on it for the next version

Third MUST is indeed a very important point that should bubble up to a requirement, but it isn't developed here.  It wants supporting evidence, which fundamentally relates to the L's.  Note that this is the reverse of previous MUST.  This one takes general characteristics of the underlying link and the needs of applications to yield a requirement on routing.  That is what these docs should do. 

[Pascal] what about this text:

""
  The logical topologies 
  ----------------------
  
   Publication
   -----------

Most of the traffic over the LLN is publish/subscribe of sensor data from the field device towards the backbone router or gateway that acts as the sink for the WSN.  The destination of the sensor data is an Infrastructure devices that sits on the backbone and is reachable via one or more backbone router. Since publishing the data is the raison d'etre for most of the sensors, it makes sense to build proactively a set of default routes between the sensors and one or more backbone router and maintain those routes at all time. Also, because of the lossy nature of the network, the routing in place should attempt to propose multiple forwarding solutions, building non congruent forwarding topologies in the form of Directed acyclic graphs oriented towards the sinks. Note that for a control application, the jitter is just as important as latency and has a potential of destabilizing control algorithms. 

   Closed loop
   -----------
   
In contrast with the general requirement of maintaining default routes towards the sinks, the need for field device to field device connectivity is only occasional, though the traffic associated might be of foremost importance. An example of field device to field device traffic is a control loop between a vibration sensor and an actuator that can stop a motor that is failing. A class 0 control loop requires guaranteed delivery and extremely tight response times. In other words, both the respect of criteria in the route computation and the quality of the maintenance of the route are critical for the field devices operation. The requirements vary with the operations of the field devices but in a general fashion field device to field device routes will often be the most critical, optimized and well-maintained.

"

QoS.

QoS as a term is gone because it's forwarding. We used the term service and changed this 200 to hundereds of ms. 
 
QoS parameters

[Pascal] is there text left to be changed for this remark?
[Pascal] Do you disagree with text about Transmission phase? If so would you propose an alternate text? 

[Pascal] This text is gone 

[Pascal] right. Maybe we need to get this out. It's just good information though

[Pascal] referenced to slotted are gone

[Pascal] we added text:
"  Industrial application data flows between field devices are not
   necessarily symmetric.  In particular, asymmetrical cost and
   unidirectional routes are common for published data and alerts, which
   represent the most part of the sensor traffic.  The routing protocol
   MUST be able to set up unidirectional or asymmetrical cost routes
   that are composed of one or more non congruent paths.
"

[Pascal] I would agree on this. If the routing has established multiple path then 
a forwarding protocol can control the bandwidth and latency in real time and that 
can be all a forwarding issue. I note that we have to work some more on 3.1

[Pascal] I would agree on this too. There is a chicken and an egg problem between routing 
an link provisionning when links can be provisionned dynamically. This is nothing new 
if you consider traffic engineering. We have to figure what we really want. I see 2 options:
- links come and go because they are used or not and L3 makes the best of it
- L3 uses virtual links that are actually instantiated when they are really needed and L2 
tries to adapt to the traffic that L3 causes to go over links
To be honest I think that the later makes more sense.

[Pascal] 4 moved into 2.2. The text you refer to is all gone now. 

[Pascal] Terms were fixed (no more slotted links) but the chicken and an egg problem also discussed in 3.1 stays

[Pascal]Text with requirements was added in 2.2.1 in the long pâragraph after the picture. 
Maybe this section (that is unchanged BTW) should move earlier in the doc to intriduce the issue?

[Pascal] Certainly something to look at when considering solution. We can dig into MANET and MANEMO history as well.

[Pascal] I agree we should remove the terms link banking industry which point at someone in particular.
There's also the discussion about keys, perimeter of security and related provisionning that needs rework.
This section should bequite improved with the help of new authors (Sicco Dwars and Tom Phinney) who were invited in.

Pascal
>-----Original Message-----
>From: [email-masked] [mailto:[email-masked]] On Behalf Of David E. Culler
>Sent: vendredi 18 avril 2008 00:37
>To: [email-masked]
>Subject: [Roll] Comments on the Industrial Requirements Draft
>

---

From: "Shitanshu Shah (svshah)" <[email-masked]>
Date: Fri, 25 Apr 2008 09:39:43 -0700

Hi Pascal,

> [Pascal] right. Maybe we need to get this out. It's just good 

I don't have a strong opinion but IMO it is good to take out forwarding 
specifics (not only it is irrelevant to routing requirements but is 
also partial specifics for service forwarding)

As mentioned in my private response to you Pascal, if the purpose is to 
highlight that forwarding should not even face the situation where if 
such action required it may affect the service requirement then it is
better to call out service requirements explicitly. Which I think you
have called out very well (btw, nice work there)

cheers
Shitanshu 

> From: [email-masked] [mailto:[email-masked]] On 
> Behalf Of Pascal Thubert (pthubert)
> Sent: Thursday, April 24, 2008 2:29 AM
> To: David E. Culler; [email-masked]
> Subject: Re: [Roll] Comments on the Industrial Requirements Draft
> 
> 
> Hi David:
> 
> I'm trying to summarize here what we did or did not do (yet) 
> about your comments here. BTW thanks a huge deal for your 
> help here. Also this response somùetimes reflects my opinion 
> and not necessarily a consensus amonst the authors.
> 
> I note that there's still work to do on 3.1 and 7 for the 
> chicken and an egg problem between routing and dynamic radio 
> links instantiations, in section 10 about security and 
> provisionning, and per your comments on rev 1.
> 
> Now that we have our first WG document, let's keep things 
> rolling.  The following are comments on the Industrial 
> Routing Requirements draft from November 9.  Hope they are helpful.
> 
> 1. Terminology
>  - defines access point, but not router.  In the ROLL WG 
> meeting there was a good deal of terminology discussion 
> because the term gateway was used in places where router was 
> intended.  Here that confusion has been avoided, but it may 
> make sense to be clear on where the R in ROLL is involved.
> 
> - Superframe - a very specific link layer mechanism.  Unclear 
> that it belongs in a routing requirements doc.
> 
> [Pascal] gone
> 
> - HART - belongs in Informative Reference section (where it 
> does appear).  Probably not in Terminology.
> - ISA - same
> 
> [Pascal] Not done yet. I would agree though; terminology 
> mostly defines new terms and refers to docs for existing terms
> 
> Slotted-link - specific choice of link layer media access 
> control mechanism.  Unclear in belongs in routing requirements doc.
> Timeslot - same
> 
> [Pascal] sure but then we need to express the same 
> reliability and security as in the WSN. At this point the 
> term "wired" was removed.
> 
> capacity analysis is for a localized region.  It does not 
> account for spatial reuse, which is expected to be common 
> place in geographically large deployments - where Routing is 
> most critical.
> 
> [Pascal] Should routing know about spatial reuse? Is there 
> text that this specifically aims at changing or is it entirely new?
> 
> discussion of typical data rates is important.  Should that 
> paragraph draw implications for route update rates?
> 
> Nine-9s discussion seems a bit misleading.  Trues that there 
> are a trillion seconds in 30 years, but that is hardly the 
> point.  This seems to suggest that there is no mechanism for 
> error detection or recovery anywhere other than plant shutdown.
> 
> First MUST relates to multi-topology routing.  It seems like 
> it would be good to develop more fully what we mean by this 
> in the industrial setting.
> 
> Second MUST seems entirely out of place.  "color 
> slotted-links" is a particular form of a particular link 
> layer mechanism.  It may be a great thing, but it hardly 
> seems like it belongs in a routing requirements doc.  There 
> is probably a requirement having to do with isolation or 
> predictability or something that this particular solution was 
> standing as a proxy for.  We should try to state the 
> requirement, rather than dictate a particular solution at this stage.
> 
> Third MUST is indeed a very important point that should 
> bubble up to a requirement, but it isn't developed here.  It 
> wants supporting evidence, which fundamentally relates to the 
> L's.  Note that this is the reverse of previous MUST.  This 
> one takes general characteristics of the underlying link and 
> the needs of applications to yield a requirement on routing.  
> That is what these docs should do. 
> 
> Most of the traffic over the LLN is publish/subscribe of 
> sensor data from the field device towards the backbone router 
> or gateway that acts as the sink for the WSN.  The 
> destination of the sensor data is an Infrastructure devices 
> that sits on the backbone and is reachable via one or more 
> backbone router. Since publishing the data is the raison 
> d'etre for most of the sensors, it makes sense to build 
> proactively a set of default routes between the sensors and 
> one or more backbone router and maintain those routes at all 
> time. Also, because of the lossy nature of the network, the 
> routing in place should attempt to propose multiple 
> forwarding solutions, building non congruent forwarding 
> topologies in the form of Directed acyclic graphs oriented 
> towards the sinks. Note that for a control application, the 
> jitter is just as important as latency and has a potential of 
> destabilizing control algorithms. 
> 
> 
> 
>    Closed loop
>    -----------
>    
> In contrast with the general requirement of maintaining 
> default routes towards the sinks, the need for field device 
> to field device connectivity is only occasional, though the 
> traffic associated might be of foremost importance. An 
> example of field device to field device traffic is a control 
> loop between a vibration sensor and an actuator that can stop 
> a motor that is failing. A class 0 control loop requires 
> guaranteed delivery and extremely tight response times. In 
> other words, both the respect of criteria in the route 
> computation and the quality of the maintenance of the route 
> are critical for the field devices operation. The 
> requirements vary with the operations of the field devices 
> but in a general fashion field device to field device routes 
> will often be the most critical, optimized and well-maintained.
> 
> "
> 
> 
> QoS.
> QoS as a term is gone because it's forwarding. We used the 
> term service and changed this 200 to hundereds of ms. 
>  
> QoS parameters
> 
> Transmission time seems to be calling for time correlated 
> sampling, i.e., NTP, rather than coordinated transmissions.  
> Are we establishing a requirement or advocating a solution?
> 
> 
> [Pascal] Do you disagree with text about Transmission phase? 
> If so would you propose an alternate text? 
> 
> Transmission priority - this seems to address forwarding, 
> rather than routing.
> 
> Page 7-8.  We should be careful not to confuse the presence 
> of a particular approach or solution with a requirement.
> 
> Page 8.  "MUST be able to set up routes that are 
> directional".  Is this to mean that A ----> B need not follow 
> the reverse of the route B -----A.  Or is this dealing with 
> the asymmetry of the flows?
> 
> [Pascal] we added text:
> 3.1 two additional MUSTs.  First seems like it needs to be 
> clarified.  I think it is trying to get at adjusting to 
> different flow rates - sampling may be very different from 
> maintenance.  This primarily a forwarding issue.  Even with 
> adaptation, it doesn't mean that it MUST be changing.
> 
> 
> [Pascal] I would agree on this. If the routing has 
> established multiple path then a forwarding protocol can 
> control the bandwidth and latency in real time and that can 
> be all a forwarding issue. I note that we have to work some 
> more on 3.1
> 
> Second MUST confuses dynamic path determination with a 
> particular link layer mechanism.  First is probably the 
> intended requirement.  Send is a particular solution.
> 
> 
> [Pascal] I would agree on this too. There is a chicken and an 
> egg problem between routing an link provisionning when links 
> can be provisionned dynamically. This is nothing new if you 
> consider traffic engineering. We have to figure what we 
> really want. I see 2 options:
> - L3 uses virtual links that are actually instantiated when 
> they are really needed and L2 tries to adapt to the traffic 
> that L3 causes to go over links To be honest I think that the 
> later makes more sense.
> 
> Section 4 para 2.  Appears to define MAC requirement, rather 
> than routing req.  Perhaps is intended to define end-to-end 
> application requirement.  Also a piece of a service 
> requirement (NTP?).
> Page 10.  Section 7.  Seems to want to deal with transient 
> loss.  Is it necessary to put it in terms of a particular 
> link mechanism?
> 
> [Pascal] Terms were fixed (no more slotted links) but the 
> chicken and an egg problem also discussed in 3.1 stays
> 
> section 8.  Demographic discussion is probably accurate, but 
> is it relevant for routing?  "Should support the wireless 
> worker" is probably a bit too vague a requirement.
> 
> Maybe this section (that is unchanged BTW) should move 
> earlier in the doc to intriduce the issue?
> 
> [Pascal] Certainly something to look at when considering 
> solution. We can dig into MANET and MANEMO history as well.
> 
> Section 10 - Claims of the relative tolerance to failure in 
> these industries should probably be removed.
> 
> [Pascal] I agree we should remove the terms link banking 
> industry which point at someone in particular.
> There's also the discussion about keys, perimeter of security 
> and related provisionning that needs rework.
> This section should bequite improved with the help of new 
> authors (Sicco Dwars and Tom Phinney) who were invited in.
> 
> 
> 
> 
> >From: [email-masked] [mailto:[email-masked]] 
> On Behalf Of 
> >David E. Culler
> >following are comments on the Industrial Routing Requirements draft 
> >from November 9.  Hope they are helpful.
> >
> >1. Terminology
> >First MUST relates to multi-topology routing.  It seems like 
> it would 
> >be good to develop more fully what we mean by this in the 
> industrial setting.
> >
> >particular form of a particular link layer mechanism.  It may be a 
> >great thing, but it hardly seems like it belongs in a routing 
> >requirements doc.  There is probably a requirement having to do with 
> >isolation or predictability or something that this 
> particular solution 
> >was standing as a proxy for.  We should try to state the 
> requirement, 
> >rather than dictate a particular solution at this stage.
> >
> >requirement, but it isn't developed here.  It wants supporting 
> >evidence, which fundamentally relates to the L's.  Note that this is 
> >the reverse of previous MUST.  This one takes general 
> characteristics 
> >of the underlying link and the needs of applications to yield a 
> >requirement on routing.  That is what these docs should do.
> >
> >QoS.
> >i.e., NTP, rather than coordinated transmissions.  Are we 
> establishing 
> >a requirement or advocating a solution?
> >
> >3.1 two additional MUSTs.  First seems like it needs to be 
> clarified.  
> >I think it is trying to get at adjusting to different flow rates - 
> >forwarding issue.  Even with adaptation, it doesn't mean 
> that it MUST 
> >be changing.
> >
> >layer mechanism.  First is probably the intended 
> requirement.  Send is 
> >a particular solution.
> >