Received: from relay.versatel.net (relay.versatel.net [62.250.3.110]) by bierator.ibr.cs.tu-bs.de (8.13.4/8.13.4/Debian-3sarge3) with SMTP id m4JLmnfx029754 for ; Mon, 19 May 2008 23:48:55 +0200 Received: (qmail 59468 invoked from network); 19 May 2008 21:48:49 -0000 Received: from unknown (HELO bwMedion) (87.215.199.34) by relay.versatel.net with SMTP; 19 May 2008 21:48:49 -0000 From: "Bert Wijnen - IETF" To: "Karen R. Sollins" , Date: Mon, 19 May 2008 23:49:00 +0200 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="US-ASCII" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.6604 (9.0.2911.0) In-Reply-To: X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.3198 Importance: Normal X-IBRFilter-SpamReport: 0.001 () BAYES_50 X-Scanned-By: MIMEDefang 2.51 on 134.169.34.9 Cc: Internet Research Steering Group , nmrg@ibr.cs.tu-bs.de Subject: [nmrg] RE: [IRSG] review of draft-irtf-nmrg-snmp-measure-04.txt X-BeenThere: nmrg@ibr.cs.tu-bs.de X-Mailman-Version: 2.1.5 Precedence: list List-Id: Network Management Research Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 May 2008 21:48:58 -0000 Karen, Juergen has posted revision 5 with the fixes/updates as you and he discussed. If you could quickly check that revision 5 is now OK with you and let us all know if so, that would be appreciated. The document is here: http://www.ietf.org/internet-drafts/draft-irtf-nmrg-snmp-measure-05.txt NMRG members, pls check and speak up if you see any issues in the changes that were made for revision 5 as a result of IRSG review. Bert Wijnen document shepherd > -----Oorspronkelijk bericht----- > Van: Karen R. Sollins [mailto:sollins@csail.mit.edu] > Verzonden: maandag 19 mei 2008 19:25 > Aan: j.schoenwaelder@jacobs-university.de; Karen R. Sollins > CC: nmrg@ibr.cs.tu-bs.de; Bert Wijnen - IETF; Internet Research Steering > Group > Onderwerp: Re: [IRSG] review of draft-irtf-nmrg-snmp-measure-04.txt > > > HI Juergen, > > It is all fine with me. I think all you need now is concurrence from > the research group. > > Cheers, > Karen > > At 11:46 AM +0200 5/19/08, Juergen Schoenwaelder wrote: > >On Sun, May 18, 2008 at 09:05:15PM -0700, Karen R. Sollins wrote: > > > >> Thanks for your thoughtful responses. I also did not think > that what I was > >> suggesting was a lot of work. At a high level, think about > a reader who > >> is not part of your group, whom you are trying to convince > that what you > >> are doing is valuable or whom you would like to convince to > do such a data > >> collection exercise. I have interspersed my comments into > your responses > >> below. > > > >I will repond to those things that are still "open" with suggestions > >on how to "close" them. > > > >KS> The second high level concern I have is that there is talk about > >KS> specific kinds of information to be collected and an interest in not > >KS> only the nature but longer term inferences with perhaps implications > >KS> for future redesign efforts in the SNMP context. I have > two levels > >KS> of concern here. The most important one is that since network > >KS> management and in particular SNMP is NOT the primary > objective of the > >KS> net (the primary objective being the transport of real payload), it > >KS> seems to me that the truly critical question with respect to network > >KS> management traffic is the impact that it has or does not > have on that > >KS> real job. To me this implies that the measurements MUST > also include > >KS> contextual information. As an example, it is probably more > important > >KS> to understand whether or not the network management traffic is > >KS> causing significant congestion for the payload traffic than the > >KS> particular mix or frequency pattern within the network management > >KS> traffic. Without out the complementary contextual information, the > >KS> whole measurement exercise seems to me to be of somewhat narrow > >KS> value. > > > >JS> The measurement may be of narrow value from your point of view but > >JS> please keep in mind that this document is coming from the Network > >JS> Management Research Group and not from a general Network Measurement > >JS> Research Group. Our goal is to understand how network management > >JS> protocols are being used because that has impact on their design and > >JS> implementation strategies. Further note that in many networks, the > >JS> management traffic is logically and sometimes even physically > >JS> separated from the normal traffic and perhaps this is the reason why > >JS> we did not even think about the question whether management traffic > >JS> has an impact on normal traffic. > > > >KS> If you want to leave it as is, then I think it would be > valuable to say as > >KS> much. Be specific about what you are not doing, because > much of the rest > >KS> of the world looks at network traffic from a broader perspective. > > > >I think the abstract and the introduction are pretty clearly spelling > >out the scope of the measurement effort. So I am not sure changes are > >needed, but see below. > > > >KS> 1. Section 1: It seems to me that there are TWO key questions with > >KS> respect to SNMP. The first is how it is being used, which in turn > >KS> leads to the points made in this section, but the second is the > >KS> impact of that traffic. I think that ought to appear in the > >KS> Introduction as well. > > > >JS> See my comments above. So far, the NMRG did not consider the > impact of > >JS> SNMP on other traffic a target of this activity. I don't want to add > >JS> such text unless I see support from the NMRG and concrete proposals > >JS> what should be added. > > > >KS> Again, as above, if you want to leave the scope as it is, > then you should > >KS> probably be up front about that, clearly leaving that work > to another time > >KS> and place. > > > >I propose to add the following paragraph just before the last > >paragraph in section 1: > > > > The measurement approach described in this document is by design > > limited to the study of SNMP traffic. Studies of other management > > protocols or the impact of management protocols such as SNMP on > > other traffic sharing the same network resources is left to future > > efforts. > > This is what I was looking for. > > >KS> 2. Section 2.1. The second paragraph begins with, "It is > recommended > >KS> to capture at least a full week of data." This is never > justified or > >KS> explained. Is one week really enough? For what? Why wouldn't > >KS> several weeks be critical, because one week might be anomalous? Why > >KS> isn't a year critical, since we know that there are annual or > >KS> seasonal differences in traffic behaviors? Typically, I find that > >KS> one-week data sets often leave me with lots of unanswered questions, > >KS> so justify this. > > > >JS> The text actually says: > >JS> > >JS> It is recommended to capture at least a full week of > data. Operators > >JS> are encouraged to capture traces over even longer periods of time. > >JS> > >JS> The text tries to establish a lower bound of one week an encourages > >JS> longer capture periods. I would love to get continuous traces but > >JS> reality is such that this is not feasible. Our idea is > simply to catch > >JS> at least the weekly behaviour. Yes, there is of course also > monthly or > >JS> yearly behaviour but I believe it is not useful to set the > bar so high > >JS> that nobody gives us appropriate traces. I personally > believe the text > >JS> is fine as is. > > > >KS> So, what I was really getting at was the question of why one week > >KS> was the minimum necessary. So, something like that it is the > >KS> minimum over which one can see the diurnal patterns in the weekly > >KS> pattern and it is understood that both for computational and > >KS> storage reasons the operators may not want to collect more. > > > >I have changed the text to the following: > > > > It is recommended to capture at least a full week of data to capture > > diurnal patterns and one cycle of weekly behavior. Operators are > > strongly encouraged to capture traces over even longer periods of > > time. > > OK by me. > > >KS> 4. Section 3.3, end of first paragraph: The sentence reads, "Some > >KS> SNMP implementations approximate networking delays by measuring > >KS> request-response times and it would be useful to understand to what > >KS> extent this is a viable approach." I agree, but traces > will not tell > >KS> you anything about whether behaviors observed in packet traces are > >KS> for this reason or some other reason. I do not believe you can get > >KS> at this question with the data you are collecting. > > > >JS> I think it is possible to analyze retransmission behaviour. Depending > >JS> on the SNMP version used (and the other versions also depending on > >JS> implementation choices), you can get information whether a > response is > >JS> just coming late for the original request or it is actually > a response > >JS> to a retranmitted request. We are not talking TCP here; we > are talking > >JS> about application layer retransmissions and SNMP has its own > msgID and > >JS> requestID fields. > > > >KS> The point I was trying to make here is that it is very > difficult to intuit > >KS> the reasons behind behaviors seen in the traffic, unless someone or > >KS> something tells you. So, you can see what the ends do, but not why. > > > >I agree, but this is a rather general observation and not necessarily > >specific to 3.3 and it is not clear to me what I should do about it. > >There is already a general remark in the last paragraph of section 2.5 > >that one has to be careful with drawing conclusions that go beyond > >what you can really get out of traces. > > OK - I'll drop it. > > > > >KS> 5. Section 3.4: Please explain why it is "interesting" > (your word) to > >KS> identify whether concurrency or sequentiality is occurring? What > >KS> will you "learn" if both are observed? And, if one is > occurring more > >KS> frequently or under specific identifiable conditions, what further > >KS> does that tell you? Just knowing that one or the other occurs is > >KS> only the tip of the iceberg, and without acknowledging the fact that > >KS> these are important and unanswered questions, just learning first > >KS> ordered details suggests you are setting the bar too low. > > > >JS> The introduction of section 3 says: > >JS> > >JS> The questions raised in the following subsections are meant to be > >JS> illustrative and no attempt has been made to provide a complete > >JS> list. > >JS> > >JS> I believe it is a good idea to first figure out whether there is an > >JS> iceberg or not (keeping your analogy) and if there is one to ask > >JS> questions how big the iceberg might be. For SNMP agent > implementations > >JS> that tend to do quite some caching, it is useful to know how well > >JS> caching strategies are working in real-world networks. The > concurrency > >JS> level an agent experiences has clear impact on that. Furthermore, it > >JS> will be useful to know how bursty the traffic tends to be or how well > >JS> managers spread the traffic over polling intervals and this is again > >JS> related to the concurrency we can extract from traces. > >JS> > >JS> I am not really sure what I should change, perhaps the word > >JS> "interesting" is the source of the trouble and I should replace this > >JS> with "valuable"? > > > >KS> What I was trying to ask was that you tell the reader a bit about > >KS> what makes it interesting. It certainly doesn't have to be > >KS> complete, but just a> hint. I don't think you should change the > >KS> word "interesting", because if it isn't interesting, you probably > >KS> shouldn't be doing it. > > > >I added the following sentence to section 3.4: > > > > The concurrency level and the amount of redundant requests has > > implications on caching strategies employed by SNMP agents. > > Fine. > > >KS> 6. Section 3.5: Please explain what you would do with the > >KS> information about which approach to table retrieval is used. Again, > >KS> what if the results tell you that both are used? And, if not, of > >KS> what use is it to know which approach is prevalent? Mighten it be > >KS> useful to know the conditions under which one or the other is used > >KS> more commonly? > > > >JS> This again has direct impact on agent implementation techniques and > >JS> caching strategies. > > > >KS> And again, just explain a little. > > > >I changed the last sentence of section 3.5 to read as follows: > > > > [...] It will be useful to know which of > > these approaches are used on production networks since this has a > > direct implication on agent implementation techniques and caching > > strategies. > > Fine. > > >/js > > > >-- > >Juergen Schoenwaelder Jacobs University Bremen gGmbH > >Phone: +49 421 200 3587 Campus Ring 1, 28759 Bremen, Germany > >Fax: +49 421 200 3103 > > > -- > > Karen R. Sollins, Ph. D. > Principal Research Scientist > M.I.T. CSAIL > The Stata Center > 32 Vassar St., 32-G818 > Cambridge, MA 02139 > V: 617/253-6006 > F: 617/253-2673 > E: sollins@csail.mit.edu > Received: from hermes.jacobs-university.de (hermes.jacobs-university.de [212.201.44.23]) by bierator.ibr.cs.tu-bs.de (8.13.4/8.13.4/Debian-3sarge3) with ESMTP id m4JJBbw2012246 for ; Mon, 19 May 2008 21:11:42 +0200 Received: from localhost (demetrius3.jacobs-university.de [212.201.44.48]) by hermes.jacobs-university.de (Postfix) with ESMTP id AB37CC0011; Mon, 19 May 2008 21:11:37 +0200 (CEST) X-Virus-Scanned: amavisd-new at jacobs-university.de Received: from hermes.jacobs-university.de ([212.201.44.23]) by localhost (demetrius3.jacobs-university.de [212.201.44.32]) (amavisd-new, port 10024) with ESMTP id RMKTyNWsATKZ; Mon, 19 May 2008 21:11:32 +0200 (CEST) Received: from elstar.local (elstar.iuhb02.iu-bremen.de [10.50.231.133]) by hermes.jacobs-university.de (Postfix) with ESMTP id 6A4F0C0003; Mon, 19 May 2008 21:11:32 +0200 (CEST) Received: by elstar.local (Postfix, from userid 501) id 9F694597C17; Mon, 19 May 2008 21:11:33 +0200 (CEST) Date: Mon, 19 May 2008 21:11:33 +0200 From: Juergen Schoenwaelder To: "Karen R. Sollins" Message-ID: <20080519191133.GB28763@elstar.local> Mail-Followup-To: "Karen R. Sollins" , nmrg@ibr.cs.tu-bs.de, Bert Wijnen - IETF , Internet Research Steering Group References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.17 (2007-11-01) X-IBRFilter-SpamReport: -2.599 () BAYES_00 X-Scanned-By: MIMEDefang 2.51 on 134.169.34.9 Cc: Internet Research Steering Group , nmrg@ibr.cs.tu-bs.de Subject: [nmrg] Re: [IRSG] review of draft-irtf-nmrg-snmp-measure-04.txt X-BeenThere: nmrg@ibr.cs.tu-bs.de X-Mailman-Version: 2.1.5 Precedence: list Reply-To: j.schoenwaelder@jacobs-university.de List-Id: Network Management Research Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 May 2008 19:11:44 -0000 On Mon, May 12, 2008 at 02:52:02PM -0400, Karen R. Sollins wrote: > Here is my full review of draft-irtf-nmrg-snmp-measure-04.txt. [...] I have posted the -05 version of the document which includes all the changes discussed here. The -05 version is already online but not yet on the tools pages (no nice colored diffs yet). /js -- Juergen Schoenwaelder Jacobs University Bremen gGmbH Phone: +49 421 200 3587 Campus Ring 1, 28759 Bremen, Germany Fax: +49 421 200 3103 Received: from hermes.jacobs-university.de (hermes.jacobs-university.de [212.201.44.23]) by bierator.ibr.cs.tu-bs.de (8.13.4/8.13.4/Debian-3sarge3) with ESMTP id m4JJBQAL012180 for ; Mon, 19 May 2008 21:11:31 +0200 Received: from localhost (demetrius3.jacobs-university.de [212.201.44.48]) by hermes.jacobs-university.de (Postfix) with ESMTP id BBCC1C000D for ; Mon, 19 May 2008 21:11:26 +0200 (CEST) X-Virus-Scanned: amavisd-new at jacobs-university.de Received: from hermes.jacobs-university.de ([212.201.44.23]) by localhost (demetrius3.jacobs-university.de [212.201.44.32]) (amavisd-new, port 10024) with ESMTP id FA6w9WOi09ls for ; Mon, 19 May 2008 21:11:21 +0200 (CEST) Received: from elstar.local (elstar.iuhb02.iu-bremen.de [10.50.231.133]) by hermes.jacobs-university.de (Postfix) with ESMTP id 5965EC0003 for ; Mon, 19 May 2008 21:11:21 +0200 (CEST) Received: by elstar.local (Postfix, from userid 501) id 99834597BFA; Mon, 19 May 2008 21:11:21 +0200 (CEST) Resent-From: j.schoenwaelder@jacobs-university.de Resent-Date: Mon, 19 May 2008 21:11:21 +0200 Resent-Message-ID: <20080519191121.GA28763@elstar.local> Resent-To: nmrg@ibr.cs.tu-bs.de Received: from merkur.jacobs-university.de ([unix socket]) by merkur (Cyrus v2.2.13) with LMTPA; Mon, 19 May 2008 21:02:30 +0200 X-Sieve: CMU Sieve 2.2 Received: from hermes.jacobs-university.de (hermes.jacobs-university.de [212.201.44.23]) by merkur.jacobs-university.de (Postfix) with ESMTP id 6F3FF46860 for ; Mon, 19 May 2008 21:02:30 +0200 (CEST) Received: from localhost (demetrius4.jacobs-university.de [212.201.44.49]) by hermes.jacobs-university.de (Postfix) with ESMTP id 69758C000D for ; Mon, 19 May 2008 21:02:30 +0200 (CEST) X-Virus-Scanned: amavisd-new at jacobs-university.de X-Spam-Flag: NO X-Spam-Score: -6.09 X-Spam-Level: X-Spam-Status: No, score=-6.09 tagged_above=-100 required=5 tests=[AWL=-0.290, BAYES_00=-2.599, RCVD_IN_DNSWL_MED=-4, SARE_SUB_RAND_LETTRS4=0.799] Received: from hermes.jacobs-university.de ([212.201.44.23]) by localhost (demetrius4.jacobs-university.de [212.201.44.32]) (amavisd-new, port 10024) with ESMTP id P8Q2fa61Q7x1 for ; Mon, 19 May 2008 21:02:23 +0200 (CEST) X-JacobsISPWhiteListed: med ietf.org DNSWLId 1703 Received: from mail.ietf.org (mail.ietf.org [64.170.98.32]) by hermes.jacobs-university.de (Postfix) with ESMTP id F103CC0002 for ; Mon, 19 May 2008 21:02:22 +0200 (CEST) Received: from [127.0.0.1] (localhost [127.0.0.1]) by core3.amsl.com (Postfix) with ESMTP id 071DC28C15C; Mon, 19 May 2008 12:00:10 -0700 (PDT) X-Original-To: i-d-announce@ietf.org Delivered-To: i-d-announce@core3.amsl.com Received: by core3.amsl.com (Postfix, from userid 0) id 6C4413A6B12; Mon, 19 May 2008 12:00:01 -0700 (PDT) From: Internet-Drafts@ietf.org To: i-d-announce@ietf.org Content-Type: Multipart/Mixed; Boundary="NextPart" Mime-Version: 1.0 Message-Id: <20080519190002.6C4413A6B12@core3.amsl.com> Date: Mon, 19 May 2008 12:00:02 -0700 (PDT) X-BeenThere: i-d-announce@ietf.org X-Mailman-Version: 2.1.9 Precedence: list Sender: i-d-announce-bounces@ietf.org Errors-To: i-d-announce-bounces@ietf.org X-IBRFilter-SpamReport: -0.406 () BAYES_05,MIME_BOUND_NEXTPART,NO_REAL_NAME X-Scanned-By: MIMEDefang 2.51 on 134.169.34.9 Subject: [nmrg] I-D Action:draft-irtf-nmrg-snmp-measure-05.txt X-BeenThere: nmrg@ibr.cs.tu-bs.de Reply-To: internet-drafts@ietf.org List-Id: Network Management Research Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 May 2008 19:11:32 -0000 --NextPart A New Internet-Draft is available from the on-line Internet-Drafts directories. Title : SNMP Traffic Measurements and Trace Exchange Formats Author(s) : Filename : draft-irtf-nmrg-snmp-measure-05.txt Pages : 23 Date : 2008-05-19 The Simple Network Management Protocol (SNMP) is widely deployed to monitor, control and (sometimes also) configure network elements. Even though the SNMP technology is well documented, it remains relatively unclear how SNMP is used in practice and what typical SNMP usage patterns are. This document describes an approach to carrying out large scale SNMP traffic measurements in order to develop a better understanding how SNMP is used in real world production networks. It describes the motivation, the measurement approach, and the tools and data formats needed to carry out such a study. This document was produced within the IRTF's Network Management Research Group (NMRG) and represents the consensus of all of the active contributors to this group. A URL for this Internet-Draft is: http://www.ietf.org/internet-drafts/draft-irtf-nmrg-snmp-measure-05.txt Internet-Drafts are also available by anonymous FTP at: ftp://ftp.ietf.org/internet-drafts/ Below is the data which will enable a MIME compliant mail reader implementation to automatically retrieve the ASCII version of the Internet-Draft. --NextPart Content-Type: Message/External-body; name="draft-irtf-nmrg-snmp-measure-05.txt"; site="ftp.ietf.org"; access-type="anon-ftp"; directory="internet-drafts" Content-Type: text/plain Content-ID: <2008-05-19115603.I-D@ietf.org> --NextPart Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ I-D-Announce mailing list I-D-Announce@ietf.org https://www.ietf.org/mailman/listinfo/i-d-announce Internet-Draft directories: http://www.ietf.org/shadow.html or ftp://ftp.ietf.org/ietf/1shadow-sites.txt --NextPart-- Received: from QMTA07.emeryville.ca.mail.comcast.net (qmta07.emeryville.ca.mail.comcast.net [76.96.30.64]) by bierator.ibr.cs.tu-bs.de (8.13.4/8.13.4/Debian-3sarge3) with ESMTP id m4JIhe3E003951 for ; Mon, 19 May 2008 20:43:46 +0200 Received: from OMTA13.emeryville.ca.mail.comcast.net ([76.96.30.52]) by QMTA07.emeryville.ca.mail.comcast.net with comcast id TTuF1Z00117UAYkA70FV00; Mon, 19 May 2008 18:43:35 +0000 Received: from Harrington73653 ([24.128.66.199]) by OMTA13.emeryville.ca.mail.comcast.net with comcast id TWjX1Z0074HwxpC8Z00000; Mon, 19 May 2008 18:43:33 +0000 X-Authority-Analysis: v=1.0 c=1 a=zOsyX00C8WEA:10 a=1Im-IAa9KzAA:10 a=j3Z76cjpAAAA:8 a=ScMFhzVftyQFUUTUuCcA:9 a=yh1Q2r7NcGlr1b-9CGw_GpB0NqEA:4 a=FvgKqOQ44qUA:10 a=JrSEOxZJtCQA:10 a=XF7b4UCPwd8A:10 From: "David B Harrington" To: References: <08cb01c8b9b2$7669edb0$0600a8c0@china.huawei.com> <20080519183102.GA28463@elstar.local> Subject: RE: [nmrg] Re: [IRSG] review of draft-irtf-nmrg-snmp-measure-04.txt Date: Mon, 19 May 2008 14:43:31 -0400 Message-ID: <08f401c8b9e0$3da02f20$0600a8c0@china.huawei.com> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Mailer: Microsoft Office Outlook 11 In-Reply-To: <20080519183102.GA28463@elstar.local> Thread-Index: Aci53oep20LMx7XATpGWtp/3XVChrwAAa/eA X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.3198 X-IBRFilter-SpamReport: 3.602 (***) BAYES_50, DNS_FROM_RFC_POST, RCVD_IN_SORBS_DUL X-Scanned-By: MIMEDefang 2.51 on 134.169.34.9 Cc: 'Internet Research Steering Group' , "'Karen R. Sollins'" , nmrg@ibr.cs.tu-bs.de X-BeenThere: nmrg@ibr.cs.tu-bs.de X-Mailman-Version: 2.1.5 Precedence: list List-Id: Network Management Research Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 May 2008 18:43:48 -0000 Thanks, dbh > -----Original Message----- > From: Juergen Schoenwaelder > [mailto:j.schoenwaelder@jacobs-university.de] > Sent: Monday, May 19, 2008 2:31 PM > To: David B Harrington > Cc: 'Karen R. Sollins'; 'Internet Research Steering Group'; > nmrg@ibr.cs.tu-bs.de > Subject: Re: [nmrg] Re: [IRSG] review of > draft-irtf-nmrg-snmp-measure-04.txt > > On Mon, May 19, 2008 at 09:15:49AM -0400, David B Harrington wrote: > > > I think something should be said that this information could be used > > by an attacker (especially an attacker internal to the organization) > > to decide/pinpoint where and how to attack. This information might > > need to be anonymized, although that would seem to defeat > the purpose > > of having the information. I don't really know what to suggest here > > other than to raise the point in the security > considerations that such > > location information might be sensitive, and could aid an attacker. > > I suggest to add the following text to the end of the security > considerations: > > The meta data associated with traces and in particular information > about the organization owning a network and the description of the > measurement point in the network topology where a trace > was collected > may be misused to decide/pinpoint where and how to attack > a network. > Meta data therefore needs to be properly protected. > > In addition, I like to replace "generate XML traces" with "generate > CSV or XML traces" in the first sentence of the second paragraph of > the security considerations text since the rest of the paragraph > applies to both trace formats and not just XML. > > /js > > -- > Juergen Schoenwaelder Jacobs University Bremen gGmbH > Phone: +49 421 200 3587 Campus Ring 1, 28759 Bremen, Germany > Fax: +49 421 200 3103 > Received: from hermes.jacobs-university.de (hermes.jacobs-university.de [212.201.44.23]) by bierator.ibr.cs.tu-bs.de (8.13.4/8.13.4/Debian-3sarge3) with ESMTP id m4JIV9P6001048 for ; Mon, 19 May 2008 20:31:14 +0200 Received: from localhost (demetrius2.jacobs-university.de [212.201.44.47]) by hermes.jacobs-university.de (Postfix) with ESMTP id 239BFC0003; Mon, 19 May 2008 20:31:09 +0200 (CEST) X-Virus-Scanned: amavisd-new at jacobs-university.de Received: from hermes.jacobs-university.de ([212.201.44.23]) by localhost (demetrius2.jacobs-university.de [212.201.44.32]) (amavisd-new, port 10024) with ESMTP id MarCvghtCy7P; Mon, 19 May 2008 20:31:02 +0200 (CEST) Received: from elstar.local (elstar.iuhb02.iu-bremen.de [10.50.231.133]) by hermes.jacobs-university.de (Postfix) with ESMTP id DC10AC0011; Mon, 19 May 2008 20:31:02 +0200 (CEST) Received: by elstar.local (Postfix, from userid 501) id 26097597A1B; Mon, 19 May 2008 20:31:02 +0200 (CEST) Date: Mon, 19 May 2008 20:31:02 +0200 From: Juergen Schoenwaelder To: David B Harrington Subject: Re: [nmrg] Re: [IRSG] review of draft-irtf-nmrg-snmp-measure-04.txt Message-ID: <20080519183102.GA28463@elstar.local> Mail-Followup-To: David B Harrington , "'Karen R. Sollins'" , 'Internet Research Steering Group' , nmrg@ibr.cs.tu-bs.de References: <08cb01c8b9b2$7669edb0$0600a8c0@china.huawei.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <08cb01c8b9b2$7669edb0$0600a8c0@china.huawei.com> User-Agent: Mutt/1.5.17 (2007-11-01) X-IBRFilter-SpamReport: -1.951 () BAYES_20 X-Scanned-By: MIMEDefang 2.51 on 134.169.34.9 Cc: 'Internet Research Steering Group' , "'Karen R. Sollins'" , nmrg@ibr.cs.tu-bs.de X-BeenThere: nmrg@ibr.cs.tu-bs.de X-Mailman-Version: 2.1.5 Precedence: list Reply-To: j.schoenwaelder@jacobs-university.de List-Id: Network Management Research Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 May 2008 18:31:19 -0000 On Mon, May 19, 2008 at 09:15:49AM -0400, David B Harrington wrote: > I think something should be said that this information could be used > by an attacker (especially an attacker internal to the organization) > to decide/pinpoint where and how to attack. This information might > need to be anonymized, although that would seem to defeat the purpose > of having the information. I don't really know what to suggest here > other than to raise the point in the security considerations that such > location information might be sensitive, and could aid an attacker. I suggest to add the following text to the end of the security considerations: The meta data associated with traces and in particular information about the organization owning a network and the description of the measurement point in the network topology where a trace was collected may be misused to decide/pinpoint where and how to attack a network. Meta data therefore needs to be properly protected. In addition, I like to replace "generate XML traces" with "generate CSV or XML traces" in the first sentence of the second paragraph of the security considerations text since the rest of the paragraph applies to both trace formats and not just XML. /js -- Juergen Schoenwaelder Jacobs University Bremen gGmbH Phone: +49 421 200 3587 Campus Ring 1, 28759 Bremen, Germany Fax: +49 421 200 3103 Received: from mercury.lcs.mit.edu (mercury.lcs.mit.edu [18.26.0.122]) by bierator.ibr.cs.tu-bs.de (8.13.4/8.13.4/Debian-3sarge3) with ESMTP id m4JHPi6M016414 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT) for ; Mon, 19 May 2008 19:25:50 +0200 Received: from [192.168.1.105] (cpe-76-168-88-197.socal.res.rr.com [76.168.88.197]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mercury.lcs.mit.edu (Postfix) with ESMTP id 91CBE6BE59D; Mon, 19 May 2008 13:25:37 -0400 (EDT) Mime-Version: 1.0 Message-Id: In-Reply-To: <20080519094639.GA27481@elstar.local> References: <20080516122042.GA19275@elstar.local> <20080519094639.GA27481@elstar.local> Date: Mon, 19 May 2008 10:25:06 -0700 To: j.schoenwaelder@jacobs-university.de, "Karen R. Sollins" From: "Karen R. Sollins" Content-Type: text/plain; charset="us-ascii" ; format="flowed" X-IBRFilter-SpamReport: 0.001 () BAYES_50 X-Scanned-By: MIMEDefang 2.51 on 134.169.34.9 Cc: Internet Research Steering Group , nmrg@ibr.cs.tu-bs.de Subject: [nmrg] Re: [IRSG] review of draft-irtf-nmrg-snmp-measure-04.txt X-BeenThere: nmrg@ibr.cs.tu-bs.de X-Mailman-Version: 2.1.5 Precedence: list List-Id: Network Management Research Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 May 2008 17:25:53 -0000 HI Juergen, It is all fine with me. I think all you need now is concurrence from the research group. Cheers, Karen At 11:46 AM +0200 5/19/08, Juergen Schoenwaelder wrote: >On Sun, May 18, 2008 at 09:05:15PM -0700, Karen R. Sollins wrote: > >> Thanks for your thoughtful responses. I also did not think that what I was >> suggesting was a lot of work. At a high level, think about a reader who >> is not part of your group, whom you are trying to convince that what you >> are doing is valuable or whom you would like to convince to do such a data >> collection exercise. I have interspersed my comments into your responses >> below. > >I will repond to those things that are still "open" with suggestions >on how to "close" them. > >KS> The second high level concern I have is that there is talk about >KS> specific kinds of information to be collected and an interest in not >KS> only the nature but longer term inferences with perhaps implications >KS> for future redesign efforts in the SNMP context. I have two levels >KS> of concern here. The most important one is that since network >KS> management and in particular SNMP is NOT the primary objective of the >KS> net (the primary objective being the transport of real payload), it >KS> seems to me that the truly critical question with respect to network >KS> management traffic is the impact that it has or does not have on that >KS> real job. To me this implies that the measurements MUST also include >KS> contextual information. As an example, it is probably more important >KS> to understand whether or not the network management traffic is >KS> causing significant congestion for the payload traffic than the >KS> particular mix or frequency pattern within the network management >KS> traffic. Without out the complementary contextual information, the >KS> whole measurement exercise seems to me to be of somewhat narrow >KS> value. > >JS> The measurement may be of narrow value from your point of view but >JS> please keep in mind that this document is coming from the Network >JS> Management Research Group and not from a general Network Measurement >JS> Research Group. Our goal is to understand how network management >JS> protocols are being used because that has impact on their design and >JS> implementation strategies. Further note that in many networks, the >JS> management traffic is logically and sometimes even physically >JS> separated from the normal traffic and perhaps this is the reason why >JS> we did not even think about the question whether management traffic >JS> has an impact on normal traffic. > >KS> If you want to leave it as is, then I think it would be valuable to say as >KS> much. Be specific about what you are not doing, because much of the rest >KS> of the world looks at network traffic from a broader perspective. > >I think the abstract and the introduction are pretty clearly spelling >out the scope of the measurement effort. So I am not sure changes are >needed, but see below. > >KS> 1. Section 1: It seems to me that there are TWO key questions with >KS> respect to SNMP. The first is how it is being used, which in turn >KS> leads to the points made in this section, but the second is the >KS> impact of that traffic. I think that ought to appear in the >KS> Introduction as well. > >JS> See my comments above. So far, the NMRG did not consider the impact of >JS> SNMP on other traffic a target of this activity. I don't want to add >JS> such text unless I see support from the NMRG and concrete proposals >JS> what should be added. > >KS> Again, as above, if you want to leave the scope as it is, then you should >KS> probably be up front about that, clearly leaving that work to another time >KS> and place. > >I propose to add the following paragraph just before the last >paragraph in section 1: > > The measurement approach described in this document is by design > limited to the study of SNMP traffic. Studies of other management > protocols or the impact of management protocols such as SNMP on > other traffic sharing the same network resources is left to future > efforts. This is what I was looking for. >KS> 2. Section 2.1. The second paragraph begins with, "It is recommended >KS> to capture at least a full week of data." This is never justified or >KS> explained. Is one week really enough? For what? Why wouldn't >KS> several weeks be critical, because one week might be anomalous? Why >KS> isn't a year critical, since we know that there are annual or >KS> seasonal differences in traffic behaviors? Typically, I find that >KS> one-week data sets often leave me with lots of unanswered questions, >KS> so justify this. > >JS> The text actually says: >JS> >JS> It is recommended to capture at least a full week of data. Operators >JS> are encouraged to capture traces over even longer periods of time. >JS> >JS> The text tries to establish a lower bound of one week an encourages >JS> longer capture periods. I would love to get continuous traces but >JS> reality is such that this is not feasible. Our idea is simply to catch >JS> at least the weekly behaviour. Yes, there is of course also monthly or >JS> yearly behaviour but I believe it is not useful to set the bar so high >JS> that nobody gives us appropriate traces. I personally believe the text >JS> is fine as is. > >KS> So, what I was really getting at was the question of why one week >KS> was the minimum necessary. So, something like that it is the >KS> minimum over which one can see the diurnal patterns in the weekly >KS> pattern and it is understood that both for computational and >KS> storage reasons the operators may not want to collect more. > >I have changed the text to the following: > > It is recommended to capture at least a full week of data to capture > diurnal patterns and one cycle of weekly behavior. Operators are > strongly encouraged to capture traces over even longer periods of > time. OK by me. >KS> 4. Section 3.3, end of first paragraph: The sentence reads, "Some >KS> SNMP implementations approximate networking delays by measuring >KS> request-response times and it would be useful to understand to what >KS> extent this is a viable approach." I agree, but traces will not tell >KS> you anything about whether behaviors observed in packet traces are >KS> for this reason or some other reason. I do not believe you can get >KS> at this question with the data you are collecting. > >JS> I think it is possible to analyze retransmission behaviour. Depending >JS> on the SNMP version used (and the other versions also depending on >JS> implementation choices), you can get information whether a response is >JS> just coming late for the original request or it is actually a response >JS> to a retranmitted request. We are not talking TCP here; we are talking >JS> about application layer retransmissions and SNMP has its own msgID and >JS> requestID fields. > >KS> The point I was trying to make here is that it is very difficult to intuit >KS> the reasons behind behaviors seen in the traffic, unless someone or >KS> something tells you. So, you can see what the ends do, but not why. > >I agree, but this is a rather general observation and not necessarily >specific to 3.3 and it is not clear to me what I should do about it. >There is already a general remark in the last paragraph of section 2.5 >that one has to be careful with drawing conclusions that go beyond >what you can really get out of traces. OK - I'll drop it. > >KS> 5. Section 3.4: Please explain why it is "interesting" (your word) to >KS> identify whether concurrency or sequentiality is occurring? What >KS> will you "learn" if both are observed? And, if one is occurring more >KS> frequently or under specific identifiable conditions, what further >KS> does that tell you? Just knowing that one or the other occurs is >KS> only the tip of the iceberg, and without acknowledging the fact that >KS> these are important and unanswered questions, just learning first >KS> ordered details suggests you are setting the bar too low. > >JS> The introduction of section 3 says: >JS> >JS> The questions raised in the following subsections are meant to be >JS> illustrative and no attempt has been made to provide a complete >JS> list. >JS> >JS> I believe it is a good idea to first figure out whether there is an >JS> iceberg or not (keeping your analogy) and if there is one to ask >JS> questions how big the iceberg might be. For SNMP agent implementations >JS> that tend to do quite some caching, it is useful to know how well >JS> caching strategies are working in real-world networks. The concurrency >JS> level an agent experiences has clear impact on that. Furthermore, it >JS> will be useful to know how bursty the traffic tends to be or how well >JS> managers spread the traffic over polling intervals and this is again >JS> related to the concurrency we can extract from traces. >JS> >JS> I am not really sure what I should change, perhaps the word >JS> "interesting" is the source of the trouble and I should replace this >JS> with "valuable"? > >KS> What I was trying to ask was that you tell the reader a bit about >KS> what makes it interesting. It certainly doesn't have to be >KS> complete, but just a> hint. I don't think you should change the >KS> word "interesting", because if it isn't interesting, you probably >KS> shouldn't be doing it. > >I added the following sentence to section 3.4: > > The concurrency level and the amount of redundant requests has > implications on caching strategies employed by SNMP agents. Fine. >KS> 6. Section 3.5: Please explain what you would do with the >KS> information about which approach to table retrieval is used. Again, >KS> what if the results tell you that both are used? And, if not, of >KS> what use is it to know which approach is prevalent? Mighten it be >KS> useful to know the conditions under which one or the other is used >KS> more commonly? > >JS> This again has direct impact on agent implementation techniques and >JS> caching strategies. > >KS> And again, just explain a little. > >I changed the last sentence of section 3.5 to read as follows: > > [...] It will be useful to know which of > these approaches are used on production networks since this has a > direct implication on agent implementation techniques and caching > strategies. Fine. >/js > >-- >Juergen Schoenwaelder Jacobs University Bremen gGmbH >Phone: +49 421 200 3587 Campus Ring 1, 28759 Bremen, Germany >Fax: +49 421 200 3103 -- Karen R. Sollins, Ph. D. Principal Research Scientist M.I.T. CSAIL The Stata Center 32 Vassar St., 32-G818 Cambridge, MA 02139 V: 617/253-6006 F: 617/253-2673 E: sollins@csail.mit.edu Received: from QMTA02.emeryville.ca.mail.comcast.net (qmta02.emeryville.ca.mail.comcast.net [76.96.30.24]) by bierator.ibr.cs.tu-bs.de (8.13.4/8.13.4/Debian-3sarge3) with ESMTP id m4JDG3Mh008908 for ; Mon, 19 May 2008 15:16:08 +0200 Received: from OMTA11.emeryville.ca.mail.comcast.net ([76.96.30.36]) by QMTA02.emeryville.ca.mail.comcast.net with comcast id TPZ91Z01D0mlR8UA206r00; Mon, 19 May 2008 13:15:56 +0000 Received: from Harrington73653 ([24.128.66.199]) by OMTA11.emeryville.ca.mail.comcast.net with comcast id TRFq1Z0024HwxpC8X00000; Mon, 19 May 2008 13:15:52 +0000 X-Authority-Analysis: v=1.0 c=1 a=zOsyX00C8WEA:10 a=1Im-IAa9KzAA:10 a=Lb1g3DJAFA9zFMzVLIoA:9 a=Fn906MrGOBrDZp-6CbwA:7 a=W1aYemK4AHE_IdzZMNnquqwlh6kA:4 a=si9q_4b84H0A:10 a=hPjdaMEvmhQA:10 a=gJcimI5xSWUA:10 From: "David B Harrington" To: "'Karen R. Sollins'" , References: <20080516122042.GA19275@elstar.local> Subject: RE: [nmrg] Re: [IRSG] review of draft-irtf-nmrg-snmp-measure-04.txt Date: Mon, 19 May 2008 09:15:49 -0400 Message-ID: <08cb01c8b9b2$7669edb0$0600a8c0@china.huawei.com> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Mailer: Microsoft Office Outlook 11 In-Reply-To: Thread-Index: Aci5Zi1tRdPfaR5BRt+zXUW2WP/OSgASdrBA X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.3198 X-IBRFilter-SpamReport: 3.602 (***) BAYES_50, DNS_FROM_RFC_POST, RCVD_IN_SORBS_DUL X-Scanned-By: MIMEDefang 2.51 on 134.169.34.9 Cc: 'Internet Research Steering Group' , nmrg@ibr.cs.tu-bs.de X-BeenThere: nmrg@ibr.cs.tu-bs.de X-Mailman-Version: 2.1.5 Precedence: list List-Id: Network Management Research Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 May 2008 13:16:12 -0000 > -----Original Message----- > From: nmrg-bounces@ibr.cs.tu-bs.de > [mailto:nmrg-bounces@ibr.cs.tu-bs.de] On Behalf Of Karen R. Sollins > Sent: Monday, May 19, 2008 12:05 AM > To: j.schoenwaelder@jacobs-university.de; Karen R. Sollins > Cc: Internet Research Steering Group; nmrg@ibr.cs.tu-bs.de > Subject: [nmrg] Re: [IRSG] review of > draft-irtf-nmrg-snmp-measure-04.txt > [...] > >> 3. Next paragraph: this is where the location question arises. > >> Without some completely standardized and self explanatory > capturing > >> of location information, any data set will be incomparable to any > >> other. > > > >I expanded "where the trace was collected" to "where the trace was > >collected (name of the network and/or name of the organization owning > >the network, description of the measurement point in the network > >topology where the trace was collected)". > > Good. > I think something should be said that this information could be used by an attacker (especially an attacker internal to the organization) to decide/pinpoint where and how to attack. This information might need to be anonymized, although that would seem to defeat the purpose of having the information. I don't really know what to suggest here other than to raise the point in the security considerations that such location information might be sensitive, and could aid an attacker. Personally, I do not know that one needs to know the organization and the network within the organization unless you are planning to do regression testing or correlating the data with information available via other means. To compare the data with data sets from different networks, then I think the measurement point in the network topology and knowing the network topology is far more important (and unlikely to be available). Given the dynamic nature of network topologies, especially in any sort of virtualized environment (Virtual LANs, routers, servers, etc.), I doubt any data set even from the same network is likely to be directly comparable over time unless an attempt is made to make the data collections directly comparable by deliberately not changing the topology. I think a statement to that effect might be more useful to those doing analyses than adding organization/network information. David Harrington dbharrington@comcast.net ietfdbh@comcast.net dharrington@huawei.com Received: from hermes.jacobs-university.de (hermes.jacobs-university.de [212.201.44.23]) by bierator.ibr.cs.tu-bs.de (8.13.4/8.13.4/Debian-3sarge3) with ESMTP id m4J9kkLM009294 for ; Mon, 19 May 2008 11:46:51 +0200 Received: from localhost (demetrius3.jacobs-university.de [212.201.44.48]) by hermes.jacobs-university.de (Postfix) with ESMTP id 19658C0002; Mon, 19 May 2008 11:46:46 +0200 (CEST) X-Virus-Scanned: amavisd-new at jacobs-university.de Received: from hermes.jacobs-university.de ([212.201.44.23]) by localhost (demetrius3.jacobs-university.de [212.201.44.32]) (amavisd-new, port 10024) with ESMTP id Ob-VXCaLdwoP; Mon, 19 May 2008 11:46:39 +0200 (CEST) Received: from elstar.local (elstar.iuhb02.iu-bremen.de [10.50.231.133]) by hermes.jacobs-university.de (Postfix) with ESMTP id 1E5F0C000D; Mon, 19 May 2008 11:46:39 +0200 (CEST) Received: by elstar.local (Postfix, from userid 501) id 343A15970AE; Mon, 19 May 2008 11:46:39 +0200 (CEST) Date: Mon, 19 May 2008 11:46:39 +0200 From: Juergen Schoenwaelder To: "Karen R. Sollins" Message-ID: <20080519094639.GA27481@elstar.local> Mail-Followup-To: "Karen R. Sollins" , nmrg@ibr.cs.tu-bs.de, Bert Wijnen - IETF , Internet Research Steering Group References: <20080516122042.GA19275@elstar.local> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.17 (2007-11-01) X-IBRFilter-SpamReport: 0.001 () BAYES_50 X-Scanned-By: MIMEDefang 2.51 on 134.169.34.9 Cc: Internet Research Steering Group , nmrg@ibr.cs.tu-bs.de Subject: [nmrg] Re: [IRSG] review of draft-irtf-nmrg-snmp-measure-04.txt X-BeenThere: nmrg@ibr.cs.tu-bs.de X-Mailman-Version: 2.1.5 Precedence: list Reply-To: j.schoenwaelder@jacobs-university.de List-Id: Network Management Research Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 May 2008 09:46:53 -0000 On Sun, May 18, 2008 at 09:05:15PM -0700, Karen R. Sollins wrote: > Thanks for your thoughtful responses. I also did not think that what I was > suggesting was a lot of work. At a high level, think about a reader who > is not part of your group, whom you are trying to convince that what you > are doing is valuable or whom you would like to convince to do such a data > collection exercise. I have interspersed my comments into your responses > below. I will repond to those things that are still "open" with suggestions on how to "close" them. KS> The second high level concern I have is that there is talk about KS> specific kinds of information to be collected and an interest in not KS> only the nature but longer term inferences with perhaps implications KS> for future redesign efforts in the SNMP context. I have two levels KS> of concern here. The most important one is that since network KS> management and in particular SNMP is NOT the primary objective of the KS> net (the primary objective being the transport of real payload), it KS> seems to me that the truly critical question with respect to network KS> management traffic is the impact that it has or does not have on that KS> real job. To me this implies that the measurements MUST also include KS> contextual information. As an example, it is probably more important KS> to understand whether or not the network management traffic is KS> causing significant congestion for the payload traffic than the KS> particular mix or frequency pattern within the network management KS> traffic. Without out the complementary contextual information, the KS> whole measurement exercise seems to me to be of somewhat narrow KS> value. JS> The measurement may be of narrow value from your point of view but JS> please keep in mind that this document is coming from the Network JS> Management Research Group and not from a general Network Measurement JS> Research Group. Our goal is to understand how network management JS> protocols are being used because that has impact on their design and JS> implementation strategies. Further note that in many networks, the JS> management traffic is logically and sometimes even physically JS> separated from the normal traffic and perhaps this is the reason why JS> we did not even think about the question whether management traffic JS> has an impact on normal traffic. KS> If you want to leave it as is, then I think it would be valuable to say as KS> much. Be specific about what you are not doing, because much of the rest KS> of the world looks at network traffic from a broader perspective. I think the abstract and the introduction are pretty clearly spelling out the scope of the measurement effort. So I am not sure changes are needed, but see below. KS> 1. Section 1: It seems to me that there are TWO key questions with KS> respect to SNMP. The first is how it is being used, which in turn KS> leads to the points made in this section, but the second is the KS> impact of that traffic. I think that ought to appear in the KS> Introduction as well. JS> See my comments above. So far, the NMRG did not consider the impact of JS> SNMP on other traffic a target of this activity. I don't want to add JS> such text unless I see support from the NMRG and concrete proposals JS> what should be added. KS> Again, as above, if you want to leave the scope as it is, then you should KS> probably be up front about that, clearly leaving that work to another time KS> and place. I propose to add the following paragraph just before the last paragraph in section 1: The measurement approach described in this document is by design limited to the study of SNMP traffic. Studies of other management protocols or the impact of management protocols such as SNMP on other traffic sharing the same network resources is left to future efforts. KS> 2. Section 2.1. The second paragraph begins with, "It is recommended KS> to capture at least a full week of data." This is never justified or KS> explained. Is one week really enough? For what? Why wouldn't KS> several weeks be critical, because one week might be anomalous? Why KS> isn't a year critical, since we know that there are annual or KS> seasonal differences in traffic behaviors? Typically, I find that KS> one-week data sets often leave me with lots of unanswered questions, KS> so justify this. JS> The text actually says: JS> JS> It is recommended to capture at least a full week of data. Operators JS> are encouraged to capture traces over even longer periods of time. JS> JS> The text tries to establish a lower bound of one week an encourages JS> longer capture periods. I would love to get continuous traces but JS> reality is such that this is not feasible. Our idea is simply to catch JS> at least the weekly behaviour. Yes, there is of course also monthly or JS> yearly behaviour but I believe it is not useful to set the bar so high JS> that nobody gives us appropriate traces. I personally believe the text JS> is fine as is. KS> So, what I was really getting at was the question of why one week KS> was the minimum necessary. So, something like that it is the KS> minimum over which one can see the diurnal patterns in the weekly KS> pattern and it is understood that both for computational and KS> storage reasons the operators may not want to collect more. I have changed the text to the following: It is recommended to capture at least a full week of data to capture diurnal patterns and one cycle of weekly behavior. Operators are strongly encouraged to capture traces over even longer periods of time. KS> 4. Section 3.3, end of first paragraph: The sentence reads, "Some KS> SNMP implementations approximate networking delays by measuring KS> request-response times and it would be useful to understand to what KS> extent this is a viable approach." I agree, but traces will not tell KS> you anything about whether behaviors observed in packet traces are KS> for this reason or some other reason. I do not believe you can get KS> at this question with the data you are collecting. JS> I think it is possible to analyze retransmission behaviour. Depending JS> on the SNMP version used (and the other versions also depending on JS> implementation choices), you can get information whether a response is JS> just coming late for the original request or it is actually a response JS> to a retranmitted request. We are not talking TCP here; we are talking JS> about application layer retransmissions and SNMP has its own msgID and JS> requestID fields. KS> The point I was trying to make here is that it is very difficult to intuit KS> the reasons behind behaviors seen in the traffic, unless someone or KS> something tells you. So, you can see what the ends do, but not why. I agree, but this is a rather general observation and not necessarily specific to 3.3 and it is not clear to me what I should do about it. There is already a general remark in the last paragraph of section 2.5 that one has to be careful with drawing conclusions that go beyond what you can really get out of traces. KS> 5. Section 3.4: Please explain why it is "interesting" (your word) to KS> identify whether concurrency or sequentiality is occurring? What KS> will you "learn" if both are observed? And, if one is occurring more KS> frequently or under specific identifiable conditions, what further KS> does that tell you? Just knowing that one or the other occurs is KS> only the tip of the iceberg, and without acknowledging the fact that KS> these are important and unanswered questions, just learning first KS> ordered details suggests you are setting the bar too low. JS> The introduction of section 3 says: JS> JS> The questions raised in the following subsections are meant to be JS> illustrative and no attempt has been made to provide a complete JS> list. JS> JS> I believe it is a good idea to first figure out whether there is an JS> iceberg or not (keeping your analogy) and if there is one to ask JS> questions how big the iceberg might be. For SNMP agent implementations JS> that tend to do quite some caching, it is useful to know how well JS> caching strategies are working in real-world networks. The concurrency JS> level an agent experiences has clear impact on that. Furthermore, it JS> will be useful to know how bursty the traffic tends to be or how well JS> managers spread the traffic over polling intervals and this is again JS> related to the concurrency we can extract from traces. JS> JS> I am not really sure what I should change, perhaps the word JS> "interesting" is the source of the trouble and I should replace this JS> with "valuable"? KS> What I was trying to ask was that you tell the reader a bit about KS> what makes it interesting. It certainly doesn't have to be KS> complete, but just a> hint. I don't think you should change the KS> word "interesting", because if it isn't interesting, you probably KS> shouldn't be doing it. I added the following sentence to section 3.4: The concurrency level and the amount of redundant requests has implications on caching strategies employed by SNMP agents. KS> 6. Section 3.5: Please explain what you would do with the KS> information about which approach to table retrieval is used. Again, KS> what if the results tell you that both are used? And, if not, of KS> what use is it to know which approach is prevalent? Mighten it be KS> useful to know the conditions under which one or the other is used KS> more commonly? JS> This again has direct impact on agent implementation techniques and JS> caching strategies. KS> And again, just explain a little. I changed the last sentence of section 3.5 to read as follows: [...] It will be useful to know which of these approaches are used on production networks since this has a direct implication on agent implementation techniques and caching strategies. /js -- Juergen Schoenwaelder Jacobs University Bremen gGmbH Phone: +49 421 200 3587 Campus Ring 1, 28759 Bremen, Germany Fax: +49 421 200 3103 Received: from relay.versatel.net (relay.versatel.net [62.250.3.110]) by bierator.ibr.cs.tu-bs.de (8.13.4/8.13.4/Debian-3sarge3) with SMTP id m4J8BVvC016482 for ; Mon, 19 May 2008 10:11:37 +0200 Received: (qmail 10502 invoked from network); 19 May 2008 08:11:31 -0000 Received: from unknown (HELO bwMedion) (87.215.199.34) by relay.versatel.net with SMTP; 19 May 2008 08:11:31 -0000 From: "Bert Wijnen - IETF" To: "Karen R. Sollins" , Date: Mon, 19 May 2008 10:11:37 +0200 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="US-ASCII" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.6604 (9.0.2911.0) In-Reply-To: X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.3198 Importance: Normal X-IBRFilter-SpamReport: 0.001 () BAYES_50 X-Scanned-By: MIMEDefang 2.51 on 134.169.34.9 Cc: Internet Research Steering Group , nmrg@ibr.cs.tu-bs.de Subject: [nmrg] RE: [IRSG] review of draft-irtf-nmrg-snmp-measure-04.txt X-BeenThere: nmrg@ibr.cs.tu-bs.de X-Mailman-Version: 2.1.5 Precedence: list List-Id: Network Management Research Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 May 2008 08:11:43 -0000 Thanks for the discussion sofar. >From what I have seen during the development of this document I believe that the scope of the document was intentionally limited to what the document describes. In other weords, we intentionally did not want to include the "impact of SNMP traffic on otehr network activity". >From the discussion below, I think what we need to do is to be somewhat more explicit about the fact that and why we have limited ourselves to that scope. Karen, please let me know if you agree with my assesment. Further Juergen has fixed a few things and made a few more changes as per below discussion. I asumme that the NMRG group does agree with this approach. If not, please speak up quickly/soon. Bert Wijnen document shepherd > -----Oorspronkelijk bericht----- > Van: Karen R. Sollins [mailto:sollins@csail.mit.edu] > Verzonden: maandag 19 mei 2008 6:05 > Aan: j.schoenwaelder@jacobs-university.de; Karen R. Sollins > CC: nmrg@ibr.cs.tu-bs.de; Bert Wijnen - IETF; Internet Research Steering > Group > Onderwerp: Re: [IRSG] review of draft-irtf-nmrg-snmp-measure-04.txt > > > HI Juergen, > > Thanks for your thoughtful responses. I also did not think that what > I was suggesting was a lot of work. At a high level, think about a > reader who is not part of your group, whom you are trying to convince > that what you are doing is valuable or whom you would like to > convince to do such a data collection exercise. I have interspersed > my comments into your responses below. > > Cheers, > Karen > > At 2:20 PM +0200 5/16/08, Juergen Schoenwaelder wrote: > >On Mon, May 12, 2008 at 02:52:02PM -0400, Karen R. Sollins wrote: > > > >> Here is my full review of draft-irtf-nmrg-snmp-measure-04.txt. > > > >Thank you very much for your substantial review. Below is how I plan > >to address your comments. For most of your comments, the changes are > >easy to do. For some of your comments, I suggest to make no changes > >and I try to explain why. Please check and let me and the RG know > >where you do not agree with my suggested plan of action. > > > >> I do not think this document is quite ready for publication, but it > >> is quite close and I don't think this should be major work. > >> > >> My review falls into two parts. The first is broad comments and the > >> second is nits. Overall, I agree with the working group that it is > >> valuable to spell out a set of details such as this, more or less a > >> "protocol" in the sense of biologically based research, describing > >> what should be done, what kind of information should be collected and > >> why and how to handle it. I have not reviewed the XML or CSV > >> representations nor the reference lists in detail. With that said, I > >> have two high level concerns with this document. > >> > >> The first is that there is confusion about whether this document > >> describes a proposed measurement effort or (closer to the reality of > >> what is in the document) a methodology by which such measurements > >> could and perhaps should be taken. So, given the bulk of the > >> document, I do not believe that it is at all explicit about the > >> particular measurement exercise that will be done. With that said, > >> there are a few introductory points that are worth changing. > >> > >> 1. The abstract includes the sentence, "This document proposes to > >> carry out large scale SNMP traffic measurements." First, the > >> document won't be carrying out anything. More importantly the > >> document describes an approach (not even a pan), so I suggest that > >> the sentence read, "This document describes an approach to carrying > >> out..." > > > >changed as suggested > > > >> 2. In Section 1, the 4th paragraph reads, "This document describes an > >> effort to collect SNMP traffic traces..." Instead, I suggest that it > >> read, "This document recommends an approach to collecting, codifying, > >> and handling SNMP traffic traces..." > > > >changed as suggested > > > >> The second high level concern I have is that there is talk about > >> specific kinds of information to be collected and an interest in not > >> only the nature but longer term inferences with perhaps implications > >> for future redesign efforts in the SNMP context. I have two levels > >> of concern here. The most important one is that since network > >> management and in particular SNMP is NOT the primary objective of the > >> net (the primary objective being the transport of real payload), it > >> seems to me that the truly critical question with respect to network > >> management traffic is the impact that it has or does not have on that > >> real job. To me this implies that the measurements MUST also include > >> contextual information. As an example, it is probably more important > >> to understand whether or not the network management traffic is > >> causing significant congestion for the payload traffic than the > >> particular mix or frequency pattern within the network management > >> traffic. Without out the complementary contextual information, the > > > whole measurement exercise seems to me to be of somewhat narrow > >> value. > > > >The measurement may be of narrow value from your point of view but > >please keep in mind that this document is coming from the Network > >Management Research Group and not from a general Network Measurement > >Research Group. Our goal is to understand how network management > >protocols are being used because that has impact on their design and > >implementation strategies. Further note that in many networks, the > >management traffic is logically and sometimes even physically > >separated from the normal traffic and perhaps this is the reason why > >we did not even think about the question whether management traffic > >has an impact on normal traffic. > > If you want to leave it as is, then I think it would be valuable to > say as much. Be specific about what you are not doing, because much > of the rest of the world looks at network traffic from a broader > perspective. > > > > >> Secondarily, the document hits on one of my little pet peeves > >> - location. There is mention that the meta-data must include "where > >> the trace was collected". That, in and of itself, is a challenging > >> problem. Is this geographic, topological, or based on other metrics? > >> If it is based on something that can change with time (e.g. ip > >> addresses or topology), then without keeping track of how those > >> metrics may have changed over time, the data set becomes useless > >> because the location becomes uninterpretable. As you all hint, it is > >> important that the collected information be archived, in order to be > >> useful at significantly later times, and thus, everything importatn > >> about the situation must be recorded. I would like to see more > >> specificity with respect to location and clearer acknowledgement of > >> its need in order to keep the collected information useful outside > >> the current context. > > > >Makes sense. > > > >> More generally, there are many cases in which there are statements > >> about what kind of information should be collected (e.g.one week > >> traces, concurrent vs. sequential requests, etc.) where there is no > >> explanation of the criticality of this information or how to go > >> beyond the immediately observable facts. > >> > >> I realize that it is generally not in the nature of RFC to provide > >> explanation as much as simple unadorned facts, but this is an unusual > >> kind of document, because it is describing a potential information > >> gathering exercise. It has some of the explanations of why things > >> are important, but it is quite incomplete. I realize that it is > >> important to collect and archive measurement sets exactly because one > >> cannot know ahead of time all the potential uses, but there should > >> either be a story about collecting everything possible or an initial > >> use for each kind of information collected. In some cases neither > >> argument is presented. > >> > >> This all leads to a number of specific questions: > >> > >> 1. Section 1: It seems to me that there are TWO key questions with > >> respect to SNMP. The first is how it is being used, which in turn > >> leads to the points made in this section, but the second is the > >> impact of that traffic. I think that ought to appear in the > >> Introduction as well. > > > >See my comments above. So far, the NMRG did not consider the impact of > >SNMP on other traffic a target of this activity. I don't want to add > >such text unless I see support from the NMRG and concrete proposals > >what should be added. > > Again, as above, if you want to leave the scope as it is, then you > should probably be up front about that, clearly leaving that work to > another time and place. > > > > >> 2. Section 2.1. The second paragraph begins with, "It is recommended > >> to capture at least a full week of data." This is never justified or > >> explained. Is one week really enough? For what? Why wouldn't > >> several weeks be critical, because one week might be anomalous? Why > >> isn't a year critical, since we know that there are annual or > >> seasonal differences in traffic behaviors? Typically, I find that > >> one-week data sets often leave me with lots of unanswered questions, > >> so justify this. > > > >The text actually says: > > > > It is recommended to capture at least a full week of data. Operators > > are encouraged to capture traces over even longer periods of time. > > > >The text tries to establish a lower bound of one week an encourages > >longer capture periods. I would love to get continuous traces but > >reality is such that this is not feasible. Our idea is simply to catch > >at least the weekly behaviour. Yes, there is of course also monthly or > >yearly behaviour but I believe it is not useful to set the bar so high > >that nobody gives us appropriate traces. I personally believe the text > >is fine as is. > > So, what I was really getting at was the question of why one week was > the minimum necessary. So, something like that it is the minimum > over which one can see the diurnal patterns in the weekly pattern and > it is understood that both for computational and storage reasons the > operators may not want to collect more. > > > > >> 3. Next paragraph: this is where the location question arises. > >> Without some completely standardized and self explanatory capturing > >> of location information, any data set will be incomparable to any > >> other. > > > >I expanded "where the trace was collected" to "where the trace was > >collected (name of the network and/or name of the organization owning > >the network, description of the measurement point in the network > >topology where the trace was collected)". > > Good. > > > > >> 4. Section 3.3, end of first paragraph: The sentence reads, "Some > >> SNMP implementations approximate networking delays by measuring > >> request-response times and it would be useful to understand to what > >> extent this is a viable approach." I agree, but traces will not tell > >> you anything about whether behaviors observed in packet traces are > >> for this reason or some other reason. I do not believe you can get > >> at this question with the data you are collecting. > > > >I think it is possible to analyze retransmission behaviour. Depending > >on the SNMP version used (and the other versions also depending on > >implementation choices), you can get information whether a response is > >just coming late for the original request or it is actually a response > >to a retranmitted request. We are not talking TCP here; we are talking > >about application layer retransmissions and SNMP has its own msgID and > >requestID fields. > > The point I was trying to make here is that it is very difficult to > intuit the reasons behind behaviors seen in the traffic, unless > someone or something tells you. So, you can see what the ends do, > but not why. > > > > >> 5. Section 3.4: Please explain why it is "interesting" (your word) to > >> identify whether concurrency or sequentiality is occurring? What > >> will you "learn" if both are observed? And, if one is occurring more > >> frequently or under specific identifiable conditions, what further > >> does that tell you? Just knowing that one or the other occurs is > >> only the tip of the iceberg, and without acknowledging the fact that > >> these are important and unanswered questions, just learning first > >> ordered details suggests you are setting the bar too low. > > > >The introduction of section 3 says: > > > > The questions raised in the following subsections are meant to be > > illustrative and no attempt has been made to provide a complete > > list. > > > >I believe it is a good idea to first figure out whether there is an > >iceberg or not (keeping your analogy) and if there is one to ask > >questions how big the iceberg might be. For SNMP agent implementations > >that tend to do quite some caching, it is useful to know how well > >caching strategies are working in real-world networks. The concurrency > >level an agent experiences has clear impact on that. Furthermore, it > >will be useful to know how bursty the traffic tends to be or how well > >managers spread the traffic over polling intervals and this is again > >related to the concurrency we can extract from traces. > > > >I am not really sure what I should change, perhaps the word > >"interesting" is the source of the trouble and I should replace this > >with "valuable"? > > What I was trying to ask was that you tell the reader a bit about > what makes it interesting. It certainly doesn't have to be complete, > but just a hint. I don't think you should change the word > "interesting", because if it isn't interesting, you probably > shouldn't be doing it. > > > > >> 6. Section 3.5: Please explain what you would do with the > >> information about which approach to table retrieval is used. Again, > >> what if the results tell you that both are used? And, if not, of > >> what use is it to know which approach is prevalent? Mighten it be > >> useful to know the conditions under which one or the other is used > > > more commonly? > > > >This again has direct impact on agent implementation techniques and > >caching strategies. > > And again, just explain a little. > > > > >> Now on to the real NITS: > >> 1. p. 1, Section 1, middle of second paragraph: > >> s/In fact, there are many speculations how SNMP/In fact, there are > >> many speculations on how SNMP/ > > > >fixed > > > >> 2. p. 6, Section 2.2, 5th paragraph: > >> s/for SNMP messages that got fragmented/for SNMP messages that > >>were fragmented/ > > > >fixed > > > >> 3. p. 7, Section 2.4, last paragraph: > >> s/Improvements in the tool chain may require to go back to the > >> original pcap traces and to rebuild all intermediate formats from > >> them./Improvements in the tool chain may require going back to the > >> original pcap traces and rebuilding all intermediate formats from > >> them./ > > > >fixed > > > >> 4. p. 7, Section 2.5, first paragraph: > >> s/all scripts used to analyze traffic traces would be/all scripts > >> used to analyze traffic traces will be/ > > > >fixed > > > >> 5. p. 8, first sentence: > >> s/A common versioned repository for analysis of scripts might be > >> useful to establish./It might be useful to establish and common, > >> versioning repository for analysis scripts./ > > > >fixed > > > >> 6. p. 6, second paragraph (includes several corrections): > >> s/it is suggested that analysis scripts are written in scripting > >> languages such as Perl using suitable Perl modules to manipulate XML > >> documents [8]. Using a scripting language such as Perl instead of > >> system programming languages such as C or C++ has the advantage to > >> reduce development time and to make scripts more accessible to > >> operators who may want to verify scripts before running them on trace > >> files which potentially contain sensitive data./it is suggested that > >> analysis scripts be written is scripting languages such as Perl using > >> suitable Perl modules to manipulate XML documents [8]. Using a > >> scripting language such as Perl instead of system programming > >> languages such as C or C++ has the advantage of reducing development > >> time and making scripts more accessible to operators who may want to > >> verify scripts before running them on trace files which may contain > >> sensitive data./ > > > >fixed > > > >> 6. p. 6, end of next paragraph: > >> s/might be the only option to deal/might be the only option > in dealing/ > > > >fixed > > > >> 7. p. 9, Section 3.2. In this case there are enough corrections in > >> close proximity that I've grouped them together. I suggest that the > >> following text: > >> > >> "SNMP is used to periodically poll devices as well as to retrieve > >> information on request of an operator or application. The periodic > >> polling leads to periodic traffic patterns while the on demand > >> information retrieval causes more aperiodic traffic patterns. It is > >> worthwhile to understand what the relationship is between the amount > >> of periodic and aperiodic traffic. It will be interesting to > >> research whether there are multiple levels of periodicity at > >> different time scales. > >> > >> The periodic polling behavior may dependent on the application and > >> the polling engine it uses. For example, some management platforms > >> allow applications to specify how long polled values may be kept in a > >> cache before it is polled again." > >> > >> have the following grammatical changes made: > >> > >> "SNMP is used to periodically poll devices as well as to retrieve > >> information at the request of an operator or application. The > >> periodic polling leads to periodic traffic patterns while on-demand > >> information retrieval causes more aperiodic traffic patterns. It is > >> worthwhile to understand what the relationship is between the amount > >> of periodic and aperiodic traffic. It will be interesting to > >> understand whether there are multiple levels of periodicity at > >> different time scales. > >> > >> Periodic polling behavior may be dependent on the application and > > > polling engine it uses. For example, some management platforms allow > >> applications to specify how long polled values may be kept in a cache > >> before they are polled again." > > > >fixed > > > >> 8. p. 10, Section 3.6, middle first paragraph: > >> s/ adopt/adapt/ > > > >fixed > > > >/js > > > >-- > >Juergen Schoenwaelder Jacobs University Bremen gGmbH > >Phone: +49 421 200 3587 Campus Ring 1, 28759 Bremen, Germany > >Fax: +49 421 200 3103 > > > -- > > Karen R. Sollins, Ph. D. > Principal Research Scientist > M.I.T. CSAIL > The Stata Center > 32 Vassar St., 32-G818 > Cambridge, MA 02139 > V: 617/253-6006 > F: 617/253-2673 > E: sollins@csail.mit.edu > Received: from mercury.lcs.mit.edu (mercury.lcs.mit.edu [18.26.0.122]) by bierator.ibr.cs.tu-bs.de (8.13.4/8.13.4/Debian-3sarge3) with ESMTP id m4J48pJv026256 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT) for ; Mon, 19 May 2008 06:08:59 +0200 Received: from [192.168.1.105] (cpe-76-168-88-197.socal.res.rr.com [76.168.88.197]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mercury.lcs.mit.edu (Postfix) with ESMTP id 5AF556BE555; Mon, 19 May 2008 00:08:48 -0400 (EDT) Mime-Version: 1.0 Message-Id: In-Reply-To: <20080516122042.GA19275@elstar.local> References: <20080516122042.GA19275@elstar.local> Date: Sun, 18 May 2008 21:05:15 -0700 To: j.schoenwaelder@jacobs-university.de, "Karen R. Sollins" From: "Karen R. Sollins" Content-Type: text/plain; charset="us-ascii" ; format="flowed" X-IBRFilter-SpamReport: 0.001 () BAYES_50 X-Scanned-By: MIMEDefang 2.51 on 134.169.34.9 Cc: Internet Research Steering Group , nmrg@ibr.cs.tu-bs.de Subject: [nmrg] Re: [IRSG] review of draft-irtf-nmrg-snmp-measure-04.txt X-BeenThere: nmrg@ibr.cs.tu-bs.de X-Mailman-Version: 2.1.5 Precedence: list List-Id: Network Management Research Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 May 2008 04:09:04 -0000 HI Juergen, Thanks for your thoughtful responses. I also did not think that what I was suggesting was a lot of work. At a high level, think about a reader who is not part of your group, whom you are trying to convince that what you are doing is valuable or whom you would like to convince to do such a data collection exercise. I have interspersed my comments into your responses below. Cheers, Karen At 2:20 PM +0200 5/16/08, Juergen Schoenwaelder wrote: >On Mon, May 12, 2008 at 02:52:02PM -0400, Karen R. Sollins wrote: > >> Here is my full review of draft-irtf-nmrg-snmp-measure-04.txt. > >Thank you very much for your substantial review. Below is how I plan >to address your comments. For most of your comments, the changes are >easy to do. For some of your comments, I suggest to make no changes >and I try to explain why. Please check and let me and the RG know >where you do not agree with my suggested plan of action. > >> I do not think this document is quite ready for publication, but it >> is quite close and I don't think this should be major work. >> >> My review falls into two parts. The first is broad comments and the >> second is nits. Overall, I agree with the working group that it is >> valuable to spell out a set of details such as this, more or less a >> "protocol" in the sense of biologically based research, describing >> what should be done, what kind of information should be collected and >> why and how to handle it. I have not reviewed the XML or CSV >> representations nor the reference lists in detail. With that said, I >> have two high level concerns with this document. >> >> The first is that there is confusion about whether this document >> describes a proposed measurement effort or (closer to the reality of >> what is in the document) a methodology by which such measurements >> could and perhaps should be taken. So, given the bulk of the >> document, I do not believe that it is at all explicit about the >> particular measurement exercise that will be done. With that said, >> there are a few introductory points that are worth changing. >> >> 1. The abstract includes the sentence, "This document proposes to >> carry out large scale SNMP traffic measurements." First, the >> document won't be carrying out anything. More importantly the >> document describes an approach (not even a pan), so I suggest that >> the sentence read, "This document describes an approach to carrying >> out..." > >changed as suggested > >> 2. In Section 1, the 4th paragraph reads, "This document describes an >> effort to collect SNMP traffic traces..." Instead, I suggest that it >> read, "This document recommends an approach to collecting, codifying, >> and handling SNMP traffic traces..." > >changed as suggested > >> The second high level concern I have is that there is talk about >> specific kinds of information to be collected and an interest in not >> only the nature but longer term inferences with perhaps implications >> for future redesign efforts in the SNMP context. I have two levels >> of concern here. The most important one is that since network >> management and in particular SNMP is NOT the primary objective of the >> net (the primary objective being the transport of real payload), it >> seems to me that the truly critical question with respect to network >> management traffic is the impact that it has or does not have on that >> real job. To me this implies that the measurements MUST also include >> contextual information. As an example, it is probably more important >> to understand whether or not the network management traffic is >> causing significant congestion for the payload traffic than the >> particular mix or frequency pattern within the network management >> traffic. Without out the complementary contextual information, the > > whole measurement exercise seems to me to be of somewhat narrow >> value. > >The measurement may be of narrow value from your point of view but >please keep in mind that this document is coming from the Network >Management Research Group and not from a general Network Measurement >Research Group. Our goal is to understand how network management >protocols are being used because that has impact on their design and >implementation strategies. Further note that in many networks, the >management traffic is logically and sometimes even physically >separated from the normal traffic and perhaps this is the reason why >we did not even think about the question whether management traffic >has an impact on normal traffic. If you want to leave it as is, then I think it would be valuable to say as much. Be specific about what you are not doing, because much of the rest of the world looks at network traffic from a broader perspective. > >> Secondarily, the document hits on one of my little pet peeves >> - location. There is mention that the meta-data must include "where >> the trace was collected". That, in and of itself, is a challenging >> problem. Is this geographic, topological, or based on other metrics? >> If it is based on something that can change with time (e.g. ip >> addresses or topology), then without keeping track of how those >> metrics may have changed over time, the data set becomes useless >> because the location becomes uninterpretable. As you all hint, it is >> important that the collected information be archived, in order to be >> useful at significantly later times, and thus, everything importatn >> about the situation must be recorded. I would like to see more >> specificity with respect to location and clearer acknowledgement of >> its need in order to keep the collected information useful outside >> the current context. > >Makes sense. > >> More generally, there are many cases in which there are statements >> about what kind of information should be collected (e.g.one week >> traces, concurrent vs. sequential requests, etc.) where there is no >> explanation of the criticality of this information or how to go >> beyond the immediately observable facts. >> >> I realize that it is generally not in the nature of RFC to provide >> explanation as much as simple unadorned facts, but this is an unusual >> kind of document, because it is describing a potential information >> gathering exercise. It has some of the explanations of why things >> are important, but it is quite incomplete. I realize that it is >> important to collect and archive measurement sets exactly because one >> cannot know ahead of time all the potential uses, but there should >> either be a story about collecting everything possible or an initial >> use for each kind of information collected. In some cases neither >> argument is presented. >> >> This all leads to a number of specific questions: >> >> 1. Section 1: It seems to me that there are TWO key questions with >> respect to SNMP. The first is how it is being used, which in turn >> leads to the points made in this section, but the second is the >> impact of that traffic. I think that ought to appear in the >> Introduction as well. > >See my comments above. So far, the NMRG did not consider the impact of >SNMP on other traffic a target of this activity. I don't want to add >such text unless I see support from the NMRG and concrete proposals >what should be added. Again, as above, if you want to leave the scope as it is, then you should probably be up front about that, clearly leaving that work to another time and place. > >> 2. Section 2.1. The second paragraph begins with, "It is recommended >> to capture at least a full week of data." This is never justified or >> explained. Is one week really enough? For what? Why wouldn't >> several weeks be critical, because one week might be anomalous? Why >> isn't a year critical, since we know that there are annual or >> seasonal differences in traffic behaviors? Typically, I find that >> one-week data sets often leave me with lots of unanswered questions, >> so justify this. > >The text actually says: > > It is recommended to capture at least a full week of data. Operators > are encouraged to capture traces over even longer periods of time. > >The text tries to establish a lower bound of one week an encourages >longer capture periods. I would love to get continuous traces but >reality is such that this is not feasible. Our idea is simply to catch >at least the weekly behaviour. Yes, there is of course also monthly or >yearly behaviour but I believe it is not useful to set the bar so high >that nobody gives us appropriate traces. I personally believe the text >is fine as is. So, what I was really getting at was the question of why one week was the minimum necessary. So, something like that it is the minimum over which one can see the diurnal patterns in the weekly pattern and it is understood that both for computational and storage reasons the operators may not want to collect more. > >> 3. Next paragraph: this is where the location question arises. >> Without some completely standardized and self explanatory capturing >> of location information, any data set will be incomparable to any >> other. > >I expanded "where the trace was collected" to "where the trace was >collected (name of the network and/or name of the organization owning >the network, description of the measurement point in the network >topology where the trace was collected)". Good. > >> 4. Section 3.3, end of first paragraph: The sentence reads, "Some >> SNMP implementations approximate networking delays by measuring >> request-response times and it would be useful to understand to what >> extent this is a viable approach." I agree, but traces will not tell >> you anything about whether behaviors observed in packet traces are >> for this reason or some other reason. I do not believe you can get >> at this question with the data you are collecting. > >I think it is possible to analyze retransmission behaviour. Depending >on the SNMP version used (and the other versions also depending on >implementation choices), you can get information whether a response is >just coming late for the original request or it is actually a response >to a retranmitted request. We are not talking TCP here; we are talking >about application layer retransmissions and SNMP has its own msgID and >requestID fields. The point I was trying to make here is that it is very difficult to intuit the reasons behind behaviors seen in the traffic, unless someone or something tells you. So, you can see what the ends do, but not why. > >> 5. Section 3.4: Please explain why it is "interesting" (your word) to >> identify whether concurrency or sequentiality is occurring? What >> will you "learn" if both are observed? And, if one is occurring more >> frequently or under specific identifiable conditions, what further >> does that tell you? Just knowing that one or the other occurs is >> only the tip of the iceberg, and without acknowledging the fact that >> these are important and unanswered questions, just learning first >> ordered details suggests you are setting the bar too low. > >The introduction of section 3 says: > > The questions raised in the following subsections are meant to be > illustrative and no attempt has been made to provide a complete > list. > >I believe it is a good idea to first figure out whether there is an >iceberg or not (keeping your analogy) and if there is one to ask >questions how big the iceberg might be. For SNMP agent implementations >that tend to do quite some caching, it is useful to know how well >caching strategies are working in real-world networks. The concurrency >level an agent experiences has clear impact on that. Furthermore, it >will be useful to know how bursty the traffic tends to be or how well >managers spread the traffic over polling intervals and this is again >related to the concurrency we can extract from traces. > >I am not really sure what I should change, perhaps the word >"interesting" is the source of the trouble and I should replace this >with "valuable"? What I was trying to ask was that you tell the reader a bit about what makes it interesting. It certainly doesn't have to be complete, but just a hint. I don't think you should change the word "interesting", because if it isn't interesting, you probably shouldn't be doing it. > >> 6. Section 3.5: Please explain what you would do with the >> information about which approach to table retrieval is used. Again, >> what if the results tell you that both are used? And, if not, of >> what use is it to know which approach is prevalent? Mighten it be >> useful to know the conditions under which one or the other is used > > more commonly? > >This again has direct impact on agent implementation techniques and >caching strategies. And again, just explain a little. > >> Now on to the real NITS: >> 1. p. 1, Section 1, middle of second paragraph: >> s/In fact, there are many speculations how SNMP/In fact, there are >> many speculations on how SNMP/ > >fixed > >> 2. p. 6, Section 2.2, 5th paragraph: >> s/for SNMP messages that got fragmented/for SNMP messages that >>were fragmented/ > >fixed > >> 3. p. 7, Section 2.4, last paragraph: >> s/Improvements in the tool chain may require to go back to the >> original pcap traces and to rebuild all intermediate formats from >> them./Improvements in the tool chain may require going back to the >> original pcap traces and rebuilding all intermediate formats from >> them./ > >fixed > >> 4. p. 7, Section 2.5, first paragraph: >> s/all scripts used to analyze traffic traces would be/all scripts >> used to analyze traffic traces will be/ > >fixed > >> 5. p. 8, first sentence: >> s/A common versioned repository for analysis of scripts might be >> useful to establish./It might be useful to establish and common, >> versioning repository for analysis scripts./ > >fixed > >> 6. p. 6, second paragraph (includes several corrections): >> s/it is suggested that analysis scripts are written in scripting >> languages such as Perl using suitable Perl modules to manipulate XML >> documents [8]. Using a scripting language such as Perl instead of >> system programming languages such as C or C++ has the advantage to >> reduce development time and to make scripts more accessible to >> operators who may want to verify scripts before running them on trace >> files which potentially contain sensitive data./it is suggested that >> analysis scripts be written is scripting languages such as Perl using >> suitable Perl modules to manipulate XML documents [8]. Using a >> scripting language such as Perl instead of system programming >> languages such as C or C++ has the advantage of reducing development >> time and making scripts more accessible to operators who may want to >> verify scripts before running them on trace files which may contain >> sensitive data./ > >fixed > >> 6. p. 6, end of next paragraph: >> s/might be the only option to deal/might be the only option in dealing/ > >fixed > >> 7. p. 9, Section 3.2. In this case there are enough corrections in >> close proximity that I've grouped them together. I suggest that the >> following text: >> >> "SNMP is used to periodically poll devices as well as to retrieve >> information on request of an operator or application. The periodic >> polling leads to periodic traffic patterns while the on demand >> information retrieval causes more aperiodic traffic patterns. It is >> worthwhile to understand what the relationship is between the amount >> of periodic and aperiodic traffic. It will be interesting to >> research whether there are multiple levels of periodicity at >> different time scales. >> >> The periodic polling behavior may dependent on the application and >> the polling engine it uses. For example, some management platforms >> allow applications to specify how long polled values may be kept in a >> cache before it is polled again." >> >> have the following grammatical changes made: >> >> "SNMP is used to periodically poll devices as well as to retrieve >> information at the request of an operator or application. The >> periodic polling leads to periodic traffic patterns while on-demand >> information retrieval causes more aperiodic traffic patterns. It is >> worthwhile to understand what the relationship is between the amount >> of periodic and aperiodic traffic. It will be interesting to >> understand whether there are multiple levels of periodicity at >> different time scales. >> >> Periodic polling behavior may be dependent on the application and > > polling engine it uses. For example, some management platforms allow >> applications to specify how long polled values may be kept in a cache >> before they are polled again." > >fixed > >> 8. p. 10, Section 3.6, middle first paragraph: >> s/ adopt/adapt/ > >fixed > >/js > >-- >Juergen Schoenwaelder Jacobs University Bremen gGmbH >Phone: +49 421 200 3587 Campus Ring 1, 28759 Bremen, Germany >Fax: +49 421 200 3103 -- Karen R. Sollins, Ph. D. Principal Research Scientist M.I.T. CSAIL The Stata Center 32 Vassar St., 32-G818 Cambridge, MA 02139 V: 617/253-6006 F: 617/253-2673 E: sollins@csail.mit.edu Received: from smtp-bedford.mitre.org (smtp-bedford.mitre.org [129.83.20.191]) by bierator.ibr.cs.tu-bs.de (8.13.4/8.13.4/Debian-3sarge3) with ESMTP id m4GES6S7019142 for ; Fri, 16 May 2008 16:28:11 +0200 Received: from smtp-bedford.mitre.org (localhost.localdomain [127.0.0.1]) by smtp-bedford.mitre.org (8.13.1/8.13.1) with ESMTP id m4GES5u8022954 for ; Fri, 16 May 2008 10:28:05 -0400 Received: from IMCFE1.MITRE.ORG (imcfe1.mitre.org [129.83.29.3]) by smtp-bedford.mitre.org (8.13.1/8.13.1) with ESMTP id m4GES5mS022950; Fri, 16 May 2008 10:28:05 -0400 Received: from IMCSRV2.MITRE.ORG ([129.83.20.164]) by IMCFE1.MITRE.ORG with Microsoft SMTPSVC(6.0.3790.1830); Fri, 16 May 2008 10:28:05 -0400 x-mimeole: Produced By Microsoft Exchange V6.5 Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Subject: RE: [OPSAWG] [nmrg] draft-irtf-nmrg-snmp-measure-04.txt oidxmlpattern change Date: Fri, 16 May 2008 10:27:05 -0400 Message-ID: <4915F014FDD99049A9C3A8C1B832004F02AEAA5E@IMCSRV2.MITRE.ORG> In-Reply-To: <07ef01c8b75f$0c1b7000$0600a8c0@china.huawei.com> X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: [OPSAWG] [nmrg] draft-irtf-nmrg-snmp-measure-04.txt oidxmlpattern change Thread-Index: Aci3UELJ3N6FvxJ6R86avAHLVyYyMgACifDAAADWtXAAAF4OEA== References: <20080516122704.GB19275@elstar.local><4915F014FDD99049A9C3A8C1B832004F02AEAA46@IMCSRV2.MITRE.ORG> <07ef01c8b75f$0c1b7000$0600a8c0@china.huawei.com> From: "Natale, Bob" To: "David Harrington" , , "Bert Wijnen" X-OriginalArrivalTime: 16 May 2008 14:28:05.0014 (UTC) FILETIME=[0DBB2B60:01C8B761] X-IBRFilter-SpamReport: 0.001 () BAYES_50 X-Scanned-By: MIMEDefang 2.51 on 134.169.34.9 Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from quoted-printable to 8bit by bierator.ibr.cs.tu-bs.de id m4GES6S7019142 X-Mailman-Approved-At: Sat, 17 May 2008 22:45:49 +0200 Cc: opsawg@ietf.org, nmrg@ibr.cs.tu-bs.de X-BeenThere: nmrg@ibr.cs.tu-bs.de X-Mailman-Version: 2.1.5 Precedence: list List-Id: Network Management Research Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 16 May 2008 14:28:14 -0000 Hi David, You are right about the fidelity basis for SNMP to ASN.1/BER, but I don't think the version diffs come into play on this point. The rule for BER encoding of the first two sub-ids (X and Y) probably always has boiled down to "(X*40)+Y" -- which was expressed as far back as Steedman's book as "40m + n", where (p. 93) he says: "This saves an octet from every value, exploiting the fact that 0 <= m <= 2, and that, except possibly for m = 2 (joint-iso-ccitt), n is a very small number." Ahhh, the good old days! :) There is a registered value for 2.41 (ITU-T Rec. X.1083 | ISO/IEC 24708: BioAPI Interworking Protocol (BIP)), approved in Sep-2007, which requires Juergen's correction...and one could imagine that it could need to be carried in an SNMP varbind. Cheers, BobN -----Original Message----- From: opsawg-bounces@ietf.org [mailto:opsawg-bounces@ietf.org] On Behalf Of David Harrington Sent: Friday, May 16, 2008 10:14 AM To: Natale, Bob; j.schoenwaelder@jacobs-university.de; 'Bert Wijnen' Cc: opsawg@ietf.org; nmrg@ibr.cs.tu-bs.de Subject: Re: [OPSAWG] [nmrg] draft-irtf-nmrg-snmp-measure-04.txt oidxmlpattern change Hi, SMIv2 is based on ASN.1-1988 not ASN.1-2002. XSDMI should maintain fidelity to SMIv2. I would think the snmp-measure document should do so also. We should be liberal in what we accept, and conservative in what we send, of course. I think XSDMI should enforce the 1988 syntax, since one expected use case is validation; it probably will make no difference to the snmp-research document, whose use case is reporting. In practice, I have never seen a 2.X in any SMIv2 MIB module. dbh > -----Original Message----- > From: opsawg-bounces@ietf.org > [mailto:opsawg-bounces@ietf.org] On Behalf Of Natale, Bob > Sent: Friday, May 16, 2008 9:48 AM > To: j.schoenwaelder@jacobs-university.de; Bert Wijnen > Cc: opsawg@ietf.org; nmrg@ibr.cs.tu-bs.de > Subject: Re: [OPSAWG] [nmrg] > draft-irtf-nmrg-snmp-measure-04.txt oid xmlpattern change > > Hi, > > Juergen is correct (per Sec. 8.19 of ITU-T Recommendation X.690, > prepared by ITU-T Study Group 17 and approved on 14 July 2002, > identical text is also published as ISO/IEC 8825-1). > > draft-ietf-opsawg-smi-datatypes-in-xsd-02.txt (to be posted any day > now!) will contain the fix Juergen provides below. > > Cheers, > BobN > > -----Original Message----- > From: nmrg-bounces@ibr.cs.tu-bs.de > [mailto:nmrg-bounces@ibr.cs.tu-bs.de] On Behalf Of Juergen > Schoenwaelder > Sent: Friday, May 16, 2008 8:27 AM > To: nmrg@ibr.cs.tu-bs.de; Bert Wijnen > Subject: [nmrg] draft-irtf-nmrg-snmp-measure-04.txt oid xml pattern > change > > Hi, > > there was recently a discussion on the XSD pattern for object > identifier values and it was pointed out that the pattern we picked up > from the SMI-XSD document is too restrictive by not allowing OIDs of > the format 2.X.* where X is larger than 39. I therefore propose that > we change the pattern used by oid.type from > > "[0-2](\.[1-3]?[0-9])(\.(0|([1-9]\d*))){0,126}" > > to > > "([0-1](\.[1-3]?[0-9]))|(2.(0|([1-9]\d*)))" ~ > "(\.(0|([1-9]\d*))){0,126}" > > Any objections against this change which is really a bugfix? Bert, do > we run into procedural trouble if we fix this or an additional N weeks > delay? (In that case, I would be tempted to go along with the buggy > version of the pattern. ;-) > > /js > > -- > Juergen Schoenwaelder Jacobs University Bremen gGmbH > Phone: +49 421 200 3587 Campus Ring 1, 28759 Bremen, Germany > Fax: +49 421 200 3103 > -- > !! This message is brought to you via the `nmrg' mailing list. > !! Please do not reply to this message to unsubscribe. To unsubscribe > or adjust > !! your settings, send a mail message to > > !! or look at https://mail.ibr.cs.tu-bs.de/mailman/listinfo/nmrg. > _______________________________________________ > OPSAWG mailing list > OPSAWG@ietf.org > https://www.ietf.org/mailman/listinfo/opsawg > _______________________________________________ OPSAWG mailing list OPSAWG@ietf.org https://www.ietf.org/mailman/listinfo/opsawg Received: from QMTA02.emeryville.ca.mail.comcast.net (qmta02.emeryville.ca.mail.comcast.net [76.96.30.24]) by bierator.ibr.cs.tu-bs.de (8.13.4/8.13.4/Debian-3sarge3) with ESMTP id m4GEDnCB014887 for ; Fri, 16 May 2008 16:13:55 +0200 Received: from OMTA06.emeryville.ca.mail.comcast.net ([76.96.30.51]) by QMTA02.emeryville.ca.mail.comcast.net with comcast id SAv71Z00216AWCUA20G700; Fri, 16 May 2008 14:13:43 +0000 Received: from Harrington73653 ([24.128.66.199]) by OMTA06.emeryville.ca.mail.comcast.net with comcast id SEDg1Z0044HwxpC8S00000; Fri, 16 May 2008 14:13:43 +0000 X-Authority-Analysis: v=1.0 c=1 a=ISx8kgmK6NcA:10 a=X5CS3wKyBVwA:10 a=j3Z76cjpAAAA:8 a=Jiz0tsSxAAAA:8 a=48vgC7mUAAAA:8 a=cQZuYZh4ME2UarBVmeoA:9 a=blMDACQhUAAelina1AQA:7 a=32j3w6sY7jJCCCOEoydU2Ar3j-0A:4 a=FvgKqOQ44qUA:10 a=JrSEOxZJtCQA:10 a=lZB815dzVvQA:10 a=50e4U0PicR4A:10 From: "David Harrington" To: "'Natale, Bob'" , , "'Bert Wijnen'" References: <20080516122704.GB19275@elstar.local> <4915F014FDD99049A9C3A8C1B832004F02AEAA46@IMCSRV2.MITRE.ORG> Subject: RE: [OPSAWG] [nmrg] draft-irtf-nmrg-snmp-measure-04.txt oid xmlpattern change Date: Fri, 16 May 2008 10:13:40 -0400 Message-ID: <07ef01c8b75f$0c1b7000$0600a8c0@china.huawei.com> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Mailer: Microsoft Office Outlook 11 In-Reply-To: <4915F014FDD99049A9C3A8C1B832004F02AEAA46@IMCSRV2.MITRE.ORG> Thread-Index: Aci3UELJ3N6FvxJ6R86avAHLVyYyMgACifDAAADWtXA= X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.3198 X-IBRFilter-SpamReport: 3.602 (***) BAYES_50, DNS_FROM_RFC_POST, RCVD_IN_SORBS_DUL X-Scanned-By: MIMEDefang 2.51 on 134.169.34.9 Cc: opsawg@ietf.org, nmrg@ibr.cs.tu-bs.de X-BeenThere: nmrg@ibr.cs.tu-bs.de X-Mailman-Version: 2.1.5 Precedence: list List-Id: Network Management Research Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 16 May 2008 14:14:01 -0000 Hi, SMIv2 is based on ASN.1-1988 not ASN.1-2002. XSDMI should maintain fidelity to SMIv2. I would think the snmp-measure document should do so also. We should be liberal in what we accept, and conservative in what we send, of course. I think XSDMI should enforce the 1988 syntax, since one expected use case is validation; it probably will make no difference to the snmp-research document, whose use case is reporting. In practice, I have never seen a 2.X in any SMIv2 MIB module. dbh > -----Original Message----- > From: opsawg-bounces@ietf.org > [mailto:opsawg-bounces@ietf.org] On Behalf Of Natale, Bob > Sent: Friday, May 16, 2008 9:48 AM > To: j.schoenwaelder@jacobs-university.de; Bert Wijnen > Cc: opsawg@ietf.org; nmrg@ibr.cs.tu-bs.de > Subject: Re: [OPSAWG] [nmrg] > draft-irtf-nmrg-snmp-measure-04.txt oid xmlpattern change > > Hi, > > Juergen is correct (per Sec. 8.19 of ITU-T Recommendation X.690, > prepared by ITU-T Study Group 17 and approved on 14 July 2002, > identical text is also published as ISO/IEC 8825-1). > > draft-ietf-opsawg-smi-datatypes-in-xsd-02.txt (to be posted any day > now!) will contain the fix Juergen provides below. > > Cheers, > BobN > > -----Original Message----- > From: nmrg-bounces@ibr.cs.tu-bs.de > [mailto:nmrg-bounces@ibr.cs.tu-bs.de] On Behalf Of Juergen > Schoenwaelder > Sent: Friday, May 16, 2008 8:27 AM > To: nmrg@ibr.cs.tu-bs.de; Bert Wijnen > Subject: [nmrg] draft-irtf-nmrg-snmp-measure-04.txt oid xml pattern > change > > Hi, > > there was recently a discussion on the XSD pattern for object > identifier values and it was pointed out that the pattern we picked up > from the SMI-XSD document is too restrictive by not allowing OIDs of > the format 2.X.* where X is larger than 39. I therefore propose that > we change the pattern used by oid.type from > > "[0-2](\.[1-3]?[0-9])(\.(0|([1-9]\d*))){0,126}" > > to > > "([0-1](\.[1-3]?[0-9]))|(2.(0|([1-9]\d*)))" ~ > "(\.(0|([1-9]\d*))){0,126}" > > Any objections against this change which is really a bugfix? Bert, do > we run into procedural trouble if we fix this or an additional N weeks > delay? (In that case, I would be tempted to go along with the buggy > version of the pattern. ;-) > > /js > > -- > Juergen Schoenwaelder Jacobs University Bremen gGmbH > Phone: +49 421 200 3587 Campus Ring 1, 28759 Bremen, Germany > Fax: +49 421 200 3103 > -- > !! This message is brought to you via the `nmrg' mailing list. > !! Please do not reply to this message to unsubscribe. To unsubscribe > or adjust > !! your settings, send a mail message to > > !! or look at https://mail.ibr.cs.tu-bs.de/mailman/listinfo/nmrg. > _______________________________________________ > OPSAWG mailing list > OPSAWG@ietf.org > https://www.ietf.org/mailman/listinfo/opsawg > Received: from smtp-bedford.mitre.org (smtp-bedford.mitre.org [129.83.20.191]) by bierator.ibr.cs.tu-bs.de (8.13.4/8.13.4/Debian-3sarge3) with ESMTP id m4GDmZGI007770 for ; Fri, 16 May 2008 15:48:40 +0200 Received: from smtp-bedford.mitre.org (localhost.localdomain [127.0.0.1]) by smtp-bedford.mitre.org (8.13.1/8.13.1) with ESMTP id m4GDmUaT008841 for ; Fri, 16 May 2008 09:48:32 -0400 Received: from IMCFE1.MITRE.ORG (imcfe1.mitre.org [129.83.29.3]) by smtp-bedford.mitre.org (8.13.1/8.13.1) with ESMTP id m4GDmUuc008834; Fri, 16 May 2008 09:48:30 -0400 Received: from IMCSRV2.MITRE.ORG ([129.83.20.164]) by IMCFE1.MITRE.ORG with Microsoft SMTPSVC(6.0.3790.1830); Fri, 16 May 2008 09:48:29 -0400 x-mimeole: Produced By Microsoft Exchange V6.5 Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Subject: RE: [nmrg] draft-irtf-nmrg-snmp-measure-04.txt oid xml pattern change Date: Fri, 16 May 2008 09:48:16 -0400 Message-ID: <4915F014FDD99049A9C3A8C1B832004F02AEAA46@IMCSRV2.MITRE.ORG> In-Reply-To: <20080516122704.GB19275@elstar.local> X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: [nmrg] draft-irtf-nmrg-snmp-measure-04.txt oid xml pattern change Thread-Index: Aci3UELJ3N6FvxJ6R86avAHLVyYyMgACifDA References: <20080516122704.GB19275@elstar.local> From: "Natale, Bob" To: , "Bert Wijnen" X-OriginalArrivalTime: 16 May 2008 13:48:29.0935 (UTC) FILETIME=[8612DFF0:01C8B75B] X-IBRFilter-SpamReport: -1.951 () BAYES_20 X-Scanned-By: MIMEDefang 2.51 on 134.169.34.9 Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from quoted-printable to 8bit by bierator.ibr.cs.tu-bs.de id m4GDmZGI007770 X-Mailman-Approved-At: Sat, 17 May 2008 22:45:38 +0200 Cc: opsawg@ietf.org, nmrg@ibr.cs.tu-bs.de X-BeenThere: nmrg@ibr.cs.tu-bs.de X-Mailman-Version: 2.1.5 Precedence: list List-Id: Network Management Research Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 16 May 2008 13:48:45 -0000 Hi, Juergen is correct (per Sec. 8.19 of ITU-T Recommendation X.690, prepared by ITU-T Study Group 17 and approved on 14 July 2002, identical text is also published as ISO/IEC 8825-1). draft-ietf-opsawg-smi-datatypes-in-xsd-02.txt (to be posted any day now!) will contain the fix Juergen provides below. Cheers, BobN -----Original Message----- From: nmrg-bounces@ibr.cs.tu-bs.de [mailto:nmrg-bounces@ibr.cs.tu-bs.de] On Behalf Of Juergen Schoenwaelder Sent: Friday, May 16, 2008 8:27 AM To: nmrg@ibr.cs.tu-bs.de; Bert Wijnen Subject: [nmrg] draft-irtf-nmrg-snmp-measure-04.txt oid xml pattern change Hi, there was recently a discussion on the XSD pattern for object identifier values and it was pointed out that the pattern we picked up from the SMI-XSD document is too restrictive by not allowing OIDs of the format 2.X.* where X is larger than 39. I therefore propose that we change the pattern used by oid.type from "[0-2](\.[1-3]?[0-9])(\.(0|([1-9]\d*))){0,126}" to "([0-1](\.[1-3]?[0-9]))|(2.(0|([1-9]\d*)))" ~ "(\.(0|([1-9]\d*))){0,126}" Any objections against this change which is really a bugfix? Bert, do we run into procedural trouble if we fix this or an additional N weeks delay? (In that case, I would be tempted to go along with the buggy version of the pattern. ;-) /js -- Juergen Schoenwaelder Jacobs University Bremen gGmbH Phone: +49 421 200 3587 Campus Ring 1, 28759 Bremen, Germany Fax: +49 421 200 3103 -- !! This message is brought to you via the `nmrg' mailing list. !! Please do not reply to this message to unsubscribe. To unsubscribe or adjust !! your settings, send a mail message to !! or look at https://mail.ibr.cs.tu-bs.de/mailman/listinfo/nmrg. Received: from hermes.jacobs-university.de (hermes.jacobs-university.de [212.201.44.23]) by bierator.ibr.cs.tu-bs.de (8.13.4/8.13.4/Debian-3sarge3) with ESMTP id m4GCRCDu015564 for ; Fri, 16 May 2008 14:27:17 +0200 Received: from localhost (demetrius4.jacobs-university.de [212.201.44.49]) by hermes.jacobs-university.de (Postfix) with ESMTP id 5B2D0C0027; Fri, 16 May 2008 14:27:12 +0200 (CEST) X-Virus-Scanned: amavisd-new at jacobs-university.de Received: from hermes.jacobs-university.de ([212.201.44.23]) by localhost (demetrius4.jacobs-university.de [212.201.44.32]) (amavisd-new, port 10024) with ESMTP id 2iSOE-1Ye0PT; Fri, 16 May 2008 14:27:06 +0200 (CEST) Received: from elstar.local (elstar.iuhb02.iu-bremen.de [10.50.231.133]) by hermes.jacobs-university.de (Postfix) with ESMTP id 1EA4DC0006; Fri, 16 May 2008 14:27:05 +0200 (CEST) Received: by elstar.local (Postfix, from userid 501) id DCF1258040A; Fri, 16 May 2008 14:27:04 +0200 (CEST) Date: Fri, 16 May 2008 14:27:04 +0200 From: Juergen Schoenwaelder To: nmrg@ibr.cs.tu-bs.de, Bert Wijnen Message-ID: <20080516122704.GB19275@elstar.local> Mail-Followup-To: nmrg@ibr.cs.tu-bs.de, Bert Wijnen MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.5.17 (2007-11-01) X-IBRFilter-SpamReport: -2.599 () BAYES_00 X-Scanned-By: MIMEDefang 2.51 on 134.169.34.9 Subject: [nmrg] draft-irtf-nmrg-snmp-measure-04.txt oid xml pattern change X-BeenThere: nmrg@ibr.cs.tu-bs.de X-Mailman-Version: 2.1.5 Precedence: list Reply-To: j.schoenwaelder@jacobs-university.de List-Id: Network Management Research Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 16 May 2008 12:27:20 -0000 Hi, there was recently a discussion on the XSD pattern for object identifier values and it was pointed out that the pattern we picked up from the SMI-XSD document is too restrictive by not allowing OIDs of the format 2.X.* where X is larger than 39. I therefore propose that we change the pattern used by oid.type from "[0-2](\.[1-3]?[0-9])(\.(0|([1-9]\d*))){0,126}" to "([0-1](\.[1-3]?[0-9]))|(2.(0|([1-9]\d*)))" ~ "(\.(0|([1-9]\d*))){0,126}" Any objections against this change which is really a bugfix? Bert, do we run into procedural trouble if we fix this or an additional N weeks delay? (In that case, I would be tempted to go along with the buggy version of the pattern. ;-) /js -- Juergen Schoenwaelder Jacobs University Bremen gGmbH Phone: +49 421 200 3587 Campus Ring 1, 28759 Bremen, Germany Fax: +49 421 200 3103 Received: from hermes.jacobs-university.de (hermes.jacobs-university.de [212.201.44.23]) by bierator.ibr.cs.tu-bs.de (8.13.4/8.13.4/Debian-3sarge3) with ESMTP id m4GCKpbM013735 for ; Fri, 16 May 2008 14:20:56 +0200 Received: from localhost (demetrius4.jacobs-university.de [212.201.44.49]) by hermes.jacobs-university.de (Postfix) with ESMTP id 5C120C0006; Fri, 16 May 2008 14:20:51 +0200 (CEST) X-Virus-Scanned: amavisd-new at jacobs-university.de Received: from hermes.jacobs-university.de ([212.201.44.23]) by localhost (demetrius4.jacobs-university.de [212.201.44.32]) (amavisd-new, port 10024) with ESMTP id 9NKZKyNDEtpJ; Fri, 16 May 2008 14:20:43 +0200 (CEST) Received: from elstar.local (elstar.iuhb02.iu-bremen.de [10.50.231.133]) by hermes.jacobs-university.de (Postfix) with ESMTP id 8D62DC002D; Fri, 16 May 2008 14:20:43 +0200 (CEST) Received: by elstar.local (Postfix, from userid 501) id 0DCA75803D8; Fri, 16 May 2008 14:20:43 +0200 (CEST) Date: Fri, 16 May 2008 14:20:42 +0200 From: Juergen Schoenwaelder To: "Karen R. Sollins" Message-ID: <20080516122042.GA19275@elstar.local> Mail-Followup-To: "Karen R. Sollins" , nmrg@ibr.cs.tu-bs.de, Bert Wijnen - IETF , Internet Research Steering Group References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.17 (2007-11-01) X-IBRFilter-SpamReport: 0.001 () BAYES_50 X-Scanned-By: MIMEDefang 2.51 on 134.169.34.9 Cc: Internet Research Steering Group , nmrg@ibr.cs.tu-bs.de Subject: [nmrg] Re: [IRSG] review of draft-irtf-nmrg-snmp-measure-04.txt X-BeenThere: nmrg@ibr.cs.tu-bs.de X-Mailman-Version: 2.1.5 Precedence: list Reply-To: j.schoenwaelder@jacobs-university.de List-Id: Network Management Research Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 16 May 2008 12:21:02 -0000 On Mon, May 12, 2008 at 02:52:02PM -0400, Karen R. Sollins wrote: > Here is my full review of draft-irtf-nmrg-snmp-measure-04.txt. Thank you very much for your substantial review. Below is how I plan to address your comments. For most of your comments, the changes are easy to do. For some of your comments, I suggest to make no changes and I try to explain why. Please check and let me and the RG know where you do not agree with my suggested plan of action. > I do not think this document is quite ready for publication, but it > is quite close and I don't think this should be major work. > > My review falls into two parts. The first is broad comments and the > second is nits. Overall, I agree with the working group that it is > valuable to spell out a set of details such as this, more or less a > "protocol" in the sense of biologically based research, describing > what should be done, what kind of information should be collected and > why and how to handle it. I have not reviewed the XML or CSV > representations nor the reference lists in detail. With that said, I > have two high level concerns with this document. > > The first is that there is confusion about whether this document > describes a proposed measurement effort or (closer to the reality of > what is in the document) a methodology by which such measurements > could and perhaps should be taken. So, given the bulk of the > document, I do not believe that it is at all explicit about the > particular measurement exercise that will be done. With that said, > there are a few introductory points that are worth changing. > > 1. The abstract includes the sentence, "This document proposes to > carry out large scale SNMP traffic measurements." First, the > document won't be carrying out anything. More importantly the > document describes an approach (not even a pan), so I suggest that > the sentence read, "This document describes an approach to carrying > out..." changed as suggested > 2. In Section 1, the 4th paragraph reads, "This document describes an > effort to collect SNMP traffic traces..." Instead, I suggest that it > read, "This document recommends an approach to collecting, codifying, > and handling SNMP traffic traces..." changed as suggested > The second high level concern I have is that there is talk about > specific kinds of information to be collected and an interest in not > only the nature but longer term inferences with perhaps implications > for future redesign efforts in the SNMP context. I have two levels > of concern here. The most important one is that since network > management and in particular SNMP is NOT the primary objective of the > net (the primary objective being the transport of real payload), it > seems to me that the truly critical question with respect to network > management traffic is the impact that it has or does not have on that > real job. To me this implies that the measurements MUST also include > contextual information. As an example, it is probably more important > to understand whether or not the network management traffic is > causing significant congestion for the payload traffic than the > particular mix or frequency pattern within the network management > traffic. Without out the complementary contextual information, the > whole measurement exercise seems to me to be of somewhat narrow > value. The measurement may be of narrow value from your point of view but please keep in mind that this document is coming from the Network Management Research Group and not from a general Network Measurement Research Group. Our goal is to understand how network management protocols are being used because that has impact on their design and implementation strategies. Further note that in many networks, the management traffic is logically and sometimes even physically separated from the normal traffic and perhaps this is the reason why we did not even think about the question whether management traffic has an impact on normal traffic. > Secondarily, the document hits on one of my little pet peeves > - location. There is mention that the meta-data must include "where > the trace was collected". That, in and of itself, is a challenging > problem. Is this geographic, topological, or based on other metrics? > If it is based on something that can change with time (e.g. ip > addresses or topology), then without keeping track of how those > metrics may have changed over time, the data set becomes useless > because the location becomes uninterpretable. As you all hint, it is > important that the collected information be archived, in order to be > useful at significantly later times, and thus, everything importatn > about the situation must be recorded. I would like to see more > specificity with respect to location and clearer acknowledgement of > its need in order to keep the collected information useful outside > the current context. Makes sense. > More generally, there are many cases in which there are statements > about what kind of information should be collected (e.g.one week > traces, concurrent vs. sequential requests, etc.) where there is no > explanation of the criticality of this information or how to go > beyond the immediately observable facts. > > I realize that it is generally not in the nature of RFC to provide > explanation as much as simple unadorned facts, but this is an unusual > kind of document, because it is describing a potential information > gathering exercise. It has some of the explanations of why things > are important, but it is quite incomplete. I realize that it is > important to collect and archive measurement sets exactly because one > cannot know ahead of time all the potential uses, but there should > either be a story about collecting everything possible or an initial > use for each kind of information collected. In some cases neither > argument is presented. > > This all leads to a number of specific questions: > > 1. Section 1: It seems to me that there are TWO key questions with > respect to SNMP. The first is how it is being used, which in turn > leads to the points made in this section, but the second is the > impact of that traffic. I think that ought to appear in the > Introduction as well. See my comments above. So far, the NMRG did not consider the impact of SNMP on other traffic a target of this activity. I don't want to add such text unless I see support from the NMRG and concrete proposals what should be added. > 2. Section 2.1. The second paragraph begins with, "It is recommended > to capture at least a full week of data." This is never justified or > explained. Is one week really enough? For what? Why wouldn't > several weeks be critical, because one week might be anomalous? Why > isn't a year critical, since we know that there are annual or > seasonal differences in traffic behaviors? Typically, I find that > one-week data sets often leave me with lots of unanswered questions, > so justify this. The text actually says: It is recommended to capture at least a full week of data. Operators are encouraged to capture traces over even longer periods of time. The text tries to establish a lower bound of one week an encourages longer capture periods. I would love to get continuous traces but reality is such that this is not feasible. Our idea is simply to catch at least the weekly behaviour. Yes, there is of course also monthly or yearly behaviour but I believe it is not useful to set the bar so high that nobody gives us appropriate traces. I personally believe the text is fine as is. > 3. Next paragraph: this is where the location question arises. > Without some completely standardized and self explanatory capturing > of location information, any data set will be incomparable to any > other. I expanded "where the trace was collected" to "where the trace was collected (name of the network and/or name of the organization owning the network, description of the measurement point in the network topology where the trace was collected)". > 4. Section 3.3, end of first paragraph: The sentence reads, "Some > SNMP implementations approximate networking delays by measuring > request-response times and it would be useful to understand to what > extent this is a viable approach." I agree, but traces will not tell > you anything about whether behaviors observed in packet traces are > for this reason or some other reason. I do not believe you can get > at this question with the data you are collecting. I think it is possible to analyze retransmission behaviour. Depending on the SNMP version used (and the other versions also depending on implementation choices), you can get information whether a response is just coming late for the original request or it is actually a response to a retranmitted request. We are not talking TCP here; we are talking about application layer retransmissions and SNMP has its own msgID and requestID fields. > 5. Section 3.4: Please explain why it is "interesting" (your word) to > identify whether concurrency or sequentiality is occurring? What > will you "learn" if both are observed? And, if one is occurring more > frequently or under specific identifiable conditions, what further > does that tell you? Just knowing that one or the other occurs is > only the tip of the iceberg, and without acknowledging the fact that > these are important and unanswered questions, just learning first > ordered details suggests you are setting the bar too low. The introduction of section 3 says: The questions raised in the following subsections are meant to be illustrative and no attempt has been made to provide a complete list. I believe it is a good idea to first figure out whether there is an iceberg or not (keeping your analogy) and if there is one to ask questions how big the iceberg might be. For SNMP agent implementations that tend to do quite some caching, it is useful to know how well caching strategies are working in real-world networks. The concurrency level an agent experiences has clear impact on that. Furthermore, it will be useful to know how bursty the traffic tends to be or how well managers spread the traffic over polling intervals and this is again related to the concurrency we can extract from traces. I am not really sure what I should change, perhaps the word "interesting" is the source of the trouble and I should replace this with "valuable"? > 6. Section 3.5: Please explain what you would do with the > information about which approach to table retrieval is used. Again, > what if the results tell you that both are used? And, if not, of > what use is it to know which approach is prevalent? Mighten it be > useful to know the conditions under which one or the other is used > more commonly? This again has direct impact on agent implementation techniques and caching strategies. > Now on to the real NITS: > 1. p. 1, Section 1, middle of second paragraph: > s/In fact, there are many speculations how SNMP/In fact, there are > many speculations on how SNMP/ fixed > 2. p. 6, Section 2.2, 5th paragraph: > s/for SNMP messages that got fragmented/for SNMP messages that were fragmented/ fixed > 3. p. 7, Section 2.4, last paragraph: > s/Improvements in the tool chain may require to go back to the > original pcap traces and to rebuild all intermediate formats from > them./Improvements in the tool chain may require going back to the > original pcap traces and rebuilding all intermediate formats from > them./ fixed > 4. p. 7, Section 2.5, first paragraph: > s/all scripts used to analyze traffic traces would be/all scripts > used to analyze traffic traces will be/ fixed > 5. p. 8, first sentence: > s/A common versioned repository for analysis of scripts might be > useful to establish./It might be useful to establish and common, > versioning repository for analysis scripts./ fixed > 6. p. 6, second paragraph (includes several corrections): > s/it is suggested that analysis scripts are written in scripting > languages such as Perl using suitable Perl modules to manipulate XML > documents [8]. Using a scripting language such as Perl instead of > system programming languages such as C or C++ has the advantage to > reduce development time and to make scripts more accessible to > operators who may want to verify scripts before running them on trace > files which potentially contain sensitive data./it is suggested that > analysis scripts be written is scripting languages such as Perl using > suitable Perl modules to manipulate XML documents [8]. Using a > scripting language such as Perl instead of system programming > languages such as C or C++ has the advantage of reducing development > time and making scripts more accessible to operators who may want to > verify scripts before running them on trace files which may contain > sensitive data./ fixed > 6. p. 6, end of next paragraph: > s/might be the only option to deal/might be the only option in dealing/ fixed > 7. p. 9, Section 3.2. In this case there are enough corrections in > close proximity that I've grouped them together. I suggest that the > following text: > > "SNMP is used to periodically poll devices as well as to retrieve > information on request of an operator or application. The periodic > polling leads to periodic traffic patterns while the on demand > information retrieval causes more aperiodic traffic patterns. It is > worthwhile to understand what the relationship is between the amount > of periodic and aperiodic traffic. It will be interesting to > research whether there are multiple levels of periodicity at > different time scales. > > The periodic polling behavior may dependent on the application and > the polling engine it uses. For example, some management platforms > allow applications to specify how long polled values may be kept in a > cache before it is polled again." > > have the following grammatical changes made: > > "SNMP is used to periodically poll devices as well as to retrieve > information at the request of an operator or application. The > periodic polling leads to periodic traffic patterns while on-demand > information retrieval causes more aperiodic traffic patterns. It is > worthwhile to understand what the relationship is between the amount > of periodic and aperiodic traffic. It will be interesting to > understand whether there are multiple levels of periodicity at > different time scales. > > Periodic polling behavior may be dependent on the application and > polling engine it uses. For example, some management platforms allow > applications to specify how long polled values may be kept in a cache > before they are polled again." fixed > 8. p. 10, Section 3.6, middle first paragraph: > s/ adopt/adapt/ fixed /js -- Juergen Schoenwaelder Jacobs University Bremen gGmbH Phone: +49 421 200 3587 Campus Ring 1, 28759 Bremen, Germany Fax: +49 421 200 3103 Received: from mercury.lcs.mit.edu (mercury.lcs.mit.edu [18.26.0.122]) by bierator.ibr.cs.tu-bs.de (8.13.4/8.13.4/Debian-3sarge3) with ESMTP id m4CIqUYS010294 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT) for ; Mon, 12 May 2008 20:52:36 +0200 Received: from [18.26.0.27] (tethys.lcs.mit.edu [18.26.0.27]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mercury.lcs.mit.edu (Postfix) with ESMTP id A7DB96BE59B; Mon, 12 May 2008 14:52:29 -0400 (EDT) Mime-Version: 1.0 Message-Id: Date: Mon, 12 May 2008 14:52:02 -0400 To: nmrg@ibr.cs.tu-bs.de, "Bert Wijnen - IETF" , "Internet Research Steering Group" From: "Karen R. Sollins" Content-Type: text/plain; charset="us-ascii" ; format="flowed" X-IBRFilter-SpamReport: 0.001 () BAYES_50 X-Scanned-By: MIMEDefang 2.51 on 134.169.34.9 X-Mailman-Approved-At: Wed, 14 May 2008 00:39:03 +0200 Subject: [nmrg] review of draft-irtf-nmrg-snmp-measure-04.txt X-BeenThere: nmrg@ibr.cs.tu-bs.de X-Mailman-Version: 2.1.5 Precedence: list List-Id: Network Management Research Group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 May 2008 18:52:37 -0000 Here is my full review of draft-irtf-nmrg-snmp-measure-04.txt. I do not think this document is quite ready for publication, but it is quite close and I don't think this should be major work. My review falls into two parts. The first is broad comments and the second is nits. Overall, I agree with the working group that it is valuable to spell out a set of details such as this, more or less a "protocol" in the sense of biologically based research, describing what should be done, what kind of information should be collected and why and how to handle it. I have not reviewed the XML or CSV representations nor the reference lists in detail. With that said, I have two high level concerns with this document. The first is that there is confusion about whether this document describes a proposed measurement effort or (closer to the reality of what is in the document) a methodology by which such measurements could and perhaps should be taken. So, given the bulk of the document, I do not believe that it is at all explicit about the particular measurement exercise that will be done. With that said, there are a few introductory points that are worth changing. 1. The abstract includes the sentence, "This document proposes to carry out large scale SNMP traffic measurements." First, the document won't be carrying out anything. More importantly the document describes an approach (not even a pan), so I suggest that the sentence read, "This document describes an approach to carrying out..." 2. In Section 1, the 4th paragraph reads, "This document describes an effort to collect SNMP traffic traces..." Instead, I suggest that it read, "This document recommends an approach to collecting, codifying, and handling SNMP traffic traces..." The second high level concern I have is that there is talk about specific kinds of information to be collected and an interest in not only the nature but longer term inferences with perhaps implications for future redesign efforts in the SNMP context. I have two levels of concern here. The most important one is that since network management and in particular SNMP is NOT the primary objective of the net (the primary objective being the transport of real payload), it seems to me that the truly critical question with respect to network management traffic is the impact that it has or does not have on that real job. To me this implies that the measurements MUST also include contextual information. As an example, it is probably more important to understand whether or not the network management traffic is causing significant congestion for the payload traffic than the particular mix or frequency pattern within the network management traffic. Without out the complementary contextual information, the whole measurement exercise seems to me to be of somewhat narrow value. Secondarily, the document hits on one of my little pet peeves - location. There is mention that the meta-data must include "where the trace was collected". That, in and of itself, is a challenging problem. Is this geographic, topological, or based on other metrics? If it is based on something that can change with time (e.g. ip addresses or topology), then without keeping track of how those metrics may have changed over time, the data set becomes useless because the location becomes uninterpretable. As you all hint, it is important that the collected information be archived, in order to be useful at significantly later times, and thus, everything importatn about the situation must be recorded. I would like to see more specificity with respect to location and clearer acknowledgement of its need in order to keep the collected information useful outside the current context. More generally, there are many cases in which there are statements about what kind of information should be collected (e.g.one week traces, concurrent vs. sequential requests, etc.) where there is no explanation of the criticality of this information or how to go beyond the immediately observable facts. I realize that it is generally not in the nature of RFC to provide explanation as much as simple unadorned facts, but this is an unusual kind of document, because it is describing a potential information gathering exercise. It has some of the explanations of why things are important, but it is quite incomplete. I realize that it is important to collect and archive measurement sets exactly because one cannot know ahead of time all the potential uses, but there should either be a story about collecting everything possible or an initial use for each kind of information collected. In some cases neither argument is presented. This all leads to a number of specific questions: 1. Section 1: It seems to me that there are TWO key questions with respect to SNMP. The first is how it is being used, which in turn leads to the points made in this section, but the second is the impact of that traffic. I think that ought to appear in the Introduction as well. 2. Section 2.1. The second paragraph begins with, "It is recommended to capture at least a full week of data." This is never justified or explained. Is one week really enough? For what? Why wouldn't several weeks be critical, because one week might be anomalous? Why isn't a year critical, since we know that there are annual or seasonal differences in traffic behaviors? Typically, I find that one-week data sets often leave me with lots of unanswered questions, so justify this. 3. Next paragraph: this is where the location question arises. Without some completely standardized and self explanatory capturing of location information, any data set will be incomparable to any other. 4. Section 3.3, end of first paragraph: The sentence reads, "Some SNMP implementations approximate networking delays by measuring request-response times and it would be useful to understand to what extent this is a viable approach." I agree, but traces will not tell you anything about whether behaviors observed in packet traces are for this reason or some other reason. I do not believe you can get at this question with the data you are collecting. 5. Section 3.4: Please explain why it is "interesting" (your word) to identify whether concurrency or sequentiality is occurring? What will you "learn" if both are observed? And, if one is occurring more frequently or under specific identifiable conditions, what further does that tell you? Just knowing that one or the other occurs is only the tip of the iceberg, and without acknowledging the fact that these are important and unanswered questions, just learning first ordered details suggests you are setting the bar too low. 6. Section 3.5: Please explain what you would do with the information about which approach to table retrieval is used. Again, what if the results tell you that both are used? And, if not, of what use is it to know which approach is prevalent? Mighten it be useful to know the conditions under which one or the other is used more commonly? Now on to the real NITS: 1. p. 1, Section 1, middle of second paragraph: s/In fact, there are many speculations how SNMP/In fact, there are many speculations on how SNMP/ 2. p. 6, Section 2.2, 5th paragraph: s/for SNMP messages that got fragmented/for SNMP messages that were fragmented/ 3. p. 7, Section 2.4, last paragraph: s/Improvements in the tool chain may require to go back to the original pcap traces and to rebuild all intermediate formats from them./Improvements in the tool chain may require going back to the original pcap traces and rebuilding all intermediate formats from them./ 4. p. 7, Section 2.5, first paragraph: s/all scripts used to analyze traffic traces would be/all scripts used to analyze traffic traces will be/ 5. p. 8, first sentence: s/A common versioned repository for analysis of scripts might be useful to establish./It might be useful to establish and common, versioning repository for analysis scripts./ 6. p. 6, second paragraph (includes several corrections): s/it is suggested that analysis scripts are written in scripting languages such as Perl using suitable Perl modules to manipulate XML documents [8]. Using a scripting language such as Perl instead of system programming languages such as C or C++ has the advantage to reduce development time and to make scripts more accessible to operators who may want to verify scripts before running them on trace files which potentially contain sensitive data./it is suggested that analysis scripts be written is scripting languages such as Perl using suitable Perl modules to manipulate XML documents [8]. Using a scripting language such as Perl instead of system programming languages such as C or C++ has the advantage of reducing development time and making scripts more accessible to operators who may want to verify scripts before running them on trace files which may contain sensitive data./ 6. p. 6, end of next paragraph: s/might be the only option to deal/might be the only option in dealing/ 7. p. 9, Section 3.2. In this case there are enough corrections in close proximity that I've grouped them together. I suggest that the following text: "SNMP is used to periodically poll devices as well as to retrieve information on request of an operator or application. The periodic polling leads to periodic traffic patterns while the on demand information retrieval causes more aperiodic traffic patterns. It is worthwhile to understand what the relationship is between the amount of periodic and aperiodic traffic. It will be interesting to research whether there are multiple levels of periodicity at different time scales. The periodic polling behavior may dependent on the application and the polling engine it uses. For example, some management platforms allow applications to specify how long polled values may be kept in a cache before it is polled again." have the following grammatical changes made: "SNMP is used to periodically poll devices as well as to retrieve information at the request of an operator or application. The periodic polling leads to periodic traffic patterns while on-demand information retrieval causes more aperiodic traffic patterns. It is worthwhile to understand what the relationship is between the amount of periodic and aperiodic traffic. It will be interesting to understand whether there are multiple levels of periodicity at different time scales. Periodic polling behavior may be dependent on the application and polling engine it uses. For example, some management platforms allow applications to specify how long polled values may be kept in a cache before they are polled again." 8. p. 10, Section 3.6, middle first paragraph: s/ adopt/adapt/ Karen Sollins -- Karen R. Sollins, Ph.D. Principal Research Scientist MIT CSAIL, The Stata Center 32 Vassar St., 32-G818 Cambridge, MA 02139, USA V: +1 617 253 6006 F: +1 617 253 2673 E: sollins@csail.mit.edu