From owner-psamp@ops.ietf.org Wed Dec 1 12:28:08 2004 Received: from psg.com (mailnull@psg.com [147.28.0.62]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id MAA00135 for ; Wed, 1 Dec 2004 12:28:08 -0500 (EST) Received: from majordom by psg.com with local (Exim 4.43 (FreeBSD)) id 1CZY8t-000NHu-U5 for psamp-data@psg.com; Wed, 01 Dec 2004 17:19:51 +0000 Received: from [144.254.224.140] (helo=ams-iport-1.cisco.com) by psg.com with esmtp (Exim 4.43 (FreeBSD)) id 1CZY8l-000NGX-Oo for psamp@ops.ietf.org; Wed, 01 Dec 2004 17:19:44 +0000 Received: from ams-core-1.cisco.com (144.254.224.150) by ams-iport-1.cisco.com with ESMTP; 01 Dec 2004 18:30:37 +0100 X-BrightmailFiltered: true X-Brightmail-Tracker: AAAAAA== Received: from cisco.com (mrwint.cisco.com [64.103.71.48]) by ams-core-1.cisco.com (8.12.10/8.12.6) with ESMTP id iB1HJY3R017641 for ; Wed, 1 Dec 2004 18:19:35 +0100 (MET) Received: from cisco.com (dhcp-rea-gp250-64-103-65-235.cisco.com [64.103.65.235]) by cisco.com (8.8.8-Cisco List Logging/8.8.8) with ESMTP id RAA00117; Wed, 1 Dec 2004 17:19:19 GMT Message-ID: <41ADFD17.2010300@cisco.com> Date: Wed, 01 Dec 2004 17:19:19 +0000 From: Stewart Bryant User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.4) Gecko/20030624 Netscape/7.1 (ax) X-Accept-Language: en-us, en MIME-Version: 1.0 To: psamp@ops.ietf.org Subject: psamp-sample-tech-05 - comments & typos Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Checker-Version: SpamAssassin 3.0.1 (2004-10-22) on psg.com X-Spam-Status: No, score=-2.6 required=5.0 tests=BAYES_00 autolearn=ham version=3.0.1 Sender: owner-psamp@ops.ietf.org Precedence: bulk Content-Transfer-Encoding: 7bit Some comments and typos re - Stewart > > Abstract > > This document describes Sampling and Filtering techniques for IP > packet selection. It provides a categorization of schemes and SB perhaps taxonomy or classification of > defines what parameters are needed to describe the most common > selection schemes. Furthermore it shows how techniques can be ------------------------------------------------------------------- > 1. Introduction > > There are two main drivers for the growth in measurement > infrastructures and their underlying technology. First, network > data rates are increasing, with a concomitant growth in SB concomitant??? > measurement data. Secondly, the growth is compounded by the > demand by measurement-based applications for increasingly fine > grained traffic measurements. Devices such as routers, which > perform the measurements, require increasingly sophisticated and > resource intensive measurement capabilities, including the > capture of packet headers or even parts of the payload, and > classification for flow analysis. All these factors can lead to > an overwhelming amount of measurement data, resulting in high SB s/amount/quantity/ ??? > demands on resources for measurement, storage, transport and > post processing. > > The sustained capture of network traffic at line rate can be > performed by specialized measurement hardware. However, the cost > of the hardware and the measurement infrastructure required to > accommodate the measurements preclude this as a ubiquitous > approach. Instead some form of data reduction at the point of > measurement is necessary, and in current practice, this > reduction is achieved at routers through packet Sampling, > Filtering, or aggregation. The motivation for Sampling is to > select a representative subset of packets that allow accurate > estimates of properties of the unsampled traffic to be formed. > The motivation for Filtering is to remove all packets that are > not of interest. Aggregation allows compact pre-defined views of > the traffic; it is not considered in this document. Examples for SB Perhaps: ...traffic. Aggregation is out of scope for this document. SB s/for/of/ > applications that benefit from packet selection are given in > [PSAMP-FW]. > > 2. PSAMP Documents Overview SB Not sure you need such an extensive overview - maybe just compactly SB call out the docs for the reader to fetch. > > [PSAMP-FW]: "A Framework for Packet Selection and Reporting" > describes the PSAMP framework for network elements > to select subsets of packets by statistical and > other methods, and to export a stream of reports > on the selected packets to a Collector. > Definitions of terminology and the use of the > terms "must", "should" and "may" in this document > are informational only. The doc will say this last sentence for itself > SB Might put the refs inline (...Reporting" [PSAMP-FW]) to save space > > 3. Terminology > > The PSAMP terminology defined here includes (and is consistent > with) all terms listed in [PSAMP-FW]. SB Not checked with [PSAMP-FW], but you should say that you use the SB terms (you do not need to state that you are consistent with them SB that will be taken as read) and not restate them here > We here define additional > terms required for the description of the packet selection > methods. SB In this section we define the additional terms needed by this document > An architecture overview and possible configurations of > PSAMP elements can be found in [PSAMP-FW]. SB You do not need to say that in the term section - you said it above > PSAMP terminology > also aims to be consistent with terms used in [IPFIX-REQ]. The SB s/also aims/is/ > relationship between some PSAMP and IPFIX terms is described in > [PSAMP-FW]. SB This can be infered from the earlier section giving the refs > > 3.1 Observation Points, Packet Streams and Packet Content > > * Observation Point SB Surely this is described elsewhere in IPFIX. If so don't repeat. > > An Observation Point is a location in the network where > packets can be observed. Examples include: > > (i) a line to which a probe is attached; > > (ii) a shared medium, such as an Ethernet-based LAN; > > (iii) a single port of a router, or set of interfaces > (physical or logical) of a router; > > (iv) an embedded measurement subsystem within an interface. > > Note that one Observation Point may be a superset of several SB s/one/an/ > other Observation Points. For example one Observation Point SB d/other/ SB s/one/an/ > can be an entire line card. This would be the superset of the SB s/can/may/ SB s/This/ This Observation Point/ > individual Observation Points at the line card's interfaces. > > > * Packet Content > > The packet content denotes the union of the packet header > (which includes link layer, network layer and other > encapsulation headers) and the packet payload. SB This is really the frame rather than the packet. Also shouldn't SB it be data link? > Note that packets selected from a stream, e.g. by Sampling, do > not necessarily possess a property by which they can be > distinguished from packets that have not been selected. For > this reason the term "stream" is favored over "flow", which is > defined as set of packets with common properties [IPFIX- > REQUIRE]. SB Sounds like we should change the name of IPFIX after all :) > > 3.2 Selection Process > > * Selection Process > > A selection process takes the Observed Packet Stream as its > input and selects a subset of that stream as its output. > > * Selection State > > A selection process may maintain state information for use by > the selection process and/or the reporting process. At a given > time, the selection state may depend on packets observed at > and before that time, and other variables. Examples include: SB Perhaps: SB The selection state may depend on the current packet, packets observed SB earlier and other state variable, for example: ---------------------------------------------------------------- > > Selection processes may change portions of the selection state SB d/portions/ SB If it changes a portion of the state, it changes the state. SB or do you mean "may change a state variable"? > as a result of processing a packet. Selection state for a > packet is to reflect the state after processing the packet. SB Not sure you need the last sentence. > > * Selector > > A Selector defines the action of a selection process on a > single packet of its input. If selected, the packet becomes an > element of the output packet stream. > > The Selector can make use of the following information in > determining whether a packet is selected: > > (i) the packet's content; SB or is it the frame content? > > (ii) information derived from the packet's treatment at the > Observation Point; > > (iii) any selection state that may be maintained by the > selection process. > > * Composite Selector > > A Composite Selector is an ordered composition of Selectors, > in which the output Packet Stream issuing from one Selector > forms the input Packet Stream to the succeeding Selector. > > * Primitive Selector > > A Selector is primitive if it is not a Composite Selector. SB Not sure why you need this. Surely it is just a single stage SB composite selector - or perhaps just a selector? ------------------------------------------------------------ > * Hash Domain > > A subset of the packet content and the packet treatment, SB is packet treatment defined in this doc? > > process for selection of samples. In accordance to [AmCa89] and > [ClPB93] we define the following basic Sampling processes: SB I think that you need to call out the list here, or call up SB the sections. > > 5.1 Systematic Sampling > ------------------------------------------------------------ > > 6.1 Field Match Filtering > > We here define a basic Filtering schemes based on the IPIFIX > flow definition. With this method a packet is selected if a > specific field in the packet equals a predefined value. Possible > filter fields are all IPFIX flow attributes specified in [IPFIX- > INFO]. Further fields can be defined by vendor specific > extensions. SB PSAMP can propose new information elements and register SB them with IANA. The sentence should therefore read "Possible filter fields are all IPFIX flow attributes specified in [IPFIX-INFO] and any additional ones registered by IANA." > A packet is selected if Field=Value. Masks and ranges are only > supported to the extend to which [IPFIX-INFO] allows them e.g. > by providing explicit fields like the netmasks for source and > destination addresses. AND operations are possible by > concatenating filters. OR operations are not supported with this > basic model. More sophisticated filters (e.g. supporting > bitmasks, ranges or OR operations etc.) can be realized as > vendor specific schemes. SB This implies that we cannot define OR operators within SB the IPFIX design, and would need a new version of IPFIX SB to do so. I do not beleive that this is correct. ------------------------------------- > > 6.2.2 Guarding Against Pitfalls and Vulnerabilities > > A concern for Hash-based Selection is whether some large set of > related packets could have an Attained Sampling Fraction > significantly different from the Configured Sampling Fraction, > either (i) through unanticipated behavior in the Hash Function, > or (ii) because the packets had been deliberately crafted to > have this property. > > The first point underlines the importance of using a Hash > Function with good mixing properties. Examples of such are CRC32 > and Hash Functions based on modular arithmetic, see 6.4 in > [Knuth98]. The statistical properties of candidate Hash > Functions need to be evaluated, preferably on packet traces > before adoption for hash-based Sampling Is this CRC32 on the whole packet of on the start of the packet? CRC32 is very CPU intensive - beyond the CPU power of most high speed packet forwarders. The CRC at the and of the packet is rarely available to the forwarder. It's worth looking at the fletcher checksum (used in OSI transport) which has many of the properties of CRC32 but is easier to calculate. > > 6.2.3 Recommendations of Specific Hash Fuctions > > We here indicate some Hash Functions that can be used for packet > selection. The description of these Hash Functions (IPSX and > BOB) can be found in the appendix or, in the case of the CRC-32 > function, in [crc32]. In [MNiD04] different Hash Functions were > compared for collision probability, the uniformity of the > distribution of selected packets and the speed of the functions. > > A detailed description of the IPSX and the BOB function is > provided in the appendix. Further Hash Functions are described > in [MNiD04]. Note that all Hash Functions were evaluated only > for IPv4. SB> What do we recommend for IPv6? SB> -------------------------------------------------------- > 9. Security Considerations > > Malicious users or attackers may be interested to hide packets SB s/may be interested to hide/may wish to hide/ > from service providers or network operators. For instance if > packet Selectors are used for accounting or intrusion detection > applications, users may want to prevent that packets are SB s/that packets are/certain packet from being/ > SB May want to say that there are confidentially risks, but SB these are the responsibility of other system components. -------------------------------------------------------- > > > 17. Appendix: Hash Functions > > 17.1 IP Shift-XOR (IPSX) Hash Function SB> is MNiD04 the definitive reference to IPSX - same Q for BOB SB> below. SB> Also what are we goingto do about IPv6? -------------------------------------------------------- > > If you are hashing n strings (ub1 **)k, do it like this: > for (i=0, h=0; i SB Need to fix up the English in the para below > By Bob Jenkins, 1996. bob_jenkins@burtleburtle.net. You may > use this code any way you wish, private, educational, or > commercial. It's free. -- to unsubscribe send a message to psamp-request@ops.ietf.org with the word 'unsubscribe' in a single line as the message text body. archive: From owner-psamp@ops.ietf.org Thu Dec 2 03:28:06 2004 Received: from psg.com (mailnull@psg.com [147.28.0.62]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id DAA19147 for ; Thu, 2 Dec 2004 03:28:05 -0500 (EST) Received: from majordom by psg.com with local (Exim 4.43 (FreeBSD)) id 1CZmB1-000A3R-CA for psamp-data@psg.com; Thu, 02 Dec 2004 08:18:59 +0000 Received: from [195.37.70.21] (helo=kyoto.netlab.nec.de) by psg.com with esmtp (Exim 4.43 (FreeBSD)) id 1CZmB0-000A3E-2R for psamp@ops.ietf.org; Thu, 02 Dec 2004 08:18:58 +0000 Received: from [10.1.1.171] (mito.netlab.nec.de [195.37.70.39]) by kyoto.netlab.nec.de (Postfix) with ESMTP id C1DAD1BAC4D for ; Thu, 2 Dec 2004 09:18:51 +0100 (CET) Date: Thu, 02 Dec 2004 09:18:54 +0100 From: Juergen Quittek To: psamp@ops.ietf.org Subject: =?ISO-8859-1?Q?draft_PSAMP_minutes_of_IETF_=A761?= Message-ID: <2147483647.1101979134@[10.1.1.171]> X-Mailer: Mulberry/3.0.3 (Mac OS X) MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1; format=flowed Content-Transfer-Encoding: quoted-printable Content-Disposition: inline X-Spam-Checker-Version: SpamAssassin 3.0.1 (2004-10-22) on psg.com X-Spam-Status: No, score=-2.6 required=5.0 tests=BAYES_00,MIME_QP_LONG_LINE autolearn=ham version=3.0.1 Sender: owner-psamp@ops.ietf.org Precedence: bulk Content-Transfer-Encoding: quoted-printable Dear all, Below please find the draft minutes of our last session. Please send comments within a week. Thanks, Juergen --=20 Juergen Quittek quittek@netlab.nec.de Tel: +49 6221 90511-15 NEC Europe Ltd., Network Laboratories Fax: +49 6221 90511-55 Kurfuersten-Anlage 36, 69115 Heidelberg, Germany http://www.netlab.nec.de =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D Minutes of the PSAMP BOF session at IETF 61 Friday November 12, 9:00 h - 11:30 h =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D Packet Sampling Working Group Chairs: Andy Bierman Juergen Quittek Minutes taken by Dave Plonka and Ralf Wolter 0. Session Summary 1. PSAMP WG Status 2. Update of Sampling Framework 3. Update od Packet Selection 4. Discussion of open PSAMP protocol issues 5. Discussion of open PSAMP Information Model issues 6. PSAMP MIB 7. Wrap up ---------------- Discussed Internet drafts A Framework for Passive Packet Measurement http://www.ietf.org/internet-drafts/draft-ietf-psamp-framework-09.txt Sampling and Filtering Techniques for IP Packet Selection http://www.ietf.org/internet-drafts/draft-ietf-psamp-sample-tech-05.txt Packet Sampling (PSAMP) Protocol Specifications (expired) draft-ietf-psamp-protocol-01.txt Information Model for Packet Sampling Exports http://www.ietf.org/internet-drafts/draft-ietf-psamp-info-02.txt Definitions of Managed Objects for Packet Sampling http://www.ietf.org/internet-drafts/draft-ietf-psamp-mib-03.txt ---------------- 0. Session Summary The PSAMP WG finished last call of its first document, the PSAMP framework. A few editorial issues were still put on the list of actions items during the meeting. The document on packet selection is still in WG last call. The call was extended, because the number of reviews was performed until the session was not sufficient. After completing the packet selection document all required input for the PSAMP MIB is available and this document can be completed next. The remaining two document, the protocol and the information model specification, are on hold, because they depend on the IPFIX protocol and information model specifications, which are not finished yet. If work in the IPFIX WG progresses as expected, the two missing documents can be discussed at the next meeting and will enter WG last call shortly after that meeting. ---------------- 1. PSAMP WG Status (Juergen) The PSAMP framework document passed WG last call. It will be forwarded to the area directors soon. The packet selection document is quite mature. The deadline of WG last call on this document is past, but since there was no serious review to be observed yet, it remains in last call. Juergen requested that positive responses to the draft should be posted as well so that there is evidence that it has been reviewed. Protocol and info model document are waiting for progress of the corresponding IPFIX docuements. Work on the PSAMP MIB can go on when last call on the packedt selection document is closed. Then the PSAMP MIB can be completed soon. ---------------- 2. PSAMP Framework Document (Nick Duffield) draft-ietf-psamp-framework-09.txt Nick explained that there were only minor changes applied to the draft since the last version. The last call is closed and should be ready for WG last call. [Benoit Claise] On the low-level filter, we only kept logical AND [in the packet selection draft] and removed logical OR because it was too hard to model, so it should be removed from this doc as well. [Tanja Zseby] I agree and think this was just overlooked. Andy reminded participants who review docs to make it known (that they reviewed it) even if there are no negative issues/problems ---------------- 3. Sampling and Filtering Techniques (Tanja Zseby) draft-ietf-psamp-sample-tech-05.txt Tanja explained that Mask/match was renamed to field match filter. Low level filter definition were substituted by a high level definition. There are further minor changes, e.g. the terminology section was updated. the Introduction and text on other PSAMP document was added as well as the AT&T IPR statement. Tanja requested that people read the document, and send feedback regardless of whether or not there are problems [Andy] Are there any volunteers willing to commit to reviewing this document in the next 2-3 weeks? I don't see a way to move forward unless there's reviews. Stewart Bryant offered to do so. ---------------- 4. PSAMP Protocol Specifications (Benoit Claise) draft-ietf-psamp-protocol-01.txt (expired) Benoit explained that there was no proegress on this document because it strongly depends on the IPFIX protocol document that is not yet finished. The draft is already expired. Benoit present a summary of recent progress with the IPFIX protocol document. For details about IPFIX, see Benoit=D5s section in the minutes of the IETF 61 IPFIX session. After completing IPFIX WG last call, work on the PSAMP protocol draft will continue. [Andy] Is this a big delay in the PSAMP schedule [Dave Plonka (IPFIX chair)] PSAMP participants should/could review the IPFIX protocol document. [Juergen] When the IPFIX protocol is complete, there will be no big technical issues for the PSAMP protocol. ---------------- 5. PSAMP Information Model (no slides) draft-ietf-psamp-info-02.txt J=3Frgen listed of open PSAMP Information Model issues. The state is similar to the protocol document, but the draft was updated in July 2004 and is not yet expired. After completing IPFIX WG last call on the IPFIX info model, work on the PSAMP IM draft will continue. A new version should be available by the next meeting. ---------------- 6. PSAMP MIB (no slides) draft-ietf-psamp-mib-03.txt Also this draft was not not updated since July 2004. But the MIB is almost complete, except for some work on packet selection, due to updates in that area. After packet selection document finishes WG last call, the MIB could go to last call as well, currently targeted for December 04. ---------------- 7. Wrap up The meeting was adjourned. Following the meeting, Juergen invited those in the room to review the drafts documents now, since there was plenty of time left in the session's time slot. -- to unsubscribe send a message to psamp-request@ops.ietf.org with the word 'unsubscribe' in a single line as the message text body. archive: From owner-psamp@ops.ietf.org Fri Dec 3 13:23:12 2004 Received: from psg.com (mailnull@psg.com [147.28.0.62]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id NAA14806 for ; Fri, 3 Dec 2004 13:23:12 -0500 (EST) Received: from majordom by psg.com with local (Exim 4.43 (FreeBSD)) id 1CaHvD-000HWf-MH for psamp-data@psg.com; Fri, 03 Dec 2004 18:12:47 +0000 Received: from [192.20.225.112] (helo=mail-yellow.research.att.com) by psg.com with esmtp (Exim 4.43 (FreeBSD)) id 1CaHvC-000HWR-Pp for psamp@ops.ietf.org; Fri, 03 Dec 2004 18:12:46 +0000 Received: from NJFPSRVEXG2KCL.research.att.com (njfpsrvexg2k2.research.att.com [135.207.21.32]) by mail-green.research.att.com (Postfix) with ESMTP id 4F4D4A7B03; Fri, 3 Dec 2004 13:12:46 -0500 (EST) content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Subject: RE: psamp filtering X-MimeOLE: Produced By Microsoft Exchange V6.0.6603.0 Date: Fri, 3 Dec 2004 13:12:46 -0500 Message-ID: <387B5A9BF31B5D43A2B18DD9F326B8E191541C@NJFPSRVEXG2KCL.research.att.com> Thread-Topic: psamp filtering Thread-Index: AcTXCfUKUAqV1FLCSsC9e7kXdvV/NQCR3Kig From: To: Cc: X-Spam-Checker-Version: SpamAssassin 3.0.1 (2004-10-22) on psg.com X-Spam-Status: No, score=-2.4 required=5.0 tests=AWL,BAYES_00,NO_REAL_NAME autolearn=no version=3.0.1 Sender: owner-psamp@ops.ietf.org Precedence: bulk Content-Transfer-Encoding: quoted-printable Maurizio, See question inline below, Nick > -----Original Message----- > From: owner-psamp@ops.ietf.org [mailto:owner-psamp@ops.ietf.org] On Behalf > Of Maurizio Molina > Sent: Tuesday, November 30, 2004 5:34 AM > To: Duffield,Nicholas G (Nick) > Cc: psamp@ops.ietf.org > Subject: Re: psamp filtering >=20 > duffield@research.att.com wrote: >=20 > >In the sampling techniques draft: > > > > 6.1 Field Match Filtering > > > > We here define a basic Filtering schemes based on the IPIFIX > > flow definition. With this method a packet is selected if a > > specific field in the packet equals a predefined value. Possible > > > > filter fields are all IPFIX flow attributes specified in [IPFIX- > > INFO]. Further fields can be defined by vendor specific > > extensions. > > A packet is selected if Field=3DValue. Masks and ranges are only > > supported to the extend to which [IPFIX-INFO] allows them e.g. > > by providing explicit fields like the netmasks for source and > > destination addresses. AND operations are possible by > > concatenating filters. OR operations are not supported with this > > > > basic model. More sophisticated filters (e.g. supporting > > bitmasks, ranges or OR operations etc.) can be realized as > > vendor specific schemes. > > > >As it stands now, this text might give the impression that AND > >operations are accomplished by composing a number of PSAMP selectors, > >each being primitive and corresponding to filtering on a single field. I > >don't think this is what is intended. (In any case, PSAMP can currently > >only combine one filter with one sampling operation at present). > > > My understanding is that through the STREAM ID you can have multiple > concatenation, and even a filter + filter concatenation (and in this > case the order is intrinsically specified). > However, I admit that from the TOC only it looks like that the only > possible composition is Cascaded Filtering->Sampling or > Sampling->Filtering (se. 8.1). But if you read the text before, it > states that it is just an example. >=20 > > > > > >Also, "concatenation" implies that an order amongst the basic filters is > >to be specified. > > > Correct, but the result will then be independent of the order. >=20 > > As Matt Grossglauser has pointed out, this might be > >useful if one wants to put a "quick" filter (e.g. on port number) first, > >to reduce the packet rate down for a "slow" filter (e.g. on routing > >state) following. So do we want to require ordering of filters within > >the combination? > > > See above: I think that if you specify a cascaded filter via the STREAM > ID, ordering is implicitly defined, but the result will then be > independent of the order. >=20 > > Or would some implementors want to implement some or > >all combinations as a single composite mask/match across fields, without > >ordering....? > > > >If order is not specified, we might want to replace > > > >"AND operations are possible by concatenating filters" > > > >with > > > >"AND operations are possible by specifying multiple basic field match > >Filters, the packet being selected if would be selected by all the > >specified filters. The resulting combination is viewed as a primitive > >PSAMP Selector." > > > > > I think that if we go for your proposal, we have to ensure that > IPFIX-INFO supports it. > I'd rather stick to the old text for SPECIFYING a filter. Then I'd add > part of your text to leave the door open to implementation of composite > filters in "one step". > My proposal: >=20 > AND operations are possible by concatenating filters through the use of > the STREAM ID (see sec. 7.1 and 8). In this case, the ordering in which > the filtering happens is implicitly defined (outer filters come after > inner filters). However, as long as the concatenation is on filters only, > the result of the cascaded filter is independent from the order, but the > order MAY be important for implementation purposes, as the first filter > will have to work at a higher rate. In any case, an implementation is not > constrained to respect the filter ordering, as long as the result is the > same, and it MAY even implement the composite filtering in filtering in > one single step. > Maurizio >=20 >=20 Do you view the concatenation as a composite selector, with each component filter a primitive selector? If so, the text in the framework document Section 5.5 needs to include this as an example of a composite selector. >=20 > >Finally, it was suggested to me at IETF 61 that the current text in the > >framework draft (it speaks of match/mask operations) should be replaced > >with the text from the sampling draft. I can do this once the latter is > >finalized. > > > >Comments? > > > >Nick > > > > > > > >-- > >to unsubscribe send a message to psamp-request@ops.ietf.org with > >the word 'unsubscribe' in a single line as the message text body. > >archive: > > > > >=20 >=20 >=20 > -- > to unsubscribe send a message to psamp-request@ops.ietf.org with > the word 'unsubscribe' in a single line as the message text body. > archive: -- to unsubscribe send a message to psamp-request@ops.ietf.org with the word 'unsubscribe' in a single line as the message text body. archive: From owner-psamp@ops.ietf.org Wed Dec 8 10:04:44 2004 Received: from psg.com (mailnull@psg.com [147.28.0.62]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id KAA02008 for ; Wed, 8 Dec 2004 10:04:44 -0500 (EST) Received: from majordom by psg.com with local (Exim 4.43 (FreeBSD)) id 1Cc3Gy-000G6W-Aa for psamp-data@psg.com; Wed, 08 Dec 2004 14:58:32 +0000 Received: from [193.63.211.19] (helo=mail.dante.org.uk) by psg.com with esmtp (Exim 4.43 (FreeBSD)) id 1Cc3Gv-000G66-Bh for psamp@ops.ietf.org; Wed, 08 Dec 2004 14:58:29 +0000 Received: from [127.0.0.1] (helo=localhost) by mail.dante.org.uk with esmtp (Exim 4.43) id 1Cc3Gs-0002Hh-Et; Wed, 08 Dec 2004 14:58:26 +0000 Received: from mail.dante.org.uk ([127.0.0.1]) by localhost (alpha [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 08763-01; Wed, 8 Dec 2004 14:58:07 +0000 (GMT) Received: from [193.63.211.62] (helo=[193.63.211.62]) by mail.dante.org.uk with esmtp (Exim 4.43) id 1Cc3GZ-0002HW-MM; Wed, 08 Dec 2004 14:58:07 +0000 Message-ID: <41B7167F.9020309@dante.org.uk> Date: Wed, 08 Dec 2004 14:58:07 +0000 From: Maurizio Molina User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7.3) Gecko/20040910 X-Accept-Language: en-us, en MIME-Version: 1.0 To: duffield@research.att.com CC: psamp@ops.ietf.org Subject: Re: psamp filtering References: <387B5A9BF31B5D43A2B18DD9F326B8E191541C@NJFPSRVEXG2KCL.research.att.com> In-Reply-To: <387B5A9BF31B5D43A2B18DD9F326B8E191541C@NJFPSRVEXG2KCL.research.att.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: amavisd-new at dante.org.uk X-Spam-Checker-Version: SpamAssassin 3.0.1 (2004-10-22) on psg.com X-Spam-Status: No, score=-2.6 required=5.0 tests=BAYES_00 autolearn=ham version=3.0.1 Sender: owner-psamp@ops.ietf.org Precedence: bulk Content-Transfer-Encoding: 7bit duffield@research.att.com wrote: >Maurizio, > >See question inline below, > >Nick > > > >>-----Original Message----- >>From: owner-psamp@ops.ietf.org [mailto:owner-psamp@ops.ietf.org] On >> >> >Behalf > > >>Of Maurizio Molina >>Sent: Tuesday, November 30, 2004 5:34 AM >>To: Duffield,Nicholas G (Nick) >>Cc: psamp@ops.ietf.org >>Subject: Re: psamp filtering >> >>duffield@research.att.com wrote: >> >> >> >>>In the sampling techniques draft: >>> >>> 6.1 Field Match Filtering >>> >>> We here define a basic Filtering schemes based on the IPIFIX >>> flow definition. With this method a packet is selected if a >>> specific field in the packet equals a predefined value. Possible >>> >>> filter fields are all IPFIX flow attributes specified in [IPFIX- >>> INFO]. Further fields can be defined by vendor specific >>> extensions. >>> A packet is selected if Field=Value. Masks and ranges are only >>> supported to the extend to which [IPFIX-INFO] allows them e.g. >>> by providing explicit fields like the netmasks for source and >>> destination addresses. AND operations are possible by >>> concatenating filters. OR operations are not supported with this >>> >>> basic model. More sophisticated filters (e.g. supporting >>> bitmasks, ranges or OR operations etc.) can be realized as >>> vendor specific schemes. >>> >>>As it stands now, this text might give the impression that AND >>>operations are accomplished by composing a number of PSAMP selectors, >>>each being primitive and corresponding to filtering on a single >>> >>> >field. I > > >>>don't think this is what is intended. (In any case, PSAMP can >>> >>> >currently > > >>>only combine one filter with one sampling operation at present). >>> >>> >>> >>My understanding is that through the STREAM ID you can have multiple >>concatenation, and even a filter + filter concatenation (and in this >>case the order is intrinsically specified). >>However, I admit that from the TOC only it looks like that the only >>possible composition is Cascaded Filtering->Sampling or >>Sampling->Filtering (se. 8.1). But if you read the text before, it >>states that it is just an example. >> >> >> >>>Also, "concatenation" implies that an order amongst the basic filters >>> >>> >is > > >>>to be specified. >>> >>> >>> >>Correct, but the result will then be independent of the order. >> >> >> >>>As Matt Grossglauser has pointed out, this might be >>>useful if one wants to put a "quick" filter (e.g. on port number) >>> >>> >first, > > >>>to reduce the packet rate down for a "slow" filter (e.g. on routing >>>state) following. So do we want to require ordering of filters within >>>the combination? >>> >>> >>> >>See above: I think that if you specify a cascaded filter via the >> >> >STREAM > > >>ID, ordering is implicitly defined, but the result will then be >>independent of the order. >> >> >> >>>Or would some implementors want to implement some or >>>all combinations as a single composite mask/match across fields, >>> >>> >without > > >>>ordering....? >>> >>>If order is not specified, we might want to replace >>> >>>"AND operations are possible by concatenating filters" >>> >>>with >>> >>>"AND operations are possible by specifying multiple basic field match >>>Filters, the packet being selected if would be selected by all the >>>specified filters. The resulting combination is viewed as a primitive >>>PSAMP Selector." >>> >>> >>> >>> >>I think that if we go for your proposal, we have to ensure that >>IPFIX-INFO supports it. >>I'd rather stick to the old text for SPECIFYING a filter. Then I'd >> >> >add > > >>part of your text to leave the door open to implementation of >> >> >composite > > >>filters in "one step". >>My proposal: >> >>AND operations are possible by concatenating filters through the use >> >> >of > > >>the STREAM ID (see sec. 7.1 and 8). In this case, the ordering in >> >> >which > > >>the filtering happens is implicitly defined (outer filters come after >>inner filters). However, as long as the concatenation is on filters >> >> >only, > > >>the result of the cascaded filter is independent from the order, but >> >> >the > > >>order MAY be important for implementation purposes, as the first >> >> >filter > > >>will have to work at a higher rate. In any case, an implementation is >> >> >not > > >>constrained to respect the filter ordering, as long as the result is >> >> >the > > >>same, and it MAY even implement the composite filtering in filtering >> >> >in > > >>one single step. >>Maurizio >> >> >> >> > >Do you view the concatenation as a composite selector, with each >component filter a primitive selector? > Yes > If so, the text in the framework >document Section 5.5 needs to include this as an example of a composite >selector. > > Since 5.5 mentions that they are only examples, it is not strictly necessary. But a third explicit example with filter+filter can be added, if you feel it clarifies. Maurizio > > > >>>Finally, it was suggested to me at IETF 61 that the current text in >>> >>> >the > > >>>framework draft (it speaks of match/mask operations) should be >>> >>> >replaced > > >>>with the text from the sampling draft. I can do this once the latter >>> >>> >is > > >>>finalized. >>> >>>Comments? >>> >>>Nick >>> >>> >>> >>>-- >>>to unsubscribe send a message to psamp-request@ops.ietf.org with >>>the word 'unsubscribe' in a single line as the message text body. >>>archive: >>> >>> >>> >>> >> >>-- >>to unsubscribe send a message to psamp-request@ops.ietf.org with >>the word 'unsubscribe' in a single line as the message text body. >>archive: >> >> > > > > -- to unsubscribe send a message to psamp-request@ops.ietf.org with the word 'unsubscribe' in a single line as the message text body. archive: From owner-psamp@ops.ietf.org Fri Dec 10 16:24:48 2004 Received: from psg.com (mailnull@psg.com [147.28.0.62]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id QAA26455 for ; Fri, 10 Dec 2004 16:24:47 -0500 (EST) Received: from majordom by psg.com with local (Exim 4.43 (FreeBSD)) id 1Ccs9y-0009He-Pq for psamp-data@psg.com; Fri, 10 Dec 2004 21:18:42 +0000 Received: from [192.20.225.112] (helo=mail-yellow.research.att.com) by psg.com with esmtp (Exim 4.43 (FreeBSD)) id 1Ccs9x-0009HN-Vs for psamp@ops.ietf.org; Fri, 10 Dec 2004 21:18:42 +0000 Received: from NJFPSRVEXG2KCL.research.att.com (njfpsrvexg2k2.research.att.com [135.207.21.32]) by mail-green.research.att.com (Postfix) with ESMTP id 6CDC6A7B02; Fri, 10 Dec 2004 16:18:41 -0500 (EST) Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable X-MimeOLE: Produced By Microsoft Exchange V6.0.6603.0 Subject: RE: some comments on draft-ietf-psamp-framework-09.txt Date: Fri, 10 Dec 2004 16:18:41 -0500 Message-ID: <387B5A9BF31B5D43A2B18DD9F326B8E1915420@NJFPSRVEXG2KCL.research.att.com> Thread-Topic: some comments on draft-ietf-psamp-framework-09.txt Thread-Index: AcTQvjYIn5VirdxNRpWtlsNMWPXAlQOPNXVw From: To: , , X-Spam-Checker-Version: SpamAssassin 3.0.1 (2004-10-22) on psg.com X-Spam-Status: No, score=-2.4 required=5.0 tests=AWL,BAYES_00,NO_REAL_NAME autolearn=no version=3.0.1 Sender: owner-psamp@ops.ietf.org Precedence: bulk Content-Transfer-Encoding: quoted-printable > >> ! > >> ! SHOULD WE MAKE IT EXPLICIT THAT OBERSVATION POINT, MEASUREMENT > >> ! PROCESS AND EXPORTING PROCESS IDS SHOULD BE CONTAINED IN EVERY > >> ! PACKET REPORT? > > > > > > > > > >Since an export packet may contain multiple packet reports, the export > >process id can be included once per export packet. > > > > > > > Did we say anywhere that exporting process IDs are included?=20 Currently, Section 6.4 says about the reporting process: 6.4 Report Interpretation Information for use in Report Interpretation must include=20 [...] (iv) identifiers for Observation Point, Measurement Process, and Exporting Process. I'm thinking now that responsibility for inserting identifiers should be split as follows:=20 . Reporting Process inserts IDs for Observation Point and Measurement Process . Exporting Process inserts IDs for Export Process. Reasoning: we don't exclude the case that a report on a given packet is sent by more than one exporting process. Example: send one copy to a local on board processor, and one copy to a remote collector. I don't see a need for the reporting process to include both exporting process identifiers in this case; instead each exporting process can insert its own ID. Also, one can imagine in some configurations that the reporting process wouldn't have the information on exporting process ID available (i.e. processes are run on different hosts). Following suggestions from a number of folks, the IDs would now explicitly be inserted per report or packet, i.e.: . packet reports each contain ID for observation point and measurement process . export packets (which may contain multiple report packets from multiple measurement processes) each contain an ID for the exporting process. Comments? Nick -- to unsubscribe send a message to psamp-request@ops.ietf.org with the word 'unsubscribe' in a single line as the message text body. archive: