Subject: psamp-sample-tech-05 - comments & typos

From: Stewart Bryant <[email-masked]>
Date: Wed, 01 Dec 2004 17:19:19 +0000

Some comments and typos re <draft-ietf-psamp-sample-tech-05.txt>

- Stewart

>
>  Abstract
>
>     This document describes Sampling and Filtering techniques for IP
>     packet selection. It provides a categorization of schemes and

SB perhaps taxonomy or classification of

>     defines what parameters are needed to describe the most common
>     selection schemes. Furthermore it shows how techniques can be
-------------------------------------------------------------------
>  1. Introduction
>
>     There are two main drivers for the growth in measurement
>     infrastructures and their underlying technology. First, network
>     data rates are increasing, with a concomitant growth in

SB concomitant???

>     measurement data. Secondly, the growth is compounded by the
>     demand by measurement-based applications for increasingly fine
>     grained traffic measurements. Devices such as routers, which
>     perform the measurements, require increasingly sophisticated and
>     resource intensive measurement capabilities, including the
>     capture of packet headers or even parts of the payload, and
>     classification for flow analysis. All these factors can lead to
>     an overwhelming amount of measurement data, resulting in high
SB s/amount/quantity/ ???
>     demands on resources for measurement, storage, transport and
>     post processing.
>
>     The sustained capture of network traffic at line rate can be
>     performed by specialized measurement hardware. However, the cost
>     of the hardware and the measurement infrastructure required to
>     accommodate the measurements preclude this as a ubiquitous
>     approach. Instead some form of data reduction at the point of
>     measurement is necessary, and in current practice, this
>     reduction is achieved at routers through packet Sampling,
>     Filtering, or aggregation. The motivation for Sampling is to
>     select a representative subset of packets that allow accurate
>     estimates of properties of the unsampled traffic to be formed.
>     The motivation for Filtering is to remove all packets that are
>     not of interest. Aggregation allows compact pre-defined views of
>     the traffic; it is not considered in this document. Examples for
SB Perhaps: ...traffic. Aggregation is out of scope for this document.
SB s/for/of/
>     applications that benefit from packet selection are given in
>     [PSAMP-FW].
>
>  2. PSAMP Documents Overview

SB Not sure you need such an extensive overview - maybe just compactly
SB call out the docs for the reader to fetch.

>
>     [PSAMP-FW]:   "A Framework for Packet Selection and Reporting"
>                    describes the PSAMP framework for network elements
>                    to select subsets of packets by statistical and
>                    other methods, and to export a stream of reports
>                    on the selected packets to a Collector.
>                    Definitions of terminology and the use of the
>                    terms "must", "should" and "may" in this document
>                    are informational only.
The doc will say this last sentence for itself

SB Might put the refs inline (...Reporting" [PSAMP-FW]) to save space

>
>  3. Terminology
>
>     The PSAMP terminology defined here includes (and is consistent
>     with) all terms listed in [PSAMP-FW].

SB Not checked with [PSAMP-FW], but you should say that you use the
SB terms (you do not need to state that you are consistent with them
SB that will be taken as read) and not restate them here

>     We here define additional
>     terms required for the description of the packet selection
>     methods.

SB In this section we define the additional terms needed by this document

>     An architecture overview and possible configurations of
>     PSAMP elements can be found in [PSAMP-FW].

SB You do not need to say that in the term section - you said it above

>     PSAMP terminology
>     also aims to be consistent with terms used in [IPFIX-REQ]. The

SB s/also aims/is/

>     relationship between some PSAMP and IPFIX terms is described in
>     [PSAMP-FW].

SB This can be infered from the earlier section giving the refs

>
>  3.1 Observation Points, Packet Streams and Packet Content
>
>     * Observation Point

SB Surely this is described elsewhere in IPFIX. If so don't repeat.

>
>        An Observation Point is a location in the network where
>        packets can be observed. Examples include:
>
>          (i)   a line to which a probe is attached;
>
>          (ii)  a shared medium, such as an Ethernet-based LAN;
>
>          (iii) a single port of a router, or set of interfaces
>                (physical or logical) of a router;
>
>          (iv)  an embedded measurement subsystem within an interface.
>
>        Note that one Observation Point may be a superset of several
SB s/one/an/
>        other Observation Points.  For example one Observation Point
SB d/other/
SB s/one/an/
>        can be an entire line card.  This would be the superset of the
SB s/can/may/
SB s/This/ This Observation Point/
>        individual Observation Points at the line card's interfaces.
>
>
>     * Packet Content
>
>        The packet content denotes the union of the packet header
>        (which includes link layer, network layer and other
>        encapsulation headers) and the packet payload.

SB This is really the frame rather than the packet. Also shouldn't
SB it be data link?

>        Note that packets selected from a stream, e.g. by Sampling, do
>        not necessarily possess a property by which they can be
>        distinguished from packets that have not been selected. For
>        this reason the term "stream" is favored over "flow", which is
>        defined as set of packets with common properties [IPFIX-
>        REQUIRE].

SB Sounds like we should change the name of IPFIX after all :)

>
>  3.2 Selection Process
>
>     * Selection Process
>
>        A selection process takes the Observed Packet Stream as its
>        input and selects a subset of that stream as its output.
>
>     * Selection State
>
>        A selection process may maintain state information for use by
>        the selection process and/or the reporting process. At a given
>        time, the selection state may depend on packets observed at
>        and before that time, and other variables. Examples include:

SB Perhaps:
SB The selection state may depend on the current packet, packets observed
SB earlier and other state variable, for example:

----------------------------------------------------------------
>
>        Selection processes may change portions of the selection state

SB d/portions/
SB If it changes a portion of the state, it changes the state.
SB or do you mean "may change a state variable"?

>        as a result of processing a packet. Selection state for a
>        packet is to reflect the state after processing the packet.

SB Not sure you need the last sentence.

>
>     * Selector
>
>        A Selector defines the action of a selection process on a
>        single packet of its input. If selected, the packet becomes an
>        element of the output packet stream.
>
>        The Selector can make use of the following information in
>        determining whether a packet is selected:
>
>          (i)   the packet's content;

SB or is it the frame content?

>
>          (ii)  information derived from the packet's treatment at the
>                Observation Point;
>
>          (iii) any selection state that may be maintained by the
>                selection process.
>
>     * Composite Selector
>
>        A Composite Selector is an ordered composition of Selectors,
>        in which the output Packet Stream issuing from one Selector
>        forms the input Packet Stream to the succeeding Selector.
>
>     * Primitive Selector
>
>        A Selector is primitive if it is not a Composite Selector.

SB Not sure why you need this. Surely it is just a single stage
SB composite selector - or perhaps just a selector?

------------------------------------------------------------

>     * Hash Domain
>
>        A subset of the packet content and the packet treatment,
SB is packet treatment defined in this doc?

>
>     process for selection of samples. In accordance to [AmCa89] and
>     [ClPB93] we define the following basic Sampling processes:

SB I think that you need to call out the list here, or call up
SB the sections.

>  5.1 Systematic Sampling
------------------------------------------------------------
>
>  6.1 Field Match Filtering
>
>     We here define a basic Filtering schemes based on the IPIFIX
>     flow definition. With this method a packet is selected if a
>     specific field in the packet equals a predefined value. Possible
>     filter fields are all IPFIX flow attributes specified in [IPFIX-
>     INFO]. Further fields can be defined by vendor specific
>     extensions.

SB PSAMP can propose new information elements and register
SB them with IANA. The sentence should therefore read

"Possible filter fields are all IPFIX flow attributes specified
in [IPFIX-INFO] and any additional ones registered by IANA."

>     A packet is selected if Field=Value. Masks and ranges are only
>     supported to the extend to which [IPFIX-INFO] allows them e.g.
>     by providing explicit fields like the netmasks for source and
>     destination addresses. AND operations are possible by
>     concatenating filters. OR operations are not supported with this
>     basic model. More sophisticated filters (e.g. supporting
>     bitmasks, ranges or OR operations etc.) can be realized as
>     vendor specific schemes.

SB This implies that we cannot define OR operators within
SB the IPFIX design, and would need a new version of IPFIX
SB to do so. I do not beleive that this is correct.

-------------------------------------
>
>  6.2.2   Guarding Against Pitfalls and Vulnerabilities
>
>     A concern for Hash-based Selection is whether some large set of
>     related packets could have an Attained Sampling Fraction
>     significantly different from the Configured Sampling Fraction,
>     either (i) through unanticipated behavior in the Hash Function,
>     or (ii) because the packets had been deliberately crafted to
>     have this property.
>
>     The first point underlines the importance of using a Hash
>     Function with good mixing properties. Examples of such are CRC32
>     and Hash Functions based on modular arithmetic, see 6.4 in
>     [Knuth98]. The statistical properties of candidate Hash
>     Functions need to be evaluated, preferably on packet traces
>     before adoption for hash-based Sampling

Is this CRC32 on the whole packet of on the start of the
packet? CRC32 is very CPU intensive - beyond the CPU power
of most high speed packet forwarders. The CRC at the and of the
packet is rarely available to the forwarder. It's worth looking
at the fletcher checksum (used in OSI transport) which has
many of the properties of CRC32 but is easier to calculate.

>
>  6.2.3   Recommendations of Specific Hash Fuctions
>
>     We here indicate some Hash Functions that can be used for packet
>     selection. The description of these Hash Functions (IPSX and
>     BOB) can be found in the appendix or, in the case of the CRC-32
>     function, in [crc32]. In [MNiD04] different Hash Functions were
>     compared for collision probability, the uniformity of the
>     distribution of selected packets and the speed of the functions.
>
>     A detailed description of the IPSX and the BOB function is
>     provided in the appendix. Further Hash Functions are described
>     in [MNiD04]. Note that all Hash Functions were evaluated only
>     for IPv4.

SB> What do we recommend for IPv6?
SB>
--------------------------------------------------------
>  9. Security Considerations
>
>     Malicious users or attackers may be interested to hide packets
SB s/may be interested to hide/may wish to hide/
>     from service providers or network operators. For instance if
>     packet Selectors are used for accounting or intrusion detection
>     applications, users may want to prevent that packets are
SB s/that packets are/certain packet from being/

SB May want to say that there are confidentially risks, but
SB these are the responsibility of other system components.

--------------------------------------------------------
>
>
>  17. Appendix: Hash Functions
>
>  17.1    IP Shift-XOR (IPSX) Hash Function

SB> is MNiD04 the definitive reference to IPSX - same Q for BOB
SB> below.
SB> Also what are we goingto do about IPv6?

--------------------------------------------------------
>
>     If you are hashing n strings (ub1 **)k, do it like this:
>     for (i=0, h=0; i<n; ++i) h = hash( k[i], len[i], h);
>

SB Need to fix up the English in the para below

>     By Bob Jenkins, 1996.  [email-masked].  You may
>     use this code any way you wish, private, educational, or
>     commercial.  It's free.

--
to unsubscribe send a message to [email-masked] with
the word 'unsubscribe' in a single line as the message text body.
archive: <http://ops.ietf.org/lists/psamp/>