Subject: Re: [babel] info-model: message log

From: "STARK, BARBARA H" <[email-masked]>
Date: Wed, 2 Jan 2019 21:14:53 +0000

Getting back to this after the holidays...

> >> Here is my suggestion for having a message log:
> >
> >> babel-message-log is:
> >> A reference or url link to a file that contains a timestamped log of
> >> messages received and sent on babel-udp-port on all interfaces. File format
> >> MUST be readable by Wireshark 1.8 or later and use the applicable file format
> >> extension in the filename. Logging is enabled / disabled by babel-message-
> >> log-enable.
> >
> > Wireshark supports multiple formats, I'm not sure that it's a good
> > idea to allow all of the formats supported by Wireshark.

Why not? Not that I'm arguing strongly for allowing all formats; just that I can't convince myself of a strong argument against it. I'm fine with a more limited set, but don't want to be too limiting. Personally, I'm fine with anything that I can get an easy-to-use, free, good parser for and that can provide all the info.

> > I'm not intimately familiar with the capture packet formats.  The
> > pcapng file format appears to be the preferred format for wireshark,
> > but the internet-draft is expired since 2014; what's more, I'm not
> > sure it's supported by libpcap.

Per https://wiki.wireshark.org/Development/LibpcapFileFormat (with a last edited date from 2015):
"It is widely accepted that the libpcap file format serves its purpose but lacks some useful features. There's a next generation pcap file format documented at the pcapng specification Git repository. The new format supplies many of the capabilities listed in "Drawbacks" above.
Wireshark currently has the ability to read and write pcapng files, and does so by default, although doesn't support all of the capabilities of the files. Libpcap 1.1.0 and later have a limited ability to read them as well, although libpcap doesn't yet supporting writing them."
 
> Would agree that the capture format is a pcap file. I will not be surprised if
> Wireshark does not understand Babel protocol, particularly if no one has
> defined it in libpcap. Not too difficult to add though.

The RFC6126 dissector exists: https://www.wireshark.org/docs/dfref/b/babel.html
I'm hopeful someone will contribute the updated dissector once the new RFC is published...

> > The traditional libpcap format is documented here:
> >
> > <url mauled by AT&T security filters>
> >
> > and it's likely to be supported by any tool in existence.  Unless
> > somebody has good technical reasons to avoid libpcap, I suggest we
> > should mandate that format.

Apparently, libpcap is said (by https://wiki.wireshark.org/Development/LibpcapFileFormat) to have the following drawbacks:
"The libpcap format is very simple, one of the reasons that it has gained such a wide usage. Unfortunately it misses a few things which would be helpful:
    nanosecond time resolution
    user comments: "shows connection breakdown starting at packet 1432"
    interface information (like the network card manufacturer)
    packet drop count (and probably other counts as well)"

I don't find any of these to be showstoppers for this proposed use. But I go back to my original premise that I'm struggling to figure out why there's a need to limit the file format.
Barbara

> >
> > -- Juliusz
> >
> > _______________________________________________
> > babel mailing list
> > [email-masked]
> 
> Mahesh Jethanandani
> [email-masked]

---

From: Juliusz Chroboczek <[email-masked]>
Date: Thu, 03 Jan 2019 17:46:20 +0100

> I will not be surprised if Wireshark does not understand Babel protocol,

Both tcpdump and Wireshark understand Babel.  That was one of the first
things we did after we got our first working prototype.

> particularly if no one has defined it in libpcap.

That has nothing to do with libpcap -- libpcap handles Babel packets as it
would any other IPv6 packets, interpreting the application-layer protocol
happens in the application (Wireshark or tcpdump).

-- Juliusz

---

From: Juliusz Chroboczek <[email-masked]>
Date: Thu, 03 Jan 2019 17:54:12 +0100

> Why not?

Suppose that I want to write a management tool for Babel routers.  I need
a small, well-defined set of dump formats that I need to implement in my
tool and be sure that I can grok the output of all correct implementations.

> Personally, I'm fine with anything that I can get an easy-to-use, free,
> good parser for and that can provide all the info.

Right.

> I don't find any of these to be showstoppers for this proposed use. But
> I go back to my original premise that I'm struggling to figure out why
> there's a need to limit the file format.

If we want tool authors to write tools that parse the format, we need to
mandate just one or two formats.  So I'd suggest one of the following:

  1. mandate pcap;
  2. mandate pcapng;
  3. allow both pcap or pcapng.

Allowing more than two formats is a bad idea IMHO.

-- Juliusz

---

From: Juliusz Chroboczek <[email-masked]>
Date: Thu, 03 Jan 2019 20:09:00 +0100

> I vote one format for implementation simplicity.  I do not really know
> which one.

pcap is much simpler.

pcapng supports nanosecond timestamps, comments, vendor-defined TLVs, and
bidirectional scanning.  While it looks reasonably easy to generate, it's
a little more challenging to parse in its full generality.  Heck, even
Wireshark doesn't :-/

(Note that picking a dump format doesn't mean we can parse it -- there's
the small matter of the encapsulation used, which could be 802.2,
radiotap, raw IPv6, or something else.  Do we need to specify that?)

-- Juliusz

---

From: "STARK, BARBARA H" <[email-masked]>
Date: Thu, 3 Jan 2019 22:42:36 +0000

> > I vote one format for implementation simplicity.  I do not really know

OK. And by "pcap", we mean libpcap per https://wiki.wireshark.org/Development/LibpcapFileFormat? 
I'm fine with that.  
"The libpcap file format (https://wiki.wireshark.org/Development/LibpcapFileFormat) with ".pcap" file extension MUST be supported for message log files."? Note that supporting message log files is optional, so this is conditionally mandatory.
 
> pcapng supports nanosecond timestamps, comments, vendor-defined TLVs,
> and bidirectional scanning.  While it looks reasonably easy to generate, it's a
> little more challenging to parse in its full generality.  Heck, even Wireshark
> doesn't :-/
> 
> (Note that picking a dump format doesn't mean we can parse it -- there's the
> small matter of the encapsulation used, which could be 802.2, radiotap, raw
> IPv6, or something else.  Do we need to specify that?)

Encapsulation is the job of the management protocol. The info model will allow the file to be exposed either as a URL (in which case encapsulation is specified by that URL; e.g., "https:\\<router IP address>/mylogfiles/babellog.pcap") or as a file known to the info model that can be gotten using a management protocol operation.

Barbara

---

From: "Joel M. Halpern" <[email-masked]>
Date: Thu, 3 Jan 2019 17:55:06 -0500

Is that URL a stable referent?  Or might the wireshark folks change what 
is documented there (as they evolve).
For an RFC, particularly for something we are mandating, the definition 
has to be stable.

Yours,
Joel

On 1/3/19 5:42 PM, STARK, BARBARA H wrote:
> https://www.ietf.org/mailman/listinfo/babel
>

---

From: "STARK, BARBARA H" <[email-masked]>
Date: Fri, 4 Jan 2019 14:24:51 +0000

> Is that URL a stable referent?  Or might the wireshark folks change what is
> documented there (as they evolve).

They've been stable for 3 years and seem to understand that changing the "libpcap" format would not be a good thing because of its support in more than just wireshark. Rather, they've tried to evolve by defining a new format (pcapng).

> For an RFC, particularly for something we are mandating, the definition has to
> be stable.

Conditionally mandatory isn't quite the same as mandatory. The condition is "if creating a message log is supported". Supporting creation of a message log is optional. Also, there's a difference between (conditionally) mandatory to use vs. support. I'm only suggesting mandatory to support this file format. I think implementations need to be free to support (and use) other file formats, as well. 

But, I agree that stability is desirable.
As I see it, options are:
1. Decide anyway that the Wireshark reference is good enough for this purpose.
2. Define a different format (won't happen; not a real option)
3. Use a different already-defined stable format (doesn't exist; not a real option)
4. Copy the libpcap format definition into this draft (with proper attribution to Wireshark; would we need to get permission from Wireshark to copy it?)
5. Remove any and all suggestions that it might be possible for a message log to be generated (while I find message logs to be valuable in trouble-shooting and am aware of many devices that successfully provide such logs with apparent ease, I'm starting to think it's more trouble than it's worth to include one as an optional element in this info model)
6. Leave the optional message log but remove all mention of how to format the file (anybody who supports the message log is likely to use libpcap anyway, and this way there's no potentially-unstable references or plagiarized content in an IETF RFC; there is the danger of lack of interoperability)

I'm open to other options. So long as they're simple. If they aren't simple, I think option 5 is the best approach. I'm not willing to put a lot of effort into this.

Barbara

> >>> I vote one format for implementation simplicity.  I do not really
> >>> know which one.
> >>
> >> pcap is much simpler.
> >
> > OK. And by "pcap", we mean libpcap per <url>
> > 
> > I'm fine with that.
> > "The libpcap file format
> (<url>) with ".pcap" file extension MUST be supported for message log
> files."? Note that supporting message log files is optional, so this is
> conditionally mandatory.
> >
> >> and bidirectional scanning.  While it looks reasonably easy to
> >> generate, it's a little more challenging to parse in its full
> >> generality.  Heck, even Wireshark doesn't :-/
> >>
> >> (Note that picking a dump format doesn't mean we can parse it --
> >> there's the small matter of the encapsulation used, which could be
> >> 802.2, radiotap, raw IPv6, or something else.  Do we need to specify
> >> that?)
> >
> > Encapsulation is the job of the management protocol. The info model will
> allow the file to be exposed either as a URL (in which case encapsulation is
> specified by that URL; e.g., "https:\\<router IP
> address>/mylogfiles/babellog.pcap") or as a file known to the info model
> that can be gotten using a management protocol operation.
> >
> > Barbara
> >

---

From: Juliusz Chroboczek <[email-masked]>
Date: Wed, 09 Jan 2019 14:15:05 +0100

> allow the file to be exposed...

That's not what I meant.  Pcap and pcapng contain a sequence of frames --
these frames can be using a number of different encapsulations.

Here are the cases I'm familiar with:

  - pcap can contain a sequence of Ethernet frames;
  - pcap can contain a sequence of 802.2 SNAP frames;
  - pcap can contain a sequence of radiotap frames which themselves
    contain whatever is carried by the 802.11 link (typically 802.2 SNAP).

Is it legal for an implementation to expose, say, PPP frames in the pcap
file?  Do we need to formalise the allowed encapsulations contained within
the pcap file, or are we going to assume implementers do something
reasonable?

(I'd argue in favour of being liberal here -- being restrictive might
cause issues if at some point Babel is carried over exotic link layers,
and after all one of Babe's strenghts is that it's link-layer-agnostic.
But I think it deserves discussion.)

-- Juliusz

---

From: "STARK, BARBARA H" <[email-masked]>
Date: Wed, 9 Jan 2019 14:21:02 +0000

> > Encapsulation is the job of the management protocol. The info model
> > will allow the file to be exposed...
> 
> Is it legal for an implementation to expose, say, PPP frames in the pcap file?
> Do we need to formalise the allowed encapsulations contained within the
> pcap file, or are we going to assume implementers do something
> reasonable?
> 
> (I'd argue in favour of being liberal here -- being restrictive might cause issues
> if at some point Babel is carried over exotic link layers, and after all one of
> Babel's strengths is that it's link-layer-agnostic.

Ah. I understand now (I hope). I see no need to be restrictive. The data link type is specified in the file header (using values from http://www.tcpdump.org/linktypes.html).  I think it's the responsibility of whatever is reading the file to be able to comprehend the link types of the Babel deployment. If you deploy Babel in an exotic environment, you need to have the tools that understand your exotic environment. I'm often accused of being liberal.