Subject: RE: [Rfid] XML vs. Text vs. Binary

From: "David Husak" <[email-masked]>
Date: Thu, 21 Jul 2005 19:56:22 -0400

First of all, let me say that I have no axe to grind wrt a text (I'll
lump XML into this category for now) vs. a binary protocol. I just want
a solution that's simple, unambiguous and efficient-enough to be
implemented on a lightweight reader.

That said, I do think that there are important factors to consider apart
from the code-size and protocol-overhead-on-the-wire issues*:

1. Direct-access vs. TLV-binary vs. sequential-text-parsing. 

In a compact, fixed binary encoding, it's easy for an implementation to
access protocol fields directly via indexed load and store, i.e. an
implementation need only access the bytes it needs, and it knows where
those bytes are. 

In a type-length-value binary encoding, an implementation can hop from
parameter to parameter by accessing only the parameter type and length
bytes. 

A text-based encoding, where parameters are typically delimited by
special character sequences, requires accessing all bytes until you find
the parameter of interest. **

I apologize for the pedagoguery of the foregoing, but I wanted to
illustrate the fact that the amount of processing and attendant latency
and latency-variation in generating and interpreting these messages can
vary considerably (from a few instructions to a few thousand
instructions) depending on the approach chosen. I'll suggest that the
latency variation incurred by a text-based encoding, especially on a
modest processor, is a real problem in the context of a protocol, like
SLRRP, with real-time control requirements.

For SLRRP, we chose the TLV-binary approach as the best compromise
between flexibility and efficiency. 

2. Utility of human-readability in the context of SLRRP.

I'll grant that for application protocols like FTP, there's real utility
in having an encoding that's human-readable and -usable, because it's
reasonable to expect a human or equivalent may regularly exercise the
protocol, and may need to do so with readily-available tools. (Though,
even in those cases the protocol is front-ended by a CLI that has a help
facility, etc.)

I think that the likelihood of SLRRP being successfully used in this
manner is very close to zero, due to the sub-second real-time nature of
the task-at-hand (reading tags in a dynamic environment) at the
layer-at-hand.

One must also recognize that a large fraction of the parameters that are
controlled in SLRRP are not numbers, they are selectors or enumerated
types, which makes it even less likely that an engineer typing UTF-8
SLRRP commands over a telnet connection can get it right without some
help/cheat-sheets/tools, in which case the value of the text encoding is
diminished.

One could argue that human-readability might be useful for debugging
during product development, early trials, etc. Maybe, but you'll
probably have or want a SLRRP codec for your sniffer in the lab
anyway... 

Another advantage of a text-based encoding that's generally acknowledged
is the possibility of using text-editing tools to manipulate and compare
the protocol contents, usually in the context of configuration models
that permit bulk dumps and restores of configuration info. This
advantage also does not seem to apply in the present case.

Net of all of that, in the present case, I just don't see that the
benefit of human-readability outweighs the cost.

3. Other layers

Which brings me to an important point, which is that many of the
text-based encoding advantages do indeed apply at higher-layers, for
example the ALE layer in the EPCglobal framework. Text-based encoding is
almost certainly the right choice at that level of abstraction, on up.

4. Management and configuration interface

At the risk of crossing the streams, it's important to recognize that
SLRRP is not a management and configuration protocol. It is a control
and data transport protocol for a real-time data acquisition device. The
requirements and the correct choice of encoding may/will vary when we
consider the two functions.

It may well be the case that we decide that the right choice for the
management or configuration interface at the SLRRP level is text-based,
and therefore we've already got the text- or XML-parser, so, modulo the
performance impact, we should just use it for the SLRRP path as well.
Maybe.

BTW, RFC3535 is a good read and appears to be quite comprehensive and
compact. Though, I'll point out the fact that SNMP creates the
requirement for specific client applications, was rated a 'neutral' by
the task force, and was not characterized as a "significant barrier" to
its use. There are, of course a number of well-understood SNMP
'minuses,' but this isn't one of them.

I must say, that based on the cursory description in RFC3535, COPS-PR
seems like it has some really interesting properties for managing RFID
reader networks. Unfortunately, it seems like, at least as of May 2003,
it had an uncertain future.

Dave

----------------------------

*I agree that these are non-issues. Memory is really cheap. And the only
thing cheaper than memory is bandwidth in an enterprise LAN.

**As an aside, the example of encoding the number '3' being more
efficient in text vs. as 32-bit integer is a bit specious. Obviously, if
you've allocated 32-bits to the field, you're also probably interested
in expressing numbers with more than four digits, in which case the
binary encoding is more compact... even before accounting for the fact
that the text-encoded '3' really looks something like
startdelimiter+parameterID+'3'+enddelimiter.

------------------------
David J. Husak
Reva Systems Corporation
[email-masked]

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]]
On
> Behalf Of Margaret Wasserman
> Sent: Wednesday, July 20, 2005 10:19 PM
> To: [email-masked]
> Subject: [Rfid] XML vs. Text vs. Binary
> 
> 
> We've had some discussion on this list about XML encoding vs. binary
> encoding...
> 
> I'm not sure that we are in agreement on all of the related points,
> but the concerns with XML seem to be code size on the reader and
> protocol overhead on the wire.  These concerns can be minimized by
> the use of a restricted XML subset, such as canonical XML (see RFC
> 3076).  However, there still seem to be some concerns along those
> lines.
> 
> The only advantages that I remember being discussed for a binary
> encoding were the complement to the concerns with XML:  smaller code
> size on the reader and less protocol overhead.
> 
> I think, though, that our discussions have missed a major benefit of
> any text-based encoding (whether a protocol-specific text encoding or
> canonical XML):  the ability to access the device using text
> processing tools (such as Perl scripts) and/or for a human to
> interact with the device directly.
> 
> In 2002, the IAB held a Network Management workshop that is
> documented in RFC 3535.  I would suggest that folks on this list read
> this report which can be found at:
> 
> http://www.ietf.org/rfc/rfc3535.txt?number=3535
> 
> One of the interesting findings of that workshop was that one of the
> significant barriers to the use of SNMP as a configuration or control
> protocol is that it uses a binary encoding, which means that
> SNMP-specific client software (an SNMP browser or manager) is needed
> to interact with the device via SNMP.  This prevents SNMP access
> using text processing tools or via human interaction.
> 
> I would not like to see the industry create the same problem with an
> RFID control protocol.
> 
> If there is real evidence that canonical XML would require too much
> code size on the reader and/or would result in an unacceptable level
> of protocol overhead, perhaps we could consider a protocol-specific
> text-based encoding (similar to FTP, SMTP and HTTP)?  I don't believe
> that parsing a well-defined text-based encoding would require much
> more code than processing a binary encoding.  And, in some cases a
> text based encoding would actually result in less data on the wire --
> for instance a 32 bit integer with the value of 3 would be encoded in
> one byte of text ("3"), while it would require 4 bytes in binary
> encoding ("0x00000003').
> 
> Personally I like canonical XML, because I think it strikes a good
> balance between being human readable and machine-parsable.  I also
> like the fact that the syntax is already well-defined.  However, I
> think that well-defined, protocol-specific textual encodings could
> also achieve that balance, perhaps with less impact in the areas of
> concern (code size and protocol overhead).
> 
> What do others think?  Should we at least consider a text-based
encoding?
> 
> Margaret
> 
> 
> 
> _______________________________________________
> Rfid mailing list
> [email-masked]
> https://www1.ietf.org/mailman/listinfo/rfid

_______________________________________________
Rfid mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/rfid

---

From: Margaret Wasserman <[email-masked]>
Date: Thu, 21 Jul 2005 21:05:10 -0400

Hi David,

At 7:56 PM -0400 7/21/05, David Husak wrote:
>I'll suggest that the

We've been around this block before, but without reaching a clear 
answer.  What are the real-time requirements for RFID reader 
control/management?  And, what are the minimum system requirements 
that we need to consider?  If we can't answer this for the entire 
industry, I think we at least need to reach a mutual understanding of 
the range of devices that we are planning to support with this 
protocol.

The stringency of the real-time requirements, and whether they exist 
at all, may be a function of the paradigm that we use for reader 
control/management.  As I stated in my earlier message, I would 
prefer a paradigm where a reader operates autonomously based on 
configured filters and policies, similar to an IP switch or router. 
I think that such a paradigm would be much more practical, much more 
scalable and much more in-line with the RFID reader applications I've 
seen, than a scenario where a large set of readers are micro-managed 
on a real-time basis by a centralized network controller.

Most of your real-time and scaling requirements seem to derive from 
your fundamental assumption that there will be a single controller 
providing real-time control for 1000's of readers, and I see no 
reason to build our protocol (or an RFID network) so that a single 
real-time controller is necessary in large-scale installations.

>and data transport protocol for a real-time data acquisition device.

If this is the case, why do you believe that the IETF has any special 
expertise to offer here?  We don't have any experience (that I'm 
aware of, anyway) in developing real-time control protocols for data 
acquisition devices (although I'm sure that someone has used SNMP for 
this purpose).  We generally develop higher-level management and 
control protocols for devices (almost exclusively network 
infrastructure devices) that perform their main functions 
independently based on filters and policies, without consulting a 
controlling entity in real-time.

I think that RFID readers could be controlled and managed at the same 
level as most networking equipment, perhaps using some of the same 
mechanisms.  You seem to thinks so, too, since you mentioned that 
COPS-PR could be an interesting solution in this space.

IMO, though, it's not consistent to argue that these devices will be 
controlled and managed like networking equipment (which is the 
primary argument for why the IETF has any pertinent expertise here), 
AND to argue that these will be very minimal devices (without enough 
processing power or memory to deal with a simple text encoding) that 
will be controlled top-down with all of their real-time decisions 
made by a network controller.  When you add the notion that the data 
and control may be communicated over a low-bandwidth serial link, you 
have completely departed from any territory in which the IETF has any 
special expertise, as far as I know.

So, which is it?

Regarding the need for specialized clients...  I understand that you, 
as a middleware vendor, prefer a worldview in which people purchase 
commercial middleware products to access RFID readers (or run 
substantial open source packages).  As a reader vendor, I obviously 
don't want my product to require a high-priced commercial middleware 
package to be useful.  And, today we certainly see a large number of 
customers who prefer to write their own clients to access RFID 
readers, often using text processing languages like Perl, or even 
shell scripts.

Even in installations that use middleware, it could be extremely 
valuable to access the reader interface directly using test and 
debugging scripts, in order to determine if there is a reader failure 
or if the problem lies elsewhere in the system.

So, I will restate my opinion that we should not lightly develop an 
RFID reader management protocol that can only be access from a 
specialized client.

To your point about enumerated types, etc.... There is no reason why 
a text interface couldn't allow well-defined strings to be used for 
those enumerations (EPC0, EPC1, EPCG2, etc...), rather than requiring 
the client (whether it is a piece of middleware, a Perl script or a 
human typing at an SSH prompt) to send numerical values.

Margaret

---

From: Marshall Rose <[email-masked]>
Date: Fri, 22 Jul 2005 08:17:40 +0530

> RFID reader management protocol that can only be access from a  

i guess we'd need to agree on the meaning of specialized before  
agreeing on this point.

> To your point about enumerated types, etc.... There is no reason  
> why a text interface couldn't allow well-defined strings to be used  
> for those enumerations (EPC0, EPC1, EPCG2, etc...), rather than  
> requiring the client (whether it is a piece of middleware, a Perl  
> script or a human typing at an SSH prompt) to send numerical values.

the problem, i think, is that the XML compromise, like most  
compromises, doesn't really solve anything; further, like most  
compromises, it works worse than most purist approaches.

it's certainly better than binary if you're going to type-in commands  
via telnet or ssh. the moment you have to start matching elements,  
and are typing more than a few of them, you need to have a tool do  
the typing.

as soon as you have tool do it, you've lost the text advantage  
because it's just as easy for a tool, perl-based or not, to spit out  
binary as xml.

my experience, which admittedly is dated, is that text works for  
debugging simple protocols like smtp. as soon as you introduce  
nesting, you lose the type-in advantage.

/mtr

ps: people who complain about the binary nature of snmp are actually  
complaining, whether they know it or not, about ASN.1/BER, which  
define how to describe and encode SNMP packets. i really should have  
listened more to chuck davin 20 years ago...

---

From: Stephane Bortzmeyer <[email-masked]>
Date: Fri, 22 Jul 2005 10:19:06 +0200

On Thu, Jul 21, 2005 at 07:56:22PM -0400,
 David Husak <[email-masked]> wrote 
 a message of 213 lines which said:

> In a compact, fixed binary encoding, it's easy for an implementation
> to access protocol fields directly via indexed load and store,
> i.e. an implementation need only access the bytes it needs, and it
> knows where those bytes are.

#define WARNING "newbie"
 
I've read quite often this statement in IETF discussions. I am not an
implementer but I wonder if this idea of blindly retrieving bits
coming from the newtork is really what the implementers do. Because it
seems there is a security issue here: since the bits come from a
possibly untrusted source (either because it is malicious or because
it is buggy), you surely validate them in some way, first, and
therefore you have to parse the incoming data, no?

---

From: Margaret Wasserman <[email-masked]>
Date: Fri, 22 Jul 2005 10:42:04 -0400

Hi Marshall,

>i guess we'd need to agree on the meaning of specialized before 

Yes, absolutely.

The distinction I would make is that "specialized" == written 
specifically for this protocol.

A TCP/IP stack, an SSL client and a canonical XML parser aren't 
specialized, because those are widely deployed technologies that are 
used for things other than this specific reader control protocol. 
They are already available for most OS's and HW platforms, there are 
embedded and non-embedded versions of them available, etc.

A parser for a SLRRP-specific text or binary protocol and/or a 
library that understands how to break higher level functions into 
specific SLRRP commands would be "specialized", by my definition.

As an example...
If SNMP's subset of ASN.1 and/or it's BER encoding had been more 
widely used, then there might have been widely deployed, 
non-specialized tools that could encode/decode SNMP PDUs (there 
aren't, unfortunately).  However, SNMP's PDU structure is specific to 
SNMP and requires a specialized tool to parse incoming messages 
and/or format outgoing messages.

IMO, the more we can leverage non-specialized, widely deployed 
technology (such as TCP/IP, SSH or TLS, XML, etc.) on both the client 
and reader sides, the better.  People will be able to build clients 
and/or readers that are stable and robust in much less time if they 
can use tools/libraries for this purpose that have already been 
written and debugged.  It lowers the barrier for entry and (most 
importantly, IMO) makes it possible for anyone with basic Perl, 
Python or Ruby programming skills to interact with a reader using our 
protocol and widely-available tools and libraries.

My preference for using canonical XML is based on three things:

(1) I think that we should focus only on network-connected RFID 
readers.  There are very limited, low-cost DSP-only readers available 
(including one from ThingMagic), but these are embedded modules that 
are accessed over serial lines from a host CPU, and they are included 
in devices like label printers and box folders.  If it is necessary 
to control the RFID reader portion of these devices over the network, 
that will have to be done through the host CPU, anyway.  It isn't 
practical to put TLS/TCP/IP on those embedded devices, so the fact 
that it isn't practical to run an XML parser on those devices is 
moot, IMO.

(2) There are embedded (or embeddable) XML parsers available (like 
expat) that can parse canonical XML using a SAX parser.  These 
parsers are neither CPU-intensive nor memory-intensive compared to a 
TCP/IP stack, DHCP, TFTP (or FTP or both), NTP, a DNS resolver, HTTP, 
TLS, OpenSSL (or your favorite encryption library), some web pages, 
SSH (or Telnet?), some command-line management/configuration tools, 
and all of the code necessary to implement EPC Global's Gen2 
specification including the anti-collision and dense reader mode 
features. I fully expect that all of this (and maybe more -- SNMP, 
MIB-2 and RFID MIBs?) will be available on every network-accessible 
Gen2 RFID reader.

(3) I do not believe that we should design RFID networks so that 
there is a real-time requirement for communication of control data 
between a controller and many readers.  But, even if we did, I cannot 
picture a world in which the XML parsing is a big latency factor in a 
system that involves network transmission of control information over 
TLS/TCP/IP on the same network that is used for the data 
communication.

I understand, though, that some people do not find XML acceptable 
because of its bandwidth requirements...  If that is a real concern 
(and I'd like to see some explanation of why it would be a real 
concern), then I'd like to see if we could address that concern 
through _how_ we encode the information in XML, or through a 
specialized text encoding.  Either approach would still maintain some 
of the benefits of XML -- like the ability to interact with the 
reader using text-processing tools without using a specialized (as 
defined above) parser on the client side.

Margaret

---

From: Marshall Rose <[email-masked]>
Date: Sat, 23 Jul 2005 09:16:35 +0530

margaret - apologies, for not quoting from your email, but i think  
the issues i raised in my original reply remain untouched, or perhaps  
are now amplified by your reply.

the problem is agreeing on a subset is that we now have to design/ 
write/debug/use an entirely different set of libraries than the  
"standard" ones. it takes both hands to count the number of different  
encoding rules that have been defined/implemented for use with ASN.1  
over the years. none of them achieved anywhere near the usage of the  
original BER, which was the thing that folks were complaining about  
to begin with.

in other words, the compromise becomes more complicated than either  
purist extreme; it  looks like one of those little white lies one  
tells in polite company that, over the course of the comversation,  
have to become increasingly more complex so as to avoid discovery  
through contradiction or non-sequitor.

this brings us back full circle: as soon as you have any level of  
nesting, human type-in becomes problematic. as soon as you decide  
that human type-in isn't mandatory, it is trivial to include a  
standard library to do the heavy lifting while the humans invoke the  
tool using textual commands.

in other words, from where i sit, XML, cXML, etc., enjoy all the  
drawbacks of both purist approaches.

/mtr

---

From: Margaret Wasserman <[email-masked]>
Date: Sat, 23 Jul 2005 09:14:45 -0400

Hi Marshall,

I think we are mostly in agreement about the technical trade-offs, 
but for some reason that doesn't lead us to the same conclusion...

>tool using textual commands.

Yes, if there is any significant nesting, I agree that a human will 
not be able to type the commands in from memory and/or by glancing at 
a reference sheet.  However, I don't see any reason why this protocol 
would require significant nesting, at least for the types of commands 
that a human is likely to use directly.

With a text encoding, a human could also have certain prepared 
commands and be able to cut and paste them into the interface -- a 
command to reset autonomous operation, for example, or to set a 
reader to its default state.  Or they could write simple scripts that 
would send specific commands or command sequences.

>drawbacks of both purist approaches.

Correct, from a "human typing" standpoint, XML, cXML and a 
specialized text encoding are all roughly equivalent.  However, in my 
opinion, they are all much better than a binary encoding.

There are two factors involved here, and all three of these choices 
(XML, binary or specialized encodings) are fundamentally different 
WRT these factors:

(1) Can the incoming stream be parsed by a widely available parser, 
or is a specialized parser necessary.  XML parsers are available for 
all likely client systems, but a specialized text encoding or  a 
binary encoding would require a specialized parser.  BTW, subsetting 
XML _does not_ require a specialized parser.  Reader vendors might 
want to includes a specialized (smaller or faster) parser and simply 
reject XML constructs that are not included in the subset, but 
clients could use a regular XML parser for this purpose.

(2) Can text processing tools be used to send control messages and 
interpret the responses, or would binary processing be required. 
Both XML and a specialized text encoding would allow text processing 
of the messages and results, while a binary encoding would require 
binary processing.

You've indicated that it is possible, with a binary encoding, for 
clients to run a tool that gives them text-based access.  That's 
true, but users would need to _get_ that tool from somewhere...  So, 
we are back to requiring specialized code on the client side to 
access this protocol.

This isn't a religious issue with me -- I've just learned the 
benefits of text-based encoding the hard way, and I don't want to 
deal with another binary protocol...  So, for the moment, we may have 
to just agree to disagree.  We'd obviously need to resolve this 
before we can move ahead, though.

Only a few of us have expressed any opinion on this subject. Are 
there others out there that have an opinion?  There are two questions 
that I'd especially like to hear answered by specific other people on 
this list:

(1) I'd like to hear from other reader vendors about whether they 
think that XML would place a prohibitive burden on their readers. It 
has been asserted that XML would have prohibitive system requirements 
for the reader.  That certainly isn't true of ThingMagic's networked 
readers.  In fact, using a standard parser, like XML, would reduce 
our development (especially debugging) and testing costs 
substantially, and would make it more likely that we would implement 
this protocol.

(2) It would be very interesting to know if RFID users would see any 
benefit to a text-based encoding.  Are there folks on this list who 
have RFID installed in real-world or even pilot installations (not 
just a few units in a test lab)?  Do you have any opinions about 
whether the ability to access the reader using text-based tools from 
a client system that doesn't have a specialized client (or library) 
installed would be useful?  Or do you see this as having little/no 
value?

Margaret