Subject: Retrieving large DNS objects

From: "Anders Rundgren" <[email-masked]>
Date: Sun, 23 Jan 2005 10:17:53 +0100

Slightly related to XML but also applies to schemes like DomainKeys
as a public key indeed is a "fat" object.

I'm NOT pretending to be an expert on this topic (or DNS at all actually)
so I humbly try to put the facts (or issues) on the table.

Historic background:
When DNS was created network resources were scarce.  Currently people are
downloading entire movies over intercontinental Internet connections.  That is,
2005 <<>> 1987.

Trend:
That it is possible to save bytes by using "smart coding" is indeed true but have
lately become out of fashion.  The reason for that is that data objects today
often are much more complex than they were in the early days of the Internet.
In addition, the "environment" has become more volatile which means that
revisions are more and more frequent.

Consequences:
Put together these things have more or less united the IT industry to use objects
that are [to some extent] self-describing and can be parsed automatically.  This
have lead to XML and XML Schema respectively.  Using XML Schemas
you would preferably assign a new namespace to a revised data type so
that there would be no question about what you actually are dealing with.
Using XML Schemas allows BOTH ends to auto-parse BTW!

Efficiency:
XML and other textual representations cannot be accused of being efficient.  There
is often a 200-500% overhead for the simple stuff (integers, flags) and a 40% for
blob data (BASE64).

THE QUESTION:
Does this (fat-object) paradigm fit DNS?
A majority of the NameDroppers seem to say no.

Here is a list of workarounds for people that would like to indeed move
the DNS infrastructure a bit closer to the "other world" (Web Services).

1.
according to Paul Vixie:
   after EDNS0, the foundations are just fine, thanks.  "more weight."

But the namedroppers seem skeptical about this and even claim that
firewalls don't accept this.  Hard to know for me.

2
Another possibility could be:
You retrieve the authoritative nameservers and in turn call
these using TCP.  That is, you don't put any major load on
the infrastructure at large only on the server you have a
contract with or run on your own.  Is that a viable scheme?

BTW, I have "ordered" an XML RR from IANA.  Putting the
famous process to test.... :-)

br
Anders rundgren

--
to unsubscribe send a message to [email-masked] with
the word 'unsubscribe' in a single line as the message text body.
archive: <http://ops.ietf.org/lists/namedroppers/>

---

From: Paul Vixie <[email-masked]>
Date: Sun, 23 Jan 2005 16:29:24 +0000

> Does this (fat-object) paradigm fit DNS?

no.

> A majority of the NameDroppers seem to say no.

and rightly so.

dns is a udp app, probably the most popular udp app in the wide area (though
on LANs, nfs and smb are going to be more popular).  since the guaranteed
minimum maximum MTU has not changed (576 for V4) and is still small (1280 for
V6), any udp app for the wide area should be trying to avoid fragmentation.

that means dns will never be useful for fat objects.  don't dispair, though:
there are a lot of thin/dense objects whose need for a reliable, autonomous,
distributed, hierarchical database is met perfectly by dns, or which will be
met perfectly by secure dns.

> firewalls don't accept this.  Hard to know for me.

some firewalls don't allow it.  these are not the norm, and when faced with
a bug report about it, firewall vendors and operators fix the problem --
dns sails on.  the fingerpointing is always toward the violator of the spec.

> contract with or run on your own.  Is that a viable scheme?

no.  tcp requires state.  state quotas are a vulnerability in the face of
a DoS attack.  tcp must remain a fallback for queries, and be reserved in
the common case for updates and zone transfers.  doesn't matter who runs
the servers, if they're supposed to stay "up" then you can't depend on
them to have room for another tcp protocol control block in the common case.

> famous process to test.... :-)

i hope they turn you down.  XML would be as bad as TXT.  without subclassing,
every consumer would have to retrieve an arbitrarily large rrset, which is
not "the dns way" to do this.  note that RP and MX share an rdata format but
are used by different applications.  some new application-specific rrtypes
should be defined as having an XML payload, probably carried in a TXT-like
rdata format.  but the rrtype should be be by application, not rdata format.

---

From: John Levine <[email-masked]>
Date: 23 Jan 2005 16:36:20 -0000

>You retrieve the authoritative nameservers and in turn call these
>using TCP.  That is, you don't put any major load on the
>infrastructure at large only on the server you have a contract with
>or run on your own.  Is that a viable scheme?

Seems pointless to me.  If you know you're going to find a server
using traditional UDP DNS, open a TCP session, give it a key, and it
gives you back a blob of XML, we already have a widely deployed scheme
called http to do that.  It's got caches, alleged session security,
content negotiation, all the trendy bells and whistles.

What would be the advantage of shoehorning the XML into DNS?

-- 
Regards,
John R. Levine, IECC, POB 727, Trumansburg NY 14886 +1 607 330 5711
[email-masked], Mayor, http://johnlevine.com, 
Member, Provisional board, Coalition Against Unsolicited Commercial E-mail

---

From: "Anders Rundgren" <[email-masked]>
Date: Sun, 23 Jan 2005 18:10:03 +0100

>no.

Not even 4K seems OK?

>i hope they turn you down.

Well, then we can dismiss iab-dns-choices right now.
From DOA to EIP :-)

>XML would be as bad as TXT.  without subclassing,
>every consumer would have to retrieve an arbitrarily large rrset

What does this really mean?  You get ALL TXT records and it is
the clients who do the actual select?

If that is NOT the case (which I doubt as I cannot find any support for
that in RFCs) the "structured prefix/suffix" is an easier solution.
Avtually you HAVE to use this for NAPTR etc. as
"all NAPTRs are not equal".

So if we anyway must prefix/suffx why not use that all the way?

Anders R

---

From: Edward Lewis <[email-masked]>
Date: Sun, 23 Jan 2005 12:13:12 -0500

>Does this (fat-object) paradigm fit DNS?

No.

It's hard to quantify in what way this proposal is wrong.

Applications have needs to search through and for data that is 
complex, dynamic, and large.  The DNS does not provide this and it 
shouldn't.

"Closed communities" and "open communities" are very different and 
have different needs.  The "science" differs between closed and open 
communities.  But there is a way to melt the two together.  The basic 
strategy is to create "lowest common denominator" (LCD) services for 
the open community and let the closed communities build upon that.

A community may be a company or it may be an application. 
Communities need not be topologically related.  This drives the need 
for a LCD open community tools.  This is why there is resistance to 
make the DNS too feature-rich.

>contract with or run on your own.  Is that a viable scheme?

No.

There's a reason why UDP is the workhorse transport.  And it would be 
unwise to design the protocol that treats name servers differently. 
(I.e., the DNS is not aware of "contracts.")

>famous process to test.... :-)

Hope you followed the process documented in RFC 2929 and RFC 2434.
-- 
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Edward Lewis                                                +1-571-434-5468
NeuStar

"A noble spirit embiggens the smallest man." - Jebediah Springfield

---

From: "Anders Rundgren" <[email-masked]>
Date: Sun, 23 Jan 2005 18:43:56 +0100

>What would be the advantage of shoehorning the XML into DNS?

To "find the server" is incorrect.
1. You find the nameserver using standard methods.
2. Now the trauma comes.  how do you identity this unique
object you are looking for?

Your solution:
goto IANA and get an RR.
wait an indefinite time for MSFT to support that in their admin GUI.
If you don't represent a major manufacturer/app you will NEVER get any support.
To *force* people to abandon these tools is a mission that I take no
interest in, irrespective of what I think of Microsoft.  Unless you find
a pleasure in creating things that can only be used by a handful
experts please do that but don't expect everyone else to follow.

Other solution:
1. Define a suitable RR ONCE.  The XML RR.
That *may* get MSFT support.  By using XML Schema import you
can create a "subclassed" GUI admin for *app-defined* XML data.
Does input validation and syntax check in spite of not knowing
what it is.

======================================
IANA and the IETF have nothing matching in the pipe-line
======================================

2. Use a structured prefix (created automatically if you have the
schema at hand) needed to access to your particular XML object.
This object may indeed indirect to http or be direct.  Which method
you end-up using has probly quite a bit with the app to do.
A dynamic thing will use http.  A static one will use direct data unless
it is really huge.

Anders

---

From: Paul Vixie <[email-masked]>
Date: Sun, 23 Jan 2005 18:05:12 +0000

> >> Does this (fat-object) paradigm fit DNS?

that depends on whether you have read and whether you agree with the
kent/mogul "fragmentation considered harmful" report, which is online:

<http://research.compaq.com/wrl/techreports/abstracts/87.3.html>

for me, 4K is a fine exception case but on average i think 1500 is
still the right upper bound for wide area udp.  remember that a
fragments of a udp datagram are transmitted back to back with no
congestion awareness (like tcp has.)  congestion awareness requires
state, and statefulness in dns is bad for any number of reasons.

as evidence that i used to be able to do technical work instead of
just arguing on this mailing list, i refer you to RFC 2671, wherein:

| 4.5.1. Note that a 512-octet UDP payload requires a 576-octet IP
|        reassembly buffer.  Choosing 1280 on an Ethernet connected
|        requestor would be reasonable.  The consequence of choosing too
|        large a value may be an ICMP message from an intermediate
|        gateway, or even a silent drop of the response message.

> Well, then we can dismiss iab-dns-choices right now.

honestly, the iab has smart and wonderful people on it, but i don't
wait on them and they don't wait on me.  i'm far more interested in
what rob austein thinks as a BIND9 architect than in what he thinks
as an IAB member.  <flame about ivory tower propellerheads elided>.

> the clients who do the actual select?

in the dark recesses of time, a smart and reasonable guy inside DEC
tried to subclass TXT by making the first "segment" be RFCXXXX and
the second segment be an IANA-assigned application identifier.  in
private communications, i explained that if this led to the existence
of something like

   FOO.DEC.COM IN TXT	RFCXXXX APP1 "whatever app1 is looking for"
		  TXT	RFCXXXX APP2 "whatever app2 is looking for"
		  TXT	RFCXXXX APP3 "whatever app3 is looking for"
		  TXT	RFCXXXX APP4 "whatever app4 is looking for"
		  TXT	RFCXXXX APP5 "whatever app5 is looking for"

then all five apps would have to retrieve the whole rrset and look
for the things they wanted, which would be hellishly expensive if
the number of apps became large or even moderate, or the amount of
data needed by any one app became large or even moderate.  i said
that it was a damned shame DNS didn't have subclassing, such that a
query could actually select a sub-rrset.  i recommended subdomains:

   APP1.FOO.DEC.COM IN TXT	"whatever app1 is looking for"
   APP2.FOO.DEC.COM IN TXT	"whatever app2 is looking for"
   APP3.FOO.DEC.COM IN TXT	"whatever app3 is looking for"
   APP4.FOO.DEC.COM IN TXT	"whatever app4 is looking for"
   APP5.FOO.DEC.COM IN TXT	"whatever app5 is looking for"

and as you all know, this is how SRV was done.  but it doesn't work
well with wildcards since wildcards aren't nonterminal.  if we had
nonterminal wildcards then TXT, and even something like XML (although
XML is just text, and an application wanting XML can just put it in),
to be used as generic containers without pain or inefficiency.

> So if we anyway must prefix/suffx why not use that all the way?

there really ought to be a FAQ on this topic.  i know we've been down this
road any number of times.  i wish i had an online whitepaper or archive
i could point you at.  but, see above.

---

From: Paul Vixie <[email-masked]>
Date: Sun, 23 Jan 2005 18:17:43 +0000

> object you are looking for?

i guess you'd put it in the GET command that you send to the indicated server.

> wait an indefinite time for MSFT to support that in their admin GUI.

that's just silly.  no implementation, not even BIND with 90% or more of
the market, has the power to halt innovation.  we added SRV.  during the
years when MSFT didn't support SRV, folks didn't use MSFT's products to
carry SRV.  eventually the market catches on.  that's how it's supposed
to work, though it's bad that all these dns-aware firewalls and NATs are
refusing to carry opaque rrtypes that they don't recognize.  even those
NAT boxes eventually caught onto SRV though, since the market demanded it.
(and soon we will begin educating the NAT market about DNSSEC metatypes.)

> If you don't represent a major manufacturer/app you will NEVER get any
> support.  To *force* people to abandon these tools is a mission that I
> take no interest in, irrespective of what I think of Microsoft.
> Unless you find a pleasure in creating things that can only be used by
> a handful experts please do that but don't expect everyone else to
> follow.

my old boss (ken olsen of DEC) once wondered why anyone would ever want
to have a computer in their home.  now DEC is gone, because compaq and
HP both knew the answer to that question.  it sounds as if you'd like to
cede control over DNS evolution to a bunch of NAT and firewall vendors.
you won't find a lot of fellow travellers here for that road.

> Does input validation and syntax check in spite of not knowing

i need to check through my personal e-mail archives and figure out who
owes me the beer i bet somebody about when the proposal would finally
come to just redo DNS in XML.

listen, though.  DNS was here 20 years before XML became "the fad", and
DNS will still be here 20 years after XML has been replaced by a dozen
more fads.

---

From: "Anders Rundgren" <[email-masked]>
Date: Sun, 23 Jan 2005 20:36:28 +0100

Paul Vixie wrote:
> <flame about ivory tower propellerheads elided>.
Propellrheads don't ask questions. They know it all by "default".
I try (not successful but anyhow) to suggest and ask to make sure
that the XML RR I-D and URI-typing I-Ds won't be too bad.

>> the clients who do the actual select?

Now it gets interesting!

>well with wildcards since wildcards aren't nonterminal.

I recommend this one:

[arguments.]"unique type id".yourdomain IN TXT "data matching type id" 
(
Although not suitable for every app.  This one work wonders for Web Services
and DomainKeys.  Supports wild-carding also.  As-is.  Like listing
all domainkeys. 

If this variant work for the app in question, I don't see any reason for
getting a specific RR.  At least not with respect to RRset. AFAIK.

>if we had nonterminal wildcards then TXT, and even something like XML (although

There are a lot of things that does not need this but I don't see any
contradictions so let there be more powerful wildcards!

Particularly if that gives us yet another reason for *NOT* using iab-dns-choices...

Anders R

---

From: "John R Levine" <[email-masked]>
Date: 23 Jan 2005 15:16:48 -0500

> object you are looking for?

Oh, my.  I thought the point I was making was so blindingly obvious that I
wouldn't have to spell it out, but I guess I do.

Your thought here is to have a scheme where you hand a service a URI, it
makes a UDP call to find the address of a second server, it contacts that
second server via TCP which gives you back a blob of XML corresponding to
that URI, is it not?  That scheme has already been invented.  It's called
http, it's used all over the world.  It needs no new DNS record types, no
new servers, no new anything, since as you have probably noted, we all use
it now.

You may argue "but it's not in the DNS."  Quite right.  It doesn't need to
be in the DNS.  Since there is no XML data in the DNS (and probably never
will be), no applications expect to do XML lookups in DNS.  They use http,
because that's where the XML is.

Coding more complex data with XML and fetching it with http is a perfectly
reasonable idea.  The IIM message signing proposal fetches per-user keys
that way.  But putting it in the DNS would be pointless and
counterproductive, since http maps URIs to XML right now,

Regards,
John Levine, [email-masked], Primary Perpetrator of "The Internet for Dummies",
Information Superhighwayman wanna-be, http://iecc.com/johnl, Mayor
"I dropped the toothpaste", said Tom, crestfallenly.

---

From: "Anders Rundgren" <[email-masked]>
Date: Sun, 23 Jan 2005 21:31:38 +0100

John,
You did not answer the things that talked in advantage to XML, here thinking
of the GUI and the support you may get from the IT industry supporting
a lot of non-experts (=MSFT).

Yahoo specifies DomainKeys in DNS.  Does this make Yahoo stupid?

XML schemas instead of TXT would add less than 100bytes to
the 200-350 already in place.  Can't see that would kill XML.

Note I don't object at all to use http.  But that does not exclude XML.
IBM have a draft on that although they have managed to make an
XML object that is specific instead of universal.  They will get no
support from the business on this one.  They should have used TXT.

Why did they do a crummy design that goes nowhere? They were afraid that
they would be banned if using TXT.

Anders

----- Original Message ----- 
From: "John R Levine" <[email-masked]>
To: "Anders Rundgren" <[email-masked]>
Cc: <[email-masked]>
Sent: Sunday, January 23, 2005 21:16
Subject: Re: Retrieving large DNS objects

Regards,

---

From: "Olaf M. Kolkman" <[email-masked]>
Date: Mon, 24 Jan 2005 11:07:50 +0100

On Sun, 23 Jan 2005 20:36:28 +0100

>  getting a specific RR.  At least not with respect to RRset. AFAIK.

I do not immediately see how this scheme supports "wild-cards" in the
presence of delegations. An I-D on this idea would actually help to
structure the discussion and would lift this from "mailistware" to
something more tangible [*]

> contradictions so let there be more powerful wildcards!

See:
http://ops.ietf.org/lists/namedroppers/namedroppers.2005/msg00182.html

-- Olaf

[*] I'd reckon that such an I-D would be more in scope for DNSOP than
DNSEXT, but we'll worry about that when its there.

---------------------------------| Olaf M. Kolkman
---------------------------------| RIPE NCC

---

From: [email-masked]
Date: Mon, 24 Jan 2005 17:32:42 +0000

> come to just redo DNS in XML.

	don't think it was beer per-se, but we did have this
	discussion.

--bill

---

From: "Anders Rundgren" <[email-masked]>
Date: Tue, 25 Jan 2005 10:11:33 +0100

>presence of delegations.

To be fully honest; I don't know either as I am not fully aware of the problem
with wild-cards and delegation.  Is it possible to get a minimal info on this?
I would very much appreciate that.

Anyway, the target apps I work with "Web Services", don't really need wild-cards,
they just look for something they know (a specific object) somewhere where they
have been (OOB) told to look (in a partner domain).  So do most public key stuff
as well although they do usually not depend on any OOB info.

> An I-D on this idea would actually help to structure the discussion and would lift
>this from "mailistware" to something more tangible [*]

I don't plan to make an I-D here assuming that IANA will (as mr. Vixie
hopes) turn down the idea adding yet another TXT-like "container" RR.

I guess "whitepaperware" is considered as even worse than "mailistware"?
XML RR whitepaper: http://web.telia.com/~u18116613/XML-RR.pdf

Namedroppers prefer schemes where you first launch a new "DNS-aware"
app using TXT or hijack SRV (or something  equally stupid), and THEN
asks IANA for "granting" you a new RR and THEN IF you are successful with
IANA, ask your "user community" (which you may have no control of at all)
to change something that actually is working after writing an RFC (which
probably document-wise is entirely different from the rest of your app). 
To make it even more appetizing, MSFT probably don't intend to support any
new RR objects in their DNS GUI admin unless it becomes ubiquitous (or is
used by Windows...).  I can't blame them.  I would do exactly the same.
That's why I suggested an alternative route.  It may not fit everybody's need
but that is valid for the just described "solution" as well.

regaords
Anders R

---

From: "Olaf M. Kolkman" <[email-masked]>
Date: Tue, 25 Jan 2005 11:36:15 +0100

On Tue, 25 Jan 2005 10:11:33 +0100

>  XML RR whitepaper: http://web.telia.com/~u18116613/XML-RR.pdf

If you expect serious review of your proposal within IETF context
writing an I-D is the way.  And to be somewhat formal, as long as
"whitepaperware" does not have the appropriate copyright notices it is
out of scope. The main reasons for that are documented in the
introduction of RFC3667 to paraprhase: allow the IETF to take
ownership.

> probably document-wise is entirely different from the rest of your app). 

I do not share that perception. Namedroppers prefers rough consensus,
running code and a stable DNS.

> but that is valid for the just described "solution" as well.

Does this group try to engineer a solid architecture that is truly
interoperable or does it try to hack around implementations that do
not care about the existing standards anyway? ---a rhetoric question--
In the end its about making trade-offs and getting rough consensus on
them. Those trade-offs are best made when they are clearly summarized
and structured in an I-D.

--Olaf

---

From: "Anders Rundgren" <[email-masked]>
Date: Tue, 25 Jan 2005 12:20:17 +0100

Olaf,

I have no problems with the IETF taking ownership.

DomainKeys, SPF, IMM, CSV etc. though all indicate that my
perception of how things actually work differs from IETF's.

Off-list several parties have acknowledged the existence of
this "migration-from-the-interim-to-the-real-solution" trauma.
It is particularly a problem for the commercial sector which
simply doesn't have the time for doing things "the right way".
The so-called right way proponents do not acknowledge the
need for an easier route.   If there was one (well maybe there is...)
it would be judged by entirely different criterions than the right
way is. The question "how do you intend to control that the URIs
are properly used and deployed" is an indication of dual standards,
because it is equally applicable to the "right way" methodology.
The proper answer is of course simply: You cannot.

It is hard to see such information be put in an I-D unless it is
really just a "camouflaged pamphlet" like iab-dns-choices which
carefully avoids mentioning the migration problem.

Advice: Pamphlets are better off in other formats.  :-)

I do understand that real IETFers do not read PDFs.
OTOH, the majority of Internet SW developers have never
read an RFC either...

Anyway, I will keep the list informed on the progress with the XML RR
IANA reg.

Anders

----- Original Message ----- 
From: "Olaf M. Kolkman" <[email-masked]>
To: "Anders Rundgren" <[email-masked]>
Cc: <[email-masked]>; <[email-masked]>
Sent: Tuesday, January 25, 2005 11:36
Subject: Re: Retrieving large DNS objects

--Olaf

---

From: Robert Elz <[email-masked]>
Date: Wed, 26 Jan 2005 10:56:58 +0700

Date:        Sun, 23 Jan 2005 16:29:24 +0000
    From:        Paul Vixie <[email-masked]>
    Message-ID:  <[email-masked]>

  | since the guaranteed
  | minimum maximum MTU has not changed (576 for V4)

Paul, you really know better I'm sure - that number isn't 576, it is 68.
576 is the "all hosts must be able to receive, reassembling if necessary,
datagrams this big" number.

The DNS packet size was never set to avoid fragmentation - it was set because
to send larger packets may mean that some hosts would be unable to receive
them, given that nothing requires that their IP stacks be able to handle that.

  | and is still small (1280 for V6),

Yes, though the v6 equivalent to 576 is 1500 I believe.

Two issues are important here - first though lots of things have grown
a lot in the past 20 years, packet sizes have not, they've barely changed
at all (factor of 3 increase probably).

  | any udp app for the wide area should be trying to avoid fragmentation.

Yes, fragmentation is not good.   It is indeed fortunate that many
people mistook the minimum required MTU as being 576, and so built
almost all links to at least that (in the past - these days nothing
that small gets deployed except on some - very few - end-links).
That had the effect that DNS packets all avoid fragmentation, which
is a very nice outcome, and something to keep encouraging, but it
neither is, nor ever was, fundamental - repeating a DNS query, when
a fragment of a reply is lost, is still cheaper than doing the whole
thing over TCP.

kre

---

From: Paul Vixie <[email-masked]>
Date: Wed, 26 Jan 2005 05:46:13 +0000

> Paul, you really know better I'm sure - that number isn't 576, it is
> 68.  576 is the "all hosts must be able to receive, reassembling if
> necessary, datagrams this big" number.
> 
> The DNS packet size was never set to avoid fragmentation - it was set
> because to send larger packets may mean that some hosts would be
> unable to receive them, given that nothing requires that their IP
> stacks be able to handle that.

i shouldn't've said "MTU".  i meant what you said, and i thought i
also said what you said.

> at all (factor of 3 increase probably).

it seems to me that the ratio of ethernet packet size to ethernet bit
frequency is now almost as granular as ATM-OC3.  10000000000/1500 vs.
155000000/53.  we all said 53 octets was too small at OC3 speeds, yet
here we are.  (i guess early experiences with teaching IP fragmentation
to fddi/ether bridges taught the ethernet industry Don't Do That, and
the rest is the future.)

> a fragment of a reply is lost, is still cheaper than doing the whole

yup.

---

From: Alex Bligh <[email-masked]>
Date: Wed, 26 Jan 2005 10:17:37 +0000

--On 26 January 2005 10:56 +0700 Robert Elz <[email-masked]> wrote:

> It is indeed fortunate that many

Lower MTU would of course be helpful in order to support DNS in
IP tunneled over DNS. (ducks...)

Alex