Subject: RE: (multi6) requirements draft comments

From: "J. Noel Chiappa" <[email-masked]>
Date: Thu, 20 Dec 2001 15:34:08 -0500

> From: "Tony Hain" <[email-masked]>

    >> What I meant was that all the people who are connected to both:

    >> - i) the same segment of the York County Cox CATV system here
    >> in Virginia,
    >>       and
    >> - ii) the same Verizon DSL office
 
    > you distinctly claimed a difference from metro, but in your example you
    > have an implicit metro context.

You clipped the next line:

    > would share the same one-level-up topological naming entity

Let me try this another way. Let's define a "chunk" of a graph as a set of
nodes, and the arcs between them, such that you can get from any node to any
other node by traversing only nodes and arcs that are part of the chunk. In
the map (graph) of the network, I can then replace that "chunk" of the
map/graph with a single node representing that "chunk"; i.e. draw a new,
simpler map/graph with one node where all those nodes used to be. This
process can recurse; i.e. I can define some "meta-chunks", and draw yet
another map/graph.

(You can fine a picture of some of this process here:

    http://users.exis.net/~jnc/tech/routing_slides.html

Look for the header "Abstraction Hierarchies", and then look at the second
diagram. I guess I should add a picture showing the simplified graph, but
I assumed it's pretty obvious what it looks like.)

Now, what I was doing above is taking a map (graph, actually) of the real
network, including all these houses Hk which might be multi-homed, and trying
to draw (nested) circles around portions of that actual connectivity
map/graph, that define "chunks" of the map/graph which are useful for making
the routing scale.

I have defined a particular "chunk", the set of end-nodes Sij, as ecompassing
the following nodes in that map/graph: the node for that particular cable
installation from that cable provider, Ci (I say "that particular ..
installation", since other cable installations belonging to that company are
elsewhere in the map), the node for that DSL office from DSL provider, Dj
(same comment - their other offices are elsewhere in the map/graph), and all
the host nodes Hk such that any Hk has a link to both Ci and Dj, along with
all the arcs among them.

Now, do the same thing for all the other local ISP's hardware segments, and
the customers who are connected to them, producing a map/graph with a
"simplified" representation of the connectivity in the local area. I can now
recurse this process, and group things into even bigger aggregates.

The *key* difference here from metro is that I *start* with the actual
connectivity map, placing no constraints on how it is connected. (E.g. there
may be no direct link from Ci to Dj, except through the customer nodes -
which of course do not forward traffic.) From that, I then asssign addresses.

This is very different from metro, which assigns addresses to things, and
then places constraints on how things can be connected - and never changes
the addresses no matter how the actual connectivity changes.

For instance, notice that if customer Hk drops cable provider Ci and signs up
with a new provider, Cq, they get a new address, since they are now part of
the set Sqj. This is distinctly different from what metro addressing does.

    >> "REAL TOPOLOGY IS KING". For routing, the *only* thing that matters is
    >> *actual* connectivity; i.e. the actual wires and routers.
    >> ...
    >> the K-K lesson, which is that the way you get the routing to scale,
    >> even in mindbogglingly large networks, is to use LOTS OF LAYERS OF
    >> ABSTRACTION.
 
    > I am sure you are well aware of the contradiction in those two
    > statements in the abstract

There is no contradiction. Perhaps you are being confused by my terminology,
where I am using the term "abstraction" in the graph sense, not the software
engineering sense.

I use the term "abstraction" of (sub) graph to mean a simplified
representation of a "chunk" of a graph - but not *neceessarily* one that can
be produced by any repeated replacement of smaller sub-"chunks" of the
"chunk" with nodes (that process, and the results, I tend to call
"aggregation").

Let me give you a concrete example. Suppose I am a small ISP, with 5 routers
total, four of them border routers, A-D, and one internal router I. I have 4
links, directly connecting each border router to I. Here are some
"simplified" representations of that "chunk" of the map/graph that you can't
produce by an aggregation process:

- a graph in which all four border nodes are connected to each other directly
- a graph in which the four border routers are connected in a ring (i.e. A to
  B, B to C, etc)

So I tend to use the word "abstraction" to mean what most people mean by
"aggregation", but as a more encompassing technical process.

In particular, in their scaling work, Kleinrock-Kamoun uses only
"aggregation" (as defined here) steps to produce simplified graphs.

    > We know how to aggregate when all the end sites play nice and constrain
    > their service to a single ISP, but they refuse to do that. We also know
    > how to implement a scalable metro approach, but ISPs refuse to do that.
    > Since the end sites are the holders of the money, the ISPs will
    > eventually loose this battle, and lacking any revolutionary technology
    > will have to implement some degree of metro.

That may well be.

FWIW, the addressing approach I have outlined allows both the users to
multihome, and the ISP's not to have to be forced to interconnect - but it
does it by using address allocation mechanisms that both sides may not like:
the users have to renumber when they change providers, and the ISP's have to
give up some control over addressing.
 
However, I have no way to control what the market will do - and indeed, my
addressing scheme may not even be an option for people.

I'm simply trying to explain to people what's fundamentally possible from the
routing side.

	Noel

---

From: "Tony Hain" <[email-masked]>
Date: Thu, 20 Dec 2001 13:44:45 -0800

>     > would share the same one-level-up topological naming entity

Sorry I should have been more precise. There is an implicit geographic
context which modifies the scope of the topology graph.

>     http://users.exis.net/~jnc/tech/routing_slides.html

Your picture is too simple to be useful in addressing the scope of this
problem. Consider that A3 is connected to 'other stuff'. Your subsiquent
abstraction to X would still be valid, but depending on where in 'other
stuff' that connection occured you might get different answers for the
topology name of X, yet the abstraction is too simple to explain why.

> Now, do the same thing for all the other local ISP's hardware
                                           ^^^^^
This focus on *local* is the basic problem. The actual topology
frequently connects both at some local level and pulls in a few very
remote ISPs. Consider MegaRandomCorp which has connections to 6 ISPs in
the SF area, but on the same set of border routers has direct circuits
to an ISP in London, Tokyo, and Sydney. At the same time RandomMegaCorp
has connections to 3 of the same SF, Tokyo, and Sydney ISPs, but a
different one in London.

> which of course do not forward traffic.) From that, I then

If you fold the above complexity into your graph along with the other
~12000 ASs that are prefix origins today, and managing the
topology-names becomes an administrative burden as topology is
constantly shifting.

> the users have to renumber when they change providers, and

Actually they have to renumber everytime their neighbors change
providers as well, because the entire graph changes around them.

Tony

---

From: Andrew Partan <[email-masked]>
Date: Fri, 21 Dec 2001 00:36:47 -0500

> topology-names becomes an administrative burden as topology is

No, not if the system does the work for you.  Managing the topology
abstractions is something the system should do, not people.

> providers as well, because the entire graph changes around them.

No, only if they change their attachment point.  If your neighor
changes his attachment point, he renumbers, not you.
	--asp

---

From: Brian E Carpenter <[email-masked]>
Date: Fri, 21 Dec 2001 11:38:59 +0100

"J. Noel Chiappa" wrote:
...
> which of course do not forward traffic.) From that, I then asssign addresses.

...which then have to be reflected globally in the DNS before they can be used.
This seems to be a real difficulty in automated address assignments.

I think we have a missing requirement:

The solutions must include an analysis of their effects, if any, on DNS updates,
including consideration of load for the global updating of the DNS, and
interactions with DNS TTL and caching.

   Brian

---

From: Andrew Partan <[email-masked]>
Date: Fri, 21 Dec 2001 11:18:49 -0500

On Fri, Dec 21, 2001 at 11:38:59AM +0100, Brian E Carpenter wrote:
> ..which then have to be reflected globally in the DNS before they can be used.

Not if you seperate EIDs from the routing goop.  [Which then leads
to the question of how you get from an EID to the routing goop.]

> The solutions must include an analysis of their effects, if any,
> on DNS updates, including consideration of load for the global
> updating of the DNS, and interactions with DNS TTL and caching.

Note that a solution could not effect the DNS at all.  I.e.: I
could see something that has the EID/routing-goop split and a
different (non-DNS) solution of how you get from EIDs to the
routing-goop.  The DNS would only deal with EIDs.
	--asp

---

From: "Tony Hain" <[email-masked]>
Date: Fri, 21 Dec 2001 09:46:58 -0800

> abstractions is something the system should do, not people.
Reality says that operations people really like to believe they know how
the routers are configured. To make the system Noel is describing work,
the entire system would have to be self configuring, else everytime a
customer connected to another provider each existing provider would have
to reconfigure their interface to that customer.

> changes his attachment point, he renumbers, not you.

If he changes the entire graph so that it does not fit neatly in the
aggrigate, then everyone who is disrupted in the more granular space
will have to readdress.

Tony

---

From: Andrew Partan <[email-masked]>
Date: Fri, 21 Dec 2001 13:14:46 -0500

On Fri, Dec 21, 2001 at 09:46:58AM -0800, Tony Hain wrote:
> the routers are configured.

No, we don't know that 100% even today.  While I may know what
neighbors I have configured on my routers today, I have no idea
what routes they are actually sending me right now.  [For some
neighbors, I may have a list of permissible routes they could send
me, but I have no idea what they are actually sending w/o going &
looking.]  For an IGP, I may only configure what interface to run
over - and let the system handle the neighbor discovery.

I let the routing system handle the details how to get from A to
B after I've set up the gross overall structure.

I can do analysis of how the system is supposed to work and what
changes in face of updates, and I can put some contraints on the
system, and adjust knobs here & there to push it more towards one
state or another, but to know what the system is actually doing at
any point in time, I'm going to have to go look.

If we are going to scale, the system has got to be better at self
organizing.  Yes there have to be contraints in there (aka 'knobs'),
but the system is going to have to take on more of the work.

When a new system rolls out, people are going to be doing a lot of
looking at it, and set it up with all of the knobs turned way down,
and do a lot of making sure that its doing the right thing.  [Been
there done that with EGP/BGP switch, ospf/isis changeovers, MPLS
deployment, others.]  But as time goes on and systems (and s/w)
proves itself out, the knobs are loosened and you let the system
do what it is supposed to.

> aggrigate, then everyone who is disrupted in the more granular space

If you move, you have to change your info, but no one else does.
	--asp

---

From: "Tony Hain" <[email-masked]>
Date: Fri, 21 Dec 2001 10:24:44 -0800

> over - and let the system handle the neighbor discovery.

Actually you made my point better than I did. Within the realm of a
trust boundary (where you would typically run an IGP) you are willing to
automate, but at the edge of that bounday, you know who is at the other
end (specifically configured) and may apply a sanity check against what
may come in from there. If we follow Noel's path, you either couldn't
know what address the neighbor was using and would have to
autoconfigure, or you have a very high cost process to manually update
everytime a customer connected to a different provider. Neither of these
fit in a typical provider/customer relationship.

> do what it is supposed to.

This is easy to say, but since the automation is crossing a trust
boundary it is much harder to sell.

Tony

---

From: "Tony Hain" <[email-masked]>
Date: Fri, 21 Dec 2001 10:35:20 -0800

> routing-goop.  The DNS would only deal with EIDs.

I agree with Brian, the requirement must include an analysis. Keep in
mind that doesn't mean there will necessarily be any impact, but it must
be addressed.

J. Noel Chiappa wrote:

> I understand that dynamic updating might be an issue, but
> that's going to be
> a problem no matter what system you keep the mapping in. I
> also understand
> that DNS has problems, and maybe we really want son-of-DNS.
> But still, why
> keep the two mappings in separate databases? What's the
> benefit, when you
> look at the entire system?

Actually this gets more to the point that I think Brian was after, the
solution must include an analysis of its effects on the entire existing
system. This would catch DNS, but also any interaction with the current
interdomain routing environment.

Tony