Subject: host based MH with a connectivity cache

From: Peter Tattam <[email-masked]>
Date: Tue, 10 Dec 2002 10:32:14 +1100 (EST)

Here are the latest thoughts I've had with the MH stuff...

This is a kind of hybrid host based and BGP MH solution.

Some preamble...

a) One of the problems I can see is that any MH solution that relies on the
originating sites view of connectivity is likely to fail.  The reason for this
is that the path and ultimate connectivity between two sites is going to be
dependendent on where they are located in relation to each other.  i.e. If a
site is multihomed, the MH address to use as a source address is going to vary
depending who it's maintaining a connection to.  This would preclude any static
advertising of MH addresses by the site (e.g. DNS or other kinds of mapping)
without an additional connectivity protocol to find the best path.

b) Another issue is the DFZ debate. Since we have applied a rule to the DFZ
that say no advertising of longer prefixes, we have effectively placed a
barrier for MH information to transit across the internet.  We should not
forget that the full MH information is *still* in the BGP system, it's just not
visible between BGP zones.  Since the information is still there, it would be a
waste not to use it on some way. 

Now some of this is hand waving - apologies if some of it won't fly.

My idea contains two parts...

first... the host based MH part (yeah I can't let go of my idea :)

1.  Use a host based MH solution as I've previously discussed.  Apply it to the
entire IP layer, not just TCP, and use the techniques for address handover as
recently discussed.  

2. MH should be restricted to prefixes only.  MH addresses with different
prefixes, but the same host part are the same address, however equivalent
addresses must still be checked.

3. Deploy a site based Connectivity Cache System (CCS) that links directly into
the BGP system, and runs parallel to this system.  It would operate on a
similar basis to DNS in that the entire BGP is not required, but just the parts
that the site is interested in reaching from it's perspective.  My guess is it
would scale in roughly the same way as DNS would.  Such a connectivty cache
should take advantage of the hiearchical nature of the MH system to provide
hints that branches of the BGP tree had come & gone.  It would by nature be a
more dynamic structure than the DNS currently is, and care should be taken to
manage the security aspects.  Typically a site would maintain 1 or 2 of these
caches in a similar way to DNS, and also a large site could maintain a
centralized cache for the use of smaller sites within it's control.  It must
also be able to determine whether a path exists between two MH addresses and
for the reasons of load balancing determine some level of fine grained path
characteristics, possibly supplemented by policy controls.

The ultimate purpose of such a cache is to restore the MH information to end
points that the strongly aggregated DFZ has thrown away.   It is important to
remember that much of the BGP information is already available in the BGP
system that is currently deployed.

4. A site would advertise it's MH prefixes into this system and the prefixes
are bound together by MH identifier.  This is very close to BGP as we know it,
but the important part is that the MH binding identifier should be visible to
the site's hosts and would be used to query the connectivity cache.

5. This is optional - the MH identifier can be used to construct a PI MH prefix
which would be advertised into the DNS system and used much in the way that an
IPv4 prefix is.  I will refer to this prefix/address as being the FORMAL
prefix/address and the MH prefixes/addresses used for transport as the ACTUAL
prefixes/addresses. A host need only ever be aware of the address formed from
this prefix in the application layer and can be used for the usual reasons
where a host identifier is required. When the address gets used, it is up to
either the host to maintain actual address pairs, or the site border routers to
translate the formal address into the actual address(es). 

I would propose that a section of the IPv6 address be allocated as a PI address
space for the specific purpose of multi homing.  Any site that is multi homed
must have a PI address prefix.  The name space by it's nature would be a flat
address space and could be constructed by concatenating a reserved MH prefix
with the existing IPv4 ASN. It would never be legal for such an address to be
used for transport outside a host/site, but must always be translated into
actual addresses once packets leave the host/site.

5(b).  If 5 is not acceptable, it is still possible for the connectivity cache
to be queried for a particular address to return all the actual MH prefixes
associated with that prefix and the reachability to each address.

6. if host based MH is to be used, it relies on the connectivity cache to get
up to date MH information and would interact roughly in the same manner as DNS,
but would require additional information if connectivity were lost for a
particular host.  The level of authntication for host based based MH would be
equivalent to ND and so could take advantage of the hop count to maintain a
reasonable level of integrity.

7. If a site based MH solution is deployed through address translation, by
virtue of (2), the amount of stateful information to be kept will be kept to a
minimum (one state per source-dest site path).  However because of preamble (a) 
I don't think it will be possible to completely remove states.  A major concern
is that this translation will need to be done at borders on expensive routers
and this has the potential not to scale.  The ideal answer is to never
translate packets once they leave a host which means the hosts must manage the
prefix modification. 

Of these points, 1,2 and 5 are fairly well understood.  3,4 & 6 are the hard
part - scalability needs to be evaluated. 7 may not scale.  Point 5 is
controversial also.

These are my thoughts - they are a bit unpolished and may indeed be things
previously proposed.  Apologies if I have just rehashed something that's been
done.

The key components of this idea are...  use of a BGP style cache that doesn't
requre the full BGP tree to be shuffled around and frees core routers to do
routing only. The other key idea is the use of a formal PI MH address which are
then translated into actual MH addresses which operate without intervention by
routers.

Regards

Peter
--
Peter R. Tattam                            [email-masked]
Managing Director,    Trumpet Software International Pty Ltd
Hobart, Australia,  Ph. +61-3-6245-0220,  Fax +61-3-62450210

---

From: marcelo bagnulo <[email-masked]>
Date: 15 Dec 2002 15:36:33 +0100

Hi Peter,

I am trying to understand your idea but I fail to understand some
points.

If i understand correctly the CSS (or CSX) stores information about
mapping between identifiers (FORMAL addresses/prefixes) and CURRENLTY
working locators (ACTUAL addresses/prefixes). Moreover, if you have a
CSS within the site A, it will store: 
- mapping information between identifiers and locators about hosts that
belong to the site A, so it can provide this information to other CSS
that can need it (this would be the server part)
- mapping information between identifiers and locators about other
sites, so it can provide it to hosts belonging to the site A (this would
be the cache part)

Then, let's say you have two sites: site A which is multi-homed and site
B which is not multi-homed.

If a host (hostB) that belongs to the site B wants to communicate with
hostA belonging to site A, (we suppose that HostB already has HostA
identifier, for instance it obtained it from the DNS) then
- hostB request mapping information about hostA from the CSS of its site
(site B) (CSSB)
- CSSB asks CSSA about the mapping info
- CSSB answer to hostB
- hostB starts the communication using one of the locators returned

Would this be correct?

If this is so,

- I do not see how does the BGP interaction works. I mean, CSSB should
return to hostB, information about locators that are currently
reachable. This means that if you send a packet from site B addressed to
that locator, an available path exits. Doesn't aggregation would
preclude you to see that?
- How do you propose to find out which is the CSS for a given address In
the example how does CSSB find out CSSA? Something like the reverse
lookup in DNS perhaps?
- The CSS seems like a critical part of the system, so i guess it should
benefit from multi-homing fault tolerance capabilities, so i guess that
an alternative method to provide its multi-homing capabilities is
required, right?
- the solutions requires at least one CSS per site (multi-homed or not)?

Thanks, marcelo

---

From: Peter Tattam <[email-masked]>
Date: Mon, 16 Dec 2002 10:14:13 +1100 (EST)

> preclude you to see that?

Yes, you are right about that & I mentioned this in the preface.  It up to the
CSSB to restore the missing BGP information via the interaction with its peer
CSS(n)s not via the BGP visible at that point in the network, possibly in a
recursive manner like the DNS operates. 

As the BGP view information traverses the net via the CSSes, the accessiblity
information should probably be modified based on the CSS's derivation of the
BGP information that it can see from that point.   This an important point as
it the best view of the other end of a connection should be the best
approximation of the BGP as given by the CSS closest to the the site being
referenced.

Now this is perhaps the hand waving and I am open to assistance here.

As far as I can envisage there would be two sets of information to be
transferred between CSSes.

There would be the initial set of mappings between formal and actual addresses
along with the weights for each path based as reconstructed by the CSS system.
This would be cached by the CSS system, and would be pulled by the local CSS in
much the same way as DNS.

Then there would be path update information which would be pushed by
neighbouring CSSes to the local one.  One would not want to have every CSS
broadcasting to the whole CSS system, so I guess one would have to have a
selective broadcasting system between CSSes.  There are techniques from other
disciplines we could borrow - perhaps something like the multicast system.  A
CSS requests from its peers that it wants to receive updates for paths which it
knows are in use, and the neighbouring peers in turn receive the same
information.

In the worst case of a fully meshed CSS, the amount of information traversing
would degenerate into the equivalent to a totally unaggregated BGP system with
the DFZ being much the same as we have in IPv4.  Only empirical data will tell
us what the true picture will be.  There is one advantage though, and that is
the CSS system load can be removed from the core routing system and could be
managed with cheaper hardware even if it ultimately resembles something like
the BGP system we currently are used to.

> the example how does CSSB find out CSSA? Something like the reverse

It is not necessary to find any given CSS, only information about prefixes. 
Each CSS has an approximation of the full DFZ which would have resulted from an
non aggregated internet but only for prefixes it is interested in.  The
appoximation is built up from the paths that the originating site needs to see
(cached information) + the definitive MH information advertised by the site(s)
that the CSS handles.  There need not be a 1-1 mapping between sites and CSS, a
CSS can serve many sites, but a site must have at least one CSS (and can
possibly have more than one).  However you have made a good point.  It is not
sufficient to just get the cached view from a neigbouring CSS.  The information
should be modified in such a way as to provide an accurate view which would be
equivalent to that seen by a BGP system which had full unaggregated information
in the DFZ.  As long as CSSes are arranged in the right way topologically, it
should be possible to determine the weights of the various MH paths between the
source and destination paths. 

> an alternative method to provide its multi-homing capabilities is

A CSS only needs to have hard wired into it all the MH paths to its peers and
an independent mechanism to determine which of these paths is accessible
(keepalives like BGP?)   The quality of traffic between CSSes should not in any
way affect the information that traverses the CSS system.  I personally believe
this is a fault in the current BGP system and the reason for the BGP storms
we have grown accustomed to dealing with.

> - the solutions requires at least one CSS per site (multi-homed or not)?

Yes..  See above.

> Thanks, marcelo

I am sorry for the hand waving - I am trying my best to understand the full
implications of how such an idea could work.  I am happy to throw the ideas
open to debate and input/enhancement from others.

Peter