Subject: Re: [RAM] Re: the separation of ID/RLOC

From: Christian Vogt <[email-masked]>
Date: Tue, 03 Jul 2007 12:23:38 +0300

>> Fast provider fail-over requires a very agile mapping function,
>> however. To me, it is not clear how either a pull or a push model,
>> or a trade-off between the two, can support this in a scalable
>> manner.
> 
> This is the key part of the mapping mechanism. I believe that at the
> first step some prototype systems based on DNS (pull model) and DHT
> (Push model)  or some hybrid model respectively must be deployed and
> test their performance,  then  decide which mechanism is a good one.

Michael,

you are certainly right.  The point I am making is that, from a delay
perspective, a pure pull model won't support efficient provider
fail-over.  A push model might; it depends on the actual mechanism.  But
still, for one edge network to update the mapping, it will have to push
the update to a large set of ingress tunnel routers.  This holds for
LISP, and for Ivip at a later transition stage.

OTOH, from a signaling overhead perspective, the pull model is more
favorable than the push model.  So there will be a trade-off, and this
does not seem easy to effect.

> The mobility support has some impact on which model will be adopted.

Right.  Edge network mobility and provider fail-over are the same in
terms of objectives and requirements IMO.

- Christian

_______________________________________________
RAM mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ram

---

From: Eliot Lear <[email-masked]>
Date: Tue, 03 Jul 2007 11:34:45 +0200

Michael,
> The mobility support has some impact on which model will be adopted. 

I think a reasonable question to ask is at what layer mobility should be 
provided.  When mobility is provided in  HIP, for instance, one needs to 
handle caching concerns within DNS and of course have appropriate 
bonafides available to authorize an appropriate record update.  In 
effect, a DNS server becomes the initial home agent, and that 
responsibility is subsequently transfered to the mobile node as handoffs 
are made.  This of course assumes that two devices don't move at the 
same time but that seems quite reasonable.

If we place mobility in LISP, the caching semantics of LISP-CONS would 
need to support it and whatever mapping binding would of course need to 
be secured, or an update would need to be made to NERD (NERD simply 
doesn't do mobility, quite frankly).

Just some thoughts.

Eliot

---

From: Christian Vogt <[email-masked]>
Date: Tue, 03 Jul 2007 13:09:41 +0300

Dino:

>> update speed and reliability of the mapping function are not just needed
>> for access network mobility.  Its also needed for provider fail-over.
> 
> It is needed as well for regular packet forwarding when nodes are
> stationary.

Exactly.  You might allow for a bit more delay in the mapping update if
a provider change is not critical because the old provider still works.
 But for fail-over, you really want to be quick.

>> Access network operators want to be able to switch providers quickly
>> when one goes down.  This is one of the main goals for multi-homing.
> 
> For what value of "quickly"? Please quantify.

I am looking at this from the perspective of a pair of communicating
hosts.  A provider change initiated by either host's edge network
operator will lead to delay and packet loss, and both should be small.

Let's take a look at the typical case of a TCP connection.  Here we will
see substantial throughput degradations after a provider change if the
delay is more than 1 RTT:  Since TCP's retransmission timeout period is
1 RTT (plus variability), the sending TCP will run into two successive
retransmission timeouts.  This means that the congestion window and
slow-start threshold will be set to 1 and 2 packets, respectively.  TCP
then cannot recover in slow-start mode (which would be efficient), but
only in congestion avoidance mode (which is highly inefficient,
especially if you start with a congestion window of 1).

If the delay caused by a provider change can be limited to 1 RTT, then
TCP will run into only a single retransmission timeout, and it will
efficiently recover in slow-start mode afterwards.  So my preferred
definition of "quickly" is 1 end-to-end RTT.

Whether or not long update delays are an issue depends, as you say, on
the change frequency and granularity.  However, any assumptions that we
make in this regards should be very careful.  Our goal is to give edge
networks more freedom in traffic engineering and provider selection.
And it would be great if we could reach this goal without restrictions
-- such as in terms of change frequency or granularity.

Best,
- Christian

---

From: Christian Vogt <[email-masked]>
Date: Tue, 03 Jul 2007 13:18:20 +0300

Eliot,

I think Michael was talking about "edge network mobility", AKA rehoming
or provider fail-over.  Host mobility support is not a primary goal for
indirection mechanisms such as LISP or Ivip.

- Christian

---

From: Tony Li <[email-masked]>
Date: Tue, 3 Jul 2007 12:18:49 -0700

> -- such as in terms of change frequency or granularity.

Isn't this issue easily resolved by advertising more than one locator?

Tony

---

From: Scott W Brim <[email-masked]>
Date: Tue, 03 Jul 2007 17:54:02 -0400

> same time but that seems quite reasonable.

You've just pointed out the problems with making DNS the HA.  DNS is
for resolving name to identifier.  HAs are for resolving identifier to
current locator.  The authorization needed to make a DNS change should
be different in scope from the one required to change state in a HA.
It's good to have the two different phases of the process somewhat
decoupled.

> doesn't do mobility, quite frankly).

The caching that needs to be done in LISP for (fast) mobility) is
luckily just in the ITRs.

See you ... Scott

---

From: Christian Vogt <[email-masked]>
Date: Wed, 04 Jul 2007 13:16:40 +0300

> Isn't this issue easily resolved by advertising more than one locator?

Tony,

if you want to an enable edge network to decide through which provider
ingress packets are received, then the edge network must be able to tell
remote edge networks one specific transit locator to which they should
send packets.

If an edge network advertises more than one transit locator, then it is
up to remote edge networks to select one of those.  This takes away the
ability for edge networks to select an ingress provider.  Additionally,
you still need a locator selection mechanism for the remote edge network
-- which may involve its own signaling overhead, delay, and packet loss.

With Six/One [1], edge networks can select both their ingress and egress
providers, and they can change them within 1 RTT without signaling.
There is also no packet loss if the old provider is still operational at
the time of the change.  And the granularity of provider selection (both
ingress and egress) is on a per-communication-session level.

[1] http://users.piuha.net/chvogt/pub/2007/draft-vogt-rrg-six-one-00.txt

- Christian

---

From: Iljitsch van Beijnum <[email-masked]>
Date: Fri, 6 Jul 2007 18:39:43 +0200

> delay is more than 1 RTT:

Two things:

1. It's doubtful that detecting, let alone restoring, a failure  
within an RTT is doable. If it is, the costs of doing so will be  
significant.

2. When you lose part of your connectivity, a substantial reduction  
in the traffic offered to the network is exactly the appropriate  
response.

With today's multihoming, it's possible to suffer delays of up to 180  
seconds before a failure is detected. The most important thing is  
that we can repair a problem before TCP times out, and if we can  
repair it before the user times out, so much the better. In my  
opinion, anything on top of that is icing and not worth additional  
effort.

---

From: Christian Vogt <[email-masked]>
Date: Mon, 09 Jul 2007 12:59:50 +0300

Iljitsch:

> 1. It's doubtful that detecting, let alone restoring, a failure within
> an RTT is doable. If it is, the costs of doing so will be significant.

You are raising an interesting point.  The process of detecting failure
of a provider may indeed be a bottleneck today.  But I would argue that
this, too, could be improved in the future.  It would therefore be wise
no to limit the efficiency of a routing and addressing solution to the
performance of state-of-the-art failure detection mechanisms.

One possible improvement would call for an edge network operator to
redirect part of its traffic to an alternative provider as the delays
incurred by the current primary provider increase.

> 2. When you lose part of your connectivity, a substantial reduction in
> the traffic offered to the network is exactly the appropriate response.

If you have an alternative provider to switch to, then it would be good
if the mapping function would allow you to do this quickly enough to not
impact ongoing communications too much.

Of course, if you don't have an alternative provider, then a reduction
in traffic would be appropriate -- and TCP would do exactly that.

- Christian

---

From: Iljitsch van Beijnum <[email-masked]>
Date: Mon, 9 Jul 2007 19:13:01 +0200

On 9-jul-2007, at 11:59, Christian Vogt wrote:

>> within

> of a provider may indeed be a bottleneck today.

Actually, this is the easy part, because you can simply send hello  
packets at a pretty high rate across a link. Or even better: let a  
watchdog timer go off when no packets have been received for a  
certain time. The latter has the advantage that you don't need to  
inject keepalives when there is real traffic and you aren't bound by  
round trip time speed. (Which isn't too bad for a single layer 2 link  
in almost all cases, though, especially if you can get rid of the  
variability by bypassing any queues for the keepalives.)

But we were talking end-to-end, or at least across a number of links.  
This means the keepalive traffic explodes and there are no longer any  
interface-specific tricks that can be done for optimization.

> incurred by the current primary provider increase.

This is susceptible to oscillations.

In my opinion, the correct way to handle this stuff is with multipath  
transport protocols.

> If you have an alternative provider to switch to, then it would be  

This sounds very much like optical restoration to me.  :-)

With IP, "backup" paths are rarely idle when the "primary" path is  
active, so loss of the path that's currently used means a reduction  
of total available bandwidth which can easily reduce this to less  
than the amount of traffic generated. So backing off until a new  
equilibrium is found is a good idea.

> impact ongoing communications too much.

If you were using a TCP-like transport protocol that works over  
multiple paths at the same time, you'd already have RTT times and  
window sizes for the other path so switching could happen without  
reducing bandwidth more than necessary. And load balancing would  
happen automatically.

---

From: Tony Li <[email-masked]>
Date: Mon, 9 Jul 2007 18:04:37 -0700

Christian,

Sorry to take so long to respond.

> remote edge networks one specific transit locator to which they should

A destination can certainly express a preference, but as always, the  
holder of the packet gets to decide on the next hop.  This may or may  
not include dropping the packet.  This is best effort and there are  
no guarantees.

> ability for edge networks to select an ingress provider.

Since a destination does not currently have that ability and the  
ability to be multi-homed today, I don't see that this is truly an  
issue.  Further, I don't see a path to a world where this could ever  
be the case.  In today's world, where only a single prefix is  
advertised, the destination can use various methods (e.g., path  
prepend) to try to induce particular path selection, but this is not  
a guarantee.  A transit provider can easily have a policy in place  
that contradicts the end-user policy, and the transit provider's  
policy will win.

In a world where a host can express a locator preference, the  
correspondent can also choose to ignore the destination's  
preferences.  This seems like it shifts control from the transit to  
the correspondent, but at no time would it ever seem to give the  
destination true and unilateral control.

> Additionally,

I think you're making an assumption that only a single locator would  
be advertised at a time, and that signaling would be required to  
transition between locators.

In fact, I was positing having multiple locators advertised  
simultaneously, with preferences expressed about their utilization.   
We do this already today with MX records and it seems to work  
passably well.

> providers, and they can change them within 1 RTT without signaling.

If I understand it correctly, a host can advertise what addresses are  
in its bunch, and a network can choose to do a prefix rewrite, which  
*should* result in the correspondent using the same addresses.   
However, the enforced symmetry is based on the actions of the  
correspondent host, which is not guaranteed.  Further, when taken  
into conjunction with source address filtering by the next hop egress  
network, it implies that the edge network is going to necessarily  
couple its ingress and egress providers on a per-packet basis.

> ingress and egress) is on a per-communication-session level.

Unfortunately, the only practical way of determining that an old  
provider (or, more generally, an old path) is non-operational is to  
lose packets.

In short, I'm not seeing anything different here: if you advertise  
multiple locators, you would seem to be doing about the best that can  
be done in terms of providing alternate path information short of  
doing an explicit path setup.  This would seem to be true of most of  
the proposals, unsurprisingly.

Tony

---

From: Eliot Lear <[email-masked]>
Date: Tue, 10 Jul 2007 08:10:58 +0200

Tony Li wrote:
> We do this already today with MX records and it seems to work passably 
> well.

LISP has just this notion, and use of NERD provides you the information 
in advance.

Eliot

---

From: Christian Vogt <[email-masked]>
Date: Tue, 10 Jul 2007 11:12:32 +0300

Iljitsch:

>> You are raising an interesting point.  The process of detecting
>> failure of a provider may indeed be a bottleneck today.
> 
>  This means the keepalive traffic explodes and there are no longer
> any interface-specific tricks that can be done for optimization.

But end-to-end keepalives are not the only solution to update remote
mapping functions.

E.g., the solution that I was having in mind is to put the edge network
in charge of selecting one of the providers it connects to.  This edge
network can find a working provider efficiently due to the topological
vicinity (as you also mentioned above).  If a provider change is needed,
mapping functions in remote edge networks can be updated on demand.

This idea does not work equally well for all architectural approaches:

- The idea is incompatible with pure host-based solutions such as Shim6,
where effectively hosts are in charge of provider selection (by choosing
IP source addresses from a specific provider).

- The idea can be realized with pure network-based solutions such as
LISP/Ivip.  The problem that remains in this case is to update remote
mapping functions /efficiently/ since the number of to-be-updated
mapping functions is large (it potentially equals the number of edge
networks).

- The idea naturally fits together with combined host/network-based
solutions such as Six/One.  Moreover, you avoid the problem of
cumbersome mapping function updates in this case.  The reason is that
the to-be-updated mapping function is with the correspondent host, and
such a mapping function gets updated automatically (and implicitly,
without extra signaling) based on the regular packet exchange.

- Christian

PS:  I'll respond to the rest of your email in a separate message.

---

From: Christian Vogt <[email-masked]>
Date: Tue, 10 Jul 2007 11:12:51 +0300

Iljitsch van Beijnum wrote:
>>  redirect part of its traffic to an alternative provider as the 
>> delays incurred by the current primary provider increase.
> 

Having worked on mobility management for several years, I fully agree
that, for efficiency reasons, the transport layer must be made aware of
connectivity changes at lower layers.  This holds for both mobility and
multi-homing.

Purely relying on multi-homing support in transport protocols, however,
would suffer the same disadvantages as Shim6.  It would limit
edge network operators in their freedom to pursue traffic engineering.

Purely network-based solutions, on the other hand, make it much harder
for a transport protocol to adapt to provider changes, because they do
not receive explicit indications of such changes.  (IP addresses are
provider-independent, after all.)

Only combined host/network-based solutions can bridge the gap.  They
allow edge network providers to do what they desire, and at the same
time provide the host with timely information on provider changes, which
is what the host's transport layer needs to adapt.

> than the amount of traffic generated.

Ok.

> So backing off until a new equilibrium is found is a good idea.

Without question.

>> if the mapping function would allow you to do this quickly enough 
>> to not impact ongoing communications too much.
> 

Yes, I agree.

Take care,
- Christian

---

From: Christian Vogt <[email-masked]>
Date: Tue, 10 Jul 2007 11:25:54 +0300

> not include dropping the packet.  This is best effort and there are no
> guarantees.

Tony,

before I reply to your email, let me make sure there is no misunderstanding.

What I was thinking is that, if edge network E communicates with edge
network F, and E directly connects to providers P1 and P2, then E should
be able to choose one of P1 and P2 for (1) egress packets that E sends
to F, and (2) ingress packets that E receives from F.

I think that we actually agree on this, right?

- Christian

---

From: Christian Vogt <[email-masked]>
Date: Tue, 10 Jul 2007 12:52:51 +0300

Tony:

> If I understand it correctly, a host can advertise what addresses are in
> its bunch, and a network can choose to do a prefix rewrite, which
> *should* result in the correspondent using the same addresses.  However,
> the enforced symmetry is based on the actions of the correspondent host,
> which is not guaranteed.  

Yes, you need support on both sides.  It's similar to TCP congestion
control, which also can be effectively disabled by a misbehaving host,
namely through acknowledgment pacing.

The difference between Six/One and TCP congestion control is that, in
Six/One, a host has no incentives to misbehave.

The requirement for compliance on both sides exists for network-based
solutions as well.  Here, you need compliance by both edge networks.

>                           Further, when taken into conjunction with
> source address filtering by the next hop egress network, it implies that
> the edge network is going to necessarily couple its ingress and egress
> providers on a per-packet basis.

Yes, Six/One enforces a symmetric path for each communication session.
You can have different paths for different sessions, but each of them
will be symmetric.

> provider (or, more generally, an old path) is non-operational is to lose
> packets.

Here I was referring not to the provider fail-over case, but to the case
where the edge network operator decides to switch traffic to a different
operator for traffic engineering reasons.  E.g., the edge network
operator might want to load-balance traffic across its border routers.

> multiple locators, you would seem to be doing about the best that can be
> done in terms of providing alternate path information short of doing an
> explicit path setup.  This would seem to be true of most of the
> proposals, unsurprisingly.

Not sure if I understand you correctly.  But in any case:  Informing a
remote mapping function about locator alternatives is not enough.  You
also need...

- a provider selection mechanism locally in an edge network that
monitors provider connections and decides when it is time to switch, and

- a mapping update mechanism that tells remote mapping functions /when/
to switch to /which/ alternative locator.

Iljitsch and I have discussed the provider selection mechanisms in a
recent email exchange (see my email "Provider Selection and Mapping
Updates" from today).  And it is the mapping update mechanism where the
delays and packet losses that I was talking about in this email exchange
come into play.

- Christian

---

From: Christian Vogt <[email-masked]>
Date: Tue, 10 Jul 2007 13:00:40 +0300

> simultaneously, with preferences expressed about their utilization.  We
> do this already today with MX records and it seems to work passably well.

Informing a remote mapping function about locator alternatives is just
one out of three mechanisms that are necessary.  As said in my previous
email, you also need...

These two additional mechanisms are the ones most critical in terms of
delay and packet loss.  I attended to this more in my previous email.

Ciao,
- Christian

---

From: Iljitsch van Beijnum <[email-masked]>
Date: Tue, 10 Jul 2007 17:31:31 +0200

> But end-to-end keepalives are not the only solution to update remote

True, but then you soon arrive at a push mechanism for the mapping/ 
reachability state, which also has downsides.

---

From: Eliot Lear <[email-masked]>
Date: Tue, 10 Jul 2007 17:44:16 +0200

Iljitsch van Beijnum wrote:
> True, but then you soon arrive at a push mechanism for the 
> mapping/reachability state, which also has downsides.

Yes, I think this is a fair point, and so one requires a bit of 
intelligence when it comes to doing keepalives.  For instance, if they 
are generated by the LISP ETR in the reverse direction with an 
indication of expected frequency then it becomes a matter of policy by 
the mapping owner as to how often they can be set, and they can be 
generated as long as there is traffic for a given EID prefix.  This same 
approach can be used with any tunneled architecture. Think of it akin to 
a DNS SOA record.

Some examples then come to mind:

    * A prefix that is used for voice might have a very high update
      frequency (10 - 20 seconds);
    * A prefix that is used by a consumer might be a lower frequency.

The key point is that the processing of these keepalives are done, at 
least in LISP, solely by ITRs and ETRs, and not by the global routing 
system or by end hosts.  So this means that there is potential 
aggregation of keepalives with no impact on the core, other than what is 
likely to be a modest amount of traffic.

But I've only really just thought a little about this issue.  Does 
anyone have a few good references to work in this space?

Eliot

Eliot

---

From: Tony Li <[email-masked]>
Date: Tue, 10 Jul 2007 08:57:40 -0700

> I think that we actually agree on this, right?

No, sorry.  E gets to decide which egress it uses.  However, F also  
gets to decide what egress it uses.  This is necessary for symmetry.   
Since F and E may not agree, ingress policy may not be respected.

If E prefers P1 as ingress and F prefers P2 as egress and there's no  
P1-P2 link, then you end up violating E's preferences.

The only way that E can guarantee its ingress policy is to not  
advertise reachability through sub-optimal providers.

Tony

---

From: Iljitsch van Beijnum <[email-masked]>
Date: Tue, 10 Jul 2007 18:11:46 +0200

On 10-jul-2007, at 17:44, Eliot Lear wrote:

> The key point is that the processing of these keepalives are done,  
> at least in LISP, solely by ITRs and ETRs, and not by the global  
> routing system or by end hosts.  So this means that there is  
> potential aggregation of keepalives with no impact on the core,  
> other than what is likely to be a modest amount of traffic.

Yes, that helps a lot. However, I still feel the need to be the voice  
of restraint in the face of additional overhead...

> anyone have a few good references to work in this space?

Please have a look at:

http://www.ietf.org/internet-drafts/draft-ietf-shim6-failure- 
detection-08.txt

The idea here is that both the situation where there is traffic in  
each direction and the situation where there is no traffic are an  
indication that there is no problem, so we don't need to do anything  
here, and no keepalives are sent. Only when we send but not receive  
there must be a problem, and if we receive but don't send the other  
side is going to think there's a problem so we send keepalives.

If you add some special case logic for common things such as TCP ACKs  
and TCP session termination, there's hardly ever any need to send  
keepalives. Only when active/quiet transitions can't be detected or  
in the unusual case that there is only traffic in one direction will  
there be keepalives.

Obviously this takes a bit more logic to implement than simply fire  
off a keepalive every few seconds. For shim6, it's important to do  
this right because you can get keepalive explosions if something goes  
down if you're not careful.

For the ETR/ITR case I would at least want to avoid keepaliving when  
there is no traffic between an ETR and an ITR.

---

From: Dino Farinacci <[email-masked]>
Date: Tue, 10 Jul 2007 23:30:33 -0700

> delay and packet loss.  I attended to this more in my previous email.

The LISP-01 draft documents the following design:

o Map-Replies contain locator-sets for an EID-prefix. Along with each  
locator is
   a priority and weight. This is used so either the ETR can force  
the ITR what
   locators to use or the ETR can allow an ITR to select the locators  
it wants within
   the scope of locators the ETR allows.

o The Map-Replies do not convey reachability status of the locators.

o The ITR, when it encapsulates packets sets the loc-reach-bits field  
to indicate
   which locators for its site are currently reachable. The loc-reach- 
bits is a bitfield
   that defines the relative location of a locator record in a Map- 
Reply. Currently the
   bitfield is 12-bits to convey reachability for 12 locators for a  
given EID-prefix.

o The ITR at a site knows reachability of it's own locators based on  
the IGP. And
   typically, the ITRs will be close to the failures, so will be able  
to have this
   info sooner than any other node in the network. As long as packets  
are being sent
   out of the site, any receiving site will get up to date  
reachability of locators.
   Failures that are close the ITR include 1) ITR failure, 2) CE-PE  
link failure, and
   3) PE router failure.

o When an ETR detects a loc-reach-bit setting that changes from 1 to  
0, it will
   not use the locator associated with that bit-setting.

Also, using NERD or CONS as a database mapping service, the mapping  
entries within them also do not provide reachability status.

This allows us to scale the Tony Li rule of "state * rate". Where we  
know the variable 'state' will be very large so we must keep the  
value of 'rate' small. Keeping reachability status out of the mapping  
database allows us to do that.

Dino

---

From: Christian Vogt <[email-masked]>
Date: Wed, 11 Jul 2007 10:38:20 +0300

> mapping/reachability state, which also has downsides.

Another option, which is the one Six/One uses, is to rely on a mechanism
that is locally executed between an edge network and its providers.  If
that mechanism decides to switch to a different provider, then it
rewrites the IP source addresses of egress packets so as to signal the
provider/address change to both end hosts.  End hosts then adopt the new
provider.

Then you don't have any signaling on an end-to-end basis.  Only possibly
some signaling between the edge network and its providers.  (Since this
signaling occurs only locally, one could do it at higher frequency.)

A host's task of finding a correspondent host's first IP address can be
done through advertising multiple addresses in DNS.  If trying multiple
IP addresses causes a bit of latency, this is typically not so critical
during session establishment than it may be in the middle of a session.
 The host will cache the correspondent host's last working IP address in
case it contacts that correspondent host again shortly afterwards.

FWIW, the mechanism I am describing here is, in a sense, a "push"
mechanism.  So it does not contradict your statement (which I think is
valid) that you end up with a push mechanism if you don't "pull" mapping
state information through keepalives.  My point here is that "push" in
this case goes without extra end-to-end signaling overhead.

- Christian

---

From: Christian Vogt <[email-masked]>
Date: Wed, 11 Jul 2007 13:34:51 +0300

Tony,

why would E prefer an ingress provider (P1 in this case) via which it
cannot reach correspondent edge networks?

Or vice versa, why would F prefer an egress provider (P2 in this case)
via which it cannot be reached by correspondent edge networks?

The only case where this would make sense is, IMO, a scenario in which
all of an edge network's providers allow only unidirectional traffic,
either ingress or egress.  That edge network would want to chose a
different provider for ingress than for egress.

You are raising an interesting point.  I wonder how often this happens,
though.  Do you have some more insight on this?  And one should also
take a look at the impact of such asymmetry on applications...

- Christian

---

From: Tony Li <[email-masked]>
Date: Wed, 11 Jul 2007 08:47:31 -0700

> via which it cannot be reached by correspondent edge networks?

I was assuming reachability, otherwise the entire question makes no  
sense.

> take a look at the impact of such asymmetry on applications...

Sorry, not much.  I'm sure that in the whole of the Internet today,  
there must be at least one case like this currently in existence.   
It's probably masked by the multihomed networks using a PI address  
instead of PA.

Tony