Subject: FW: Comments on draft-morelli-v6ops-ipv6-ix-00.txt

From: Morelli Mario <[email-masked]>
Date: Mon, 02 Aug 2004 06:17:33 +0200

Hi Kurt,

Sorry for the late answer I have been really overbooked last week.

Please see our replies in line.

Regards,

Mario and the rest of co-authors

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On
Behalf Of Kurt Erik Lindqvist
Sent: Sunday, July 25, 2004 11:13 AM
To: [email-masked]
Subject: Comments on draft-morelli-v6ops-ipv6-ix-00.txt

I guess/hope it's ok to send comments on this draft to v6ops.

 >   Internet Exchanges (IX) have played a key role in the development
of
 >   Internet, organizing and coordinating the traffic interchange among
 >   Internet Service Providers (ISP).

Actaully, I know of few, if any IXes that have coordinated the exchange
of traffic. This is purely a business decision of the involved ISPs.

[Mario] Well, what we did in our draft is a pure description of the
general IX model studied inside the Euro6IX project trying to put in
evidence the most innoovative aspects. 

I undersntand that all IXs organize and coordinate the exchange of
traffic in the sense that they provide all the facilities the ISPs need
to interchange traffic, although it could be true that they do not
organize how the traffic is exchanged. 

In the case of IXs with route servers, I think it is very true that the
IX organizes and coordinates the traffic interchange, as that is the job
of BGP peerings managed by the RS.

We should change the reading of our sentence to be more precise.

 >   Traditionally, IXs have been based
 >   on layer 2 infrastructures, being the layer 3 services managed by 
the
 >   participant ISPs.

I don't like the use of "services". IXes are based on layer 2 as that is
their business model. ISPs selling transit are dealing with Layer 3
exchange of traffic.

[Mario] It is true that today Layer 2 IX is the most frequent business
model, but technically we see several reasons to move towards business
models based on Layer 3. So in this sense, this business model could
offer new  services from both the technical and business
perspectives....if "services" is not the suitable wording, we may agree,
but we do not find any better description by now..

 >   However, IPv6 hierarchical and aggregatable routing and addressing
 >   model comes to enhance the IX functionalities by proposing to
 >   directly assign addresses to IX customer's networks.  The customers
 >   can connect with one or several upstream providers and have a
 >   separated addressing space, dependent on the IX instead of the
 >   providers, in order to facilitate multihoming or avoid renumbering
 >   procedures when changing the provider.

I am not really following what you are saying here. IPv4 is also
aggregatable, although not hierarchial. Hierarchy for IPv6 only comes
into play as end-customers can only get addresses from their provider
and there is no PI space.

[Mario]In the model that we are proposing, we want to assign to the IX
an IPv6 prefix (e.g. /32) that in its turn he could reassign to its
connected cusomters. In our view, this is not related to PI. The IX
Customer is not tied to any specific ISP on that IX.

[Mario] Moreover, in the model that we are proposing, IX is not a
transit provider but the IX acts as a local/regional ISP for its
customers.

I _think_ that what you are saying is that the IX would act as the LIR
for the end-customers and that the membership ISPs agree to accept
address assignment out of the IX address block from the various members.
To me it's not clear who would announce the covering aggregate route?
The IX? Then the IX is a transit provider competing with it's own
members...

[Mario] There are three different actors in the model: the providers,
providing connectivity to Internet, the IX that provides addresses and
coordinates all, and, optionally, another actor providing L2
connectivity between clients and the IX. The aggregate route (IX prefix)
is announced by the IX itself (in particular, by the L3MF).

 >   In addition, being an IX a central point where traffic is
 >   concentrated, several networks and application services benefit
from
 >   the fact of being partial or totally offered from an IX, opening
the
 >   IX to the world of new advanced services and functionalities like
 >   security, AAA, QoS, multicast, mobility, PKI, DNSSEC or policy
 >   provision, that could also facilitate the deployment of IPv6 and
 >   their required transition mechanisms.

IXes have been providing common services for years. Netnod have provided
i.root-servers.net, the offical Swedish time through NTP, a RIPE whois
copy etc since 1998. k.root-servers.net at Linx is another example. Etc,
etc.

[Mario] I think that you caught the point. They offered "common"
services, what we propose is to use the IX to provide for  a broader set
of services (like for example multicast, security, QoS and so on). Even
if these services are already present in some IXes, this is not
described until now.

 >   A NAP is basically an enhancement of the IX concept that, apart
from
 >   a place to host bilateral peering arrangements between similar
 >   providers, it takes the role of a transit purchase venue, where
 >   regional ISPs can acquire transit services from long-haul or
transit
 >   providers.

I don't really agree. I see the two terms used interchangable.

[Mario]  Even if you are righ from a practical point of view,
theoretically our sentence has been taken from "Interconnection, Peering
and Settlements-Part I" by Geoff Huston, Telstra
(http://www.cisco.com/warp/public/759/ipj_2-1/ipj_2-1_ps1.html). There
was an historical distintion between IXs and NAPs, that maybe has
dissapeared now. 

 >   On the other hand, IPv6 proposes a strictly hierarchical routing
and
 >   addressing model that essentially follows the principles stated in
 >   CIDR [1]: hierarchical assignment of addresses and routing based on
 >   aggregation.  The addresses assigned to an organization depend on 
the
 >   point they connect to the Internet.  As a consequence, if the site
 >   changes its provider, its global prefix must be changed according
to
 >   its new location in the global topology.

...which is more or less through also for IPv4.

[Mario] So we agree?

 >   This new IPv6 IX based addressing model, as well as the advantages 
of
 >   locating network and application services inside the IXs, bring new
 >   possibilities for the design of new advanced IPv6 Internet
Exchanges
 >   architectures, opening the providers market to new opportunities
and
 >   actors.

I think you will find that most ISPs see "new actors" as something
negative :-)

[Mario] We do not enter into business considerations in IETF, but in our
Euro6IX work we are already working on this issue. We believe that there
are valid reasons for this.

 >3.  Reference Scenario
 >
 >   The following figure describes the main reference scenario of the
 >   advanced IPv6 Internet Exchange model defined in this document.
 >
 >                 Long Haul Providers
 >                 +------+   +------+
 >                 | LHP1 |...| LHPm |
 >                 +------+   +------+
 >                      |        |
 >   +------------------|--------|----------------+
 >   |               +----+   +----+              |
 >   |    IX         | R1 |...| Rm |              |
 >   |               +----+   +----+              |
 >   |                  |        |                | IX Customers
 >   |+----------+  +-----------------+  +------+ | +-------+
 >   ||          |  |                 |--| CER1 |-|-| IXLC1 |
 >   ||  Value   |  |   Layer Three   |  +------+ | +-------+
 >   ||  Added   |--|    Mediation    |      :    |     :
 >   || Services |  |    Function     |  +------+ | +-------+
 >   ||          |  |                 |--| CERn |-|-| IXLCn |
 >   |+----------+  +-----------------+  +------+ | +-------+
 >   |                  |       |                 |
 >   |               +----+   +----+              |
 >   |               | R1 |...| Rk |              |
 >   |               +----+   +----+              |
 >   +------------------|-------|-----------------+
 >                      |       |
 >                +--------+   +--------+
 >                | RP1    |   | RPk    |
 >                |        |   |        |
 >                | +-----+|...| +-----+|
 >                | |IXRC1||   | |IXRCk||
 >                | +-----+|   | +-----+|
 >                +--------+   +--------+
 >                  Regional Providers

First note. If Long-haul providers are to mean Tier-1 or transit
providers, I have yet to find one that connects to an IX. I think
KPNQwest's connections to Linx and AMS-IX where the last ones. Also note
that it's in the interest of Tier-1s NOT to connect to IXes. And if they
are present (and we can argue about the difference) they are for the
sole purpose of selling transit.

[Mario] In our model LHP is not equal to Tier1 provider. A LHP is just a
provider that wants to sell connectivity to Internet to the IX
customers. In fact the difference between a LHP and Regional Provider is
not important for our model.  

 >   The IX is made of:
 >
 >   o  A classical L2 infrastructure (i.e, a high speed LAN), not
 >      represented on the figure.

 >   o  ISP routers (R), that connect ISP networks to the IX.
 >
 >   o  Customer Exchange Routers (CER), that connect local customer
 >      networks to the IX.

I am not following this. What is the difference betweek a ISP router and
a CER?

[Mario] ISP router is the router an ISP deploys on an IX to have
presence there. The CER is the router that connects an IX customer to an
IX.

 >   2.  IX Remote Customers (IXRC), which are not directly connected to
 >       the IX but to a regional provider that is present on the IX.

You mean transit customers of the reginal provider?

[Mario] An IXRC is basically a customer of the regional provider with
the only difference that it uses addresses from the IX range instead of
the regional provider's range. The only purpose of an IXRC is to
facilitate it the provider change without renumbering.

 >5.  Overview of the new advanced IPv6 IX model
 >
 >   In a classical model, an IX is not normally opened to the direct
 >   connection of customers (i.e.  large corporate networks or small 
ISPs
 >   or whatever).  Instead of that, customers are connected to ISP
 >   networks, which are present on the IXs.

This is not true. I would say the vast majority of connections to the
worlds IXes are small ISPs. By far. Large ISPs in the form of Tier-1s or
"Tier-1 wanna bees" are extremely rare. Corporate networks vary. In
Sweden there are none. In Norway they are plenty. In the rest of Europe
they are some AFAIK.

[Mario] We do not agree on this issue. We know about IXes where big
companies/ISP are present. Probably is a geographical/cultural issue.

I think you are using the term "ISP" in very varying roles through out
the document BTW.

[Mario] That is true. We should work on categorize better the types of
providers.

 >   In those cases where customers are directly connected to an IX,
they
 >   typically subscribe an agreement with one or several Long Haul
 >   Providers present on the IX to route their traffic and announce 
their
 >   prefix addresses.

Well, then they are not really directly connected, are they? :-)
Actaully following the next paragraph, I would simply call them
"customers".

[Mario] maybe we should clarify that "directly connected" means a L2
connection.

 >   To solve this problem, the advanced IX model presented in this
 >   document uses a different approach based on the possibility (new
for
 >   an IX) to directly assign IPv6 addresses to the customers.
 >   Connectivity provision and IPv6 address assignment are now
separated
 >   issues and they are no longer both linked to the LHP.

Actually they wheren't in the past either. They where linked to who you
choose to use as your LIR. In most IPv6 cases this is the same entity
that announces the covering route for the block from where assignments
are made.

[Mario] What we said is that in this model, the IX becomes the LIR and
the connectivity providion is done by one of the Providers connected to
that IX.

 >   In this way, if an IX customer wants to change its service provider
 >   (e.g.  because it gets a better service or price from another LHP),
 >   it does not need to change its own addresses, as they have been
 >   assigned by the IX and not by the service provider.

....as long as the new provider is a customer of that IX.

[Mario] Exactly.

 >   group, for instance, in case of distributed IX).

I only know of two distributed IXes. One in Japan and a Swedish provider
that sells something they call distributed IX, that I think most would
call either Internet transit or simply Internetaccess. I 
might be
wrong. Distributed IXes have the bad habit of competing directly with
their customers transmission service revenues. Which ISPs normally tend
to dislike even more than competing with the IP related sales.

[Mario] This is a business issue and not IETF issue.

 >5.3  Server Farm
 >
 >   The new model here proposed foresees that services are placed
inside
 >   an IX.  This is a revolutionary concept that permits the
 >development

Hrm. It wasn't even very revolutionary in 1998....

[Mario] Maybe you are right but it has not been documented until now.
Moreover we are not aware about anything similar to L3MF even with IPv4.
The work of v6ops is to document operational issues and experiences with
IPv6 and this is what we are doing in this document.

 >   In order to keep routing scalability, IX prefix/es must be
announced
 >   aggregated to the IPv6 Internet through the ISPs peering on the IX.
 >   In principle, unaggregated prefixes assigned to IX customers must 
not
 >   be redistributed outside the IX (only to routers present on the
IX).
 >   This fact imposes the limitation that all incoming traffic (that
is,
 >   traffic destined to IX addresses) will follow the same path,
 >   independently of the IX customer it is directed to.

Who will pay/select that/those provider(s)? If it is the IX, then what
is the difference between the IX and a ISP?

[Mario] The customer has the power to select/pay what he wants but
obtains, thanks to the IX, a broader set of options.

 >   As presented in section 3, two types of IX customers are defined:
 >
 >   1.  IX Local Customers (IXLC), which have a direct layer 3 
connection
 >       to a router in the IX (named CER), either managed by the 
customer
 >       or by the IX administrator.  These customers can be connected
to
 >       the IX using different technologies like point to point links,
 >       Frame relay, MPLS VPNs, etc.  Customer routers (CER) can be
 >       dedicated to a customer or can be shared between several.
 >       However, in the shared case, all the preferences and routing
 >       policies will be common to all customers sharing the router.
 >
 >   2.  IX Remote Customers (IXRC), which have not direct layer 3
 >       connection with any router in the IX.  They are customers of
one
 >       of the providers present on the IX and so they are connected to
 >       the ISP network at some place different from the IX, but they 
use
 >       addresses form the IX prefix.  In this case, the ISP has to
cope
 >       with the distribution of the prefix assigned to the customer
 >       through its routing system, in order to have the customer
 >       accessible from the IX.

I fail to see the interest in direct vs remote "layer 3" connectivity.

[Mario] Basically we want to leave everyone the possibility to get the
access it wants to have. For instance, if they only want Layer 2 they
still have this service option in the same IX.

 >   By using a route server the scalability of the IX is greatly
 >   improved, because, being "n" the number of ISPs present in the IX,
 >   the number of BGP peering is O(n) and not O(n2) as it is with a
mesh
 >   topology.  Moreover, the addition of new members to the IX is
 >   simplified, as only a new peering between new ISP router and the
 >   route server has to be configured.

It also assumes that the connected ISPs have a "open" peering policy and
are willing to peer with every other connected member. This might or
might not be true. "Traditional" IXes are neutral to this and have
(mostly) no view on member peering policy.

[Mario] You are right but both options are possible.

 >6.1.3  QoS

Intra-provider QoS is hard to agree on a dedicated point-to-point link.
Less alone a shared medium IX. And this has nothing to do with the IX
architecture.

[Mario] That is part of the innovation  :-)

Summary, I think that you describe a transit ISP that would allocate
addresses to it's downstreams. Which if I remember correctly was the
original TLA meaning in the address architecture.

[Mario] It is not a transit provider, but it can allocate addresses to
its customers (they maybe downstream providers or corporates). You are
right, in general, our document follows the original "TLA meaning" to
further describe the technical extended possibilities.

- - kurtis -

Gruppo Telecom Italia - Direzione e coordinamento di Telecom Italia S.p.A.

====================================================================
CONFIDENTIALITY NOTICE
This message and its attachments are addressed solely to the persons
above and may contain confidential information. If you have received
the message in error, be informed that any use of the content hereof
is prohibited. Please return it immediately to the sender and delete
the message. Should you have any questions, please send an e_mail to 
[email-masked]. Thank you
====================================================================

---

From: Kurt Erik Lindqvist <[email-masked]>
Date: Mon, 2 Aug 2004 23:11:54 +0200

> organize how the traffic is exchanged.

IXes provide a layer 2 fabric.

> of BGP peerings managed by the RS.

There is a slight distinction here. Route-servers will help the ISPs 
that want to peer and exchange traffic with everyone else present at 
the IX. But _ONLY_ then. I am still free to decide not to peer with 
anyone that is present.

> but we do not find any better description by now..

First, this goes kind of badly with your own statement that this is the 
IETF and we do not care about business models. I disagree that what you 
are proposing have any future as a business model, I also disagree that 
this would be a good business model. And I fail to see the technical 
reasons for doing this. Actually, I think it creates a lot of 
unnecessary complexity.

> Customer is not tied to any specific ISP on that IX.

No, but what I am saying is that there is hierarchy in IPv6, as opposed 
to in large parts of IPv4) as there is no PI IPv6 space. Further, you 
say that a customer is not tied to a particular ISP. The reason is that 
the IX has taken over the role as the ISP, and the customer in turn is 
tied to that particular IX. And the customer will required to use the 
upstreams and the polices of the choice of the IX. Depending on who the 
IX chooses as it upstream.

> transit provider but the IX acts as a local/regional ISP for its

So the IT IS a upstream for the customers. IT IS a competitor to it's 
own members. And the IX will need a upstream and all customers will 
have to follow the choice of the IX.

> is announced by the IX itself (in particular, by the L3MF).

I still don't understand who would announce the covering aggregate and 
who transport that traffic. And how that party would get payed. AFAICS 
the other ISPs will be transit customers of the IX that will have ISP 
announce the coving route as their upstream.

> if these services are already present in some IXes, this is not

Eh, Multicast has been on IXes since years, as have other services. The 
hardest part with doing intra-provider QoS is the commercial agreement. 
There is nothing in what you propose that you can not do at a IPv4 
based L2 exchange. If you are proposing "walled gardens" of for example 
connecting certain web-services etc to the IX, then that is just more 
customers moving from the ISPs to the IX.

> was an historical distintion between IXs and NAPs, that maybe has

There was, but I don't see that any more.

> ...which is more or less through also for IPv4.

Yes, but is not IPv6 specific at the moment.

> are valid reasons for this.

Then I suggest you remove the last statement from that text.

> not important for our model.

Ok, but if I an ISP, how do I know that I will only transport traffic 
to my own customers? I will need to announce the covering route, and 
will therefor potentially carry all traffic to the IX, also for all the 
other ISPs. How will I get paid for that?

> presence there. The CER is the router that connects an IX customer to 

And with "IX customer" here you mean "a business connected to the IX 
that is not in the business of reselling connectivity to a third 
party"?

> facilitate it the provider change without renumbering.

...as long as I change to a provider that is a customer of the IX. And 
as the IX is a direct competitor to the ISPs, I am not convinced that 
would be the case.

> companies/ISP are present. Probably is a geographical/cultural issue.

Ok, I only know of the +30 IXes that are members of Euro-IX and some of 
the IXes in Asia.

> the connectivity providion is done by one of the Providers connected to

But you do not describe routing, or how the ISPs guarantee that they 
are not carrying traffic for non-customers.

> [Mario] This is a business issue and not IETF issue.

So where is the technical motivation for a distributed IX? Especially 
that would be superior to a "standard" non-distributed layer 2 
exchange?

> IPv6 and this is what we are doing in this document.

That you are documenting an idea doesn't make it revolutionary. And I 
can remember seeing in your draft that this is documentation of issues 
and experiences from deployment? If it is that should go in there, and 
be made clear.

> [Mario] You are right but both options are possible.

I know you will say "this is a business issue and not for the IETF", 
but it hasn't proven to be a working method to make money so far.

> [Mario] That is part of the innovation  :-)

I have quite a few years of experience in peering negotiations for a 
Tier-1 provider. I think I am safe to say that you haven't understood 
the problem. But maybe that is just me.

- - kurtis -