Subject: [DMM] Fwd: New Version Notification for draft-matsushima-stateless-uplane-vepc-00.txt

From: Ryuji Wakikawa <[email-masked]>
Date: Wed, 10 Jul 2013 09:12:14 -0700

Hi 

We submit a new document.  Your comments are appreciated! 

thanks,
ryuji

Begin forwarded message:

> From: [email-masked]
> Subject: New Version Notification for draft-matsushima-stateless-uplane-vepc-00.txt
> Date: July 10, 2013 9:09:44 AM PDT
> To: Ryuji Wakikawa <[email-masked]>, Satoru Matsushima <[email-masked]>
> 
> 
> A new version of I-D, draft-matsushima-stateless-uplane-vepc-00.txt
> has been successfully submitted by Satoru Matsushima and posted to the
> IETF repository.
> 
> Filename:	 draft-matsushima-stateless-uplane-vepc
> Revision:	 00
> Title:		 Stateless user-plane architecture for virtualized EPC (vEPC)
> Creation date:	 2013-07-10
> Group:		 Individual Submission
> Number of pages: 19
> URL:             http://www.ietf.org/internet-drafts/draft-matsushima-stateless-uplane-vepc-00.txt
> Status:          http://datatracker.ietf.org/doc/draft-matsushima-stateless-uplane-vepc
> Htmlized:        http://tools.ietf.org/html/draft-matsushima-stateless-uplane-vepc-00
> 
> 
> Abstract:
>   We envision a new mobile architecture for the future Evolved Packet
>   Core (EPC).  The new architecture is designed to support the
>   virtualization scheme called NFV (Network Function Virtualization).
>   In our architecture, the user plane of EPC is decoupled from the
>   control-plane and uses routing information to forward packets of
>   mobile nodes.  Although the EPC control plane will run on hypervisor,
>   our proposal does not modify the signaling of the EPC control plane.
>   The benefits of our architecture are 1) scalability, 2) flexibility
>   and 3) Manageability.  How to run the EPC control plane on NFV is out
>   of our focus in this document.
> 
> 
> 
> 
> The IETF Secretariat
>

---

From: Jouni Korhonen <[email-masked]>
Date: Thu, 11 Jul 2013 09:48:38 +0300

Thanks for winding up a good old discussion on this specific GTP
topic. I have not yet read the I-D in a close detail but I have
some initial questions - so there might be some gaps in my
understanding of the I-D. Previously when similar architectures
were exercised the question of backward compatibility always came
up. One of the sticking points was the established practices to use
of GTP-U header extensions to carry subscriber specific data within
the core (i.e. between SGW/SGSN and PGW/GGSN). Have you thought 
of this? Another question/note I have is that now quite a bit of
tasks that used to belong to PGW/GGSN are now assigned to EPC-E
(like policy control). Has this been seen as an issue?

- Jouni

On Jul 10, 2013, at 7:12 PM, Ryuji Wakikawa <[email-masked]> wrote:

> _______________________________________________
> dmm mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/dmm

---

From: Liu Dapeng <[email-masked]>
Date: Thu, 11 Jul 2013 16:12:14 +0800

Hi Ryuji,

Thanks for posting an very interesting draft. Build distributed
mobility using BGP has been discussed in DMM before. For example:
http://tools.ietf.org/html/draft-mccann-dmm-flatarch-00.

My comments:
1. How to deal with routing fluctuation problem? Using BGP in Boeing
on-board system is quite different from mobile network. Boeing
on-board system normally does not have too much mobile node (the
flight is only one mobile node), but for the mobile network, there
could be hundreds of millions of mobile nodes keep moving and will
result in lots of routing updates.

2. Billing and policy enforcement is not discussed in the draft. Do
you have any thoughts on this?

Thanks,
Dapeng Liu

-- 

------
Best Regards,
Dapeng Liu

---

From: Liu Dapeng <[email-masked]>
Date: Thu, 11 Jul 2013 18:34:46 +0800

Another comment is the roaming issue. It is not covered in the draft.

But anyway, I agree with the motivation as discussed in section 2.1
and we also have submitted a SDN mobility discussion draft:
http://tools.ietf.org/html/draft-liu-sdn-mobility-00.

Thanks,
Regards,
Dapeng Liu

-- 

------
Best Regards,
Dapeng Liu

---

From: Ryuji Wakikawa <[email-masked]>
Date: Thu, 11 Jul 2013 08:11:14 -0700

Hi Jouni

Thanks for comments. 

Our proposal keeps the control plane untouched. All the signalling and functions stay same in U-Plane.
However If the information is carried in GTP-U header, the information is missed in our system.  
Can you point out what kind of data is that? The pointer to 3GPP specification is helpful too?
For the tasks in EPC-E, we don't have particular solution at this moment, but we are expecting some solution from network service chaining. I heard there will be a bog at the coming IETF. 

regards,
ryuji

---

From: Ryuji Wakikawa <[email-masked]>
Date: Thu, 11 Jul 2013 08:24:08 -0700

Hi Liu

For routing update matter, I will let my colleague to answer this. 
The routing update is happened just within an operator's network. 
The route for UE is just overwritten with the new information.   I believe BGP can easily handle these updates. 

We haven't discussed billing and roaming. but what we are thinking now is following. 
If special treatment is needed (like roaming), we keep using the original EPC functions available at NFV. Packets are routed to EPC U-/C-planes on NFV. 
For billing, we need some mechanism between U-/C- plane and didn't identify a solution yet. We will try to identify it at the next revision. 

thanks
ryuji

---

From: Behcet Sarikaya <[email-masked]>
Date: Thu, 11 Jul 2013 12:05:35 -0500

Hi Ryuji,

Thanks for submitting this draft. I read it and I also the comments.

Your draft is inline with the direction I have been advocating for dmm, I
think we should work on such issues.

I think you should present the draft in Berlin.

Regards,

Behcet

On Wed, Jul 10, 2013 at 11:12 AM, Ryuji Wakikawa
<[email-masked]>wrote:

> > Subject: New Version Notification for
> draft-matsushima-stateless-uplane-vepc-00.txt
> > URL:
> http://www.ietf.org/internet-drafts/draft-matsushima-stateless-uplane-vepc-00.txt
> > Status:
> http://datatracker.ietf.org/doc/draft-matsushima-stateless-uplane-vepc
> > Htmlized:
> http://tools.ietf.org/html/draft-matsushima-stateless-uplane-vepc-00
> >
> >
> > Abstract:

---

From: Satoru Matsushima <[email-masked]>
Date: Fri, 12 Jul 2013 04:52:46 +0900

Hi Liu,

Thanks for your good question. We, bgp operator are struggle fast
convergence for bgp that most of the issues come from best path
recalculation on bgp speaker. In the proposed architecture we expect no
such kind of recalc but just simple update to epc-e from vepc. BGP is an
incremental routing protocol so that bgp should be capable to continuously
update route to mobile nodes as they moving. Please noted that it is
happened only on RAN facing side on epc-e, not on ipv6 core network side.
So routing in ipv6 core can be much stable.

cheers,
--satoru

On Fri, Jul 12, 2013 at 12:24 AM, Ryuji Wakikawa
<[email-masked]>wrote:

> If special treatment is needed (like roaming), we keep using the original
> EPC functions available at NFV. Packets are routed to EPC U-/C-planes on
> NFV.
> For billing, we need some mechanism between U-/C- plane and didn't
> identify a solution yet. We will try to identify it at the next revision.
>
> thanks
> ryuji
>

---

From: Liu Dapeng <[email-masked]>
Date: Fri, 12 Jul 2013 09:59:58 +0800

2013/7/12 Satoru Matsushima <[email-masked]>:
> recalculation on bgp speaker. In the proposed architecture we expect no such
> kind of recalc but just simple update to epc-e from vepc. BGP is an

[Dapeng] You mean the MN's prefix can be converged in the core network? You can
do this when the MN moves in a small area, but if the MN moves out
from that area,
it is difficult to converge.

Thanks,
Dapeng Liu

 So
> routing in ipv6 core can be much stable.
>
> cheers,
> --satoru
>
>
>
>
> On Fri, Jul 12, 2013 at 12:24 AM, Ryuji Wakikawa <[email-masked]>
> wrote:
>>
>> Hi Liu
>>

--

------
Best Regards,
Dapeng Liu

---

From: Satoru Matsushima <[email-masked]>
Date: Fri, 12 Jul 2013 16:20:55 +0900

Liu,

MN's prefixes could be converged into an aggregated route. It is routing.
So MN are always expected within the area where it covers as long as the
aggregated route existed in the core network.

cheers,
--satoru

On Fri, Jul 12, 2013 at 10:59 AM, Liu Dapeng <[email-masked]> wrote:

> such
> >> The route for UE is just overwritten with the new information.   I
> believe
> >> BGP can easily handle these updates.
> >>
> >> We haven't discussed billing and roaming. but what we are thinking now
> is
> >> following.

---

From: Liu Dapeng <[email-masked]>
Date: Fri, 12 Jul 2013 17:11:39 +0800

Hi Satoru,

Then the question is how big this area will be to support all the
users of an operator?

regards,
Dapeng Liu

2013/7/12 Satoru Matsushima <[email-masked]>:
> Liu,
>
> MN's prefixes could be converged into an aggregated route. It is routing. So
> MN are always expected within the area where it covers as long as the

-- 

------
Best Regards,
Dapeng Liu

---

From: Satoru Matsushima <[email-masked]>
Date: Sat, 13 Jul 2013 00:13:24 +0900

Liu,

Yes, scalability issue is not discussed on the draft. Thanks for good
clarification.
Since EPC-E has two side of network that one is RAN side and another is
core side, scalability issue is not only on this architecture. The core
side could be the size of grobe as the Internet scale. But RAN side, an
operator would separate to some networks by region, or population,
whatever. So the question is to about how big the area will be to support
is a requirement of each operator.

There could be multiple EPC-E routers are deployed to support  multiple
RAN, or to solve scalability issue which huge number of MN population that
exceed the capacity of single EPC-E. This architecture should cover all of
these scenarios by routing basis strategy, so we'll update the draft in a
next version.

thanks!
--satoru

On Fri, Jul 12, 2013 at 6:11 PM, Liu Dapeng <[email-masked]> wrote:

> >> >> >>> A new version of I-D,
> >> >> >>> has been successfully submitted by Satoru Matsushima and posted
> to
> >> >> >>> the
> >> >> >>> IETF repository.
> >> >> >>>
> >> >> >>>  and 3) Manageability.  How to run the EPC control plane on NFV
> is
> >> >> >>> out

---

From: Peter McCann <[email-masked]>
Date: Sat, 13 Jul 2013 15:49:50 +0000

Hi, Ryuji, Satoru,

If I understand your draft correctly, you are encoding the PGW ID and the
TEID into the prefix portion of an IPv6 address.  You seem to allocate
16 bits for the TEID.  I thought that TEIDs in 3GPP were 32 bits.  Do you
have enough space?  Could you instead encode the TEID in the Interface
Identifier part of the IPv6 address?

Perhaps I have misunderstood how the routing update is supposed to work,
but why do you need to encode the UE's prefix or address at all in the
Next-Hop IPv6 address?  You have the UE's prefix as the Destination, and
the Next-Hop can just point to the tunnel, correct?

-Pete

Ryuji Wakikawa wrote:
>> A new version of I-D, draft-matsushima-stateless-uplane-vepc-00.txt has
>> been successfully submitted by Satoru Matsushima and posted to the IETF
>> repository.
>> 
>> Filename:	 draft-matsushima-stateless-uplane-vepc Revision:	 00
>> Creation date:	 2013-07-10 Group:		 Individual Submission Number of
>> pages: 19 URL:             http://www.ietf.org/internet-drafts/draft-
>> matsushima-stateless-uplane-vepc-00.txt Status:         
>> http://datatracker.ietf.org/doc/draft-matsushima- stateless-uplane-vepc

---

From: Satoru Matsushima <[email-masked]>
Date: Sun, 14 Jul 2013 02:20:53 +0900

Hi Pete,

Thanks for your comments.

> Identifier part of the IPv6 address?

Oops. Yes, you are correct. I'll update it soon.
Using TEID is one of the candidates of generating unique MN's prefix as described in stateless-pd draft.
(http://tools.ietf.org/html/draft-savolainen-stateless-pd-01)

Full of 32bits space is not necessary because embedded ID could be shortened by taking a part 16 bits or more longer from an TEID as long as its uniqueness is kept. We assume that the RAN side TEID assigned by SGW are encoded to MN's prefix. 

> the Next-Hop can just point to the tunnel, correct?

UE's prefix is not encoded in the next-hop. The next-hop indicates tunnel endpoint in the way of BGP remote next-hop draft. Please see "http://tools.ietf.org/html/draft-vandevelde-idr-remote-next-hop-03"

cheers,
--satoru

---

From: Jouni <[email-masked]>
Date: Sat, 13 Jul 2013 23:13:06 +0300

> Full of 32bits space is not necessary because embedded ID could be shortened by taking a part 16 bits or more longer from an TEID as long as its uniqueness is kept. We assume that the RAN side TEID…

TEIDs do not have structure, thus I would be careful starting to take shorter subsets out of it. 
There is quite a bit of vendor specific information already encoded into TEIDs so the text need
to be clear it does not assume anything about TEID or then state each deployment & prefix "structure"
is deployment/vendor specific.

- Jouni

---

From: Peter McCann <[email-masked]>
Date: Mon, 15 Jul 2013 18:49:05 +0000

Hi, Satoru,

Satoru Matsushima wrote:
>> If I understand your draft correctly, you are encoding the PGW ID
>> and the TEID into the prefix portion of an IPv6 address.  You seem
>> to allocate
>> 16 bits for the TEID.  I thought that TEIDs in 3GPP were 32 bits.
>> Do you have enough space?  Could you instead encode the TEID in the
>> Interface Identifier part of the IPv6 address?
> 
> Oops. Yes, you are correct. I'll update it soon. Using TEID is one of
> the candidates of generating unique MN's prefix as described in
> stateless-pd draft.
> Full of 32bits space is not necessary because embedded ID could be
> shortened by taking a part 16 bits or more longer from an TEID as long
> as its uniqueness is kept. We assume that the RAN side TEID assigned
> by SGW are encoded to MN's prefix.
> 
>> 
>> Perhaps I have misunderstood how the routing update is supposed to
>> work, but why do you need to encode the UE's prefix or address at
>> all in the Next-Hop IPv6 address?  You have the UE's prefix as the
>> Destination, and the Next-Hop can just point to the tunnel, correct?
> 
> UE's prefix is not encoded in the next-hop. The next-hop indicates
> tunnel endpoint in the way of BGP remote next-hop draft. Please see
> "http://tools.ietf.org/html/draft-vandevelde-idr-remote-next-hop-03"

I guess I don't understand the "PDN Prefix" and "subnet ID" fields that
you have in Figure 4.  You state:

   Each PDN is assumed to have single or several prefixes (called PDN
   prefix) used to generate UE's address.  Followed by the PDN prefix in
   Figure 4, there is 16-bit TEID assigned for a UE's session at SGW on
   the control plane.  TEID is 16 bits identifier in GTP header to
   distinguish each bearer.  The remaining bits are filled by subnet ID.
   The prefix is allocated per UE and used for address assignment by
   SLAAC or DHCPv6.

Is Figure 4 supposed to be the Next Hop or the Destination?  I assumed
it was Next Hop because it has the TEID encoded in it.  However, some
of the above paragraph leads me to believe it is about the UE's user
plane IP address.

-Pete

>>>> has been successfully submitted by Satoru Matsushima and posted to
>>>> the IETF repository.
>>>> 
>>>> pages: 19 URL:             http://www.ietf.org/internet-
>>>> drafts/draft- matsushima-stateless-uplane-vepc-00.txt Status:
>>>> http://datatracker.ietf.org/doc/draft-matsushima- stateless-uplane-
>>>> vepc Htmlized:        http://tools.ietf.org/html/draft-matsushima-
>>>> stateless-uplane-vepc-00
>>>> 
>>>> 
>>>> Abstract:
>>>> mobile nodes.  Although the EPC control plane will run on
>>>> hypervisor,  our proposal does not modify the signaling of the EPC
> control plane.
>>>> and 3) Manageability.  How to run the EPC control plane on NFV is
>>>> out  of our focus in this document.
>>>> 
>>>> 
>>>> 
>>>> The IETF Secretariat
>>>> 
>>>

---

From: Satoru Matsushima <[email-masked]>
Date: Tue, 16 Jul 2013 16:25:33 +0900

Hi Peter,

> of the above paragraph leads me to believe it is about the UE's user

The figure 4 illustrates UE's prefix so it shows the destination. 
Encoding TEID is just a seed to create UE's prefix in the stateless-pd manner.

We hasn't illustrate yet how to encode the TEID in next-hop. 
But as we mentioned in the document, we expect BGP remote next-hop as the way
to encode it in the sub-TLV of the remote next-hop attribute.

cheers,
--satoru

---

From: Peter McCann <[email-masked]>
Date: Tue, 16 Jul 2013 14:27:16 +0000

Satoru Matsushima wrote:
>> I guess I don't understand the "PDN Prefix" and "subnet ID" fields
>> that you have in Figure 4.  You state:
>> 

Doesn't the TEID change on every eNB handover?  I thought the UE's prefix
should remain constant across handovers.

> But as we mentioned in the document, we expect BGP remote next-hop as
> the way to encode it in the sub-TLV of the remote next-hop attribute.

I would think that the Next-Hop would change on mobility events, but the
Destination would remain the same (UE prefix).  Why is this not the case?

---

From: Bruno Mongazon-Cazavet <[email-masked]>
Date: Tue, 16 Jul 2013 17:10:27 +0200

> should remain constant across handovers.

Yes TEID changes on handover and when UE exits Idle-Mode (it has 
previously entered).

---

From: Satoru Matsushima <[email-masked]>
Date: Wed, 17 Jul 2013 01:09:54 +0900

>> should remain constant across handovers.
If you use stateless-pd rule for UE's prefix generation, the embedded TEID
is SGW assigned one. So the delegated prefix is stable during across eNB
handover. Please see the section 3.3 in update version of the draft.
Anycast routing no longer requires hand-over between SGW in virtualized EPC.

> Yes TEID changes on handover and when UE exits Idle-Mode (it has

Thanks, we haven't mention about in the case of Idle-Mode exit. will update
with it in a next version. It would be in control-plane awareness section.

>> Destination would remain the same (UE prefix).  Why is this not the case?
Yes, exactly.

cheers,
--satoru

---

From: Peter McCann <[email-masked]>
Date: Wed, 17 Jul 2013 17:11:42 +0000

Hi, Satoru,

Satoru Matsushima wrote:
> On Wed, Jul 17, 2013 at 12:10 AM, Bruno Mongazon-Cazavet 
> <[email-masked]> wrote:
> 
> 
> 				   Each PDN is assumed to have single or several prefixes (called
> PDN 				   prefix) used to generate UE's address. Followed by the PDN
> prefix in 				   Figure 4, there is 16-bit TEID assigned for a UE's
> session at SGW on 				   the control plane.  TEID is 16 bits identifier
> in GTP header to 				   distinguish each bearer.  The remaining bits are
> filled by subnet ID. 				   The prefix is allocated per UE and used for
> address assignment by 				   SLAAC or DHCPv6. 				Is Figure 4 supposed
> to be the Next Hop or the Destination?  I assumed 				it was Next Hop
> because it has the TEID encoded in it.  However, some 				of the above
> paragraph leads me to believe it is about the UE's user 				plane IP
> address.
> 
> 
> 
> 		Doesn't the TEID change on every eNB handover?  I thought the UE's
> prefix 		should remain constant across handovers.
> 
> 
> 
> 
> If you use stateless-pd rule for UE's prefix generation, the embedded 
> TEID is SGW assigned one. So the delegated prefix is stable during 
> across eNB handover. Please see the section 3.3 in update version of 
> the draft. Anycast routing no longer requires hand-over between SGW in 
> virtualized EPC.
> 
> 
> 
> Thanks, we haven't mention about in the case of Idle-Mode exit. will 
> update with it in a next version. It would be in control-plane 
> awareness section.

I think taken together, these facts would mean that the TEID space is
shared across all EPC-E devices (they all have the same anycast address)
and also that a TEID would need to remain allocated even to idle UEs.

Would you eventually run out of TEID space?

> 			We hasn't illustrate yet how to encode the TEID in next-hop. 			But
> as we mentioned in the document, we expect BGP remote next-hop as 			the
> way to encode it in the sub-TLV of the remote next-hop attribute.
> 
> 
> 		I would think that the Next-Hop would change on mobility events, but
> the 		Destination would remain the same (UE prefix).  Why is this not
> the case?
> 
> 
> 
> Yes, exactly.

Ok, so the Destination IPv6 address needs to remain the same during the lifetime
of a UE session, even across idle transitions and mobility events, right?

-Pete

---

From: Satoru Matsushima <[email-masked]>
Date: Thu, 18 Jul 2013 19:40:32 +0900

Hi Pete,

On Thu, Jul 18, 2013 at 2:11 AM, Peter McCann <[email-masked]>wrote:

> >                       Encoding TEID is just a seed to create UE's prefix
> in the
> > stateless-pd                  manner.
> >
> >

To be honest, I'm not sure whether the 32bits space of TEID could be
exhausted in almost deployment cases. :)
But when any logical(TEID, etc.,) or physical(bandwidth, etc.,) resources
are exhausted, you can build additional set of EPC-Es which share another
anycast address. Please see section 4 of the draft.

> of a UE session, even across idle transitions and mobility events, right?

Right.

regards,
--satoru