Subject: [v6ops] current solutions for incoming connections in CGN environments

From: Tassos Chatzithomaoglou <[email-masked]>
Date: Wed, 18 Jul 2012 19:29:51 +0300

Hi,

I'm looking for opinions and recommendations regarding the known limitations with incoming connections & port forwarding 
in CGN-like environments (preferably DS-Lite), in environments with multiple layers of NAT or when NAT is not taking 
place on the CPE.
If i'm not mistaken, currently you cannot have such a functionality based on standards.
There seem to be some solutions on the way, but i'm looking for something that is available now, something that you 
would implement now if you had to go live in a few months.
i.e. PCP seems to take too long, and besides some CPE vendors that offer beta firmware based on the drafts, i haven't 
seen any major router vendors implementing the PCP server function officially (besides vendor's beta demo at IETF 81).

All operators using CGN, how do you currently solve this problem?
Any plans to write an informational RFC based on this?
Or is PCP the only solution and everyone is waiting for it?

-- 
Tassos

---

From: Cameron Byrne <[email-masked]>
Date: Wed, 18 Jul 2012 09:34:31 -0700

On Wed, Jul 18, 2012 at 9:29 AM, Tassos Chatzithomaoglou
<[email-masked]> wrote:
> Hi,
>
> I'm looking for opinions and recommendations regarding the known limitations
> with incoming connections & port forwarding in CGN-like environments
> (preferably DS-Lite), in environments with multiple layers of NAT or when
> NAT is not taking place on the CPE.
> There seem to be some solutions on the way, but i'm looking for something
> that is available now, something that you would implement now if you had to
> go live in a few months.
> i.e. PCP seems to take too long, and besides some CPE vendors that offer
> beta firmware based on the drafts, i haven't seen any major router vendors
> implementing the PCP server function officially (besides vendor's beta demo
> at IETF 81).
>

Since this is the v6ops lists, the solution is don't invest in more
life support for IPv4.

IPv6 does not have these problems.

If your service is DS-lite, customers should have a very capable IPv6
connection and they can receive incoming connection over IPv6.

CB

---

From: james woodyatt <[email-masked]>
Date: Wed, 18 Jul 2012 10:23:58 -0700

On Jul 18, 2012, at 09:34 , Cameron Byrne <[email-masked]> wrote:
> 
> If your service is DS-lite, customers should have a very capable IPv6 connection and they can receive incoming connection over IPv6.

Unless the CPE gateway has RFC 6092 Simple Security enabled--which MAY be the default configuration in many cases--in which case inbound flows other than IKE and IPsec ESP are rejected unless there is an integrated PCP service available to hosts for mapping other ports and protocols according to REC-48.

   REC-48  Internet gateways with IPv6 simple security capabilities
           SHOULD implement a protocol to permit applications to solicit
           inbound traffic without advance knowledge of the addresses of
           exterior nodes with which they expect to communicate.

If there are IPv6 CPE gateways, commercially available now with RFC 6092 Simple Security filtering and an integrated PCP server for meeting REC-48, then I'd like to know about them.  In particular, I'd like to know about any such gateways that have Simple Security enabled in the default configuration.

--
james woodyatt <[email-masked]>
member of technical staff, core os networking

---

From: Victor Kuarsingh <[email-masked]>
Date: Wed, 18 Jul 2012 13:24:17 -0400

> connection and they can receive incoming connection over IPv6.

I would tend to agree with Cameron here.  Not sure how much support you
will get on this list to solve this IPv4 issue.

This is one of the many issues with CGN.  I would try an reach out to the
vendors and see if they have any specific capabilities on their platforms
to incoming connections to work, but the larger issue we saw when analyzing
this was getting it to be automated (customer portal, or other capability
to set the port forwarding rules etc).

We are waiting for PCP and have accepted the gap for now.

Perhaps if you have some number of customers who need incoming port
forwarding, then if possible, keep them off CGN (you may not have that
option).  If the port forwarding is required to support functions inside
your network, you can try and by-pass the CGN (but I had assumed you need
this for general incoming connections which would originate outside your
network to your customers).

regards,

Victor K

>
> CB
> _______________________________________________
> v6ops mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/v6ops
>

---

From: Gert Doering <[email-masked]>
Date: Wed, 18 Jul 2012 19:46:25 +0200

Hi,

On Wed, Jul 18, 2012 at 07:29:51PM +0300, Tassos Chatzithomaoglou wrote:
> I'm looking for opinions and recommendations regarding the known
> limitations with incoming connections & port forwarding
> in CGN-like environments (preferably DS-Lite), in environments
> with multiple layers of NAT or when NAT is not taking
> place on the CPE.

I've been told the solution is called "IPv6".

(And I'm actually quite serious about that: do you really want to invest
more and more money on a dead protocol, or do you want to show your 
customers some incentives why IPv6 is really what they want to have?)

Gert Doering
        -- NetMaster
-- 
have you enabled IPv6 on something today...?

SpaceNet AG                        Vorstand: Sebastian v. Bomhard
Joseph-Dollinger-Bogen 14          Aufsichtsratsvors.: A. Grundner-Culemann
D-80807 Muenchen                   HRB: 136055 (AG Muenchen)
Tel: +49 (89) 32356-444            USt-IdNr.: DE813185279

---

From: Tassos Chatzithomaoglou <[email-masked]>
Date: Wed, 18 Jul 2012 21:19:31 +0300

The problem is that although we have enabled IPv6 in our network, we don't expect traffic 
patterns to change in the near future... which means IPv4's usage will remain at high 
levels for some years.
IPv6 traffic is still limited inside our network (and everywhere) in comparison to IPv4. 
And since we need to support our customer growth, we need to invest in IPv4 connectivity 
too for the time being.

We already have started implementing an semi-automated way keeping DS-Lite "unfriendly" 
customers outside the CGN using our TR-069 platform, but it's getting more and more 
complicated as we dig into it.
If PCP is the only way to solve this issue in the near future, then pushing our vendors to 
this direction (after it becomes a standard) is in the right direction.
I was just hoping that other operators have already met and solved this issue.
IPv6 is definitely the final and best solution and we are ready for it in most areas. We 
just need another (intermediate) solution for now, to keep customers happy and growing at 
the same time.

--
Tassos

Victor Kuarsingh wrote on 18/7/2012 20:24:
>
> On Wed, Jul 18, 2012 at 12:34 PM, Cameron Byrne <[email-masked] 
> <mailto:[email-masked]>> wrote:
>
>     <[email-masked] <mailto:[email-masked]>> wrote:
>     > Hi,
>     >
> I would tend to agree with Cameron here.  Not sure how much support you will get on this 
> list to solve this IPv4 issue.
>
> This is one of the many issues with CGN.  I would try an reach out to the vendors and 
> see if they have any specific capabilities on their platforms to incoming connections to 
> work, but the larger issue we saw when analyzing this was getting it to be automated 
> (customer portal, or other capability to set the port forwarding rules etc).
>
> Perhaps if you have some number of customers who need incoming port forwarding, then if 
> possible, keep them off CGN (you may not have that option).  If the port forwarding is 
> required to support functions inside your network, you can try and by-pass the CGN (but 
> I had assumed you need this for general incoming connections which would originate 
> outside your network to your customers).
>
> regards,
>
> Victor K
>
>
>     CB
>     [email-masked] <mailto:[email-masked]>

---

From: Gert Doering <[email-masked]>
Date: Wed, 18 Jul 2012 21:10:00 +0200

Hi,

> just need another (intermediate) solution for now, to keep customers happy and growing at 

There is no way to make the customers happy with IPv4+CGN.

Gert Doering
        -- NetMaster

---

From: Tassos Chatzithomaoglou <[email-masked]>
Date: Wed, 18 Jul 2012 23:33:13 +0300

Gert, i'm more than open to proposals.
Our IPv6 is completed, our DS-Lite will be completed soon, our customers are growing, our 
IPv4 addresses will be over soon...and IPv4 content (and traffic?) is still ~99%.
Am i missing something obvious?

I'm a fanatic supporter of IPv6, but as long as IPv6-only solutions do not cover us (and 
our customers), we'll have to keep investing on IPv6 + IPv4.
We could very easily have chosen the IPv4-only way of doing CGN things, but we chose 
DS-Lite in order to prepare for the future.
If there is a better solution (currently available), i'm more than happy to discuss it 
(broadcast or unicast).
Unless, if buying IPv4 addresses is considered a better solution. But then you avoid CGN 
and IPv6.

--
Tassos

---

From: Gert Doering <[email-masked]>
Date: Wed, 18 Jul 2012 22:39:02 +0200

Hi,

> Am i missing something obvious?

You'll be surprised how much traffic google and facebook create, as soon
as your access customers have ipv6-on-by-default.  Much more than 1%.

But when talking about NAT and "mapping ports back to your customers",
it's fully and totally irrelevant how much content is there on IPv6 - if
it's "I want to connect home!", having the customer on IPv6 is sufficient,
it doesn't matter whether www.somebigsite.com needs to go through an
IPv4 NAT444 or other abominations.

Gert Doering
        -- NetMaster

---

From: Tore Anderson <[email-masked]>
Date: Thu, 19 Jul 2012 07:58:59 +0200

Hi Tassos,

* Tassos Chatzithomaoglou

> Our IPv6 is completed, our DS-Lite will be completed soon, our customers
> are growing, our IPv4 addresses will be over soon...and IPv4 content
> (and traffic?) is still ~99%.

No. If it's any consolation, it's not only you - we're all in the same
shitty situation. By deploying IPv6 you are way ahead of most people in
dealing with it though, congratulations!

That said, according to Google's statistics at
http://www.google.com/ipv6/statistics.html#tab=per-country-ipv6-adoption
there are essentially no IPv6 traffic seen from Greece, which is odd as
Forthnet is a large source of Greek eyeball traffic (second only to OTE
in my netflow stats). Is your IPv6 implementation inside a walled garden
or is there any other reason why it isn't show up in public stats like
Google's?

> I'm a fanatic supporter of IPv6, but as long as IPv6-only solutions do
> not cover us (and our customers), we'll have to keep investing on IPv6 +
> IPv4.
> We could very easily have chosen the IPv4-only way of doing CGN things,
> but we chose DS-Lite in order to prepare for the future.
> If there is a better solution (currently available), i'm more than happy
> to discuss it (broadcast or unicast).

I suggest you look into the MAP/4RD stuff. Like DS-Lite and NAT444, it
facilitates IPv4 address sharing between subscribers, but it does it
without a CGN component - each subscriber gets an official IPv4 address
routed all the way to his CPE like today, but a limited set of TCP/UDP
ports. From this port range, the user can set up port forwardings using
UPnP or manual CPE config just like he can today. You can also, if I
understand correctly, provision an "entire" IPv4 address over MAP/4RD
too, for example as a "premium" paid-for service for customers who
insist on having specific well-known ports available to them.

MAP rides on top of IPv6 in much the same way as DS-Lite, so since
you've already implemented IPv6, you've already done with the hard part.
Another very nice thing about it compared to DS-Lite is that it is
entirely stateless in the core (just simple NAT44 in the CPEs like today).

I think it fails your «currently available» criteria, though. But...

> Unless, if buying IPv4 addresses is considered a better solution. But
> then you avoid CGN and IPv6.

...the RIPE NCC still has more than 10 million IPv4 addresses in its
free pool, and you can still allocate normally from it. So by making an
allocation now, you will have bought yourself three months of time
before you have to light up your DS-Lite CGN, in which you can wait for
(or even better - help finalise!) PCP and/or MAP. I wouldn't abandon
DS-Lite though, having a contingency plan ready to go the moment you
need it is certainly smart.

I don't understand why a strategy of buying IPv4 addresses instead of
deploying any IPv4 address sharing solution would make you «avoid IPv6»,
though. IPv4 life-support and IPv6 deployments seem orthogonal to me,
except that the former may benefit from the latter in the case of
DS-Lite, MAP, and similar.

Best regards,
-- 
Tore Anderson
Redpill Linpro AS - http://www.redpill-linpro.com

---

From: Lorenzo Colitti <[email-masked]>
Date: Thu, 19 Jul 2012 17:15:31 +0900

On Thu, Jul 19, 2012 at 5:33 AM, Tassos Chatzithomaoglou <
[email-masked]> wrote:

> are growing, our IPv4 addresses will be over soon...and IPv4 content (and
> traffic?) is still ~99%.

Yes, you are. If your IPv4 traffic is 99%, then your IPv6 project is not
"completed". There is much more than 1% of content and traffic on IPv6. As
AT&T says: "For our IPv6 enabled customers, we’re seeing more than 20
percent of traffic transition to IPv6":
http://www.attinnovationspace.com/innovation/story/a7782696 . If you don't
have enough IPv6 traffic, you probably haven't deployed IPv6 to enough
users.

One solution to your problem is as follows:

1. Deploy IPv6 to a large percentage of your users
2. Take away the public IPv4 addresses for those users and put them behind
a NAT.

If a user wants inbound connectivity (and many probably don't know they do,
or don't care), then give them a public IPv4 address (either for free or
for an extra fee).

This appears to be what AT&T is doing: public numbers suggest that they
have more than 4% of IPv6 users, and there are hints on the forums that
they are asking their customers to renumber their internal networks out of
10/8 (which suggests carrier-operated NAT).

So before you tell us why this is impossible and will never work, bear in
mind that one of the largest ISPs in the world is doing it :-)

Regards,
Lorenzo

---

From: Tassos Chatzithomaoglou <[email-masked]>
Date: Thu, 19 Jul 2012 23:31:55 +0300

> or is there any other reason why it isn't show up in public stats like

We enable IPv6 on CPEs in batches (we've met many strange issues in our pilot and we want to be careful), so it will 
take a while until we have a considerable amount of traffic shown there. At the same time we are testing (and enhancing) 
our IPv6 address allocation system, so we cannot move very fast, unless everything is working fine.
> I think it fails your «currently available» criteria, though. But...
MAP/4RD will probably take a while until it gets into CPEs.
DS-Lite, which is older, got in very recently.
> DS-Lite, MAP, and similar.

Marketing-wise, the acceptance for IPv6 was driven solely by the exhaustion of IPv4 addresses. "If we buy more, we don't 
(currently) need IPv6. It's like buying more time to get more customers."
Please don't ask my opinion about this...

--
Tassos

---

From: Tassos Chatzithomaoglou <[email-masked]>
Date: Thu, 19 Jul 2012 23:55:37 +0300

Lorenzo Colitti wrote on 19/07/2012 11:15:
> On Thu, Jul 19, 2012 at 5:33 AM, Tassos Chatzithomaoglou <[email-masked] <mailto:[email-masked]>> wrote:
>
>     Our IPv6 is completed, our DS-Lite will be completed soon, our customers are growing, our IPv4 addresses will be
>     over soon...and IPv4 content (and traffic?) is still ~99%.
> Yes, you are. If your IPv4 traffic is 99%, then your IPv6 project is not "completed". There is much more than 1% of 
> content and traffic on IPv6. As AT&T says: "For our IPv6 enabled customers, we’re seeing more than 20 percent of 
> traffic transition to IPv6": http://www.attinnovationspace.com/innovation/story/a7782696 . If you don't have enough 
> IPv6 traffic, you probably haven't deployed IPv6 to enough users.
>
As i already told to Tore, we still haven't enabled IPv6 to enough users' CPEs. I expect this number to grow in the next 
few weeks.

> 2. Take away the public IPv4 addresses for those users and put them behind a NAT.
>
> If a user wants inbound connectivity (and many probably don't know they do, or don't care), then give them a public 
> IPv4 address (either for free or for an extra fee).
>
This is what we are planning to do (in a semi-automated way), but it goes against our marketing-moto: "Every customer 
should have connectivity to/from everywhere every-time, over everything.".
> This appears to be what AT&T is doing: public numbers suggest that they have more than 4% of IPv6 users, and there are 
> hints on the forums that they are asking their customers to renumber their internal networks out of 10/8 (which 
> suggests carrier-operated NAT).
>
> So before you tell us why this is impossible and will never work, bear in mind that one of the largest ISPs in the 
> world is doing it :-)
>
It would be good to know if they are running out of IPv4 addresses like us, which i believe they don't, since they were 
lucky enough to get many in the beginning.

--
Tassos

---

From: Arturo Servin <[email-masked]>
Date: Thu, 19 Jul 2012 17:11:46 -0400

Probably they are. If you are talking about 12/8 this one is only 16M address, far away less of the amount of customers that ATT probably has and need an IP.

regards,
as

On 19 Jul 2012, at 16:55, Tassos Chatzithomaoglou wrote:

>> This appears to be what AT&T is doing: public numbers suggest that they have more than 4% of IPv6 users, and there are hints on the forums that they are asking their customers to renumber their internal networks out of 10/8 (which suggests carrier-operated NAT).
>> 
>> So before you tell us why this is impossible and will never work, bear in mind that one of the largest ISPs in the world is doing it :-)
>> 
> It would be good to know if they are running out of IPv4 addresses like us, which i believe they don't, since they were lucky enough to get many in the beginning.

---

From: Daniel Roesen <[email-masked]>
Date: Mon, 23 Jul 2012 08:06:45 +0200

On Wed, Jul 18, 2012 at 10:23:58AM -0700, james woodyatt wrote:
> In particular, I'd like to know about any such gateways that have
> Simple Security enabled in the default configuration.

AVM Fritz!Box. But you can configure holes to specific hosts+ports
in the LAN.

Best regards,
Daniel

-- 
CLUE-RIPE -- Jabber: [email-masked] -- dr@IRCnet -- PGP: 0xA85C8AA0

---

From: Wuyts Carl <[email-masked]>
Date: Mon, 23 Jul 2012 08:21:05 +0200

Same for the Technicolor CPE.  By default: drop unsollicited inbound traffic and (optionally) punch holes for specific traffic if needed.

Regs
Carl

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Daniel Roesen
Sent: maandag 23 juli 2012 8:07
To: [email-masked]
Subject: Re: [v6ops] current solutions for incoming connections in CGN environments

> Simple Security enabled in the default configuration.

AVM Fritz!Box. But you can configure holes to specific hosts+ports in the LAN.

Best regards,
Daniel

--
CLUE-RIPE -- Jabber: [email-masked] -- dr@IRCnet -- PGP: 0xA85C8AA0 _______________________________________________
v6ops mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/v6ops

---

From: Tim Chown <[email-masked]>
Date: Mon, 23 Jul 2012 15:00:44 +0100

On 19 Jul 2012, at 09:15, Lorenzo Colitti wrote:

> On Thu, Jul 19, 2012 at 5:33 AM, Tassos Chatzithomaoglou <[email-masked]> wrote:
> Our IPv6 is completed, our DS-Lite will be completed soon, our customers are growing, our IPv4 addresses will be over soon...and IPv4 content (and traffic?) is still ~99%.
> Yes, you are. If your IPv4 traffic is 99%, then your IPv6 project is not "completed". There is much more than 1% of content and traffic on IPv6. As AT&T says: "For our IPv6 enabled customers, we’re seeing more than 20 percent of traffic transition to IPv6": http://www.attinnovationspace.com/innovation/story/a7782696 . If you don't have enough IPv6 traffic, you probably haven't deployed IPv6 to enough users.

I have seen other reports/measurements of over 40%, and in some cases 50%, of web content (by traffic volume, not by sites enabled) available over IPv6.  W6L has blown away that 1% stat. 

Tim

---

From: Francis Dupont <[email-masked]>
Date: Tue, 24 Jul 2012 12:45:45 +0200

In your previous mail you wrote:

>  limitations with incoming connections & port forwarding in CGN-like
>  environments (preferably DS-Lite), in environments with multiple
>  layers of NAT or when NAT is not taking place on the CPE.

=> UPnP IGD v2 (with the whole new framework, BTW not the v1) does the
job but I don't know if it is popular for CGN vendors. BTW it was published
~20 months ago...

> If i'm not mistaken, currently you cannot have such a functionality

=> define "standards" (:-)...

Regards

[email-masked]

PS: I know the IPv6 part (IPv6 firewall control, more CPE oriented)
was implemented. BTW this has no equivalent in PCP yet.

---

From: "Rajiv Asati (rajiva)" <[email-masked]>
Date: Wed, 25 Jul 2012 20:57:13 +0000

> MAP rides on top of IPv6 in much the same way as DS-Lite, so since you've
> already implemented IPv6, you've already done with the hard part.
> Another very nice thing about it compared to DS-Lite is that it is entirely
> stateless in the core (just simple NAT44 in the CPEs like today).
> 
> I think it fails your <currently available> criteria, though. But...

Turns out that there are available implementations (on routers and CPE/home gateways) that will be demoed at the IETF next week. There was an email about it on softwire WG.

Cheers,
Rajiv

> From: [email-masked] [mailto:[email-masked]] On Behalf
> Of Tore Anderson
> Sent: Thursday, July 19, 2012 1:59 AM
> To: Tassos Chatzithomaoglou
> Cc: [email-masked] WG
> > Our IPv6 is completed, our DS-Lite will be completed soon, our
> > customers are growing, our IPv4 addresses will be over soon...and IPv4
> > content (and traffic?) is still ~99%.
> No. If it's any consolation, it's not only you - we're all in the same shitty
> situation. By deploying IPv6 you are way ahead of most people in dealing
> with it though, congratulations!
> 
> Forthnet is a large source of Greek eyeball traffic (second only to OTE in my
> netflow stats). Is your IPv6 implementation inside a walled garden or is
> there any other reason why it isn't show up in public stats like Google's?
> 
> > not cover us (and our customers), we'll have to keep investing on IPv6
> > + IPv4.
> > We could very easily have chosen the IPv4-only way of doing CGN
> > things, but we chose DS-Lite in order to prepare for the future.
> > If there is a better solution (currently available), i'm more than
> > happy to discuss it (broadcast or unicast).
> 
> facilitates IPv4 address sharing between subscribers, but it does it without a
> CGN component - each subscriber gets an official IPv4 address routed all the
> way to his CPE like today, but a limited set of TCP/UDP ports. From this port
> range, the user can set up port forwardings using UPnP or manual CPE config
> just like he can today. You can also, if I understand correctly, provision an
> "entire" IPv4 address over MAP/4RD too, for example as a "premium" paid-
> for service for customers who insist on having specific well-known ports
> available to them.
> 
> MAP rides on top of IPv6 in much the same way as DS-Lite, so since you've
> already implemented IPv6, you've already done with the hard part.
> Another very nice thing about it compared to DS-Lite is that it is entirely
> stateless in the core (just simple NAT44 in the CPEs like today).
> 
> I think it fails your <currently available> criteria, though. But...
> 
> ...the RIPE NCC still has more than 10 million IPv4 addresses in its free pool,
> and you can still allocate normally from it. So by making an allocation now,
> you will have bought yourself three months of time before you have to light
> up your DS-Lite CGN, in which you can wait for (or even better - help
> finalise!) PCP and/or MAP. I wouldn't abandon DS-Lite though, having a
> contingency plan ready to go the moment you need it is certainly smart.
> 
> deploying any IPv4 address sharing solution would make you <avoid IPv6>,
> except that the former may benefit from the latter in the case of DS-Lite,
> MAP, and similar.
> 
> Best regards,
> --
> Tore Anderson

---

From: Tassos Chatzithomaoglou <[email-masked]>
Date: Thu, 26 Jul 2012 10:40:08 +0300

> job but I don't know if it is popular for CGN vendors. BTW it was published
I must admit that i haven't looked too much at UPnP stuff, but isn't that supposed to work 
between the host and the CPE on the LAN side?
Are there any UPnP IGD implementations having to do with the connection between the CPE 
and the CGN on the WAN side?

> => define "standards" (:-)...

I'm actually referring to protocols than have become proposed standard RFCs.

Regards,
Tassos

---

From: Francis Dupont <[email-masked]>
Date: Thu, 26 Jul 2012 15:18:35 +0200

In your previous mail you wrote:

>  I must admit that i haven't looked too much at UPnP stuff, but
>  isn't that supposed to work between the host and the CPE on the LAN
>  side?

=> yes but there is nothing which makes it to not work between the
host(s) and the CGN when you have replaced the multicast discovery by
something more explicit.

>  Are there any UPnP IGD implementations having to do with the
>  connection between the CPE and the CGN on the WAN side?

=> why bother with the CPE? the NAT is the CGN so it is the device you
want to control (note you can go to A+P solutions like light weight
DS-Lite too, or simply IPv6...)  and UPnP IGD v2 provides this.

BTW for real world solutions you have the portal (i.e., a web
interface where customers can configure port forwarding entries) which
is very common. Note it is not a bad idea to just have one (or a few)
static entrie(s) for a ssh or TLS tunnel port where the incoming
connections are multiplexed in a secure way: not secure dynamic
forwarding entries can be stolen as CGNs don't use stable storage to
keep them (*)...

Regards

(*): in fact for "legal" logs CGNs have stable storage but ISPs which
deploy CGNs usually don't care about customer security or similar
service quality...