Subject: [conex] ConEx & DDoS

From: Bob Briscoe <[email-masked]>
Date: Tue, 03 Aug 2010 20:42:22 +0100

Chris,

During the ConEx w-g session last Tuesday in Maastricht you suggested 
we should not include DDoS mitigation as a use-case for ConEx. I was 
willing to agree as we don't need to court controversy.

However, the co-authors of draft-moncaster-conex-concepts-uses have 
asked me to float the idea that, altho we won't include DDoS as a 
use-case in its own right, we should mention it as an extreme case of 
two other use-cases. Let me explain...

The use-cases we plan to include are:
/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\
5. Use cases (Highlighting that this is neither an exhaustive list nor a
prescriptive list...)
  5.1  ConEx for better traffic Control
   a. Targeting the right traffic
   b. Encouraging (and eventually enforcing) better CC
  5.2 ConEx for better traffic monitoring
   a. For compliance with SLAs
   b. For assessing performance of your provider
   c. Monitoring congestion hotspots for targeted upgrades
   d. Monitoring congestion anomalies (Equipment problems or helping 
identify DDoS)
/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\

We could include mentions of DDoS along the following lines (no need 
to word-smith - I'm just trying to outline the concepts).

5.1b Encouraging (and eventually enforcing) better CC
"  ConEx information can be used as a control metric for making
    traffic control decisions, such as deciding which traffic to
    prioritise or to identify and block sources of persistent and
    damaging congestion.

    Simple ingress policer mechanisms, such as those described in
    [Policing-freedom] and [re-ecn-motive], could control the
    overall volume of congestion entering a network from each user.
    Such a policer could lead to a number of beneficial outcomes:

    o Heavy users might be encouraged to shift their usage away from
      peak times in order to be able to transfer more data without
      triggering a response from the policer;
    o Users might be encouraged to use software that shifts its usage away
      from congestion peaks (shifting in time whether by hours or seconds
      [LEDBAT] or shifting to less congested routes [MPTCP]), again to
      transfer more data without triggering the policer;
    o Developers of operating systems might be encouraged to supply such
      software as the default;
    o If certain applications did not use a congestion responsive transport
      and caused high levels of congestion-bit-rate, the policer would
      eventually force the bit-rate to reduce in response to congestion.

    It is believed that a system of ConEx policers could be built that
    can verify the integrity of ConEx information and remove traffic
    that does not comply with the protocol [re-ecn-motive]. If such
    robustness against cheating is indeed possible, a ConEx policer would
    mitigate DDoS flooding attacks, at least to some extent, merely as a
    function of its ability to enforce a response to persistent and
    excessive congestion.

    It would be foolhardy to claim that it will be possible to make ConEx
    invulnerable to all cheats and attacks, even though it has been hard to
    attack it so far. Nonetheless, ConEx would still be useful even if a
    specific deployment of ConEx policers contained vulnerabilities. ConEx
    could still prevent congestion collapse due to careless ommission of
    congestion control, or due to release of software containing an
    accidental congestion control bug.
"

5.2d: Monitoring congestion anomalies
"  One of the most useful things ConEx provides is the ability to
    monitor the amount of congestion entering a network. Thus ConEx
    would add congestion to the information used by existing anomaly
    detection systems, thus greatly improving their ability to discriminate
    between pathological and benign anomalies. Such congestion-carrying
    anomalies might be due to accidental misconfigurations in another
    network, or deliberate malicious attacks.

    ConEx provides the additional benefit that it exposes congestion
    information as packets enter a network, not only at the point of
    congestion. Therefore it could be feasible for anomaly detection
    systems to use ConEx information to detect and shut down dangerous
    floods of congestion at the point where traffic enters a network.
"

Do you think circumspect wording about DDoS like the above would 
still trigger an allergic reaction from some readers? Would this sort 
of text allay your concerns? Or are you adamant that there should be 
no mention of DDoS at all?

Bob

________________________________________________________________
Bob Briscoe,                                BT Innovate & Design

---

From: Christopher Morrow <[email-masked]>
Date: Tue, 3 Aug 2010 23:37:47 -0400

(I think I sub'd to the list with this address...)

On Tue, Aug 3, 2010 at 3:42 PM, Bob Briscoe <[email-masked]> wrote:
> Chris,
>
> During the ConEx w-g session last Tuesday in Maastricht you suggested we
> should not include DDoS mitigation as a use-case for ConEx. I was willing to
> agree as we don't need to court controversy.

yup, no use ratholing if it's not central to the discussion.

> However, the co-authors of draft-moncaster-conex-concepts-uses have asked me
> to float the idea that, altho we won't include DDoS as a use-case in its own
> right, we should mention it as an extreme case of two other use-cases. Let
> me explain...
>
>  d. Monitoring congestion anomalies (Equipment problems or helping identify
> DDoS)
> We could include mentions of DDoS along the following lines (no need to
> word-smith - I'm just trying to outline the concepts).
>

this, I think, falls into the category of things that Lee from TWC is
interested in (and maybe his buddies at comcast/cox as well),
understanding what traffic can suffer more loss without causing more
end-user pain, and shifting that traffic to said profile.

>   Such a policer could lead to a number of beneficial outcomes:

these seem to be of the flavor of things Comcast 'powerboost' does, at
the CPE actually (if I understand their brand of magic correctly).

>     eventually force the bit-rate to reduce in response to congestion.

Today this is (the last bullet) the equivalent of classifying a type
of traffic (port/protocol/src/dst as classification hints) into a
'high loss' bucket/queue and just dropping faster/more of it. If the
traffic is TCP you should get drops, backoff, sawtooth behaviour until
you reach steady-state (maybe?) slower transfers.

>   function of its ability to enforce a response to persistent and

So, there are surely cases of 'ddos' (or DoS) that include loud
speakers... There are also many instances of DDoS that are many
hundreds of thousands of (or millions) of very quiet voices that in
total cause the DoS/DDoS effect.

If you look at a system of marking of congestion information,
depending upon where that marking happens, and on the traffic in
question, there's no guarantee at all that the sources will be
squelched.

For example, imagine a DDoS of 1 RST pkt (could also do this with 1
icmp-error-type message) per second from 1 million hosts across the
whole of the network (a 'botnet' for instance). There will never be
any packet sent back to the originators, the rate will be low enough
that unless there is coincident traffic to the victim from these hosts
(in the same protocol/port profile probably) no signal will ever be
seen that leads to squelching of traffic. The victim still suffers
~1mpps of traffic, and actually conex/CC just makes the problem far
worse for the victim as all of his real-user traffic will be marked
(over time) and thus squelched out while the DDoS continues to flow
unabated.

>   accidental congestion control bug.

I think that I nullified the above paragraph actually... except for a
software bug case, though I'd argue that in the case of the UW NTP
server incident ConEx/CC wouldn't have helped there either as the
broken software would still have been broken and the real-users of the
system would have just been CC"d out of existence.

>   network, or deliberate malicious attacks.

I'm on board with letting folks know there is congestion, I'm not
convinced making decisions based on this is simple/helpful, yet.
Adding this information to packets, provided I don't need new
silicon/cpu-cycles/state to do this doesn't strike me as hard, I can
imagine that most backbone providers wont' care and probably won't
implement the marking, but... I could be wrong.

Marking something purely 'congested path' or no though isn't very
helpful (there are almost always) many hops along a path and many
paths a packet/flow may take. At a single point in time one part of a
path may be congested, but there isn't any guarantee that part will be
affected even by the 1RTT time required to do something. (last-mile
problems aside, nothing fixes a 56k modem except... more bandwidth)

Being able to use information about congestion along a path to more
efficiently use the available bandwidth is a fine plan, less
re-transmits is a good plan.

>   congestion. Therefore it could be feasible for anomaly detection

ok, this is a point I need clarification on. Where is the congestion
it exposes?
  'there is congestion'
     or
  'at hop 12 there was congestion, you are at hop 22, fyi' (honestly
it'd be better to tell me 'as 3 is congested' I think)

If the information as traffic enters my network is 'somewhere there is
congestion!' I'm not sure I can do much aside from buffer (not going
to happen for very long buffers are expensive) or WRED/drop packets.
If the information is that at a router-hop 10 hops back there was
congestion I may be able to either WRED traffic to that destination or
maybe shuttle packets on slightly longer paths if they are available
(though that seems overly complex as a solution, to me).

> Do you think circumspect wording about DDoS like the above would still
> trigger an allergic reaction from some readers? Would this sort of text
> allay your concerns? Or are you adamant that there should be no mention of
> DDoS at all?

it's too easy to rathole on ddos, there are far too many ways to
create problems with it, and today it's not really that much of a
problem. There are tools in existence today to deal with the vast
majority of ddos problems, it really isn't a huge problem provided you
prepare and understand the threat(s).

I suppose in summary: I'm not adamant about it one way or the other,
but I can see that it'll end up distracting from your actual
point/topic and thus cost you cycles you could have spent explaining
why conex/cc isn't just gussied up 'longdistance toll settlement for
IP' (for instance, though I think conex itself just marking packets
doesn't really fall into the quoted phrase).

-chris

---

From: Bob Briscoe <[email-masked]>
Date: Fri, 06 Aug 2010 16:51:26 +0100

Chris,

>end-user pain, and shifting that traffic to said profile.

It's no coincidence that Rich Woundy is a co-author.

Sure, Comcast have a current solution, but Rich is the first to say 
that it gives no encouragement (and actually punishes) approaches 
like LEDBAT, because it attributes blame for high utilisation by 
volume rather than congestion-volume. There's a list of other things 
Rich presented in the ConEx BoF that makes ConEx worth doing beyond 
what Comcast currently do.
<http://www.ietf.org/proceedings/76/slides/conex-3.pdf>

Whatever, I only included this text as a lead-up to the DDoS text later...

>the CPE actually (if I understand their brand of magic correctly).

No, powerboost is very different.

[BTW, I and others did a start-up called Qariba back in 2001 which 
built a powerboost-like feature for cable networks, with a network 
API so it could be initiated from a CDN - we called it Broadband-800 
by analogy to 800 phone calls, because the server end was effectively 
temporarily buying more access capacity on the end-user's behalf. We 
had a lot of interest in the US cable industry at the time, but we 
got hit by the bubble bursting.]

Powerboost gives you access to extra capacity irrespective of whether 
you will contribute more to congestion. ConEx is much simpler and more generic.

>question, there's no guarantee at all that the sources will be

In the following I'm going to talk about re-ECN, rather than ConEx, 
because we haven't defined ConEx yet...

>any packet sent back to the originators,

(BTW, re-ECN doesn't need any packet sent back to the originators - 
you might have misunderstood the design.)

>the rate will be low enough
>seen that leads to squelching of traffic.

Yes, of course we've thought of this sort of attack. Even if we 
hadn't thought of this, since 2005 the security/DoS community have 
been thinking up attacks against re-ECN (pre-ConEx), so it would have 
been hard to miss such an obvious one.

If each bot was behind a 100Mb/s link, and everyday congestion levels 
were typically 0.2%, a reasonable config of basic re-ECN policer (I 
can give typical numbers if you want) would allow a 
congestion-bit-rate of 10Mb/s of *marked* packets for a minute, then 
30kb/s of sustained marked traffic after that.

If your attack were a flooding attack with MTU size data packets 
(12,000b), rather than an RST attack (which can be detected and 
ignored), each bot would have to congestion mark all the packets to 
have any flooding effect. If each bot could draw congestion from its 
policer allowance at 30kb/s (see above), a basic re-ECN policer would 
allow it to sustain an attack at 1 pkt every 0.4sec - a little more 
than twice as fast as your attack.

Agreeing on the 'largest' botnet ever seen isn't easy, but the 
Mariposa botnet dismantled earlier this year involved ~12M separate 
IP addresses. No-one can know whether that implied 12M separate 
machines, but I doubt it - source address spoofing could have been 
used. This is relevant because a re-ECN policer would sit at the 
physical access - nothing in re-ECN depends on valid source 
addresses. If each machine was spoofing 24 other addresses (a total 
guess), that's about 500,000 real machines.

Let's assume it gets much harder to marshall a larger army than the 
one said to be the largest yet. Then, a basic re-ECN deployment would 
contain an attack to about 30kb/s x 500,000 = 15Gb/s (which is 
similar to your scenario of 12kb/s x 1M = 12Gb/s).

In summary, just a basic re-ECN policer that isn't even looking for 
DoS can raise the bar to require a botnet about 3,000 times bigger 
than without re-ECN [because that's the ratio of the peak traffic 
each site can send (100Mb/s) divided by the averaged rate of losses 
in normal traffic (30kb/s)].

Yes, this isn't particularly impressive. But it's not meant to be. 
This is just what you get without even trying to deal with DDoS specifically.

i) The most obvious thing to add to a basic re-ECN policer would be a 
simple anomaly detector triggered by packets to one destination 
prefix with >10% of them marked for expected congestion. That could 
trigger a much more stringent limit just for that destination prefix.

ii) If the victim network deployed ConEx monitors around its border, 
attack traffic would really stand out obviously because nearly all 
attack packets would be marked for rest-of-path congestion. These 
monitors could then trigger similar tight limits on 
congestion-bit-rate to that destination.

iii) I've assumed a homogeneous botnet. In practice many would be on 
slower access links, and many would be within larger sites (e.g. 
campus nets) sharing an aggregated congestion allowance at the access 
from the campus to the Internet. I think my rough calculation is 
closer to 'worst case' than 'typical'.

>The victim still suffers

Nope. Importantly, normal /unsustained/ traffic to that destination 
would take a while (1 minute in my example) to hit the tighter 
congestion policer limits. So regular usage would be reasonably 
unaffected. Whereas the bots have to sustain the attack to be effective.

>system would have just been CC"d out of existence.

Nope. ConEx congestion policers would be run by the network operator 
and effectively take over congestion control if hosts fail to do it 
for a sustained period.

Yes, you could get bugs in ConEx policers. But it's unlikely (though 
not impossible) that would happen at the same time as a widespread 
bug in hosts. Safety in diversity.

Yes, the ConEx marking process on the hosts might be bugged. But, as 
with DoS protection, if a host or router is being flooded and has to 
drop stuff, it is easy to preferentially drop unmarked ConEx packets 
(or non-ConEx packets) first.

Agreed, this takes a leap-of-faith to believe a network might deploy 
all this, but it only has to deploy its own protections for itself, 
which is a good deployment property.

> >   network, or deliberate malicious attacks.

I must also add that re-ECN is only handling the flooding aspect of 
an attack, not the /information/ in the packets (the RST flag in your 
example). Of course a DDoS protection system needs to take this 
information into account.

What 5.2d says is that all ConEx aims to do is add rest-of-path 
congestion to that information, which is a powerful addition to the 
network's visibility. Particularly because info in the payload need 
not be visible to the network at all.

>silicon/cpu-cycles/state to do this doesn't strike me as hard,

Good

>I can

No network has to implement anything if they don't want to. It still 
works for others who do. I also don't imagine backbone providers will 
care about this.

It's easier for a backbone to throw capacity at the problem when 
they're running a few large links rather than loads of smaller links. 
So the smaller links around the network edge are always going to 
bottleneck congestion on behalf of cores and backbones.

>problems aside, nothing fixes a 56k modem except... more bandwidth)

I think you might have misunderstood. ConEx isn't intended to change 
who does congestion control. Hosts still do that on fast timescales.

ConEx is merely intended to allow the network to count up how much 
congestion is still contributed to by hosts, so the network can judge 
how effectively the host is doing congestion control. That can 
ultimately be used to take over control (via a network-based 
congestion policer) if the host is persistently being profligate 
(whether through selfishness, malice or accident).

>it'd be better to tell me 'as 3 is congested' I think)

Re-ECN deliberately doesn't tell anyone exactly where the congestion 
is - that would reveal too much. Whatever your viewing point, it only 
tells you how much congestion there is upstream of you and how much downstream.

Rather than re-write all that's ever been written on ConEx in one 
email, can I point you at a section of a paper to get this?:
         Section 4.2 of
         "Using Self-interest to Prevent Malice;
         Fixing the Denial of Service Flaw of the Internet"
         <http://www.bobbriscoe.net/projects/refb/index.html#refb_dplinc>

This is the only paper where the full re-ECN protocol is described 
but in outline form, to save you having to wade through the detailed 
protocol spec. It's also the only paper about re-ECN and DDoS (other 
than my later PhD thesis).

Plenty of other papers describe the two main codepoints of the protocol:
- Expected whole path congestion, W [inserted by the sender and immutable]
- (ECN) congestion experienced upstream so far, U [marks added by routers]
These are enough to answer your question above: a monitor at any 
point on the path can meter both W & U, so it can calculate expected 
downstream congestion on the rest-of-the-path as well: D = W - U.

But only the above DDoS paper explains the third and last part of the 
protocol (initial credit) which enables re-ECN to work without any 
feedback at all - you need that for the DDoS case.

>(though that seems overly complex as a solution, to me).

See above: We're not expecting the network to do fine-grained 
congestion control per flow - that's for the host to decide. We're 
wanting to look at the sum of all the congestion that traffic from a 
site (household, campus) is contributing to; everything on the 
Internet side of its access boundary.

Then at the next network border, the pair of networks at that border 
can see how much congestion is each side of that border - for all 
traffic, whereever it is destined. And so on.

That's all you need - necessary and sufficient - to regulate 
profligate users (or bugged, or malicious) and to regulate networks 
that allow their users to be profligate (or bugged or malicious).

>doesn't really fall into the quoted phrase).

Yes, I understand you're trying to help us avoid interminable arguments.

I guess what I'm saying is: We don't just change IP for chuckles. The 
job of ConEx is to limit congestion caused by profligacy, malice and 
accidents. If it can only do that against people who don't try too 
hard to push back, we should not spend our precious time on ConEx.

IOW, we /should/ have some level of argument about whether ConEx can 
achieve what it aims to achieve. Just not too much at this early 
stage, when we haven't even defined the ConEx protocol (as opposed to 
the re-ECN protocol).

Bob

---

From: "Mcdysan, David E" <[email-masked]>
Date: Fri, 13 Aug 2010 08:47:18 -0400

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] 
> On Behalf Of Bob Briscoe
> Sent: Friday, August 06, 2010 11:51 AM
> To: Christopher Morrow
> Cc: [email-masked]
> Subject: Re: [conex] ConEx & DDoS
> 
> Chris,
> 
> > > During the ConEx w-g session last Tuesday in Maastricht you 
> > > suggested we should not include DDoS mitigation as a use-case for 
> > > ConEx. I was
> > willing to

Several pages snipped.

> hard to push back, we should not spend our precious time on ConEx.

Please translate "hard to push back."  Does this mean respond to an exposed (indication) of congestion?

> achieve what it aims to achieve. 

Hopefully, we can reach agreement and not continue arguing about this. I am not sure there is yet a shared vision on what Conext aims to achieve. :-)

I am looking for the use case document to capture an agreement on what Conex aims to achieve. Do others in the group have a different view?

> Just not too much at this early

---

From: Christopher Morrow <[email-masked]>
Date: Fri, 13 Aug 2010 09:49:03 -0400

(still need to reply to bob...)

> I am looking for the use case document to capture an agreement on what Conex aims to achieve. Do others in the group have a different view?

I would also like to see this... now back to reading bob's message (for me).

-chris

---

From: Christopher Morrow <[email-masked]>
Date: Tue, 17 Aug 2010 14:18:08 -0400

it would help (me at least) in this discussion if we think of a
notional network like:

           user
             |
           cpe
            |
         bras
        /     \
    core1    core2 (inside a single ASN)
      \        /
       peer/transit edge (exit/entrance to another ASN)
         |
       PE
         |
        CE
         |
       server

(apologies for bad ascii art)

On Fri, Aug 6, 2010 at 11:51 AM, Bob Briscoe <[email-masked]> wrote:
>> yup, no use rat-holing if it's not central to the discussion.

oops, we rat-holed... this sort of proves my point about ddos
discussions wrt conex though.
I would stick with (for conex): "Expose information about congestion
on the network path, permit better/more-intelligent discard profiles
to be used along the network paths."

Where 'better/more-intelligent' is really: "Drop traffic that's less
important to the end-users."

Lee/Rich I think explained this (to me) as: "make sure my Internet
gaming works perfectly, it is OK to slow down a large video file
transfer." I believe plus-net in the UK does some of this today? (I
may have the wrong provider... but it's a competitor of BT's I
believe)

>> > asked me
>> > identify
> Sure, Comcast have a current solution, but Rich is the first to say that it
> gives no encouragement (and actually punishes) approaches like LEDBAT,
> because it attributes blame for high utilisation by volume rather than
> congestion-volume. There's a list of other things Rich presented in the
> ConEx BoF that makes ConEx worth doing beyond what Comcast currently do.

better understanding of what contributes to congestion in 'your
network' and what traffic your customers don't mind missing a few
packets of (at the benefit of their real-time needs) is good.

> [BTW, I and others did a start-up called Qariba back in 2001 which built a
> powerboost-like feature for cable networks, with a network API so it could
> be initiated from a CDN - we called it Broadband-800 by analogy to 800 phone
> calls, because the server end was effectively temporarily buying more access
> capacity on the end-user's behalf. We had a lot of interest in the US cable
> industry at the time, but we got hit by the bubble bursting.]
>
> Powerboost gives you access to extra capacity irrespective of whether you
> will contribute more to congestion. ConEx is much simpler and more generic.

so... 'PowerBoost' provides higher QOS/traffic-rate for a period of
time, it doesn't account for what damage that change has to other
traffic on my cable-link, the link to the head-end or the other links
in the network... it can be triggered at the CPE or the BRAS (I'd bet
that anywhere else in the Comcast network, to take a specific example,
probably doesn't matter).

I think that's the same thing I said earlier.

> In the following I'm going to talk about re-ECN, rather than ConEx, because
> we haven't defined ConEx yet...
>
>
> (BTW, re-ECN doesn't need any packet sent back to the originators - you
> might have misunderstood the design.)
>
> Yes, of course we've thought of this sort of attack. Even if we hadn't
> thought of this, since 2005 the security/DoS community have been thinking up
> attacks against re-ECN (pre-ConEx), so it would have been hard to miss such
> an obvious one.
>
> If each bot was behind a 100Mb/s link, and everyday congestion levels were
> typically 0.2%, a reasonable config of basic re-ECN policer (I can give

where is this congestion level? on the 100mbps link? or somewhere in
the local ASN? or external to that ASN?
Looking at the ascii-network-art at the top:

user -> cpe?
cpe -> bras?
bras -> core?
core -> peering-edge?
other-net -> other-net ? (ran out of space/time on the art, call this
'middle of the Internet')
PE -> CE ?
CE -> server?

where the congestion is will determine what sort of measures can be
applied today. In the future though if the end-stations know there is
congestion they could choose to modify their behavior to better
utilize the network...

> typical numbers if you want) would allow a congestion-bit-rate of 10Mb/s of
> *marked* packets for a minute, then 30kb/s of sustained marked traffic after
> that.
>
> If your attack were a flooding attack with MTU size data packets (12,000b),
> rather than an RST attack (which can be detected and ignored), each bot

the 'and ignored' part is a little misleading since 1 rst/second at
the source isn't noticeable, 1m rst/second at the victim is another
story. There will be effects on the network at the victim end,
'ignore' here is an oversimplification. (don't think it's worthwhile
rat-holing here)

> would have to congestion mark all the packets to have any flooding effect.
> If each bot could draw congestion from its policer allowance at 30kb/s (see
> above), a basic re-ECN policer would allow it to sustain an attack at 1 pkt
> every 0.4sec - a little more than twice as fast as your attack.
>
> Agreeing on the 'largest' botnet ever seen isn't easy, but the Mariposa
> botnet dismantled earlier this year involved ~12M separate IP addresses.
> No-one can know whether that implied 12M separate machines, but I doubt it -
> source address spoofing could have been used. This is relevant because a

the numbers for mariposa I think were from C&C check-ins, so these
required 3-way handshakes and weren't spoofed. I believe there's some
rotation in the address usage (dhcp-like effects) but in the end 1m or
12m isn't really important. (also not important to rat-hole here)

> re-ECN policer would sit at the physical access - nothing in re-ECN depends
> on valid source addresses. If each machine was spoofing 24 other addresses
> (a total guess), that's about 500,000 real machines.
>
> Let's assume it gets much harder to marshall a larger army than the one said
> to be the largest yet. Then, a basic re-ECN deployment would contain an
> attack to about 30kb/s x 500,000 = 15Gb/s (which is similar to your scenario
> of 12kb/s x 1M = 12Gb/s).

it's getting easier to marshal more hosts, not harder, again though
not important to rathole.

> In summary, just a basic re-ECN policer that isn't even looking for DoS can

where is the policer deployed?

o if at the source-host I still say it's hard to tell 1pps from
another, if there is no congestion between src and BRAS there's no
reason to mark anything there.

o if at the destination/victim, then policing traffic to the victim
can be done if the traffic is significantly different from normal
traffic. Effects on legitimate traffic will be hard to avoid if the
attack traffic looks enough like real/legitimate traffic.

I don't think this is really even important enough to rathole on...

> raise the bar to require a botnet about 3,000 times bigger than without
> re-ECN [because that's the ratio of the peak traffic each site can send
> (100Mb/s) divided by the averaged rate of losses in normal traffic
> (30kb/s)].
>
> Yes, this isn't particularly impressive. But it's not meant to be. This is
> just what you get without even trying to deal with DDoS specifically.
>
> i) The most obvious thing to add to a basic re-ECN policer would be a simple
> anomaly detector triggered by packets to one destination prefix with >10% of
> them marked for expected congestion. That could trigger a much more
> stringent limit just for that destination prefix.

so, more state on the router? or is this another inline device doing
the work? Factor in cost in opex/capex to implement/maintain this...
at verizon studies/eng work showed ~500k/BRAS would be required to
implement in-line 'dpi' systems on 1g links, today those links are
likely 10g and I don't think the cost has reduced. at 500k/BRAS I
really should just turn up another 20g of capacity to the BRAS.
Downstream from the BRAS I am limited by physics/last-mile issues so
we're back to the top of the thread: "If you can mark/know what
traffic is more relevant to real-time ops at the customer-end then you
can make a better decision about what packets to drop"

>
> ii) If the victim network deployed ConEx monitors around its border, attack
> traffic would really stand out obviously because nearly all attack packets
> would be marked for rest-of-path congestion. These monitors could then
> trigger similar tight limits on congestion-bit-rate to that destination.

this is costly, very costly, see above. It's not always obvious which
traffic is contributing to the actual problem, the traffic most likely
won't be marked until I mark it at ingress (at the peering edge),
which means I need to know what to mark, which the victim
host/customer-network will have to work out, or have me help them work
out.

> iii) I've assumed a homogeneous botnet. In practice many would be on slower
> access links, and many would be within larger sites (e.g. campus nets)
> sharing an aggregated congestion allowance at the access from the campus to
> the Internet. I think my rough calculation is closer to 'worst case' than
> 'typical'.

perhaps, not worth rat-holing though.

> Nope. Importantly, normal /unsustained/ traffic to that destination would
> take a while (1 minute in my example) to hit the tighter congestion policer
> limits. So regular usage would be reasonably unaffected. Whereas the bots
> have to sustain the attack to be effective.

botnets sustain attacks, it's what they do (among sending spam or
other things, not relevant to this discussion). Having sat through
many multi-gigabit per second attacks over the last many years...
sustaining a dos/ddos isn't really a problem for the attacker. User
traffic is always affected in these cases, if the traffic from the
attack looks like the normal customer traffic. If there's a method to
differentiate attack from customer traffic (rst packet, icmp packet)
the upstream can just drop the odd-traffic. They don't need anything
except the tools they have today to do this, and maintenance of more
state isn't a help.

Unless the victim can signal the upstream that 'traffic of this type'
or 'packets fitting this profile' should be marked for discard/drop
before other packets ... there's no real win. This, though, does
resemble the start of the thread: "Tell me what traffic is important
to your real-time ops, I'll apply a drop profile to all other traffic
when congestion occurs."

> Nope. ConEx congestion policers would be run by the network operator and
> effectively take over congestion control if hosts fail to do it for a
> sustained period.

where are these run? at the BRAS? at the Peering-Edge? on the CPE?

> Yes, you could get bugs in ConEx policers. But it's unlikely (though not
> impossible) that would happen at the same time as a widespread bug in hosts.
> Safety in diversity.
>
> Yes, the ConEx marking process on the hosts might be bugged. But, as with
> DoS protection, if a host or router is being flooded and has to drop stuff,
> it is easy to preferentially drop unmarked ConEx packets (or non-ConEx
> packets) first.

sure.

> Agreed, this takes a leap-of-faith to believe a network might deploy all
> this, but it only has to deploy its own protections for itself, which is a
> good deployment property.

where are the protections deployed, cost and maintenance will be
important to the deployment path...

> I must also add that re-ECN is only handling the flooding aspect of an
> attack, not the /information/ in the packets (the RST flag in your example).
> Of course a DDoS protection system needs to take this information into
> account.
>
> What 5.2d says is that all ConEx aims to do is add rest-of-path congestion
> to that information, which is a powerful addition to the network's
> visibility. Particularly because info in the payload need not be visible to
> the network at all.

ok, what 'rest of path'? Is this per-asn "congestion" or per-hop
"congestion" information? how many bits (which bits?) in the ip-header
are going to carry this information?

> No network has to implement anything if they don't want to. It still works
> for others who do. I also don't imagine backbone providers will care about
> this.
>
> It's easier for a backbone to throw capacity at the problem when they're
> running a few large links rather than loads of smaller links. So the smaller
> links around the network edge are always going to bottleneck congestion on
> behalf of cores and backbones.
>
>
> I think you might have misunderstood. ConEx isn't intended to change who
> does congestion control. Hosts still do that on fast timescales.

part of this confusion is that the wording (even in this thread)
points to networks doing the congestion control. That's one reason I
put the horrid ascii art at the top :) "Where does this congestion
control happen?"

> congestion is still contributed to by hosts, so the network can judge how
> effectively the host is doing congestion control. That can ultimately be
> used to take over control (via a network-based congestion policer) if the
> host is persistently being profligate (whether through selfishness, malice
> or accident).

ok, but is the host causing just itself problems? or all users on a
shared link? if all users on a shared link discarding packets only
helps if the end-host obeys the drop effects. In the end, if you
operate a large shared media last-mile (wireless network) ... the end
stations still must compete for last-mile access. If everyone is well
behaved then of course more marking data and more polite
machines/users will permit better utilization of the link(s).

In the case of a wireless network (forgetting the cases where the
"network" can un-enroll/force-disconnect an end-station/user)
loud/impolite end-users can still cause problems for their shared
network breathren, there's nothing that can stop that, sadly... so
maybe we shouldn't rat-hole here either. For most last-mile cases the
goal ought to be to carry more good traffic on a link, reduce
retransmits and other overhead. ConEx should, I think, be able to
signal the end-user/station that it can do better, it should also help
the network (BRAS or peering-edge maybe) what traffic to prefer to
discard in times of stress.

> Re-ECN deliberately doesn't tell anyone exactly where the congestion is -
> that would reveal too much. Whatever your viewing point, it only tells you
> how much congestion there is upstream of you and how much downstream.
>
> Rather than re-write all that's ever been written on ConEx in one email, can
> I point you at a section of a paper to get this?:
>        Section 4.2 of

I'll go have a read.

> This is the only paper where the full re-ECN protocol is described but in
> outline form, to save you having to wade through the detailed protocol spec.
> It's also the only paper about re-ECN and DDoS (other than my later PhD
> thesis).
>
> These are enough to answer your question above: a monitor at any point on
> the path can meter both W & U, so it can calculate expected downstream
> congestion on the rest-of-the-path as well: D = W - U.
>
> protocol (initial credit) which enables re-ECN to work without any feedback
> at all - you need that for the DDoS case.
>
>
> See above: We're not expecting the network to do fine-grained congestion
> control per flow - that's for the host to decide. We're wanting to look at
> the sum of all the congestion that traffic from a site (household, campus)
> is contributing to; everything on the Internet side of its access boundary.

'access boundary' == BRAS?

> Then at the next network border, the pair of networks at that border can see
> how much congestion is each side of that border - for all traffic, whereever
> it is destined. And so on.

ok, clearly this is the Peering-Edge, fine.

> That's all you need - necessary and sufficient - to regulate profligate
> users (or bugged, or malicious) and to regulate networks that allow their
> users to be profligate (or bugged or malicious).

'allow their users' is a little pejorative... I may have a network
with 500k users all with 1g links (full duplex). You may have a 1G
link to me, if your customer turns up a 'popular' site (say the
victoria-secret fashion show live stream, for a canonical example) ...
there will be issues between our networks. The only path to resolution
is to force traffic on alternate paths or upgrade links between these
2 networks, or to just chose to drop traffic and force 'bad service'
on 2 parts of the population. VictoriaSecret happened to pick the last
option in 2000/2001 (I forget the right year).

I suppose with better marking/exposure of the problem one/both sides
could have chosen to drop other traffic, that may have improved
things... pathological cases aside though, I still say that better use
of the network by end-stations is good. Better info about what to drop
first is also good.

> Yes, I understand you're trying to help us avoid interminable arguments.

yes, though this thread wasn't my best effort :(

> I guess what I'm saying is: We don't just change IP for chuckles. The job of
> ConEx is to limit congestion caused by profligacy, malice and accidents. If
> it can only do that against people who don't try too hard to push back, we
> should not spend our precious time on ConEx.

or rephrased a tad: "If we can help make the utilization of the
network more optimal.." (by letting the network and users of the
network decide which packets to discard first)

> IOW, we /should/ have some level of argument about whether ConEx can achieve
> what it aims to achieve. Just not too much at this early stage, when we
> haven't even defined the ConEx protocol (as opposed to the re-ECN protocol).

ok.

-chris

---

From: Kevin Mason <[email-masked]>
Date: Wed, 18 Aug 2010 12:03:50 +1200

Cheers
Kevin Mason

> From: [email-masked] [mailto:[email-masked]] On Behalf Of
> Christopher Morrow
> Sent: Wednesday, 18 August 2010 6:18 a.m.
> To: Bob Briscoe
> Cc: [email-masked]

[Kevin Mason] This would be helpful, but I suggest we change the labels and add a core on the other side to make it a little more generic. There may be some debate about the actual labels.
>            receiver
        Customer policy point (e.g. BRAS, GGSN, DOCSIS equivalent)
>         /     \
>     coreR1    coreR2 (inside a single ASN)
>       \        /
          /    \
      coreS1   coreS2
         \     /
        Customer policy point (e.g. PE, another BRAS for peer to peer)
>        sender>
> we
> >> > to float the idea that, altho we won't include DDoS as a use-case in
> its
> >> > own
> >> > right, we should mention it as an extreme case of two other use-
> cases.
> >> > Let
> >> > me explain...
> >> >
> to
> you
> were
> (12,000b),
> simple
> > anomaly detector triggered by packets to one destination prefix with
> >10% of
> attack
> slower
> would
> >> >   invulnerable to all cheats and attacks, even though it has been
> hard
> >> > to
> with
> congestion
> works
> how
> feedback
> still
> >> > allay your concerns? Or are you adamant that there should be no
> mention
> >> > of
> >> > DDoS at all?
> >>
> job of
> achieve
> _______________________________________________
> conex mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/conex

---

From: Christopher Morrow <[email-masked]>
Date: Tue, 17 Aug 2010 22:08:05 -0400

> [Kevin Mason] This would be helpful, but I suggest we change the labels and add a core on the other side to make it a little more generic. There may be some debate about the actual labels.

seems fine to me... in fact maybe sticking a not-ascii-art picture up
I'll do tonight, my ascii-art-foo is horrid :(

---

From: Christopher Morrow <[email-masked]>
Date: Wed, 18 Aug 2010 01:11:26 -0400

> I'll do tonight, my ascii-art-foo is horrid :(

<http://docs.as701.net/conex/conex-net-example.png>

with the dot, in case you wanted to make it better/clearer/move-it-elsewhere:

<http://docs.as701.net/conex/conex-net.dot>

-Chris