Subject: [quicwg/base-drafts] MUST send new data or retransmit data if possible (#3057)

From: ianswett <[email-masked]>
Date: Mon, 23 Sep 2019 07:58:15 -0700

Fixes #3056
You can view, comment on, or merge this pull request online at:

  https://github.com/quicwg/base-drafts/pull/3057

-- Commit Summary --

  * MUST send new data or retransmit data if possible

-- File Changes --

    M draft-ietf-quic-recovery.md (11)

-- Patch Links --

https://github.com/quicwg/base-drafts/pull/3057.patch
https://github.com/quicwg/base-drafts/pull/3057.diff

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/3057

---

From: Benjamin Saunders <[email-masked]>
Date: Mon, 23 Sep 2019 08:16:13 -0700

Ralith commented on this pull request.

Will the larger PTO packets this produces (often MTU-sized instead of minimally-sized) adversely affect performance under heavy congestion?

Do we also need to add logic guiding the level at which a PTO is sent? Quinn currently always sends PTOs at the highest encryption level for which keys are known, which under the current logic would make loss of any part of the server's first flight unrecoverable.

> @@ -526,12 +526,11 @@ as a probe, unless there is no data available to send.  An endpoint MAY send up
 to two full-sized datagrams containing ack-eliciting packets, to avoid an
 expensive consecutive PTO expiration due to a single lost datagram.
 
-It is possible that the sender has no new or previously-sent data to send.  As
-an example, consider the following sequence of events: new application data is
-sent in a STREAM frame, deemed lost, then retransmitted in a new packet, and
-then the original transmission is acknowledged.  In the absence of any new
-application data, a PTO timer expiration now would find the sender with no new
-or previously-sent data to send.
+When the PTO timer expires, and there is new or previously sent data, it MUST
+be sent.  It is possible the sender has no new or previously-sent data to send.
+As an example, consider the following sequence of events: new application data
+is sent in a STREAM frame, deemed lost, then retransmitted in a new packet,
+and then the original transmission is acknowledged.

This example only applies to implementations that can usefully process acknowledgements for packets they've already deemed lost and retransmitted the contents of, which I don't think is required. Is there a more common case?

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/3057#pullrequestreview-291851055

---

From: ianswett <[email-masked]>
Date: Mon, 23 Sep 2019 08:22:36 -0700

ianswett commented on this pull request.

This is a fairly common case in our implementation, so I think it's a good example.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/3057#discussion_r327177944

---

From: ianswett <[email-masked]>
Date: Mon, 23 Sep 2019 08:24:29 -0700

Under "Handshakes and New Paths", it says "Data at Initial encryption MUST be retransmitted before Handshake data and data at Handshake encryption MUST be retransmitted before any ApplicationData".  I moved that down to this section, since I think it fits slightly better here.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/3057#issuecomment-534149774

---

From: ianswett <[email-masked]>
Date: Mon, 23 Sep 2019 08:27:06 -0700

@ianswett pushed 1 commit.

409286b111330780882fe2ae17bbe405d9f25b09  Update draft-ietf-quic-recovery.md

-- 
You are receiving this because you are subscribed to this thread.
View it on GitHub:
https://github.com/quicwg/base-drafts/pull/3057/files/295e0c9afc7eb80454f3fd0bd9d64d4879f85573..409286b111330780882fe2ae17bbe405d9f25b09

---

From: ianswett <[email-masked]>
Date: Mon, 23 Sep 2019 08:29:14 -0700

In terms of packet size, this is similar to what the old Handshake text recommended and what PTO should have said all along, as it aligns well with TCP's TLP and RTO behavior.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/3057#issuecomment-534151991

---

From: Benjamin Saunders <[email-masked]>
Date: Mon, 23 Sep 2019 08:31:48 -0700

Maybe note that this is optional behavior? I know if I was reading this for the first time, I'd try to work out where it was required and get confused.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/3057#discussion_r327183006

---

From: Benjamin Saunders <[email-masked]>
Date: Mon, 23 Sep 2019 08:34:34 -0700

The previous text read like it only described retransmission of lost data, and this now reads like it only describes retransmission in PTOs. Are both intended?

Is there a restriction anywhere that retransmitted data must not be sent at a higher encryption level than it was originally sent, or similar?

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/3057#issuecomment-534154481

---

From: ianswett <[email-masked]>
Date: Mon, 23 Sep 2019 08:42:12 -0700

All of this text is in the PTO section, so only applies to PTO.

I think transport is usually the best place to talk about where data is retransmitted, but it might be worth an editorial note that data is never retransmitted at a different packet number space than the original one.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/3057#issuecomment-534157897

---

From: ianswett <[email-masked]>
Date: Mon, 23 Sep 2019 09:25:16 -0700

It probably should be recommended behavior, and we should probably have some text on it.  Filed #3058 to discuss if and what to do to improve that.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/3057#discussion_r327209845

---

From: Martin Thomson <[email-masked]>
Date: Mon, 23 Sep 2019 16:21:27 -0700

martinthomson commented on this pull request.

-When there is no data to send, the sender SHOULD send a PING or other
-ack-eliciting frame in a single packet, re-arming the PTO timer.
+It is possible the sender has no new or previously-sent data to send.
+As an example, consider the following sequence of events: new application data
+is sent in a STREAM frame, deemed lost, then retransmitted in a new packet,
+and then the original transmission is acknowledged.  When there is no data to
+send, the sender SHOULD send a PING or other ack-eliciting frame in a single
+packet, re-arming the PTO timer.

I don't get this example.  In this example, you have sent something, it was received, end of story.

The new text above says that you can't arm PTO if you have nothing to send (MUST NOT even).  Which did you mean?

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/3057#pullrequestreview-292104940

---

From: Benjamin Saunders <[email-masked]>
Date: Mon, 23 Sep 2019 16:34:20 -0700

In this case, the second packet is ack-eliciting and has not been acknowledged, eventually leading to a probe timeout. However, there is no unsent or sent-but-unacknowledged ack-eliciting data available to send, so a `PING` is fabricated instead.

Perhaps "there is new or previously sent data" should be more specific?

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/3057#discussion_r327373408

---

From: Tatsuhiro Tsujikawa <[email-masked]>
Date: Mon, 23 Sep 2019 18:15:48 -0700

tatsuhiro-t commented on this pull request.

I think this kind of situation only happens when client sent Initial and received its ACK but either sever Initial or Handshake was lost.  Client then literally has nothing to send.  And forced to send ack-eliciting frame (and controversially, draft-23 does not allow PING here).

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/3057#discussion_r327391007

---

From: ianswett <[email-masked]>
Date: Mon, 23 Sep 2019 18:18:00 -0700

In this example, the point is that in order to remove the retransmission from byte in flight, you SHOULD send a PING and when it's acknowledged, you'll know whether the retransmission was lost or not.  Ideally it'll be ACKed on its own before the PTO fires.

The above text is about the server's amplification factor and not arming the alarm when "If no data can be sent"

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/3057#discussion_r327391121

---

From: Benjamin Saunders <[email-masked]>
Date: Mon, 23 Sep 2019 19:06:13 -0700

> @@ -526,15 +524,16 @@ as a probe, unless there is no data available to send.  An endpoint MAY send up

Does "previously sent" here mean in-flight/unacknowledged, or literally any historical data?

Does "data" mean ack-eliciting frames specifically?

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/3057#pullrequestreview-292141439

---

From: ianswett <[email-masked]>
Date: Mon, 23 Sep 2019 19:11:41 -0700

df8c4feb8412cdc5060f427eb3e64672db56e0db  unacknowledged data

-- 
You are receiving this because you are subscribed to this thread.
View it on GitHub:
https://github.com/quicwg/base-drafts/pull/3057/files/409286b111330780882fe2ae17bbe405d9f25b09..df8c4feb8412cdc5060f427eb3e64672db56e0db

---

From: ianswett <[email-masked]>
Date: Mon, 23 Sep 2019 19:14:36 -0700

I changed it to previously sent unacknowledged data, because that is the intent, though it's also a bit verbose.

Data is used fairly widely to mean ack-eliciting frames in this document.  I'm not sure if that is something we should codify in the definitions section?

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/3057#discussion_r327400945

---

From: Martin Thomson <[email-masked]>
Date: Mon, 23 Sep 2019 19:40:01 -0700

> @@ -496,10 +496,8 @@ be considered an RTT sample.
 
 Until the server has validated the client's address on the path, the amount of
 data it can send is limited, as specified in Section 8.1 of {{QUIC-TRANSPORT}}.
-Data at Initial encryption MUST be retransmitted before Handshake data and
-data at Handshake encryption MUST be retransmitted before any ApplicationData
-data.  If no data can be sent, then the PTO alarm MUST NOT be armed until
-data has been received from the client.
+If no data can be sent, then the PTO alarm MUST NOT be armed until data has
+been received from the client.

Do you mean "until the path has been validated"?  "data" is highly non-specific.

Still not working for me.

I get the need for the client to send extra Initial packets again in the case where it has nothing to say, but that should be a special provision.

If p1 is acknowledged and p2 contained nothing new, then let it lie.  You don't need to know whether it was lost or not.

I get that this packet will stay outstanding unless you have something new to say, but you are only sending a PING to deal with an accounting discrepancy.  That only becomes relevant when you go to send the next packet.  Is there any way that this sort of blip can end up consuming the entire congestion window?

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/3057#pullrequestreview-292146858

---

From: Benjamin Saunders <[email-masked]>
Date: Mon, 23 Sep 2019 20:02:24 -0700

On review, maybe the existing requirement that probes be ACK-eliciting is enough. Nothing wrong with retransmitting some unacknowledged ACKs alongside your PING.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/3057#discussion_r327409083

---

From: ianswett <[email-masked]>
Date: Tue, 24 Sep 2019 11:34:38 -0700

0236874f3a51eb4166b9070a21f1acd8dae432b1  Update draft-ietf-quic-recovery.md

-- 
You are receiving this because you are subscribed to this thread.
View it on GitHub:
https://github.com/quicwg/base-drafts/pull/3057/files/df8c4feb8412cdc5060f427eb3e64672db56e0db..0236874f3a51eb4166b9070a21f1acd8dae432b1

---

From: ianswett <[email-masked]>
Date: Tue, 24 Sep 2019 11:35:12 -0700

It's path validation or receiving enough data.  I updated the paragraph, PTAL.

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/3057#discussion_r327771954