Subject: Re: [ppsp] PPSP encoding

From: "Bakker, A." <[email-masked]>
Date: Fri, 30 Sep 2011 08:06:12 +0000

Rui Cruz wrote:
> I our implementation, reflected in our proposed draft, the peer is a tiny HTTP server and client, implements the 
> PPSP protocols for signaling over HTTP, as well as the media transfer protocol is over HTTP, and we are able 
> to stream high quality media (including scalable media) even in harsh network conditions. 

Hi

I really question whether HTTP is the right choice. First, this means the system uses at least 2 TCP connections 
per pair of peers, where one would suffice to get bidirectional traffic. Moreover, one can question (and we do) 
whether TCP is the best suited transport layer protocol. It has fixed congestion control, so improvements like 
IETF's LEDBAT (=manage user's upload bandwidth such that P2P moves out of the way when the user e.g. 
uploads photos) are not possible. Furthermore, NAT traversal is complex, there is a high per-connection memory 
footprint and finally, it pays a price to offer facilities (in-order and reliable) that you do not necessarily need in a 
P2P protocol.

Hence, we have a strong preference for UDP. As put forward at the meeting, we would really like to hear
people's opinions on whether they prefer transport over "raw UDP" or would like PPSP to be a profile of RTP,
in the same way as Secure RTP. Like SRTP i.e., swift would add a layer below RTP that protects the content, 
and does P2P signalling, but which is transparent to the RTP consumer.

Regards,
      Arno

---

From: Njaal Borch <[email-masked]>
Date: Mon, 3 Oct 2011 14:17:22 +0200

Hi

> footprint and finally, it pays a price to offer facilities (in-order and reliable) that you do not necessarily need in a

In my opinion, connectivity issues with TCP makes it highly unsuitable
for any P2P based standard, as it will either work suboptimally or
depend on external factors for success.  A UDP based solution with
built in support for NAT and firewall piercing will ensure that
available resources can be harvested in pretty much all
configurations.

Also of great importance is to keep resource usage of "idle swarms" to
a minimum to ensure that enough peers can provide resources when
needed also for narrow content, allowing it to be efficient also for
user generated or long tail content that is seldomly accessed.

Regards,
Njål
---
Dr. Njål Borch
Northern Research Institute (NORUT)
Tromsø, Norway

---

From: Rui Cruz <[email-masked]>
Date: Mon, 3 Oct 2011 15:03:34 +0100

Hi,

One aspect that needs to be clarified is the one related to the streaming method.

When addressing bitstreaming, UDP would be a natural and obvious choice.

However, when the streaming distribution relies on a "chunk" (media segment) transfer scheme (instead of a bitstream), whereby the original media data is chopped into small media segments with a short duration (of a few seconds), decodable on packet and key frame boundaries, TCP (with HTTP) may come as the natural choice, compatible with the Internet infrastructure (CDNs, etc), with reliable transfers.
The number of messages exchanged is also highly reduced due the (duration) size of the media segments, in terms of media buffer demands.

This is precisely what we have been observing in media streaming in the Internet, now standardized by ISO as DASH, for HTTP adaptive streaming.

In our implementation, for signaling and transport (peer and tracker protocols), all messages are of request/reply nature, therefore, only one TCP connection is established for that purpose, between peers or between peers and trackers. Trackers, as servers, only receive requests from peers, even for reports (via HTTP_POST). Peers also provide information about their IP addresses (IPv4 and IPV6) to trackers.
Therefore, trackers (knowing the public address of peers and their private addresses also) are the obvious choice for providing the adequate means and information (PPSP-ICE parameters) for NAT traversal purposes, with candidates carried on PPSP messages.

I do not foresee PPSP as a re-invention or clone of BitTorrent.

Regards,
Rui

---

From: Arno Bakker <[email-masked]>
Date: Tue, 4 Oct 2011 11:37:18 +0200

On 03/10/2011 16:03, Rui Cruz wrote:
>
> In our implementation, for signaling and transport (peer and tracker protocols),
 > all messages are of request/reply nature, therefore, only one
 > TCP connection is established for that purpose, between peers or

Hi

the root of the problem is that HTTP is a client-server protocol.
Hence, IMHO it is unnatural to use it as a basis for a streaming
protocol where both parties must be able to request services from
the other independently. In your proposal I don't see any mechanism
to deal with the asymmetry of HTTP, so I don't understand your
remark that your implementation only needs 1 TCP connection per
pair of peers. Concretely, how do you allow peer A and
peer B to simultaneously do a GET_CHUNK for each other's data?

Needing two HTTP connections to overcome the asymmetry means
double trouble for NAT firewall traversal. Not only does a
peer A need to connect to peer B, but B must also connect back
to A. Without HTTP, establishing a single TCP connection would
suffice. Also I think requiring two connections will not pass
the review by the Area Director and higher as we go to
standardization.

Until bidirectional HTTP comes along, I think we should rule out
HTTP as carrier protocol on that ground alone, however unfortunate that
may be for compatibility (leaving out a discussion on how compatible a
protocol is when it defines a new set of messages that happened
to be carried over HTTP.).

Ruling out HTTP leaves us with TCP or UDP as candidates. As argued in
my previous post, TCP itself is less suited for a swarm of hosts
sharing the same content. I think LEDBAT is an important feature
for end-user acceptation, and it can only be offered in combination
with UDP. Using UDP also allows one to exploit the fact that peers
are sharing the same data, meaning a peer can request retransmission
from any peer, not just the previous sender.

Note that swift has been designed to be used over any protocol,
including TCP, but we have a strong preference for UDP.

> I do not foresee PPSP as a re-invention or clone of BitTorrent.

Can you please elaborate your viewpoint?

Regards,
      Arno

---

From: Rui Cruz <[email-masked]>
Date: Tue, 4 Oct 2011 11:54:20 +0100

On 04/10/2011, at 10:37, Arno Bakker <[email-masked]> wrote:

> the other independently.
Indeed, that is the point, they request services and data independently, not requiring a permanent session established.
> In your proposal I don't see any mechanism
> pair of peers.
One connection per pair of peers for the Request/Reply.
> Concretely, how do you allow peer A and
The process is asynchronous, so, there is no need to have permanent bilateral sessions.
> to A.
You are right, but we are dealing with well known HTTP, and NAT/Firewall traversal is less problematic in terms of service availability or filtering rules.
> Without HTTP, establishing a single TCP connection would
As I said connections may not be simultaneous for the duration of a request/reply.
> to be carried over HTTP.).
With TCP (and HTTP) this method of web streaming is just a multi-source method, where each peer behaves as a streaming server and a peer requests the segments of interest from other peers claiming to have them.
> with UDP.

> Using UDP also allows one to exploit the fact that peers
> from any peer, not just the previous 
I do dot understand, as peers are allways able to request the data from any other peer, not just the "previous"
> _______________________________________________
> ppsp mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/ppsp

---

From: Njaal Borch <[email-masked]>
Date: Tue, 4 Oct 2011 13:11:40 +0200

Hi,

> Indeed, that is the point, they request services and data independently, not requiring a permanent session established.

But HTTP is a client-server protocol where the client sends a request
and the server replies with a reply.  If this goes both ways (for two
peers to exchange data), you will either need two connections, go for
a half-duplex solution where one node must wait for the "http channel"
to be available or do nasty multiplexing of multiple HTTP sessions
over a single TCP connection.  Example:  Node A sends request for data
D1 to Node B, which starts sending D1 as a reply.  Node B now wants to
request data D4 from Node A, but can't mix a GET (or any other
request) into the data stream currently occupying the HTTP connection.

I for one have difficulties understanding why you would want to use an
extended HTTP protocol over just TCP, or even better, UDP combined
with a variety of congestion controls, and it does not seem possible
to me to create a clean system based on a full duplex HTTP connection.

> One connection per pair of peers for the Request/Reply.

In other words, you go for a half-duplex solution?

> The process is asynchronous, so, there is no need to have permanent bilateral sessions.

But HTTP is synchronous - if you in some way change HTTP to be
asynchronous, one can easily assume that backwards compatibility with
HTTP is broken.

> You are right, but we are dealing with well known HTTP, and NAT/Firewall traversal is less problematic in terms of service availability or filtering rules.

Not less problematic than a UDP based protocol designed to handle NAT
and Firewall traversals without any filtering rules presumably.

Regards,
N

---

From: Arno Bakker <[email-masked]>
Date: Tue, 4 Oct 2011 13:16:08 +0200

On 04/10/2011 12:54, Rui Cruz wrote:
> Indeed, that is the point, they request services and data
 > independently, not requiring a permanent session established.

Hi

does this mean you establish a new TCP connection for every
request, i.e. do not use persistent connections? This means
one has to do NAT/firewall traversal every time. Furthermore,
this adds latency to the request. In my view of a peer-to-peer
protocol, pairs of peers are continuously requesting pieces from
eachother in parallel (e.g. using tit-4-tat for freeriding
prevention). Or otherwise, are continuously exchanging information
about what chunks they have available, such that when new
interesting data becomes available they can request it.
In sum, peers should be permanently connected bidirectionally.

> I do dot understand, as peers are allways able to request
 > the data from any other peer, not just the "previous"

I'm referring to TCP's reliability mechanism being tied to
the sender for retransmits. When using UDP and sharing the
same content one has more flexibility whom to rerequest from.

Regards,
     Arno

---

From: Rui Cruz <[email-masked]>
Date: Tue, 4 Oct 2011 22:58:36 +0200

Hi

On 04/10/2011, at 13:11, Njaal Borch <[email-masked]> wrote:

> and the server replies with a reply.
Correct
>  If this goes both ways (for two
As I said, the time between requests for the same peers may be much larger than the RTT (the HTTP streaming segments have a much bigger size than RTP packets (considering a media segment with a duration of several seconds), and not all peers may have the required media data (in the case of SVC or stereoscopic MVC, not all layers/views of a media segment may be available at the same peer). As such, a peer requests to several peers the layers of a media segment in order to have the maximum quality (it enables a faster response to rate variations in the network). But similar reasoning can be done for for multiple bitrates of AVC, if we want the capability of adaptive streaming.
There are no nasty things done with HTTP, nor any need to multiplexing.
> to me to create a clean system based on a full duplex HTTP connection.
There is no extended HTTP protocol. The proposed protocol facilitates coping with congestion by allowing on-the-fly adaptation, performed by adding or subtracting layers (in the case of SVC or MVC) or bitrates (in the case of multibitrate AVC) to match the capabilities of the network at every time instant.
> In other words, you go for a half-duplex 
No. Each peer serves the request from other peers, and makes request to other serving peers.
> asynchronous, one can easily assume that backwards compatibility with
I said that the PROCESS is asynchronous, not the protocol.
> and Firewall traversals without any filtering rules presumably.
Yes, same level of difficulty.

> N

Regards
Rui

---

From: Rui Cruz <[email-masked]>
Date: Tue, 4 Oct 2011 23:40:05 +0200

Cumprimentos/Regards,
Rui Cruz

Sent from my iPad

On 04/10/2011, at 13:16, Arno Bakker <[email-masked]> wrote:

> request, i.e. do not use persistent connections?
We use TCP persistent connections for media transport.

> This means
> prevention).
Due to its barter nature TFT is not adequate for media streaming. Even worse for the case of Live streaming.

> Or otherwise, are continuously exchanging information
Peers do need need to announce via a gossip type protocol the chunks/segments they just got.
It is the interested peer that requests chunkmap updates to the other peers in order to decide which of them have the adequate data, as the streaming mode is not bitstream (media data is not transported in single packets, but in small segments containing a set of media access units).

> same content one has more flexibility whom to rerequest from.
You are reasoning on a single quality, single bitrate media streaming. I am reasoning on a flexible, multi-source, adaptive quality, multibitrate streaming, in which not all serving peers are in posession of the complete media (the chunkmap is multidimensional), and so, peers are always requesting segment layers to different peers, assuming there are more than a couple of pees in the swarm, being one of them the seeder. Note that for scalable and adaptive media, in order to play back media data of a specific interval, all successfully received adequate layers of a segment, belonging to that time interval, have to be combined together before the segment is delivered to the decoder, meaning that the received media quality is variable. The requested layers are received without errors (due to TCP reliability), and so, unless there is a strong congestion in the network, leading to buffer starvation, the media can be played without interruption or defects (artifacts, etc.).

---

From: Csaba Kiraly <[email-masked]>
Date: Wed, 05 Oct 2011 12:33:08 +0200

Hi,

I hope you don't mind if I add my 2c to the conversation. Since I've never expressed my thoughts here before, let me start with a quick intro: I'm Csaba Kiraly, working on the NAPA-WINE project, on the WineStreamer and PeerStreamer applications.

Let me first of all say that currently we use UDP. Not because TCP is bad or evil, but because it seems to be more suitable for our specific system goals and design. Our target is largely dynamic low delay streaming. These are the two parameters we are optimizing for, which seems to be rather different from the system Rui speaks about. As a consequence, our chunks are not seconds long but 1..5 frames (40..200 ms) long. Our p2p neighbourhood relations can be many and they can be short in time, passing only a small number of chunks.

Under these conditions, we can evaluate *TCP vs. UDP* as follows:

1, short-lived TCP connections are known to behave badly providing low average throughput. UDP doesn't have these problems, although you are expected to throw in some "TCP-friendliness". However, since TCP's solutions are not working well in this context, UDP might be a better starting point here.

2, large numbers of short-lived TCP connections are also known to behave badly for congestion control. Thus, you anyway need  some other congestion control mechanisms, which at that point can go over UDP as well.

3, even if a chunk is one frame, partial chunks can be useful. Maybe not re-propagated, but it can be used in local playout. UDP gives us this possibility, a short-lived TCP that stalls don't.

4, we do not need reliable delivery, at least not at the level that TCP provides. Actually, we don't even need it for signaling. If e.g. a chunk offer is lost, the peer will receive another one from someone else, etc.

5, UDP seems to be blocked by more firewalls than TCP. The main reason for this originally was the possible aggressiveness of UDP, but IMHO it has long been forgotten and it is simply applied as a system administration practice. This points in the direction of using
TCP.

*Summarizing*: for our system design, UDP seems a better fit.

About using *HTTP+TCP vs. something+TCP*: once TCP has been decided, there is the choice whether to use HTTP or design a custom protocol. Lets see some arguments pro and contra:

Reasons to use*HTTP+TCP* could be many:

1, the argument I hear the most is firewall traversal. There is no doubt it is true that today HTTP proxies are still dumb and thus systems can use HTTP+TCP as a backup connectivity solution, or even as their main connectivity solution. IMHO this will change as applications use more-and-more HTTP for sending their nasty traffic over the net.

2, libraries+integration: there are many HTTP libraries which makes it rather easy to create an HTTP based prototype

3, header fields (metadata): HTTP does provide some meta information in its headers, which are even extensible. This has some relevance in designing the protocol for the application (an extensible header mechanism is already available without designing one), and it also has some relevance to the proxies that try to understand/filter the traffic.

Reasons to use *TCP with something else* instead:

1, clearly, HTTP provides a restricted use of TCP. The model for bidirectional communication is much more restrictive than what would be allowed by TCP.

2, stability: although it doesn't seem like that, HTTP is a complex protocol. Many of the HTTP libraries have buggy or badly performing, or at least not well tested implementations of some of the less frequently used operation modes. Unfortunately, when you try to stretch the boundaries of HTTP, you happen to use these features, so libraries can easily become a pain instead of a help.

3, network overhead: A custom protocol can easily be designed with less overhead than TCP

4, processing overhead: many of the HTTP libraries are bloated, and indeed if one wants to support all the niche cases, it has to be quite heavy.

5, TCP sockets are not that difficult compared to all the code needed to set-up and tune a HTTP library correctly.

*To summarize*: if I would have to choose the one and only application layer protocol + transport protocol combo for a P2P application, today I would be forced to use HTTP simply for the reason of connectivity. With this choice, I seem to sacrifice low delay and some performance, but gain so many more possible clients that it seems to worth it. However, I don't think there is a need to chose one common denominator at such a high level in the protocol stack. I prefer to use something that performs better and use HTTP as a back-off.

regards,
Csaba

On 10/04/2011 10:58 PM, Rui Cruz wrote:
> Hi
>
> On 04/10/2011, at 13:11, Njaal Borch <[email-masked] <mailto:[email-masked]>> wrote:
>
>> Hi,
>>
>> On 4 October 2011 12:54, Rui Cruz <[email-masked] <mailto:[email-masked]>> wrote:
>>> On 04/10/2011, at 10:37, Arno Bakker <[email-masked] <mailto:[email-masked]>> wrote:

---

From: Marco Mellia <[email-masked]>
Date: Wed, 5 Oct 2011 13:45:53 +0200

Ciao Csaba,

Let me add some more though about the chunks size issue:

Large chunks (several seconds) will definitively cause _long_ delay and impair the real-time property of the stream.

If a chunk has to travel K peers before being received, you have to pay a "store&forward" delay at each hop/peer. So with chunks of 3s and K=5-10 hops in the overlay, the delay will be in the order of 15-30s. This without signaling/scheduling delay, and assuming an upload rate similar to the video rate.

In case upload capacity is smaller than the stream rate (as for low speed peers), or in case the upload capacity is shared by the concurrent transmission of several chunks (as it happens for TCP), the transmission time of a large chunk will be much higher...
Is that what you like?

At the end, small chunks (40-200ms as Csaba is suggesting) allow to keep the store&forward delay much smaller.
Chunks of 40-200ms correspond to few packets. Thus, only UDP seems reasonable here.
It's not surprising the pretty much all p2p-tv software so far is preferring UDP versus TCP...

Said so, HTTP seems the worst solution. It's so bloated and cumbersome to have HTTP to send 5 packets...

-- 
Marco Mellia - Assistant Professor
Dipartimento di Elettronica
Politecnico di Torino
Corso Duca Degli Abruzzi 24
10129 - Torino - IT
Tel: +39-011-090-4173
Cel: +39-331-6714789
Skype: mgmellia
http://www.tlc-networks.polito.it/mellia

On Oct 5, 2011, at 12:33 PM, Csaba Kiraly wrote:

> Under these conditions, we can evaluate TCP vs. UDP as follows:
> 
> Summarizing: for our system design, UDP seems a better fit.
> 
> About using HTTP+TCP vs. something+TCP: once TCP has been decided, there is the choice whether to use HTTP or design a custom protocol. Lets see some arguments pro and contra:
> 
> Reasons to use HTTP+TCP could be many:
> 
> Reasons to use TCP with something else instead:
> 
> To summarize: if I would have to choose the one and only application layer protocol + transport protocol combo for a P2P application, today I would be forced to use HTTP simply for the reason of connectivity. With this choice, I seem to sacrifice low     delay and some performance, but gain so many more possible clients that it seems to worth it. However, I don't think there is a need to chose one common denominator at such a high level in the protocol stack. I prefer to use something that performs better and use HTTP as a back-off.
> 
> regards,
> Csaba
>

---

From: Arno Bakker <[email-masked]>
Date: Thu, 6 Oct 2011 09:41:33 +0200

Hi

for some reason we don't see to be on the same wavelength ;o)
If peer A has chunk X that interest peer B and peer B has
chunk Y that interest peer A in your system they will need
to establish two HTTP connections, one from A to B to
do GET_CHUNK Y and one from B to A to do GET_CHUNK X.

> worse for the case of Live streaming.

What do you use for preventing freeriding instead?

> It is the interested peer that requests chunkmap updates to the other
> peers in order to decide which of them have the adequate data, as the
> streaming mode is not bitstream (media data is not transported in single
> packets, but in small segments containing a set of media access units).
>

Can you elaborate a bit more on how this process works? The above 
suggests you may have to query multiple peers before finding the
chunk you want? Doesn't that cause delays?

> You are reasoning on a single quality, single bitrate media streaming.

No, I'm not actually. This is irrespective of single, SVC or MDC. It is
about whether the transport layer takes care of reliability (TCP) or 
that you handle reliability in the application layer (i.e. PPSP) where
you have more flexibility. In particular, in this case in the
application layer we have multiple sources are available for
rerequesting lost data, as also remarked by Csaba Kiraly.

Regards,
     Arno

---

From: Arno Bakker <[email-masked]>
Date: Thu, 6 Oct 2011 09:44:32 +0200

> We use TCP persistent connections for media transport.

Forgot a remark: Or do you mean that the situation in which peer A and B 
have interest in each other's chunks does not occur frequently?

Regards,
     Arno

---

From: Rui Cruz <[email-masked]>
Date: Thu, 6 Oct 2011 11:02:43 +0200

On 06/10/2011, at 09:44, Arno Bakker wrote:

> Forgot a remark: Or do you mean that the situation in which peer A and B have interest in each other's chunks does not occur frequently?
> 
Requests between peer A and peer B may overlap during a short time interval, but they may be for different purposes like  signaling and media transfer and the connections started and released in different moments.

Regards,
Rui Cruz

---

From: Rui Cruz <[email-masked]>
Date: Thu, 6 Oct 2011 12:07:23 +0200

> do GET_CHUNK Y and one from B to A to do GET_CHUNK X.
For both media data transfer and signaling, it may happen indeed.
> What do you use for preventing freeriding instead?
Depending on the streaming mode (Live, Time-shifted/VoD) and scenarios (related with "business models", namely those being worked within the EU FP7 SARACEN Project), different schemes can be applied and are currently under development in that project. For PPSP it is important the contextual awareness of the incentive scheme to use (for scheduling and control), although the development of these schemes is not in the PPSP charter. 

> Can you elaborate a bit more on how this process works? The above suggests you may have to query multiple peers before finding the
Depending on the streaming method the chunk availability exchange between peers also differs. The Tracker plays a role in the peer selection process, contributing to the "optimization" of the neighbor set for a certain peer, which will then request the current chunkmaps from a subset of that neighbor set. The process is quite fast as the interested peer just filters from the chunkmaps the chunks available within the current time-window. The next updates of the chunkmaps will happen in case of missing chunks of future time-windows (eventually several seconds after the last update).
For Live streaming, the Media Presentation Description (content manifest/index) plays an important role, as the file is being updated while the media is being released (encoded) for streaming.   
> about whether the transport layer takes care of reliability (TCP) or that you handle reliability in the application layer (i.e. PPSP) where

Regards,
Rui Cruz

---

From: Arno Bakker <[email-masked]>
Date: Fri, 14 Oct 2011 09:30:01 +0200

Hi

> For both media data transfer and signaling, it may happen indeed.

I'm afraid it will happen a lot. In a P2P system one wants to
promote sharing, so ideally each peer should have something to give
to another peer, such that most traffic is served by the peers
and in a fair manner.

>
> Depending on the streaming mode (Live, Time-shifted/VoD) and scenarios
> (related with "business models", namely those being worked within the EU
> FP7 SARACEN Project), different schemes can be applied and are currently
> under development in that project. For PPSP it is important the
> contextual awareness of the incentive scheme to use (for scheduling and
> control), although the development of these schemes is not in the PPSP
> charter.
>

Interesting. Is there documentation on this work, an EU deliverable perhaps?

> Depending on the streaming method the chunk availability exchange
> between peers also differs. The Tracker plays a role in the peer
> selection process, contributing to the "optimization" of the neighbor
> set for a certain peer, which will then request the current chunkmaps
> from a subset of that neighbor set. The process is quite fast as the
> interested peer just filters from the chunkmaps the chunks available
> within the current time-window. The next updates of the chunkmaps will
> happen in case of missing chunks of future time-windows (eventually
> several seconds after the last update).
> For Live streaming, the Media Presentation Description (content
> manifest/index) plays an important role, as the file is being updated
> while the media is being released (encoded) for streaming.

So the manifest tells you about the existence of new chunks (live) and
then you poll your peers whether they have them already or not. How 
close in time to the live source do your clients typically get?

Regards,
       Arno