Subject: [ppsp] philosophy of peer-to-peer

From: Dave Cottlehuber <[email-masked]>
Date: Tue, 22 Oct 2013 14:44:30 +0200

Hi folks,

At the end of the IETF87, I mentioned that there seemed to be a philosophical difference in how various people saw Peer-to-peer protocols working, and that this was reflected in the positions taken during some of our discussions. Part of our implicit expectations around how P2P should work is the use cases we see for it, to which I will contribute my ideas in a separate thread.

I hope we can collect our thoughts on this prior to IETF88 and have very productive discussions as a result; being aware of our bias and perspective should be a useful counterpoint.

TL;DR I see P2P, and by extension PPSP, as a means to provide similar levels of reliability to traditional TCP or HTTP-like stream protocols, over a distributed network datastore, at significantly lower cost. And to achieve that successfully, P2P makes a set of trade-offs, sacrificing direct control of endpoints and packets, for emergent (statistical) properties of the overlaid P2P network, that from a statistical perspective, turn out to be sufficient in practise, despite a reduced number of super-peers.

Hopefully the rest makes my position more clear:

WRT to P2P in the wider picture, I think of 3 main "user stories":

- the network provider, interested in shifting data out to the edge of the network, for cost & efficiency reasons, rather than through the core, and ultimately in Billing everybody
- the user thinking of reliability, & privacy - bit torrent based sharing, without a central network or node, encrypted in transit, and unobservable wrt to content and user base, or dropbox, with extreme convenience and negligable security or privacy.
- the content provider, interested in getting a message or file out, possibly in real time, only to those who are authorised to consume it, possibly who paid for it

Note that there is not a fundamental trade-off between these three -- for example, a person sharing a video of a riot in real time that is subsequently going live from a neutral TV station (contrasting with the alleged calm view provided by the State media) represents a content provider who is extremely interested in ensuring that both privacy and the right to consumption are appropriately secured and private. Authorisation from the content provider does not of and in itself dictate a monopoly telco model. End users may also be interested in having P2P algorithms stretch into their homes so that content they are accessing is fast, and can easily switch between devices locally.

WRT to expectations from a P2P solution, as a developer I am thinking:

P2P for me, by definition, is a trade-off between control & efficiency, and takes advantage of the ever improving reliability in modern networks that means less & less data is lost at lower levels, and less is damaged in transit. It hopefully follows that:

Control: I am not interested in tracking per-peer state, such as what blocks are available, and what data volume has been sent, within the tracker protocol itself, due to the added complexity & size of a database that needs to live primarily in RAM for speed but large enough to require distributed nodes. In fact ideally I'd prefer not to need a separate protocol at all (I believe this discussion has already taken place previously BTW). If I am reaching 1 million consumers (a decent sporting event in Europe or elsewhere) I don't want to try to synchronise and maintain received message state for 1000s of messages within the tracker for each peer, at the cost of Gigabytes of transactional RAM, that additionally will require flushing to disk for later analysis. This memory & activity represents a significant waste of bandwidth between peer->tracker (which negates the point of P2P in the first place), and also represents non-negligable overhead. This type of information is not required for operating the P2P network itself, and the potential size of a database that maps blocks to peers, and tracks peers to blocks, is non trivial in practice.

Quality: some loss of packets is expected, and I'm not concerned about that - with some buffering, and a set of good peers, this will rarely impact the user significantly. From a QoS perspective, if I want to ensure 100% transmission of packets to an endpoint, I might as well use streaming HTTP, websockets etc rather than put a lot of that tracking/management functionality into the tracker. The entire point of P2P is to replace reliable point-to-point topologies with an unreliable but significantly more redundant peer overlay. The expectation is that due to cooperative sharing of data messages, the overall reliability from a statistical PoV for and end user does not change noticeably for the user/consumer.

No apologies for the length; how does this read to the rest of you? Does this match your views, or do you see things differently?

-- 
Dave Cottlehuber
Sent with Airmail