From nobody Tue Jan 8 12:23:00 2019 Return-Path: X-Original-To: ledger@ietfa.amsl.com Delivered-To: ledger@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 86093131022 for ; Tue, 8 Jan 2019 12:22:58 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.988 X-Spam-Level: X-Spam-Status: No, score=-1.988 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, T_MIXED_ES=0.01, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=hopebailie.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id FU17yZT6xe3T for ; Tue, 8 Jan 2019 12:22:56 -0800 (PST) Received: from mail-oi1-x22c.google.com (mail-oi1-x22c.google.com [IPv6:2607:f8b0:4864:20::22c]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 9256A130F96 for ; Tue, 8 Jan 2019 12:22:56 -0800 (PST) Received: by mail-oi1-x22c.google.com with SMTP id x202so4401271oif.13 for ; Tue, 08 Jan 2019 12:22:56 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=hopebailie.com; s=google; h=mime-version:from:date:message-id:subject:to; bh=kVN87pKv8ErvBD4YPYV8NF7O62Ye1r0i6JF620GCf5U=; b=V7/8XuqgfsFLC6vQddCav1O5GA4PgQvXI5wf/+mLjYmI6HUNVaRI61y9HMeXf96yz5 Oev4lfnUGfK0fGoWe1rAkXL7JNOxsRnc7y2tSxZF2O+BjrWVkPlYC26+D4v5RjPo+ADq kC2G+BB261qZX8i3fMUCLW9CkFgAy1/lneo4o= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=kVN87pKv8ErvBD4YPYV8NF7O62Ye1r0i6JF620GCf5U=; b=ipv9N25juA5VGXeZtaD4yD/JlnOWtidaDzmstMjQFEJbtXrSNibZ9lTLUaSsglMCwU mu9mPl6+rcu58MUky9vg03TZkbKfyfheTlny/4aPEyF9vjXg9ME+G/wnmIbjLkCCFq9e V0RwhjZAMd+SEmnkEDof9JBjEckxr8VeMBifzEwC4xgf7O10SDb+RUHqMKxg7fwPxqGw QtH+52Pn2LuPWzwxapW1RksMcSt1XtNkrz/yMOjsSU/LlNKNzHCSOstgIb6ae2Jqtz7V mtF3w3qWp/FJrpnHTintToKd6GVXS7QXU1uUXXNLgAIKv1Q+eBWICKVLREBlH+Cgra+f G0KQ== X-Gm-Message-State: AJcUukdz/Fn23KBeSiIGI1tDXItaoJTxnd2m0I8IDVOSOyOGfxUEoJHf XPeD7xbcRiLtIHYuAAoSWP9RcJKyOxwXxclMbf+8Ug== X-Google-Smtp-Source: ALg8bN5hUW6+mrYr8ospmOEE0Dd3LmD5/PoQa94nlTAS1rOdMJXz0pAWN9Kn109FCyh1OmXOjDKC5ANTTpcrLG7RMKQ= X-Received: by 2002:a54:4f86:: with SMTP id g6mr1995710oiy.205.1546978975616; Tue, 08 Jan 2019 12:22:55 -0800 (PST) MIME-Version: 1.0 From: Adrian Hope-Bailie Date: Tue, 8 Jan 2019 22:22:41 +0200 Message-ID: To: Interledger Community Group , Interledger Mailing List - IETF , Hyperledger Quilt Mailing List Content-Type: multipart/mixed; boundary="000000000000524148057ef81ce5" Archived-At: Subject: [Ledger] AGENDA - Interledger Call - 09 January - 3pm UTC X-BeenThere: ledger@ietf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Discussion of interledger, originally a protocol stack for moving digital assets \(making payments\) between accounts operating on different payment networks or ledgers." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Jan 2019 20:22:58 -0000 --000000000000524148057ef81ce5 Content-Type: multipart/alternative; boundary="000000000000524141057ef81ce3" --000000000000524141057ef81ce3 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Hi all, First call of the year. It would be great to hear from the community what your plans are for the year and what you'd like to see from the maintainers and the various ILP related projects. I did some work late last year on a new "ledger protocol" - basically a replacement for BTP - which I would like to present for discussion but other than that, no other agenda items have been proposed. Join from PC, Mac, Linux, iOS or Android: https://zoom.us/s/377506344 Or iPhone one-tap: US: +14086380968,,377506344# or +16468769923,,377506344# Or Telephone: Dial(for higher quality, dial a number based on your current location): US: +1 408 638 0968 <+1%20408%20638%200968> or +1 646 876 9923 <+1%20646%20876%209923> or +1 669 900 6833 <+1%20669%20900%206833> Meeting ID: 377 506 344 International numbers available: https://zoom.us/u/cbrUAzE3W --000000000000524141057ef81ce3 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Hi all,

First call of the year. It woul= d be great to hear from the community what your plans are for the year and = what you'd like to see from the maintainers and the various ILP related= projects.

I did some work late last year on a new= "ledger protocol" - basically a replacement for BTP - which I wo= uld like to present for discussion but other than that, no other agenda ite= ms have been proposed.

Join from PC, Mac, Li= nux, iOS or Android:=C2=A0h= ttps://zoom.us/s/377506344

= Or iPhone one-tap:
=C2=A0=C2=A0=C2=A0=C2=A0US: +14086= 380968,,377506344# or +16468769923,,377506344#
Or Telephone:<= /div>
=C2=A0=C2=A0=C2=A0=C2=A0Dia= l(for higher quality, dial a number based on your current location):=C2=A0<= /font>
=C2=A0=C2= =A0=C2=A0=C2=A0US:=C2=A0+1 408 638 0968=C2=A0or=C2=A0+1 646 876 9923=C2=A0or=C2=A0+1 6= 69 900 6833=C2=A0
= =C2= =A0=C2=A0=C2=A0=C2=A0Meeting ID: 377 506 344
=C2=A0=C2=A0=C2=A0=C2=A0Internation= al numbers available:=C2=A0https://zoom.us/u/cbrUAzE= 3W
--000000000000524141057ef81ce3-- --000000000000524148057ef81ce5 Content-Type: text/calendar; charset="US-ASCII"; name="ilp-2019-01-09.ics" Content-Disposition: attachment; filename="ilp-2019-01-09.ics" Content-Transfer-Encoding: base64 Content-ID: X-Attachment-Id: f_jqo7bes40 QkVHSU46VkNBTEVOREFSDQpWRVJTSU9OOjIuMA0KQ0FMU0NBTEU6R1JFR09SSUFODQpCRUdJTjpW RVZFTlQNCkRUU1RBTVA6MjAxOTAxMDlUMDAwNDU0Wg0KVUlEOjIwMTkwMTA5QGludGVybGVkZ2Vy Lm9yZw0KRFRTVEFSVDoyMDE5MDEwOVQxNTAwMDBaDQpEVEVORDoyMDE5MDEwOVQxNjAwMDBaDQpT VU1NQVJZOkludGVybGVkZ2VyIENvbW11bml0eSBDYWxsDQpVUkw6aHR0cHMlM0ElMkYlMkZ6b29t LnVzJTJGcyUyRjM3NzUwNjM0NA0KREVTQ1JJUFRJT046Sm9pbiBmcm9tIFBDXCwgTWFjXCwgTGlu dXhcLCBpT1Mgb3IgQW5kcm9pZDogaHR0cHM6Ly96b29tLnVzL3MvMzc3NTA2MzQ0XG5cbk9yIGlQ aG9uZSBvbmUtdGFwOlxuICAgIFVTOiArMTQwODYzODA5NjhcLFwsMzc3NTA2MzQ0IyBvciArMTY0 Njg3Njk5MjNcLFwsMzc3NTA2MzQ0I1xuXG5PciBUZWxlcGhvbmU6XG4gICAgRGlhbChmb3IgaGln aGVyIHF1YWxpdHlcLCBkaWFsIGEgbnVtYmVyIGJhc2VkIG9uIHlvdXIgY3VycmVudCBsb2NhdGlv bik6IFxuICAgIFVTOiArMSA0MDggNjM4IDA5Njggb3IgKzEgNjQ2IDg3NiA5OTIzIG9yICsxIDY2 OSA5MDAgNjgzMyBcbiAgICBNZWV0aW5nIElEOiAzNzcgNTA2IDM0NFxuICAgIEludGVybmF0aW9u YWwgbnVtYmVycyBhdmFpbGFibGU6IGh0dHBzOi8vem9vbS51cy91L2NiclVBekUzV1xuDQpFTkQ6 VkVWRU5UDQpFTkQ6VkNBTEVOREFS --000000000000524148057ef81ce5-- From nobody Tue Jan 22 11:17:30 2019 Return-Path: X-Original-To: ledger@ietfa.amsl.com Delivered-To: ledger@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 763EC130F1B for ; Tue, 22 Jan 2019 11:17:28 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.998 X-Spam-Level: X-Spam-Status: No, score=-1.998 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=hopebailie.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id zQwY32V8i-sA for ; Tue, 22 Jan 2019 11:17:26 -0800 (PST) Received: from mail-ot1-x32c.google.com (mail-ot1-x32c.google.com [IPv6:2607:f8b0:4864:20::32c]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 4CCAD130FA2 for ; Tue, 22 Jan 2019 11:17:25 -0800 (PST) Received: by mail-ot1-x32c.google.com with SMTP id w25so24663192otm.13 for ; Tue, 22 Jan 2019 11:17:25 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=hopebailie.com; s=google; h=mime-version:from:date:message-id:subject:to; bh=3sAk4yKoRWqzi+O7k/hamueBbptXBpgvYhUhiTELQZg=; b=bpfrlz0fXqe9dxk39Q+94YTH88VHVvSpx8b7optEr7k/SNuxtn858pJUZmyR2EvUie McCMbkp63yNqiEf3VK+VtnIRnq6qEFM+O9JmQfMRHyS4zK+QfiXsbTy0iANDql8nMLUJ HwYQBDz/zerr5oVRlkJJ9851a9IuOKsUXrNSU= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=3sAk4yKoRWqzi+O7k/hamueBbptXBpgvYhUhiTELQZg=; b=cufz4drx92Vm3qlVARpqjCO9dP9J/5dM0Pi301o+L9F+7SUAOqlM1lLL7CQbsqiFmP FCRdYq0Sk5m52Jo0+PmP4Ooda4LAgu+0gdLH07ExZWLIrYdObla+PtjD5omS2aSHrxQs iFHzysO6POlLoEVDK521YKNOa40oJcl8jXS0+EoAoJd1uurHe8L1rbQb8HGm5XuqhLTY v9Kngl2jV7oNuWJJGoD0Bh3IElhB2uSJsBTy/TXO1OpcZE8PLqIJ9h0P/uql+RABGEf/ 4a0WNbkSvyekei65CI7uMbzo4Vn1oQZ5jdaVeA0NxaE33vSz02XSiNEaPfaxywCkxe8c 7bhg== X-Gm-Message-State: AJcUukc2wr+HroKD/vC5P/H/h5FKgISn82zTXMReGt4GKxo6Qz6VgDg4 2UiYGdjTYhEb//owZTEsAx6WiN1gQvE+JmRUMVaXzg== X-Google-Smtp-Source: ALg8bN4ujKjuaRg8ME3vU11JFrtP82J21byH2TOJ+7tt2ORUtxU9fni3EVFtyGYKSzaF5IX8qmQ6BTLqWRdHjJ+zUHs= X-Received: by 2002:a9d:61c8:: with SMTP id h8mr21747672otk.279.1548184644366; Tue, 22 Jan 2019 11:17:24 -0800 (PST) MIME-Version: 1.0 From: Adrian Hope-Bailie Date: Tue, 22 Jan 2019 21:17:11 +0200 Message-ID: To: Interledger Community Group , Interledger Mailing List - IETF , Hyperledger Quilt Mailing List Content-Type: multipart/mixed; boundary="000000000000c66c11058010d36c" Archived-At: Subject: [Ledger] AGENDA - Interledger Call - 23 January - 3pm UTC X-BeenThere: ledger@ietf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Discussion of interledger, originally a protocol stack for moving digital assets \(making payments\) between accounts operating on different payment networks or ledgers." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 22 Jan 2019 19:17:28 -0000 --000000000000c66c11058010d36c Content-Type: multipart/alternative; boundary="000000000000c66c09058010d36a" --000000000000c66c09058010d36a Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Hi all, Evan has been working on a new HTTP-based bilateral protocol which he'll be chatting about on the call tomorrow. Look out for a follow up mail from him with some details. As usual, if you have any other topics to bring to the call please do! Join from PC, Mac, Linux, iOS or Android: https://zoom.us/s/377506344 Or iPhone one-tap: US: +14086380968,,377506344# or +16468769923,,377506344# Or Telephone: Dial(for higher quality, dial a number based on your current location): US: +1 408 638 0968 <+1%20408%20638%200968> or +1 646 876 9923 <+1%20646%20876%209923> or +1 669 900 6833 <+1%20669%20900%206833> Meeting ID: 377 506 344 International numbers available: https://zoom.us/u/cbrUAzE3W --000000000000c66c09058010d36a Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Hi= all,

Evan has been working on a new HTTP-based bilatera= l protocol which he'll be chatting about on the call tomorrow. Look out= for a follow up mail from him with some details.

= As usual, if you have any other topics to bring to the call please do!


--000000000000c66c09058010d36a-- --000000000000c66c11058010d36c Content-Type: text/calendar; charset="US-ASCII"; name="ilp-2019-01-23.ics" Content-Disposition: attachment; filename="ilp-2019-01-23.ics" Content-Transfer-Encoding: base64 Content-ID: X-Attachment-Id: f_jr8537kd0 QkVHSU46VkNBTEVOREFSDQpWRVJTSU9OOjIuMA0KQ0FMU0NBTEU6R1JFR09SSUFODQpCRUdJTjpW RVZFTlQNCkRUU1RBTVA6MjAxOTAxMjNUMDAwNDU0Wg0KVUlEOjIwMTkwMTIzQGludGVybGVkZ2Vy Lm9yZw0KRFRTVEFSVDoyMDE5MDEyM1QxNTAwMDBaDQpEVEVORDoyMDE5MDEyM1QxNjAwMDBaDQpT VU1NQVJZOkludGVybGVkZ2VyIENvbW11bml0eSBDYWxsDQpVUkw6aHR0cHMlM0ElMkYlMkZ6b29t LnVzJTJGcyUyRjM3NzUwNjM0NA0KREVTQ1JJUFRJT046Sm9pbiBmcm9tIFBDXCwgTWFjXCwgTGlu dXhcLCBpT1Mgb3IgQW5kcm9pZDogaHR0cHM6Ly96b29tLnVzL3MvMzc3NTA2MzQ0XG5cbk9yIGlQ aG9uZSBvbmUtdGFwOlxuICAgIFVTOiArMTQwODYzODA5NjhcLFwsMzc3NTA2MzQ0IyBvciArMTY0 Njg3Njk5MjNcLFwsMzc3NTA2MzQ0I1xuXG5PciBUZWxlcGhvbmU6XG4gICAgRGlhbChmb3IgaGln aGVyIHF1YWxpdHlcLCBkaWFsIGEgbnVtYmVyIGJhc2VkIG9uIHlvdXIgY3VycmVudCBsb2NhdGlv bik6IFxuICAgIFVTOiArMSA0MDggNjM4IDA5Njggb3IgKzEgNjQ2IDg3NiA5OTIzIG9yICsxIDY2 OSA5MDAgNjgzMyBcbiAgICBNZWV0aW5nIElEOiAzNzcgNTA2IDM0NFxuICAgIEludGVybmF0aW9u YWwgbnVtYmVycyBhdmFpbGFibGU6IGh0dHBzOi8vem9vbS51cy91L2NiclVBekUzV1xuDQpFTkQ6 VkVWRU5UDQpFTkQ6VkNBTEVOREFS --000000000000c66c11058010d36c-- From nobody Tue Jan 22 15:11:37 2019 Return-Path: X-Original-To: ledger@ietfa.amsl.com Delivered-To: ledger@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 1E6A213118E for ; Tue, 22 Jan 2019 15:11:35 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.999 X-Spam-Level: X-Spam-Status: No, score=-1.999 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=ripple.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 5PylqYpbi_Qc for ; Tue, 22 Jan 2019 15:11:32 -0800 (PST) Received: from mail-qt1-x835.google.com (mail-qt1-x835.google.com [IPv6:2607:f8b0:4864:20::835]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 223A0131175 for ; Tue, 22 Jan 2019 15:11:31 -0800 (PST) Received: by mail-qt1-x835.google.com with SMTP id n32so292355qte.11 for ; Tue, 22 Jan 2019 15:11:31 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ripple.com; s=google; h=date:from:to:cc:message-id:in-reply-to:references:subject :mime-version; bh=de+yoFbv+0jWV5Ogp8gonCXK6wn6eesq1N0YlAPwAD0=; b=SCLdn2qTu0cG1IT8VBYOLk4GQXZ//OowTdTxPOWSv6bu9ZIOofU4F7at7zk9DXn0tH wZOV2+Qg+iyyOWaMeE8mRJsixW1cmcGgezqNzFUoUvjiSMkFtlODtsC0O0xStQhjsVla 9Ss0UAwI/TfheEPqE7ohR0zRXuftNIAysCGl4= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:message-id:in-reply-to :references:subject:mime-version; bh=de+yoFbv+0jWV5Ogp8gonCXK6wn6eesq1N0YlAPwAD0=; b=CQjS82iidgdDiv0iHd6rvnk7ZYAEd4vOMM5p49QnzfVHN9Bl94R4ZSv9wnyNJmyMNu bLVamFyIpBlBXWrPxtuH0Fd9Irls1VUmFi63c7ZzTVXRa3LGP7dRaDc1ZlXbuYFCtY// 1UNBVgzuYBcylEpS98tSpHrCeId6kCFdWdWDsToS4a3M3Xq+OQPK9V07WQDsY63Hq2gW V2UN7NqMfBi4clzCjRzLY4QbKTOETx3/I99uB09MaSoCHz0VdUvFBh9//fUm9MeVAkoj /EAJ16HpxvYTVo4QdbTJy7koDxbxoywmTV7WFKGSjFyQYEFnSKIsfSwVdtY5WLO6Wb6U qUNw== X-Gm-Message-State: AJcUukdFEvCKMc1njjZPEOItgoaPMiLEr/yotRn5I6bozXFNreQg5nFl HqBdVXNIjzMMbrKeBtJxGJf8izNHfqw= X-Google-Smtp-Source: ALg8bN6f3gRoWZk5l04whzyk4DAD+MuL+xvLVbC7l3H2NPHrLhg2IzeyPdP4+SGycZmqn67yhwZqBA== X-Received: by 2002:ac8:43d0:: with SMTP id w16mr33381728qtn.78.1548198691023; Tue, 22 Jan 2019 15:11:31 -0800 (PST) Received: from arch (inet-64-112-177-6.bos.netblazr.com. [64.112.177.6]) by smtp.gmail.com with ESMTPSA id c17sm89955544qtb.14.2019.01.22.15.11.29 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 22 Jan 2019 15:11:30 -0800 (PST) Date: Tue, 22 Jan 2019 18:11:28 -0500 From: Evan Schwartz To: Adrian Hope-Bailie Cc: Interledger Community Group , Interledger Mailing List - IETF , Hyperledger Quilt Mailing List Message-ID: <1548198536.local-2e8cb3d8-634d-v1.5.5-b7939d38@getmailspring.com> In-Reply-To: References: X-Mailer: Mailspring MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="5c47a320_369c73a_61fa" Archived-At: Subject: Re: [Ledger] AGENDA - Interledger Call - 23 January - 3pm UTC X-BeenThere: ledger@ietf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Discussion of interledger, originally a protocol stack for moving digital assets \(making payments\) between accounts operating on different payment networks or ledgers." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 22 Jan 2019 23:11:35 -0000 --5c47a320_369c73a_61fa Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit Content-Disposition: inline I just wrote up this blog post describing my proposal for making connectors stateless and switching bilateral communication to HTTP: Thoughts on Scaling Interledger Connectors (https://medium.com/@emschwartz/thoughts-on-scaling-interledger-connectors-7e3cad0dab7f): (https://medium.com/@emschwartz/thoughts-on-scaling-interledger-connectors-7e3cad0dab7f) How I Learned to Stop Worrying and Love HTTP (https://medium.com/@emschwartz/thoughts-on-scaling-interledger-connectors-7e3cad0dab7f) Looking forward to hearing your thoughts and discussing on tomorrow's call! Evan On Jan 22 2019, at 2:17 pm, Adrian Hope-Bailie wrote: > Hi all, > > Evan has been working on a new HTTP-based bilateral protocol which he'll be chatting about on the call tomorrow. Look out for a follow up mail from him with some details. > > As usual, if you have any other topics to bring to the call please do! > > Join from PC, Mac, Linux, iOS or Android: https://zoom.us/s/377506344 (http://email.zoom.us/track/click/30854053/zoom.us?p=eyJzIjoiM0w2aVdDeVIxQS1QMWFUY2dOWDNqRF9FXzg4IiwidiI6MSwicCI6IntcInVcIjozMDg1NDA1MyxcInZcIjoxLFwidXJsXCI6XCJodHRwczpcXFwvXFxcL3pvb20udXNcXFwvc1xcXC8zNzc1MDYzNDRcIixcImlkXCI6XCI1NzY5ZmNlY2IyMTA0YjJlOGY5NTdjZDNkMzU3MDQxMlwiLFwidXJsX2lkc1wiOltcImY3NTEyMWVkZGI2NDQ2YjA4YzI1ODAxNjdiMWQ5ODlhM2MyY2FmMTZcIl19In0) > > Or iPhone one-tap: > US: +14086380968,,377506344# or +16468769923,,377506344# > > Or Telephone: > Dial(for higher quality, dial a number based on your current location): > US: +1 408 638 0968 (tel:+1%20408%20638%200968) or +1 646 876 9923 (tel:+1%20646%20876%209923) or +1 669 900 6833 (tel:+1%20669%20900%206833) > Meeting ID: 377 506 344 > > International numbers available: https://zoom.us/u/cbrUAzE3W (http://email.zoom.us/track/click/30854053/zoom.us?p=eyJzIjoiTkhxZ1pWcWZEbDlWZy1VaTBYTnFqX2RwcUs0IiwidiI6MSwicCI6IntcInVcIjozMDg1NDA1MyxcInZcIjoxLFwidXJsXCI6XCJodHRwczpcXFwvXFxcL3pvb20udXNcXFwvdVxcXC9jYnJVQXpFM1dcIixcImlkXCI6XCI1NzY5ZmNlY2IyMTA0YjJlOGY5NTdjZDNkMzU3MDQxMlwiLFwidXJsX2lkc1wiOltcImY3NTEyMWVkZGI2NDQ2YjA4YzI1ODAxNjdiMWQ5ODlhM2MyY2FmMTZcIl19In0) > > > > > > > _______________________________________________ > Ledger mailing list > Ledger@ietf.org > https://www.ietf.org/mailman/listinfo/ledger > --5c47a320_369c73a_61fa Content-Type: text/html; charset="utf-8" Content-Transfer-Encoding: quoted-printable Content-Disposition: inline
I just wrote up this blog post describing my proposal for making con= nectors stateless and switching bilateral communication to HTTP: Thoughts on Scaling Interledger Connectors: How I Learned to Stop Worrying and Love HTTP

Looking forward to hearing your thoughts and discussing o= n tomorrow's call=21
Evan

On Jan 22 2019, at 2:17 pm, Adrian Hope-Bailie <adr= ian=40hopebailie.com> wrote:
Hi all,

Evan has been = working on a new HTTP-based bilateral protocol which he'll be chatting ab= out on the call tomorrow. Look out for a follow up mail from him with som= e details.

= As usual, if= you have any other topics to bring to the call please do=21

<= br>
=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F= =5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F= =5F=5F=5F=5F
Ledger mailing list
Ledger=40ietf.org
https://www.ietf.org/mailman/listinfo/ledger
--5c47a320_369c73a_61fa-- From nobody Wed Jan 23 05:08:25 2019 Return-Path: X-Original-To: ledger@ietfa.amsl.com Delivered-To: ledger@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 20EDA126F72 for ; Wed, 23 Jan 2019 05:08:23 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.999 X-Spam-Level: X-Spam-Status: No, score=-1.999 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=hopebailie.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id oeytIg3wXxKf for ; Wed, 23 Jan 2019 05:08:20 -0800 (PST) Received: from mail-ot1-x341.google.com (mail-ot1-x341.google.com [IPv6:2607:f8b0:4864:20::341]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 40BE11228B7 for ; Wed, 23 Jan 2019 05:08:20 -0800 (PST) Received: by mail-ot1-x341.google.com with SMTP id u16so1814121otk.8 for ; Wed, 23 Jan 2019 05:08:20 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=hopebailie.com; s=google; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=/R4dic7wZ2W+wYICV/Kl6+rxrbZqDmtnFJG+rRgmIyI=; b=E48Lj/ezvHe1+ouiRmV9jLE+ieaui49CR6mi2TjxjuKsT3ysivn4+9izMQn94wawpc MpPHDznCEbLhfaXeFFTIs/fnnyZn/WdPuiPZWNAvceUcv3ryKOkEI7YAjGwb26bQWbY5 jmRDjwSnzw5WwElcyZ1cglIjz0LkGS6yI/62Y= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=/R4dic7wZ2W+wYICV/Kl6+rxrbZqDmtnFJG+rRgmIyI=; b=EW8Srfq8W80RH5AmlwnYPsVQ6Wh3Uc2SAqLZZWtr1IgU5yu5YAaRUtph+dZUO4injX w/D0x5e81x8eQoGJmnR9ixY8+E2rS+UTdRHJfQ+rZY17lRcEPKjvItHZmEN/ERNM+Pqv CyEa6C+sS87xF9M6EAlyBGECkkLN/a+dNi1TqemZ63qmLnyvn3r20fSX6hyxjaLBxCwZ rTOxxhO+oRcxn2hj8ZHLAPDEdX9LrW15xlT8/PAG5iGrCvKpPP5cXlziCUVsQW36uIVV zGvsJNqVjG2IWapIFQgkp/35CchOtVVpOK4DUjz7m+sh/eFnaZ9LvQsgwhkBaIKqWQOD 2MyQ== X-Gm-Message-State: AJcUukcKoMjeqpIwxzyOXvFcNZ3GH0EJPjLzcjCmifWUprgMYTMORIjc fIO8YSdOlmLJNIXaV4lmRoYQslmuQkI2YTeEAfBPfw== X-Google-Smtp-Source: ALg8bN7/NxOI+v/DHlVdXDDHQDQqk/GRFziDibCwbhu1VjhXwCjUg867FUbnMuNS1mxYnagC8KwPGOhwsezUGLGWDjE= X-Received: by 2002:a9d:7cd9:: with SMTP id r25mr1322625otn.110.1548248899078; Wed, 23 Jan 2019 05:08:19 -0800 (PST) MIME-Version: 1.0 References: <1548198536.local-2e8cb3d8-634d-v1.5.5-b7939d38@getmailspring.com> In-Reply-To: <1548198536.local-2e8cb3d8-634d-v1.5.5-b7939d38@getmailspring.com> From: Adrian Hope-Bailie Date: Wed, 23 Jan 2019 15:08:06 +0200 Message-ID: To: Evan Schwartz Cc: Interledger Community Group , Interledger Mailing List - IETF , Hyperledger Quilt Mailing List Content-Type: multipart/alternative; boundary="000000000000a763d505801fc9d4" Archived-At: Subject: Re: [Ledger] AGENDA - Interledger Call - 23 January - 3pm UTC X-BeenThere: ledger@ietf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Discussion of interledger, originally a protocol stack for moving digital assets \(making payments\) between accounts operating on different payment networks or ledgers." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 23 Jan 2019 13:08:23 -0000 --000000000000a763d505801fc9d4 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Hey Evan, Thanks for sharing. Some thoughts on your blog post, we can discuss further on the call: Many of the issues you highlight with BTP are not issues with WebSockets themselves but more with how we have used it. E.g. We should be hosting WebSocket servers that listen on a single port and path and then use a proper auth during the handshake. If you do that, the subsequent messages sent over the underlying TCP connection are very light on framing and meta-data. >From my research WebSockets are one of the most efficient and well standardised framing protocols available over a raw TCP socket. Binary WebSocket frames have very little overhead and all of the session context is dealt with up front in the handshake. If you compare this to HTTP (and HTTP2) you have a lot of overhead in the headers (i.e. sent with every message). Request/reply matching is a key feature requirement of bilateral protocols in ILP but I think we've overblown the complexity required in implementations when you consider that an ILP request/reply pair should be very short lived and that by simply using TCP you already have many of the delivery guarantees you need. HTTP is only capable of request/reply matching because it only allows a request to be sent after a previous reply has been received on the same connection. i.e. You can only send requests serially. Any ILP connections using HTTP 1 or 1.1 will be very inefficient unless they open multiple connections. HTTP/2 gets over this by mutiplexing the connection through streams. But under the hood this is just another framing protocol (like WebSockets) and giving those frames a stream id. Users of the HTTP/2 connection then still need to explicitly create new streams for each request/reply pair if they want to avoid head-of-line blocking. It's likely that the way this is implemented in clients is likely to be highly optimised given the popularity of HTTP/2, but given that streams have a complex state machine that the client must manage it seems unlikely that it can be optimised further than a simple request/reply protocol using a correlation id. (i.e. simply adding a correlation id to the header of a packet and sending in a binary WebSocket frame is definitely more efficient in theory but in practice will come down to implementations). The one aspect of WebSockets that may be hurting performance is the masking which means that payloads need to be pre-processed before being sent. That said, it's a very efficient algorithm so we'd need to test it to be sure it's an issue. A concern I have with the cluster of stateless connectors is the latency you'll introduce if the connector must do a balance check against a DB for each packet. This can be optimised to a point but in reality the most efficient way to do this will be to ensure the stickiness at the load balancer routes packets from the same peer to the same connector. Not sure this will be useful in a scenario where both peers are routing high volumes of packets (e.g. Coil and Strata). That said, I think decoupling the messaging from the clearing and settlement (FYI: forwarding ILP packets isn't clearing, clearing involves reconciliation and netting, but that's a different issue) will be a huge improvement to the architecture. As I've mentioned before, this is the common pattern in most payments. We've been discussing how we could implement this in the reference connector and hope to do some experiments in this regard when we are back from the Mojaloop convening next week. Lastly, I don't think hosting an HTTP server should be treated as a light requirement. I'd still like to be able to accept payment on a connection that I establish to my ILSP (i.e. I am the client) and have some third-party host my SPSP server. Assuming that the ILP receiver and SPSP server are the same thing is a mistake in my opinion and imposes unnecessary limitations on the architecture. Question: Why not transmit the ILP packets with the packet headers as HTTP headers and the data payload as the body? In place byte replacement for packet forwarding is not going to help much if you are framing and de-framing the packets off the wire anyway. All in, there are some great ideas in the blog but I don't think they all depend on changing the bilateral protocol (although I do still want to propose some better options than BTP). I'm keen to get agreement on a basic JS interface that is agnostic to the underlying wire protocol and then we can ship HTTP, WebSocket and any other implementation (I'll try and implement your HTTP protocol in JS as soon as it's stable). Adrian On Wed, 23 Jan 2019 at 01:11, Evan Schwartz wrote: > I just wrote up this blog post describing my proposal for making > connectors stateless and switching bilateral communication to HTTP: Thoug= hts > on Scaling Interledger Connectors > > : > > How I Learned to Stop Worrying and Love HTTP > > > Looking forward to hearing your thoughts and discussing on tomorrow's cal= l! > Evan > > On Jan 22 2019, at 2:17 pm, Adrian Hope-Bailie > wrote: > > Hi all, > > Evan has been working on a new HTTP-based bilateral protocol which he'll > be chatting about on the call tomorrow. Look out for a follow up mail fro= m > him with some details. > > As usual, if you have any other topics to bring to the call please do! > > Join from PC, Mac, Linux, iOS or Android: https://zoom.us/s/377506344 > > > Or iPhone one-tap: > US: +14086380968,,377506344# or +16468769923,,377506344# > > Or Telephone: > Dial(for higher quality, dial a number based on your current > location): > US: +1 408 638 0968 <+1%20408%20638%200968> or +1 646 876 9923 > <+1%20646%20876%209923> or +1 669 900 6833 <+1%20669%20900%206833> > Meeting ID: 377 506 344 > International numbers available: https://zoom.us/u/cbrUAzE3W > > > _______________________________________________ > Ledger mailing list > Ledger@ietf.org > https://www.ietf.org/mailman/listinfo/ledger > > --000000000000a763d505801fc9d4 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Hey Evan,

Thanks for sharing. Some thou= ghts on your blog post, we can discuss further on the call:

<= /div>
Many of the issues you highlight with BTP are not issues with Web= Sockets themselves but more with how we have used it. E.g. We should be hos= ting WebSocket servers that listen on a single port and path and then use a= proper auth during the handshake. If you do that, the subsequent messages = sent over the underlying TCP connection are very light on framing and meta-= data.

From my research WebSockets are one of the m= ost efficient and well standardised framing protocols available over a raw = TCP socket. Binary WebSocket frames have very little overhead and all of th= e session context is dealt with up front in the handshake. If you compare t= his to HTTP (and HTTP2) you have a lot of overhead in the headers (i.e. sen= t with every message).

Request/reply matching is a= key feature requirement of bilateral protocols in ILP but I think we'v= e overblown the complexity required in implementations when you consider th= at an ILP request/reply pair should be very short lived and that by simply = using TCP you already have many of the delivery guarantees you need.
<= div>
HTTP is only capable of request/reply matching because i= t only allows a request to be sent after a previous reply has been received= on the same connection. i.e. You can only send requests serially. Any ILP = connections using HTTP 1 or 1.1 will be very inefficient unless they open m= ultiple connections.=C2=A0

HTTP/2 gets over this b= y mutiplexing the connection through streams. But under the hood this is ju= st another framing protocol (like WebSockets) and giving those frames a str= eam id. Users of the HTTP/2 connection then still need to explicitly create= new streams for each request/reply pair if they want to avoid head-of-line= blocking. It's likely that the way this is implemented in clients is l= ikely to be highly optimised given the popularity of HTTP/2, but given that= streams have a complex state machine that the client must manage it seems = unlikely that it can be optimised further than a simple request/reply proto= col using a correlation id. (i.e. simply adding a correlation id to the hea= der of a packet and sending in a binary WebSocket frame is definitely more = efficient in theory but in practice will come down to implementations).

The one aspect of WebSockets that may be hurting perf= ormance is the masking which means that payloads need to be pre-processed b= efore being sent. That said, it's a very efficient algorithm so we'= d need to test it to be sure it's an issue.

A = concern I have with the cluster of stateless connectors is the latency you&= #39;ll introduce if the connector must do a balance check against a DB for = each packet. This can be optimised to a point but in reality the most effic= ient way to do this will be to ensure the stickiness at the load balancer r= outes packets from the same peer to the same connector. Not sure this will = be useful in a scenario where both peers are routing high volumes of packet= s (e.g. Coil and Strata).

That said, I think decou= pling the messaging from the clearing and settlement (FYI: forwarding ILP p= ackets isn't clearing, clearing involves reconciliation and netting, bu= t that's a different issue) will be a huge improvement to the architect= ure. As I've mentioned before, this is the common pattern in most payme= nts. We've been discussing how we could implement this in the reference= connector and hope to do some experiments in this regard when we are back = from the Mojaloop convening next week.

Lastly, I d= on't think hosting an HTTP server should be treated as a light requirem= ent. I'd still like to be able to accept payment on a connection that I= establish to my ILSP (i.e. I am the client) and have some third-party host= my SPSP server. Assuming that the ILP receiver and SPSP server are the sam= e thing is a mistake in my opinion and imposes unnecessary limitations on t= he architecture.

Question: Why not transmit the IL= P packets with the packet headers as HTTP headers and the data payload as t= he body? In place byte replacement for packet forwarding is not going to he= lp much if you are framing and de-framing the packets off the wire anyway.<= /div>

All in, there are some great ideas in the blog but= I don't think they all depend on changing the bilateral protocol (alth= ough I do still want to propose some better options than BTP).=C2=A0
<= div>
I'm keen to get agreement on a basic JS interface th= at is agnostic to the underlying wire protocol and then we can ship HTTP, W= ebSocket and any other implementation (I'll try and implement your HTTP= protocol in JS as soon as it's stable).

Adria= n




On Wed, 23 Jan 2019 at 01:11, Evan Schwartz = <evan@ripple.com> wrote:
I just wrote up this blog post describing my proposal for making connecto= rs stateless and switching bilateral communication to HTTP: Thoughts on S= caling Interledger Connectors: How I Learned to Stop Worrying and Love HTTP<= /a>

Looking forward to hearing your thoughts and discussing o= n tomorrow's call!
Evan

On Jan 2= 2 2019, at 2:17 pm, Adrian Hope-Bailie <adrian@hopebailie.com> wrote:
Hi all,

Evan has been working on a new HTTP-base= d bilateral protocol which he'll be chatting about on the call tomorrow= . Look out for a follow up mail from him with some details.

As usual, if you have any other topics to bring to the call please d= o!

Join from PC, Mac, Linux, iOS or Android= :=C2=A0https://zoom.us/s/377506344

Or iPhone one-tap:
=C2=A0=C2=A0= =C2=A0=C2=A0US: +14086380968,,377506344# or +16468769923,,377506344#=

Or Telephone:<= /div>
=C2=A0=C2=A0= =C2=A0=C2=A0Dial(for higher quality, dial a number based on your current lo= cation):=C2=A0
=C2=A0=C2=A0=C2=A0=C2=A0US:=C2=A0+1 408 638 0968= =C2=A0or=C2=A0+1 646 876 9923<= /font>=C2=A0or=C2= =A0+1 669= 900 6833=C2=A0=
=C2=A0=C2=A0= =C2=A0=C2=A0Meeting ID: 377 506 344
=C2=A0=C2=A0= =C2=A0=C2=A0International numbers available:=C2=A0https://zoom.us/u/cbrUAzE3W=

_______________________________________________
=
Ledger mailing list
--000000000000a763d505801fc9d4-- From nobody Wed Jan 23 05:15:57 2019 Return-Path: X-Original-To: ledger@ietfa.amsl.com Delivered-To: ledger@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 65DB812D84C for ; Wed, 23 Jan 2019 05:15:55 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.043 X-Spam-Level: X-Spam-Status: No, score=-2.043 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIMWL_WL_MED=-0.142, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=viagenie-ca.20150623.gappssmtp.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id uZ9tV7mF0P6K for ; Wed, 23 Jan 2019 05:15:52 -0800 (PST) Received: from mail-qk1-x732.google.com (mail-qk1-x732.google.com [IPv6:2607:f8b0:4864:20::732]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 8FBE21228B7 for ; Wed, 23 Jan 2019 05:15:52 -0800 (PST) Received: by mail-qk1-x732.google.com with SMTP id d15so1080696qkj.0 for ; Wed, 23 Jan 2019 05:15:52 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=viagenie-ca.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=aNCCgMovqrDcMmYV8OVApT74+bbQcXyX/YYDWIdRNvY=; b=tnLmGQ08Q/xzFIRRFAnQX3xR+kJ/6RitgvQiMdUQefQJ57pVTt9eTzBd19+uPYY/16 S7Zq243uJDCHD7s0W8qT0vqSJs53V1xfVGP6EtaDoaz9VyCPChElCLSMKoasBTYxiBi4 jgWPcrrc3vDQr/CNtLOjSaadiT5CnlJ+0WJrj9B0Ddb2zGVcklIjOShifp/acLlhJOR0 IjJN97NEKToL5ccFGmqpa06I4M0m35fTT6Uhqcjt4Ojh+q2ksknp6VYXzSL/f3YnE8Tx juKwxiCkPmPejOn+eTG/30HaGlN7CDY+du0QmvmoZsBfj6c+RyxDuhU90xc2q+hbLoRf cf6g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=aNCCgMovqrDcMmYV8OVApT74+bbQcXyX/YYDWIdRNvY=; b=MfJACyZ2fYUmwkCGqOM5dwbZ2XQ8D+5+Fvg91MKxlK8oEXMNJPEbYYdElsMrvGue3p wvnXMjMZiCFWDt0QZkHgpXeoqoy31FoeNabVcvRkAMpaEFMnKb1FkXGMREYxJN68bKy5 0DCToe4zuhM/rAZ8u7uQ3InBGpR+vGcxkPhptaCUqbfx7UtO+6/nAZzFaQBmYOFQSJbn sjt6zMqCuiYQmn50u/Nf07WFJ+KtfpqZ+Qj32FupGV6QKFf4nVoGM7py/pYTcJq5+ay5 JKdzLlGiAtZhRNLYtjArOedVF1mUyP5ybTormZeG7kWVNypjH0evSbIsbrhPS8ZyhgSm ETqw== X-Gm-Message-State: AJcUukf2cBUFs4bVZPUcWXLhHNEEbhQZjH7AHlwEaPGr4IduBxMjX0sz +7hr8G2nJy13+4vb9ogXaR08Hg== X-Google-Smtp-Source: ALg8bN4hDdqfhNLKRZSxjU3a6lOAQRzH5VvoWNChSIbEoF9g/7t7FiohzYPEFe0KBa57Y2UHRdWMzA== X-Received: by 2002:a37:ec5:: with SMTP id 188mr1764900qko.146.1548249351487; Wed, 23 Jan 2019 05:15:51 -0800 (PST) Received: from [206.123.31.196] (modemcable040.161-162-184.mc.videotron.ca. [184.162.161.40]) by smtp.gmail.com with ESMTPSA id r5sm56581715qke.33.2019.01.23.05.15.50 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 23 Jan 2019 05:15:50 -0800 (PST) From: "Marc Blanchet" To: "Adrian Hope-Bailie" Cc: "Evan Schwartz" , "Interledger Community Group" , "Interledger Mailing List - IETF" , "Hyperledger Quilt Mailing List" Date: Wed, 23 Jan 2019 08:15:49 -0500 X-Mailer: MailMate (1.12.4r5594) Message-ID: <81C3A496-0F4A-40C3-B6F0-FC775A54318F@viagenie.ca> In-Reply-To: References: <1548198536.local-2e8cb3d8-634d-v1.5.5-b7939d38@getmailspring.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: quoted-printable Archived-At: Subject: Re: [Ledger] AGENDA - Interledger Call - 23 January - 3pm UTC X-BeenThere: ledger@ietf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Discussion of interledger, originally a protocol stack for moving digital assets \(making payments\) between accounts operating on different payment networks or ledgers." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 23 Jan 2019 13:15:55 -0000 On 23 Jan 2019, at 8:08, Adrian Hope-Bailie wrote: > Hey Evan, > > Thanks for sharing. Some thoughts on your blog post, we can discuss = > further > on the call: > > Many of the issues you highlight with BTP are not issues with = > WebSockets > themselves but more with how we have used it. E.g. We should be = > hosting > WebSocket servers that listen on a single port and path and then use a > proper auth during the handshake. If you do that, the subsequent = > messages > sent over the underlying TCP connection are very light on framing and > meta-data. > > From my research WebSockets are one of the most efficient and well > standardised framing protocols available over a raw TCP socket. Binary > WebSocket frames have very little overhead and all of the session = > context > is dealt with up front in the handshake. If you compare this to HTTP = > (and > HTTP2) you have a lot of overhead in the headers (i.e. sent with every > message). > > Request/reply matching is a key feature requirement of bilateral = > protocols > in ILP but I think we've overblown the complexity required in > implementations when you consider that an ILP request/reply pair = > should be > very short lived and that by simply using TCP you already have many of = > the > delivery guarantees you need. > > HTTP is only capable of request/reply matching because it only allows = > a > request to be sent after a previous reply has been received on the = > same > connection. i.e. You can only send requests serially. Any ILP = > connections > using HTTP 1 or 1.1 will be very inefficient unless they open multiple > connections. > > HTTP/2 gets over this by mutiplexing the connection through streams. = > But > under the hood this is just another framing protocol (like WebSockets) = > and > giving those frames a stream id. Users of the HTTP/2 connection then = > still > need to explicitly create new streams for each request/reply pair if = > they > want to avoid head-of-line blocking. It's likely that the way this is > implemented in clients is likely to be highly optimised given the > popularity of HTTP/2, but given that streams have a complex state = > machine > that the client must manage it seems unlikely that it can be optimised > further than a simple request/reply protocol using a correlation id. = > (i.e. > simply adding a correlation id to the header of a packet and sending = > in a > binary WebSocket frame is definitely more efficient in theory but in > practice will come down to implementations). however, http is the =C2=AB=C2=A0substrate=C2=A0=C2=BB for everything on = Internet = nowadays and therefore one benefits of all services available through = cloud providers that are added every day. Therefore, by using http and = that offering, you can really scale up much more easily. my 2 cents, Marc. > > The one aspect of WebSockets that may be hurting performance is the = > masking > which means that payloads need to be pre-processed before being sent. = > That > said, it's a very efficient algorithm so we'd need to test it to be = > sure > it's an issue. > > A concern I have with the cluster of stateless connectors is the = > latency > you'll introduce if the connector must do a balance check against a DB = > for > each packet. This can be optimised to a point but in reality the most > efficient way to do this will be to ensure the stickiness at the load > balancer routes packets from the same peer to the same connector. Not = > sure > this will be useful in a scenario where both peers are routing high = > volumes > of packets (e.g. Coil and Strata). > > That said, I think decoupling the messaging from the clearing and > settlement (FYI: forwarding ILP packets isn't clearing, clearing = > involves > reconciliation and netting, but that's a different issue) will be a = > huge > improvement to the architecture. As I've mentioned before, this is the > common pattern in most payments. We've been discussing how we could > implement this in the reference connector and hope to do some = > experiments > in this regard when we are back from the Mojaloop convening next week. > > Lastly, I don't think hosting an HTTP server should be treated as a = > light > requirement. I'd still like to be able to accept payment on a = > connection > that I establish to my ILSP (i.e. I am the client) and have some > third-party host my SPSP server. Assuming that the ILP receiver and = > SPSP > server are the same thing is a mistake in my opinion and imposes > unnecessary limitations on the architecture. > > Question: Why not transmit the ILP packets with the packet headers as = > HTTP > headers and the data payload as the body? In place byte replacement = > for > packet forwarding is not going to help much if you are framing and > de-framing the packets off the wire anyway. > > All in, there are some great ideas in the blog but I don't think they = > all > depend on changing the bilateral protocol (although I do still want to > propose some better options than BTP). > > I'm keen to get agreement on a basic JS interface that is agnostic to = > the > underlying wire protocol and then we can ship HTTP, WebSocket and any = > other > implementation (I'll try and implement your HTTP protocol in JS as = > soon as > it's stable). > > Adrian > > > > > On Wed, 23 Jan 2019 at 01:11, Evan Schwartz wrote: > >> I just wrote up this blog post describing my proposal for making >> connectors stateless and switching bilateral communication to HTTP: = >> Thoughts >> on Scaling Interledger Connectors >> >> : >> >> How I Learned to Stop Worrying and Love HTTP >> >> >> Looking forward to hearing your thoughts and discussing on tomorrow's = >> call! >> Evan >> >> On Jan 22 2019, at 2:17 pm, Adrian Hope-Bailie = >> >> wrote: >> >> Hi all, >> >> Evan has been working on a new HTTP-based bilateral protocol which = >> he'll >> be chatting about on the call tomorrow. Look out for a follow up mail = >> from >> him with some details. >> >> As usual, if you have any other topics to bring to the call please = >> do! >> >> Join from PC, Mac, Linux, iOS or Android: https://zoom.us/s/377506344 >> >> >> Or iPhone one-tap: >> US: +14086380968,,377506344# or +16468769923,,377506344# >> >> Or Telephone: >> Dial(for higher quality, dial a number based on your current >> location): >> US: +1 408 638 0968 <+1%20408%20638%200968> or +1 646 876 9923 >> <+1%20646%20876%209923> or +1 669 900 6833 <+1%20669%20900%206833> >> Meeting ID: 377 506 344 >> International numbers available: https://zoom.us/u/cbrUAzE3W >> >> >> _______________________________________________ >> Ledger mailing list >> Ledger@ietf.org >> https://www.ietf.org/mailman/listinfo/ledger >> >> From nobody Wed Jan 23 05:40:16 2019 Return-Path: X-Original-To: ledger@ietfa.amsl.com Delivered-To: ledger@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id A1073130E63 for ; Wed, 23 Jan 2019 05:40:14 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2 X-Spam-Level: X-Spam-Status: No, score=-2 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=ripple.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 3feq_2-lMa-V for ; Wed, 23 Jan 2019 05:40:10 -0800 (PST) Received: from mail-qk1-x731.google.com (mail-qk1-x731.google.com [IPv6:2607:f8b0:4864:20::731]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 8967512DF71 for ; Wed, 23 Jan 2019 05:40:10 -0800 (PST) Received: by mail-qk1-x731.google.com with SMTP id 189so1092494qkj.8 for ; Wed, 23 Jan 2019 05:40:10 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ripple.com; s=google; h=date:from:to:cc:message-id:in-reply-to:references:subject :mime-version; bh=c07dTY2VQkbYt1yj1qk5gDyDq/h2KFxLZHCUvetky0w=; b=p+9mMloYDKbOjoXzu3ivP0MgJG+6jDbHHXyOX8mVQUZF+4gIjGkHoaRFF6+XfQxgKY N1yCC0UbG3hrZpO64U8CXkmPiRtbx+60HOBInZ6jvTebTRLPwQuKd01ihBcTr6AG2cZ6 nzTHvUaUIsq5OD95JHXgRMxsciaZjgbSLF7Vk= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:message-id:in-reply-to :references:subject:mime-version; bh=c07dTY2VQkbYt1yj1qk5gDyDq/h2KFxLZHCUvetky0w=; b=pCHVG25M2kIamxJuVhBcBeuJHQrkFCGjOHEocoCCJtqqPLrm2LN2J4b+eZcNWcValV 2DvIPyfe5wexvs4HcZaoCrUX+APbqgHfT32pKLVzitxq0186Mq0sQExF0ulpfkrzuaF8 uQJRd9cTk19xHOBa55QCfXwkHivgCVCz5Xzj2C6P22Tp8gDfhRxUpzrx+NFIkn9fWp1t wHDC11H3H8IaTEHA8qeb7JUm8iMO28XQW9Asq09ZvwLTkPqMh/fLv4uRmslCcJbkNmyC v7lAwmGubdIcaV2qyBxfW43Ybbz+K+aAFPofeOFjL8WpfE8QGVDgpxVH5GUM1xsIzJa9 ybAw== X-Gm-Message-State: AJcUukeiegyzyDHvrDxODswQooow6Htk64nJjLpAYqan1ctfrOlI+qPg FUOlagz1h4DmVhKlCiS6ZTFF2kLeX3s= X-Google-Smtp-Source: ALg8bN7anVABWrVx7LQZaQq1rPCrvJ1JVxMlvQW440hdSIVWiYgsuyFXZDTR4CI9vA4WtIoHNFgXcg== X-Received: by 2002:a37:81c4:: with SMTP id c187mr1799961qkd.114.1548250809328; Wed, 23 Jan 2019 05:40:09 -0800 (PST) Received: from arch (inet-64-112-177-6.bos.netblazr.com. [64.112.177.6]) by smtp.gmail.com with ESMTPSA id q72sm64849109qki.24.2019.01.23.05.40.07 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 23 Jan 2019 05:40:08 -0800 (PST) Date: Wed, 23 Jan 2019 08:40:07 -0500 From: Evan Schwartz To: Marc Blanchet Cc: Adrian Hope-Bailie , Interledger Community Group , Interledger Mailing List - IETF , Hyperledger Quilt Mailing List Message-ID: <1548249396.local-4f884aaf-9cf4-v1.5.5-b7939d38@getmailspring.com> In-Reply-To: <81C3A496-0F4A-40C3-B6F0-FC775A54318F@viagenie.ca> References: <81C3A496-0F4A-40C3-B6F0-FC775A54318F@viagenie.ca> X-Mailer: Mailspring MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="5c486eb7_2f0bb0fd_43f3" Archived-At: Subject: Re: [Ledger] AGENDA - Interledger Call - 23 January - 3pm UTC X-BeenThere: ledger@ietf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Discussion of interledger, originally a protocol stack for moving digital assets \(making payments\) between accounts operating on different payment networks or ledgers." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 23 Jan 2019 13:40:15 -0000 --5c486eb7_2f0bb0fd_43f3 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Content-Disposition: inline Here are a couple of points that I meant to emphasize in the piece: I do not think we need to (or are necessarily going to be able to) get ri= d of BTP. I think BTP is fine for the client-to-server use case. However,= it seems less appropriate for the server-to-server use case and I think = something else is needed for that specific case. One part I really like about the idea of using HTTP2 instead of WebSocket= s is that it would allow us to decouple the connection management logic f= rom the application logic. Right now, our actual implementations need to = figure out if they have an open WebSocket with a specific peer and route = to the correct socket. If we were using HTTP with Keep-Alive or HTTP2, we= 'd be letting the client libraries figure out for us who to keep connecti= ons open with. > however, http is the =C2=AB substrate =C2=BB for everything on Internet= nowadays and therefore one benefits of all services available through cl= oud providers that are added every day. Therefore, by using http and that= offering, you can really scale up much more easily. I wholeheartedly agree=21 > Users of the HTTP/2 connection then still need to explicitly create new= streams for each request/reply pair if they want to avoid head-of-line b= locking. I don't think that's true. Even if the language supports a lower-level in= terface, you wouldn't program to that directly. =46or example, in Node.js= you'd probably use something like http2-client (https://www.npmjs.com/pa= ckage/http2-client) and in Rust you'd use reqwest (https://github.com/sea= nmonstar/reqwest). > A concern I have with the cluster of stateless connectors is the latenc= y you'll introduce if the connector must do a balance check against a DB = for each packet. This can be optimised to a point but in reality the most= efficient way to do this will be to ensure the stickiness at the load ba= lancer routes packets from the same peer to the same connector. Not sure = this will be useful in a scenario where both peers are routing high volum= es of packets (e.g. Coil and Strata). This is worth digging into more. A couple of thoughts: I don't think we're anywhere near the throughput where we'd max out Redis= Connectors with very high volume and trust may not track balance limits a= t all and may just settle up the amount after based on the packet log (I = was envisioning the balance limits being optional) > Lastly, I don't think hosting an HTTP server should be treated as a lig= ht requirement. I'd still like to be able to accept payment on a connecti= on that I establish to my ILSP (i.e. I am the client) and have some third= -party host my SPSP server. Practically speaking, which third party are you going to have hosting you= r SPSP server=3F It's important to realize that anyone hosting your SPSP = server could steal money from you by changing the ILP address and shared = secret they return I'm not saying the SPSP server and receiver must be the same entity. Howe= ver, even in the case you describe, I'd guess you'd be running that recei= ver on a server anyway so you could similarly accept HTTP requests there Even if some power users might have separate SPSP servers and STREAM rece= ivers, I think most people will end up having that infrastructure hosted = by the same entity > Question: Why not transmit the ILP packets with the packet headers as H= TTP headers and the data payload as the body=3F In place byte replacement= for packet forwarding is not going to help much if you are framing and d= e-framing the packets off the wire anyway. That was the direction I went with ILP3. (https://github.com/emschwartz/i= lp3) Stefan hates that idea so I'll let you duke it out with him. In short, I think you could but it would mean somewhat needlessly retrans= mitting the strings =22ILP-Destination=22, =22ILP-Amount=22, etc. Because= of the way HTTP2 header compression works, you get most of the benefit w= hen the header keys and values are exactly the same each time (not true f= or the expiry and condition). > I'm keen to get agreement on a basic JS interface that is agnostic to t= he underlying wire protocol and then we can ship HTTP, WebSocket and any = other implementation (I'll try and implement your HTTP protocol in JS as = soon as it's stable). Don't we already have a JS interface (the plugin)=3F I don't see any valu= e in having a separate format for correlating requests and responses if w= e are going to put the messages on a protocol that provides that correlat= ion. It seems more useful for something like WebSockets or raw TLS socket= s. As I said at the top of this email though, I would advocate against ma= king another WebSocket-based protocol since we already have one. On Jan 23 2019, at 8:15 am, Marc Blanchet w= rote: > > > On 23 Jan 2019, at 8:08, Adrian Hope-Bailie wrote: > > Hey Evan, > > Thanks for sharing. Some thoughts on your blog post, we can discuss > > further > > on the call: > > > > Many of the issues you highlight with BTP are not issues with > > WebSockets > > themselves but more with how we have used it. E.g. We should be > > hosting > > WebSocket servers that listen on a single port and path and then use = a > > proper auth during the handshake. If you do that, the subsequent > > messages > > sent over the underlying TCP connection are very light on framing and= > > meta-data. > > > > =46rom my research WebSockets are one of the most efficient and well > > standardised framing protocols available over a raw TCP socket. Binar= y > > WebSocket frames have very little overhead and all of the session > > context > > is dealt with up front in the handshake. If you compare this to HTTP > > (and > > HTTP2) you have a lot of overhead in the headers (i.e. sent with ever= y > > message). > > > > Request/reply matching is a key feature requirement of bilateral > > protocols > > in ILP but I think we've overblown the complexity required in > > implementations when you consider that an ILP request/reply pair > > should be > > very short lived and that by simply using TCP you already have many o= f > > the > > delivery guarantees you need. > > > > HTTP is only capable of request/reply matching because it only allows= > > a > > request to be sent after a previous reply has been received on the > > same > > connection. i.e. You can only send requests serially. Any ILP > > connections > > using HTTP 1 or 1.1 will be very inefficient unless they open multipl= e > > connections. > > > > HTTP/2 gets over this by mutiplexing the connection through streams. > > But > > under the hood this is just another framing protocol (like WebSockets= ) > > and > > giving those frames a stream id. Users of the HTTP/2 connection then > > still > > need to explicitly create new streams for each request/reply pair if > > they > > want to avoid head-of-line blocking. It's likely that the way this is= > > implemented in clients is likely to be highly optimised given the > > popularity of HTTP/2, but given that streams have a complex state > > machine > > that the client must manage it seems unlikely that it can be optimise= d > > further than a simple request/reply protocol using a correlation id. > > (i.e. > > simply adding a correlation id to the header of a packet and sending > > in a > > binary WebSocket frame is definitely more efficient in theory but in > > practice will come down to implementations). > > > > however, http is the =C2=AB substrate =C2=BB for everything on Internet= > nowadays and therefore one benefits of all services available through > cloud providers that are added every day. Therefore, by using http and > that offering, you can really scale up much more easily. > > my 2 cents, > Marc. > > > > The one aspect of WebSockets that may be hurting performance is the > > masking > > which means that payloads need to be pre-processed before being sent.= > > That > > said, it's a very efficient algorithm so we'd need to test it to be > > sure > > it's an issue. > > > > A concern I have with the cluster of stateless connectors is the > > latency > > you'll introduce if the connector must do a balance check against a D= B > > for > > each packet. This can be optimised to a point but in reality the most= > > efficient way to do this will be to ensure the stickiness at the load= > > balancer routes packets from the same peer to the same connector. Not= > > sure > > this will be useful in a scenario where both peers are routing high > > volumes > > of packets (e.g. Coil and Strata). > > > > That said, I think decoupling the messaging from the clearing and > > settlement (=46YI: forwarding ILP packets isn't clearing, clearing > > involves > > reconciliation and netting, but that's a different issue) will be a > > huge > > improvement to the architecture. As I've mentioned before, this is th= e > > common pattern in most payments. We've been discussing how we could > > implement this in the reference connector and hope to do some > > experiments > > in this regard when we are back from the Mojaloop convening next week= . > > > > Lastly, I don't think hosting an HTTP server should be treated as a > > light > > requirement. I'd still like to be able to accept payment on a > > connection > > that I establish to my ILSP (i.e. I am the client) and have some > > third-party host my SPSP server. Assuming that the ILP receiver and > > SPSP > > server are the same thing is a mistake in my opinion and imposes > > unnecessary limitations on the architecture. > > > > Question: Why not transmit the ILP packets with the packet headers as= > > HTTP > > headers and the data payload as the body=3F In place byte replacement= > > for > > packet forwarding is not going to help much if you are framing and > > de-framing the packets off the wire anyway. > > > > All in, there are some great ideas in the blog but I don't think they= > > all > > depend on changing the bilateral protocol (although I do still want t= o > > propose some better options than BTP). > > > > I'm keen to get agreement on a basic JS interface that is agnostic to= > > the > > underlying wire protocol and then we can ship HTTP, WebSocket and any= > > other > > implementation (I'll try and implement your HTTP protocol in JS as > > soon as > > it's stable). > > > > Adrian > > > > > > > > On Wed, 23 Jan 2019 at 01:11, Evan Schwartz wrote= : > > > I just wrote up this blog post describing my proposal for making > > > connectors stateless and switching bilateral communication to HTTP:= > > > Thoughts > > > on Scaling Interledger Connectors > > > > > > : > > > > > > How I Learned to Stop Worrying and Love HTTP > > > > > > > > > Looking forward to hearing your thoughts and discussing on tomorrow= 's > > > call=21 > > > Evan > > > > > > On Jan 22 2019, at 2:17 pm, Adrian Hope-Bailie > > > > > > wrote: > > > > > > Hi all, > > > Evan has been working on a new HTTP-based bilateral protocol which > > > he'll > > > be chatting about on the call tomorrow. Look out for a follow up ma= il > > > from > > > him with some details. > > > > > > As usual, if you have any other topics to bring to the call please > > > do=21 > > > > > > Join from PC, Mac, Linux, iOS or Android: https://zoom.us/s/3775063= 44 > > > > > > > > > Or iPhone one-tap: > > > US: +14086380968,,377506344=23 or +16468769923,,377506344=23 > > > > > > Or Telephone: > > > Dial(for higher quality, dial a number based on your current > > > location): > > > US: +1 408 638 0968 <+1%20408%20638%200968> or +1 646 876 9923 > > > <+1%20646%20876%209923> or +1 669 900 6833 <+1%20669%20900%206833> > > > Meeting ID: 377 506 344 > > > International numbers available: https://zoom.us/u/cbrUAzE3W > > > > > > > > > =5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F= =5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F > > > Ledger mailing list > > > Ledger=40ietf.org > > > https://www.ietf.org/mailman/listinfo/ledger > > > > --5c486eb7_2f0bb0fd_43f3 Content-Type: text/html; charset="utf-8" Content-Transfer-Encoding: quoted-printable Content-Disposition: inline
Here are a couple of points that I meant to emphasize in the piece:
  • I do not th= ink we need to (or are necessarily going to be able to) get rid of BTP. I= think BTP is fine for the client-to-server use case. However, it seems l= ess appropriate for the server-to-server use case and I think something e= lse is needed for that specific case.
  • One part I reall= y like about the idea of using HTTP2 instead of WebSockets is that it wou= ld allow us to decouple the connection management logic from the applicat= ion logic. Right now, our actual implementations need to figure out if th= ey have an open WebSocket with a specific peer and route to the correct s= ocket. If we were using HTTP with Keep-Alive or HTTP2, we'd be letting th= e client libraries figure out for us who to keep connections open with.

> however, http is th= e =C2=AB substrate =C2=BB for everything on Internet nowad= ays and therefore one benefits of all services available through clo= ud providers that are added every day. Therefore, by using http and = that offering, you can really scale up much more easily. 

I wholeheartedly agree=21

> Users of the HTTP/2 connection then still need to ex= plicitly create new streams for each request/reply pair if they want to a= void head-of-line blocking.

I don't th= ink that's true. Even if the language supports a lower-level interface, y= ou wouldn't program to that directly. =46or example, in Node.js you'd pro= bably use something like http2= -client and in Rust you'd use reqw= est.

> A concern I have wi= th the cluster of stateless connectors is the latency you'll introduce if= the connector must do a balance check against a DB for each packet. This= can be optimised to a point but in reality the most efficient way to do = this will be to ensure the stickiness at the load balancer routes packets= from the same peer to the same connector. Not sure this will be useful i= n a scenario where both peers are routing high volumes of packets (e.g. C= oil and Strata).

This is worth digging= into more. A couple of thoughts:
  • I don't think we're a= nywhere near the throughput where we'd max out Redis
  • C= onnectors with very high volume and trust may not track balance limits at= all and may just settle up the amount after based on the packet log (I w= as envisioning the balance limits being optional)
>= ; Lastly, I don't think hosting an HTTP server = should be treated as a light requirement. I'd still like to be able to ac= cept payment on a connection that I establish to my ILSP (i.e. I am the c= lient) and have some third-party host my SPSP server.

  • Practically speaking, which third party are you g= oing to have hosting your SPSP server=3F It's important to realize that a= nyone hosting your SPSP server could steal money from you by changing the= ILP address and shared secret they return
  • I'm not say= ing the SPSP server and receiver must be the same entity. Howeve= r, even in the case you describe, I'd guess you'd be running that receive= r on a server anyway so you could similarly accept HTTP requests there
  • Even if some power users might have separate SPSP server= s and STREAM receivers, I think most people will end up having that infra= structure hosted by the same entity
> Question: Why not transmit the ILP packets with the packet he= aders as HTTP headers and the data payload as the body=3F In place byte r= eplacement for packet forwarding is not going to help much if you are fra= ming and de-framing the packets off the wire anyway.=

That was the direction I went with ILP3. Stefan hates that idea so I'll let you duke it out with him= .

In short, I think you could but it would mean somewhat ne= edlessly retransmitting the strings =22ILP-Destination=22, =22ILP-Amount=22= , etc. Because of the way HTTP2 header compression works, you get most of= the benefit when the header keys and values are exactly the sam= e each time (not true for the expiry and condition).

> <= span style=3D=22color:rgb(35, 31, 32)=22>I'm keen to get agreement on a basic JS interfac= e that is agnostic to the underlying wire protocol and then we can ship H= TTP, WebSocket and any other implementation (I'll try and implement your = HTTP protocol in JS as soon as it's stable).
Don't we already have a JS interface (the plugin)=3F I don't see a= ny value in having a separate format for correlating requests and respons= es if we are going to put the messages on a protocol that provides that c= orrelation. It seems more useful for something like WebSockets or raw TLS= sockets. As I said at the top of this email though, I would advocate aga= inst making another WebSocket-based protocol since we already have one.
On Jan 23 2019, at 8:1= 5 am, Marc Blanchet <marc.blanchet=40viagenie.ca> wrote:


On 23 Jan 2019, at 8:08, Adrian Hope-Bailie wrot= e:

Hey Evan,

Thanks for sharing. = Some thoughts on your blog post, we can discuss
further
on the call:

Many of the issues you highlight with BTP a= re not issues with
WebSockets
themselves but more wit= h how we have used it. E.g. We should be
hosting
WebS= ocket servers that listen on a single port and path and then use a
<= div>proper auth during the handshake. If you do that, the subsequent
messages
sent over the underlying TCP connection are very= light on framing and
meta-data.

=46rom my resear= ch WebSockets are one of the most efficient and well
standardis= ed framing protocols available over a raw TCP socket. Binary
We= bSocket frames have very little overhead and all of the session
context
is dealt with up front in the handshake. If you compar= e this to HTTP
(and
HTTP2) you have a lot of overhead= in the headers (i.e. sent with every
message).

R= equest/reply matching is a key feature requirement of bilateral
protocols
in ILP but I think we've overblown the complexity re= quired in
implementations when you consider that an ILP request= /reply pair
should be
very short lived and that by si= mply using TCP you already have many of
the
delivery = guarantees you need.

HTTP is only capable of request/reply = matching because it only allows
a
request to be sent = after a previous reply has been received on the
same
= connection. i.e. You can only send requests serially. Any ILP
c= onnections
using HTTP 1 or 1.1 will be very inefficient unless = they open multiple
connections.

HTTP/2 gets over = this by mutiplexing the connection through streams.
But
under the hood this is just another framing protocol (like WebSockets)=
and
giving those frames a stream id. Users of the HT= TP/2 connection then
still
need to explicitly create = new streams for each request/reply pair if
they
want = to avoid head-of-line blocking. It's likely that the way this is
implemented in clients is likely to be highly optimised given the
=
popularity of HTTP/2, but given that streams have a complex state
machine
that the client must manage it seems unlikely t= hat it can be optimised
further than a simple request/reply pro= tocol using a correlation id.
(i.e.
simply adding a c= orrelation id to the header of a packet and sending
in a
<= div>binary WebSocket frame is definitely more efficient in theory but in<= /div>
practice will come down to implementations).
=

however, http is the =C2=AB substrate =C2=BB for e= verything on Internet
nowadays and therefore one benefits of al= l services available through
cloud providers that are added eve= ry day. Therefore, by using http and
that offering, you can rea= lly scale up much more easily.

my 2 cents,

Ma= rc.


The one aspect of WebSockets that may b= e hurting performance is the
masking
which means that= payloads need to be pre-processed before being sent.
That
said, it's a very efficient algorithm so we'd need to test it to be=
sure
it's an issue.

A concern I have w= ith the cluster of stateless connectors is the
latency
you'll introduce if the connector must do a balance check against a DB<= /div>
for
each packet. This can be optimised to a point but= in reality the most
efficient way to do this will be to ensure= the stickiness at the load
balancer routes packets from the sa= me peer to the same connector. Not
sure
this will be = useful in a scenario where both peers are routing high
volumes<= /div>
of packets (e.g. Coil and Strata).

That said, I t= hink decoupling the messaging from the clearing and
settlement = (=46YI: forwarding ILP packets isn't clearing, clearing
involve= s
reconciliation and netting, but that's a different issue) wil= l be a
huge
improvement to the architecture. As I've = mentioned before, this is the
common pattern in most payments. = We've been discussing how we could
implement this in the refere= nce connector and hope to do some
experiments
in this= regard when we are back from the Mojaloop convening next week.

=
Lastly, I don't think hosting an HTTP server should be treated as a<= /div>
light
requirement. I'd still like to be able to accep= t payment on a
connection
that I establish to my ILSP= (i.e. I am the client) and have some
third-party host my SPSP = server. Assuming that the ILP receiver and
SPSP
serve= r are the same thing is a mistake in my opinion and imposes
unn= ecessary limitations on the architecture.

Question: Why not= transmit the ILP packets with the packet headers as
HTTP
=
headers and the data payload as the body=3F In place byte replacemen= t
for
packet forwarding is not going to help much if = you are framing and
de-framing the packets off the wire anyway.=

All in, there are some great ideas in the blog but I don't= think they
all
depend on changing the bilateral prot= ocol (although I do still want to
propose some better options t= han BTP).

I'm keen to get agreement on a basic JS interface= that is agnostic to
the
underlying wire protocol and= then we can ship HTTP, WebSocket and any
other
imple= mentation (I'll try and implement your HTTP protocol in JS as
s= oon as
it's stable).

Adrian




=
On Wed, 23 Jan 2019 at 01:11, Evan Schwartz <evan=40ripple.com>= ; wrote:

I just wrote up this blog post describ= ing my proposal for making
connectors stateless and switching b= ilateral communication to HTTP:
Thoughts
on Scaling I= nterledger Connectors
<https://medium.com/=40emschwartz/thou= ghts-on-scaling-interledger-connectors-7e3cad0dab7f>
:
=
<https://medium.com/=40emschwartz/thoughts-on-scaling-interledger= -connectors-7e3cad0dab7f>
How I Learned to Stop Worrying and= Love HTTP
<https://medium.com/=40emschwartz/thoughts-on-sca= ling-interledger-connectors-7e3cad0dab7f>

Looking forwar= d to hearing your thoughts and discussing on tomorrow's
call=21=
Evan

On Jan 22 2019, at 2:17 pm, Adrian Hope-Bai= lie
<adrian=40hopebailie.com>
wrote:

<= div>Hi all,

Evan has been working on a new HTTP-based bilat= eral protocol which
he'll
be chatting about on the ca= ll tomorrow. Look out for a follow up mail
from
him w= ith some details.

As usual, if you have any other topics to= bring to the call please
do=21

Join from PC, Mac= , Linux, iOS or Android: https://zoom.us/s/377506344
<http:/= /email.zoom.us/track/click/30854053/zoom.us=3Fp=3DeyJzIjoiM0w2aVdDeVIxQS1= QMW=46UY2dOWDNqR=469=46Xzg4IiwidiI6MSwicCI6IntcInVcIjozMDg1NDA1MyxcInZcIj= oxL=46widXJsXCI6XCJodHRwczpcX=46wvX=46xcL3pvb20udXNcX=46wvc1xcXC8zNzc1MDY= zNDRcIixcImlkXCI6XCI1NzY5ZmNlY2IyMTA0YjJlOGY5NTdjZDNkMzU3MDQxMlwiL=46widX= JsX2lkc1wiOltcImY3NTEyMWVkZGI2NDQ2YjA4YzI1ODAxNjdiMWQ5ODlhM2MyY2=46mMTZcI= l19In0>

Or iPhone one-tap:
US: +14086380968,,3= 77506344=23 or +16468769923,,377506344=23

Or Telephone:
Dial(for higher quality, dial a number based on your current
=
location):
US: +1 408 638 0968 <+1%20408%20638%200968&g= t; or +1 646 876 9923
<+1%20646%20876%209923> or +1 669 9= 00 6833 <+1%20669%20900%206833>
Meeting ID: 377 506 344
International numbers available: https://zoom.us/u/cbrUAzE3W
<http://email.zoom.us/track/click/30854053/zoom.us=3Fp=3DeyJzIj= oiTkhxZ1pWcWZEbDlWZy1VaTBYTn=46qX2RwcUs0IiwidiI6MSwicCI6IntcInVcIjozMDg1N= DA1MyxcInZcIjoxL=46widXJsXCI6XCJodHRwczpcX=46wvX=46xcL3pvb20udXNcX=46wvdV= xcXC9jYnJVQXp=46M1dcIixcImlkXCI6XCI1NzY5ZmNlY2IyMTA0YjJlOGY5NTdjZDNkMzU3M= DQxMlwiL=46widXJsX2lkc1wiOltcImY3NTEyMWVkZGI2NDQ2YjA4YzI1ODAxNjdiMWQ5ODlh= M2MyY2=46mMTZcIl19In0>

=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F= =5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F= =5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F
Ledger mailing list
Ledger=40ietf.org
https://www.ietf.org/mailman/listinfo/ledger=
--5c486eb7_2f0bb0fd_43f3-- From nobody Wed Jan 23 06:36:54 2019 Return-Path: X-Original-To: ledger@ietfa.amsl.com Delivered-To: ledger@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id B33E012426E for ; Wed, 23 Jan 2019 06:36:53 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2 X-Spam-Level: X-Spam-Status: No, score=-2 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=ripple.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id nrXut_KJ48O7 for ; Wed, 23 Jan 2019 06:36:50 -0800 (PST) Received: from mail-qt1-x82f.google.com (mail-qt1-x82f.google.com [IPv6:2607:f8b0:4864:20::82f]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 68C88123FFD for ; Wed, 23 Jan 2019 06:36:50 -0800 (PST) Received: by mail-qt1-x82f.google.com with SMTP id r14so2589409qtp.1 for ; Wed, 23 Jan 2019 06:36:50 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ripple.com; s=google; h=date:from:to:cc:message-id:in-reply-to:references:subject :mime-version; bh=hWa+X4xd1Cpv/deoLNHgneOWNbPf7E7+vQzSZVtBGWk=; b=SrAPi0hmZoSsB0/kfoQVVIU9mIfcxOLxYwqlKFZiPAI1fepFzgrjBwOOukOEl2/1Of lN/NZsCLf5zwBhQb/bRrZxelKk5yxpB9Z423VrLvWR6R/1iObjLkpYGLxpEyOhbGPsmM IAdZkjZGc5btQXBzyRjkdzQg2mM4V3GuTAhZ4= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:message-id:in-reply-to :references:subject:mime-version; bh=hWa+X4xd1Cpv/deoLNHgneOWNbPf7E7+vQzSZVtBGWk=; b=K/+sWDEIgem3durzpa/71JOxvWkhNGBBlf6F0IZhFloxjhAzM+TJNypESj1KyU2/WQ QZNkfSiLAPVyyTZF26KEX6tPiFolksmgXrt+kLVftKsSqPpso1kB/xHVL4Y4VTfAy26j 4W0vIKYpzHvnOnO1I2A0sl448uNyUu7RT5OewnDUgUMHyIRizGQJOgbizWrASYWFzMIY N4cPNAHxE5VI7B2is3WsLFNhhpKW1YUv19PKL7ywCtf93ELqYAR/5DIgCKkXs+uKGkgS aZbCqDb0sOM1KJYuOPiqBmpvw4TIWDbq1oInk3vR24Di58Zn4HLDG80CQFy4moOtZWAf M7EA== X-Gm-Message-State: AJcUukebcVLUKgKqGmvsW7+W2pSYAtCuvYinnxv5QnSQmoK0MuzHLGjv LTX8TKwDSa7WcZ9XqxkaTaM5Jg== X-Google-Smtp-Source: ALg8bN4LhzvLhWyIqK2z3dLiRoXesO/mLaUzr1E5PPgetxvGbtmW4anvPfya4LhDSedGcJY5EJUDwQ== X-Received: by 2002:a0c:c584:: with SMTP id a4mr2074827qvj.227.1548254209306; Wed, 23 Jan 2019 06:36:49 -0800 (PST) Received: from arch (inet-64-112-177-6.bos.netblazr.com. [64.112.177.6]) by smtp.gmail.com with ESMTPSA id l51sm68009664qtk.11.2019.01.23.06.36.48 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 23 Jan 2019 06:36:48 -0800 (PST) Date: Wed, 23 Jan 2019 09:36:47 -0500 From: Evan Schwartz To: Jeff Martinez Cc: Marc Blanchet , Adrian Hope-Bailie , Interledger Community Group , Interledger Mailing List - IETF , Hyperledger Quilt Mailing List Message-ID: <1548254101.local-20c667e3-d20b-v1.5.5-b7939d38@getmailspring.com> In-Reply-To: References: X-Mailer: Mailspring MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="5c487bff_4d82389b_43f3" Archived-At: Subject: Re: [Ledger] AGENDA - Interledger Call - 23 January - 3pm UTC X-BeenThere: ledger@ietf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Discussion of interledger, originally a protocol stack for moving digital assets \(making payments\) between accounts operating on different payment networks or ledgers." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 23 Jan 2019 14:36:54 -0000 --5c487bff_4d82389b_43f3 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Content-Disposition: inline Most likely. No matter what messaging technologies we use, attackers will= figure out how to DDoS connectors and receivers. Another reason I like t= he idea of using HTTP is that it puts us in the same boat as every websit= e on the internet and would make it easier for connectors to use off-the-= shelf DDoS protection tools or services. On Jan 23 2019, at 9:34 am, Jeff Martinez wrote: > Will there be any issues with cyber attacks such as DDoS attacks which = can affect the stream=3F > > > > > > On Wed, Jan 23, 2019, 5:42 AM Evan Schwartz wrote: > > Here are a couple of points that I meant to emphasize in the piece: > > I do not think we need to (or are necessarily going to be able to) ge= t rid of BTP. I think BTP is fine for the client-to-server use case. Howe= ver, it seems less appropriate for the server-to-server use case and I th= ink something else is needed for that specific case. > > > > One part I really like about the idea of using HTTP2 instead of WebSo= ckets is that it would allow us to decouple the connection management log= ic from the application logic. Right now, our actual implementations need= to figure out if they have an open WebSocket with a specific peer and ro= ute to the correct socket. If we were using HTTP with Keep-Alive or HTTP2= , we'd be letting the client libraries figure out for us who to keep conn= ections open with. > > > > > > > > > however, http is the =C2=AB substrate =C2=BB for everything on Inte= rnet nowadays and therefore one benefits of all services available throug= h cloud providers that are added every day. Therefore, by using http and = that offering, you can really scale up much more easily. > > I wholeheartedly agree=21 > > > Users of the HTTP/2 connection then still need to explicitly create= new streams for each request/reply pair if they want to avoid head-of-li= ne blocking. > > I don't think that's true. Even if the language supports a lower-leve= l interface, you wouldn't program to that directly. =46or example, in Nod= e.js you'd probably use something like http2-client (https://www.npmjs.co= m/package/http2-client) and in Rust you'd use reqwest (https://github.com= /seanmonstar/reqwest). > > > A concern I have with the cluster of stateless connectors is the la= tency you'll introduce if the connector must do a balance check against a= DB for each packet. This can be optimised to a point but in reality the = most efficient way to do this will be to ensure the stickiness at the loa= d balancer routes packets from the same peer to the same connector. Not s= ure this will be useful in a scenario where both peers are routing high v= olumes of packets (e.g. Coil and Strata). > > This is worth digging into more. A couple of thoughts: > > I don't think we're anywhere near the throughput where we'd max out R= edis > > > > Connectors with very high volume and trust may not track balance limi= ts at all and may just settle up the amount after based on the packet log= (I was envisioning the balance limits being optional) > > > > > > > Lastly, I don't think hosting an HTTP server should be treated as a= light requirement. I'd still like to be able to accept payment on a conn= ection that I establish to my ILSP (i.e. I am the client) and have some t= hird-party host my SPSP server. > > > > Practically speaking, which third party are you going to have hosting= your SPSP server=3F It's important to realize that anyone hosting your S= PSP server could steal money from you by changing the ILP address and sha= red secret they return > > I'm not saying the SPSP server and receiver must be the same entity. = However, even in the case you describe, I'd guess you'd be running that r= eceiver on a server anyway so you could similarly accept HTTP requests th= ere > > > > Even if some power users might have separate SPSP servers and STREAM = receivers, I think most people will end up having that infrastructure hos= ted by the same entity > > > > > > > Question: Why not transmit the ILP packets with the packet headers = as HTTP headers and the data payload as the body=3F In place byte replace= ment for packet forwarding is not going to help much if you are framing a= nd de-framing the packets off the wire anyway. > > > > That was the direction I went with ILP3. (https://github.com/emschwar= tz/ilp3) Stefan hates that idea so I'll let you duke it out with him. > > In short, I think you could but it would mean somewhat needlessly ret= ransmitting the strings =22ILP-Destination=22, =22ILP-Amount=22, etc. Bec= ause of the way HTTP2 header compression works, you get most of the benef= it when the header keys and values are exactly the same each time (not tr= ue for the expiry and condition). > > > I'm keen to get agreement on a basic JS interface that is agnostic = to the underlying wire protocol and then we can ship HTTP, WebSocket and = any other implementation (I'll try and implement your HTTP protocol in JS= as soon as it's stable). > > Don't we already have a JS interface (the plugin)=3F I don't see any = value in having a separate format for correlating requests and responses = if we are going to put the messages on a protocol that provides that corr= elation. It seems more useful for something like WebSockets or raw TLS so= ckets. As I said at the top of this email though, I would advocate agains= t making another WebSocket-based protocol since we already have one. > > On Jan 23 2019, at 8:15 am, Marc Blanchet wrote: > > > > > > > > > On 23 Jan 2019, at 8:08, Adrian Hope-Bailie wrote: > > > > Hey Evan, > > > > Thanks for sharing. Some thoughts on your blog post, we can discu= ss > > > > further > > > > on the call: > > > > > > > > Many of the issues you highlight with BTP are not issues with > > > > WebSockets > > > > themselves but more with how we have used it. E.g. We should be > > > > hosting > > > > WebSocket servers that listen on a single port and path and then = use a > > > > proper auth during the handshake. If you do that, the subsequent > > > > messages > > > > sent over the underlying TCP connection are very light on framing= and > > > > meta-data. > > > > > > > > =46rom my research WebSockets are one of the most efficient and w= ell > > > > standardised framing protocols available over a raw TCP socket. B= inary > > > > WebSocket frames have very little overhead and all of the session= > > > > context > > > > is dealt with up front in the handshake. If you compare this to H= TTP > > > > (and > > > > HTTP2) you have a lot of overhead in the headers (i.e. sent with = every > > > > message). > > > > > > > > Request/reply matching is a key feature requirement of bilateral > > > > protocols > > > > in ILP but I think we've overblown the complexity required in > > > > implementations when you consider that an ILP request/reply pair > > > > should be > > > > very short lived and that by simply using TCP you already have ma= ny of > > > > the > > > > delivery guarantees you need. > > > > > > > > HTTP is only capable of request/reply matching because it only al= lows > > > > a > > > > request to be sent after a previous reply has been received on th= e > > > > same > > > > connection. i.e. You can only send requests serially. Any ILP > > > > connections > > > > using HTTP 1 or 1.1 will be very inefficient unless they open mul= tiple > > > > connections. > > > > > > > > HTTP/2 gets over this by mutiplexing the connection through strea= ms. > > > > But > > > > under the hood this is just another framing protocol (like WebSoc= kets) > > > > and > > > > giving those frames a stream id. Users of the HTTP/2 connection t= hen > > > > still > > > > need to explicitly create new streams for each request/reply pair= if > > > > they > > > > want to avoid head-of-line blocking. It's likely that the way thi= s is > > > > implemented in clients is likely to be highly optimised given the= > > > > popularity of HTTP/2, but given that streams have a complex state= > > > > machine > > > > that the client must manage it seems unlikely that it can be opti= mised > > > > further than a simple request/reply protocol using a correlation = id. > > > > (i.e. > > > > simply adding a correlation id to the header of a packet and send= ing > > > > in a > > > > binary WebSocket frame is definitely more efficient in theory but= in > > > > practice will come down to implementations). > > > > > > > > > > > > however, http is the =C2=AB substrate =C2=BB for everything on Inte= rnet > > > nowadays and therefore one benefits of all services available throu= gh > > > cloud providers that are added every day. Therefore, by using http = and > > > that offering, you can really scale up much more easily. > > > > > > my 2 cents, > > > Marc. > > > > > > > > The one aspect of WebSockets that may be hurting performance is t= he > > > > masking > > > > which means that payloads need to be pre-processed before being s= ent. > > > > That > > > > said, it's a very efficient algorithm so we'd need to test it to = be > > > > sure > > > > it's an issue. > > > > > > > > A concern I have with the cluster of stateless connectors is the > > > > latency > > > > you'll introduce if the connector must do a balance check against= a DB > > > > for > > > > each packet. This can be optimised to a point but in reality the = most > > > > efficient way to do this will be to ensure the stickiness at the = load > > > > balancer routes packets from the same peer to the same connector.= Not > > > > sure > > > > this will be useful in a scenario where both peers are routing hi= gh > > > > volumes > > > > of packets (e.g. Coil and Strata). > > > > > > > > That said, I think decoupling the messaging from the clearing and= > > > > settlement (=46YI: forwarding ILP packets isn't clearing, clearin= g > > > > involves > > > > reconciliation and netting, but that's a different issue) will be= a > > > > huge > > > > improvement to the architecture. As I've mentioned before, this i= s the > > > > common pattern in most payments. We've been discussing how we cou= ld > > > > implement this in the reference connector and hope to do some > > > > experiments > > > > in this regard when we are back from the Mojaloop convening next = week. > > > > > > > > Lastly, I don't think hosting an HTTP server should be treated as= a > > > > light > > > > requirement. I'd still like to be able to accept payment on a > > > > connection > > > > that I establish to my ILSP (i.e. I am the client) and have some > > > > third-party host my SPSP server. Assuming that the ILP receiver a= nd > > > > SPSP > > > > server are the same thing is a mistake in my opinion and imposes > > > > unnecessary limitations on the architecture. > > > > > > > > Question: Why not transmit the ILP packets with the packet header= s as > > > > HTTP > > > > headers and the data payload as the body=3F In place byte replace= ment > > > > for > > > > packet forwarding is not going to help much if you are framing an= d > > > > de-framing the packets off the wire anyway. > > > > > > > > All in, there are some great ideas in the blog but I don't think = they > > > > all > > > > depend on changing the bilateral protocol (although I do still wa= nt to > > > > propose some better options than BTP). > > > > > > > > I'm keen to get agreement on a basic JS interface that is agnosti= c to > > > > the > > > > underlying wire protocol and then we can ship HTTP, WebSocket and= any > > > > other > > > > implementation (I'll try and implement your HTTP protocol in JS a= s > > > > soon as > > > > it's stable). > > > > > > > > Adrian > > > > > > > > > > > > > > > > On Wed, 23 Jan 2019 at 01:11, Evan Schwartz wrote: > > > > > I just wrote up this blog post describing my proposal for makin= g > > > > > connectors stateless and switching bilateral communication to H= TTP: > > > > > Thoughts > > > > > on Scaling Interledger Connectors > > > > > > > > > > : > > > > > > > > > > How I Learned to Stop Worrying and Love HTTP > > > > > > > > > > > > > > > Looking forward to hearing your thoughts and discussing on tomo= rrow's > > > > > call=21 > > > > > Evan > > > > > > > > > > On Jan 22 2019, at 2:17 pm, Adrian Hope-Bailie > > > > > > > > > > wrote: > > > > > > > > > > Hi all, > > > > > Evan has been working on a new HTTP-based bilateral protocol wh= ich > > > > > he'll > > > > > be chatting about on the call tomorrow. Look out for a follow u= p mail > > > > > from > > > > > him with some details. > > > > > > > > > > As usual, if you have any other topics to bring to the call ple= ase > > > > > do=21 > > > > > > > > > > Join from PC, Mac, Linux, iOS or Android: https://zoom.us/s/377= 506344 > > > > > > > > > > > > > > > Or iPhone one-tap: > > > > > US: +14086380968,,377506344=23 or +16468769923,,377506344=23 > > > > > > > > > > Or Telephone: > > > > > Dial(for higher quality, dial a number based on your current > > > > > location): > > > > > US: +1 408 638 0968 <+1%20408%20638%200968> or +1 646 876 9923 > > > > > <+1%20646%20876%209923> or +1 669 900 6833 <+1%20669%20900%2068= 33> > > > > > Meeting ID: 377 506 344 > > > > > International numbers available: https://zoom.us/u/cbrUAzE3W > > > > > > > > > > > > > > > =5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F= =5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F= =5F > > > > > Ledger mailing list > > > > > Ledger=40ietf.org (mailto:Ledger=40ietf.org) > > > > > https://www.ietf.org/mailman/listinfo/ledger > > > > > > > > > > > > > > > --5c487bff_4d82389b_43f3 Content-Type: text/html; charset="utf-8" Content-Transfer-Encoding: quoted-printable Content-Disposition: inline
Most likely. No matter what messaging technologies we use, attackers= will figure out how to DDoS connectors and receivers. Another reason I l= ike the idea of using HTTP is that it puts us in the same boat as every w= ebsite on the internet and would make it easier for connectors to use off= -the-shelf DDoS protection tools or services.

On Jan 23 2019, at 9:34 am, Jeff Martinez &l= t;jmar42=40gmail.com> wrote:
Will ther= e be any issues with cyber attacks such as DDoS attacks which can affect = the stream=3F




=

On Wed, Jan 23, 2019, 5:42 AM Evan Schwart= z <evan=40ripple.com> wrote:
Here are a couple of points that I meant to emphasize in the piece:
  • I do not think we need to (or are necessarily going to be a= ble to) get rid of BTP. I think BTP is fine for the client-to-server use = case. However, it seems less appropriate for the server-to-server use cas= e and I think something else is needed for that specific case.
    • =
  • One part I really like about the idea of using HTTP2 instead of = WebSockets is that it would allow us to decouple the connection managemen= t logic from the application logic. Right now, our actual implementations= need to figure out if they have an open WebSocket with a specific peer a= nd route to the correct socket. If we were using HTTP with Keep-Alive or = HTTP2, we'd be letting the client libraries figure out for us who to keep= connections open with.

> however, http is the =C2=AB substrate =C2=BB for everythi= ng on Internet nowadays and therefore one benefits of all services a= vailable through cloud providers that are added every day. Therefore= , by using http and that offering, you can really scale up much more= easily. 

I wholeheartedly agree=21=

> Users of the HTTP/2 connect= ion then still need to explicitly create new streams for each request/rep= ly pair if they want to avoid head-of-line blocking.=

I don't think that's true. Even if the language supports a= lower-level interface, you wouldn't program to that directly. =46or exam= ple, in Node.js you'd probably use something like http2-client and in Rust you'd use reqwest.

> A concern I have with the cluster of stateless connectors is the la= tency you'll introduce if the connector must do a balance check against a= DB for each packet. This can be optimised to a point but in reality the = most efficient way to do this will be to ensure the stickiness at the loa= d balancer routes packets from the same peer to the same connector. Not s= ure this will be useful in a scenario where both peers are routing high v= olumes of packets (e.g. Coil and Strata).

<= div>This is worth digging into more. A couple of thoughts:
  • <= div>I don't think we're anywhere near the throughput where we'd max out R= edis
  • Connectors with very high volume and trust may no= t track balance limits at all and may just settle up the amount after bas= ed on the packet log (I was envisioning the balance limits being optional= )
    • > Lastly, I don't thin= k hosting an HTTP server should be treated as a light requirement. I'd st= ill like to be able to accept payment on a connection that I establish to= my ILSP (i.e. I am the client) and have some third-party host my SPSP se= rver.

    • Practically speaking, wh= ich third party are you going to have hosting your SPSP server=3F It's im= portant to realize that anyone hosting your SPSP server could steal money= from you by changing the ILP address and shared secret they return
      =
    • I'm not saying the SPSP server and receiver must b= e the same entity. However, even in the case you describe, I'd guess you'= d be running that receiver on a server anyway so you could similarly acce= pt HTTP requests there
    • Even if some power users might = have separate SPSP servers and STREAM receivers, I think most people will= end up having that infrastructure hosted by the same entity
      • > Question: Why not transmit the ILP= packets with the packet headers as HTTP headers and the data payload as = the body=3F In place byte replacement for packet forwarding is not going = to help much if you are framing and de-framing the packets off the wire a= nyway.

      That was the direction I went w= ith ILP3. Stefan hates that idea so I'll l= et you duke it out with him.

      In short, I think you could bu= t it would mean somewhat needlessly retransmitting the strings =22ILP-Des= tination=22, =22ILP-Amount=22, etc. Because of the way HTTP2 header compr= ession works, you get most of the benefit when the header keys and values are exactly the same each time (not true for the expiry and con= dition).

      > I'm keen to get agr= eement on a basic JS interface that is agnostic to the underlying wire pr= otocol and then we can ship HTTP, WebSocket and any other implementation = (I'll try and implement your HTTP protocol in JS as soon as it's stable).=

      Don't we already have a JS interface = (the plugin)=3F I don't see any value in having a separate format for cor= relating requests and responses if we are going to put the messages on a = protocol that provides that correlation. It seems more useful for somethi= ng like WebSockets or raw TLS sockets. As I said at the top of this email= though, I would advocate against making another WebSocket-based protocol= since we already have one.
      On Jan 23 2019, at 8:15 am, Marc Bl= anchet <marc.blanchet=40viagenie.ca>= wrote:


      On 23 Jan 2019, at 8:08, Adria= n Hope-Bailie wrote:

      Hey Evan,

      Th= anks for sharing. Some thoughts on your blog post, we can discuss
      further
      on the call:

      Many of the issues you hi= ghlight with BTP are not issues with
      WebSockets
      thems= elves but more with how we have used it. E.g. We should be
      host= ing
      WebSocket servers that listen on a single port and path and= then use a
      proper auth during the handshake. If you do that, t= he subsequent
      messages
      sent over the underlying TCP c= onnection are very light on framing and
      meta-data.

      =46rom my research WebSockets are one of the most efficient and well
      standardised framing protocols available over a raw TCP socket. B= inary
      WebSocket frames have very little overhead and all of the= session
      context
      is dealt with up front in the handsh= ake. If you compare this to HTTP
      (and
      HTTP2) you have= a lot of overhead in the headers (i.e. sent with every
      message= ).

      Request/reply matching is a key feature requirement of b= ilateral
      protocols
      in ILP but I think we've overblown= the complexity required in
      implementations when you consider t= hat an ILP request/reply pair
      should be
      very short li= ved and that by simply using TCP you already have many of
      the
      delivery guarantees you need.

      HTTP is only capable= of request/reply matching because it only allows
      a
      r= equest to be sent after a previous reply has been received on the
      same
      connection. i.e. You can only send requests serially. A= ny ILP
      connections
      using HTTP 1 or 1.1 will be very i= nefficient unless they open multiple
      connections.

      HTTP/2 gets over this by mutiplexing the connection through streams.
      But
      under the hood this is just another framing protocol= (like WebSockets)
      and
      giving those frames a stream i= d. Users of the HTTP/2 connection then
      still
      need to = explicitly create new streams for each request/reply pair if
      th= ey
      want to avoid head-of-line blocking. It's likely that the wa= y this is
      implemented in clients is likely to be highly optimis= ed given the
      popularity of HTTP/2, but given that streams have = a complex state
      machine
      that the client must manage i= t seems unlikely that it can be optimised
      further than a simple= request/reply protocol using a correlation id.
      (i.e.
      simply adding a correlation id to the header of a packet and sending
      in a
      binary WebSocket frame is definitely more efficient= in theory but in
      practice will come down to implementations).<= /div>


      however, http is the =C2=AB substrate=  =C2=BB for everything on Internet
      nowadays and therefore = one benefits of all services available through
      cloud providers = that are added every day. Therefore, by using http and
      that off= ering, you can really scale up much more easily.

      my 2 cents= ,

      Marc.


      The one aspect of Web= Sockets that may be hurting performance is the
      masking
      which means that payloads need to be pre-processed before being sent.
      That
      said, it's a very efficient algorithm so we'd nee= d to test it to be
      sure
      it's an issue.

      = A concern I have with the cluster of stateless connectors is the
      latency
      you'll introduce if the connector must do a balance c= heck against a DB
      for
      each packet. This can be optimi= sed to a point but in reality the most
      efficient way to do this= will be to ensure the stickiness at the load
      balancer routes p= ackets from the same peer to the same connector. Not
      sure
      =
      this will be useful in a scenario where both peers are routing high<= /div>
      volumes
      of packets (e.g. Coil and Strata).

      <= div>That said, I think decoupling the messaging from the clearing and
      settlement (=46YI: forwarding ILP packets isn't clearing, clearing=
      involves
      reconciliation and netting, but that's a di= fferent issue) will be a
      huge
      improvement to the arch= itecture. As I've mentioned before, this is the
      common pattern = in most payments. We've been discussing how we could
      implement = this in the reference connector and hope to do some
      experiments=
      in this regard when we are back from the Mojaloop convening ne= xt week.

      Lastly, I don't think hosting an HTTP server shoul= d be treated as a
      light
      requirement. I'd still like t= o be able to accept payment on a
      connection
      that I es= tablish to my ILSP (i.e. I am the client) and have some
      third-p= arty host my SPSP server. Assuming that the ILP receiver and
      SP= SP
      server are the same thing is a mistake in my opinion and imp= oses
      unnecessary limitations on the architecture.

      Question: Why not transmit the ILP packets with the packet headers as
      HTTP
      headers and the data payload as the body=3F In pla= ce byte replacement
      for
      packet forwarding is not goin= g to help much if you are framing and
      de-framing the packets of= f the wire anyway.

      All in, there are some great ideas in th= e blog but I don't think they
      all
      depend on changing = the bilateral protocol (although I do still want to
      propose som= e better options than BTP).

      I'm keen to get agreement on a = basic JS interface that is agnostic to
      the
      underlying= wire protocol and then we can ship HTTP, WebSocket and any
      oth= er
      implementation (I'll try and implement your HTTP protocol in= JS as
      soon as
      it's stable).

      Adrian



      On Wed, 23 Jan 2019 at 01:11, Evan Schwartz <evan=40ripple.com> wrote:

      I just wr= ote up this blog post describing my proposal for making
      connect= ors stateless and switching bilateral communication to HTTP:
      Th= oughts
      on Scaling Interledger Connectors
      :
      How I Learned to Stop = Worrying and Love HTTP

      Looking f= orward to hearing your thoughts and discussing on tomorrow's
      ca= ll=21
      Evan

      On Jan 22 2019, at 2:17 pm, Adrian Hop= e-Bailie
      wrote:

      Hi all,

      Evan has been worki= ng on a new HTTP-based bilateral protocol which
      he'll
      be chatting about on the call tomorrow. Look out for a follow up mail
      from
      him with some details.

      As usual, if = you have any other topics to bring to the call please
      do=21
      Join from PC, Mac, Linux, iOS or Android: https:/= /zoom.us/s/377506344

      Or iPhone one-tap= :
      US: +14086380968,,377506344=23 or +16468769923,,377506344=23<= /div>
      Or Telephone:
      Dial(for higher quality, dial a num= ber based on your current
      location):
      US: +1 408 638 0= 968 <+1%20408%20638%200968> or +1 646 876 9923
      <+1%206= 46%20876%209923> or +1 669 900 6833 <+1%20669%20900%206833>
      Meeting ID: 377 506 344
      International numbers available: = https://zoom.us/u/cbrUAzE3W

      =5F= =5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F= =5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F
      Ledger mailing list
    --5c487bff_4d82389b_43f3-- From nobody Wed Jan 23 10:53:52 2019 Return-Path: X-Original-To: ledger@ietfa.amsl.com Delivered-To: ledger@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id B535F130EF1 for ; Wed, 23 Jan 2019 10:53:43 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.999 X-Spam-Level: X-Spam-Status: No, score=-1.999 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=hopebailie.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id qk1NbFkzaxcL for ; Wed, 23 Jan 2019 10:53:38 -0800 (PST) Received: from mail-ot1-x32b.google.com (mail-ot1-x32b.google.com [IPv6:2607:f8b0:4864:20::32b]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id C7E5D1312A7 for ; Wed, 23 Jan 2019 10:27:42 -0800 (PST) Received: by mail-ot1-x32b.google.com with SMTP id 81so2829881otj.2 for ; Wed, 23 Jan 2019 10:27:42 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=hopebailie.com; s=google; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=9Bnbj2PeshKeE1nBHtD2v6XaE7roLYxW0iCYI/RK5Kk=; b=jkhC+nGVZKK6/Fm3TgqJkpZj4wPOI1pqp9M54Xzmn8GFnef89TcUhMdSJwMitzRcYW Nv+3j+P+M2/QKGnPyvkKhR/hp861Biebc2w5cmhPAFVhUlJUvFwgjpRkommyXlAfwu5C 1H4haLkNsrmWVTamnhaH8CcQA19LuRnyQwzOI= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=9Bnbj2PeshKeE1nBHtD2v6XaE7roLYxW0iCYI/RK5Kk=; b=uHRVFmQtsm5yfIdGoTjPh36d663EUk5JKZ88diYs8K51L0CO9ORIxLAE2MhVVUwBQF 8EXtMD8W/AZVROhk0rpF0soCPIKDIeDze5Zpb3ICv7eIMwT3iq9MhTIpPoy53RbEH0Xe uF16xHOsJJRy+/P/g+bGrqC+t/0mLP8OPNrI/qknmPqNy5tj8L27Ek70uQH8xAxw+MsA 2skR1vcOXTiEUKpDY4HAEwzOwqgfr90lqVZPTM7k37IXWXJaUfrfj6jMQgcYVXksIrKk 8msxcKV29NaEM2HrT4JzgqE6LgKv8ua+6EG3Kgvh/IjHtvdFKfh0nVAWawGXdyGe4Hnz M56A== X-Gm-Message-State: AJcUukf2+ZCiO0rcQInwGdCFfV4B/8PO/EIbmd4eNwJ7ddNxkJ2myzKF CIi9skNPwr5MHbq7gzLelYASG6AJ12ynJ3RJAFXasw== X-Google-Smtp-Source: ALg8bN69TdeHPceL/sWjRYYBGR7IXfTBAMAr6OeqjNbw/WcJYF0Q4vUhfOBXsEsegGse1wCo7DPt15PxYBQCtj/S2ls= X-Received: by 2002:a9d:6a41:: with SMTP id h1mr2187392otn.332.1548268061504; Wed, 23 Jan 2019 10:27:41 -0800 (PST) MIME-Version: 1.0 References: <1548254101.local-20c667e3-d20b-v1.5.5-b7939d38@getmailspring.com> In-Reply-To: <1548254101.local-20c667e3-d20b-v1.5.5-b7939d38@getmailspring.com> From: Adrian Hope-Bailie Date: Wed, 23 Jan 2019 20:27:27 +0200 Message-ID: To: Evan Schwartz Cc: Jeff Martinez , Marc Blanchet , Interledger Community Group , Interledger Mailing List - IETF , Hyperledger Quilt Mailing List Content-Type: multipart/alternative; boundary="000000000000d2cd170580243fd9" Archived-At: Subject: Re: [Ledger] AGENDA - Interledger Call - 23 January - 3pm UTC X-BeenThere: ledger@ietf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Discussion of interledger, originally a protocol stack for moving digital assets \(making payments\) between accounts operating on different payment networks or ledgers." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 23 Jan 2019 18:53:44 -0000 --000000000000d2cd170580243fd9 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable An interesting project that could be undertaken if we have a solid HTTP-based protocol would be a native module for nginx or a similar webserver that operates as a simple connector. It could simply route packets based on an in-memory routing table that is updated by another process (which is also the target of peer.* packets) On Wed, 23 Jan 2019 at 16:36, Evan Schwartz wrote: > Most likely. No matter what messaging technologies we use, attackers will > figure out how to DDoS connectors and receivers. Another reason I like th= e > idea of using HTTP is that it puts us in the same boat as every website o= n > the internet and would make it easier for connectors to use off-the-shelf > DDoS protection tools or services. > > On Jan 23 2019, at 9:34 am, Jeff Martinez wrote: > > Will there be any issues with cyber attacks such as DDoS attacks which ca= n > affect the stream? > > > > > > On Wed, Jan 23, 2019, 5:42 AM Evan Schwartz wrote: > > Here are a couple of points that I meant to emphasize in the piece: > > - I do not think we need to (or are necessarily going to be able to) > get rid of BTP. I think BTP is fine for the client-to-server use case. > However, it seems less appropriate for the server-to-server use case a= nd I > think something else is needed for that specific case. > - One part I really like about the idea of using HTTP2 instead of > WebSockets is that it would allow us to decouple the connection manage= ment > logic from the application logic. Right now, our actual implementation= s > need to figure out if they have an open WebSocket with a specific peer= and > route to the correct socket. If we were using HTTP with Keep-Alive or > HTTP2, we'd be letting the client libraries figure out for us who to k= eep > connections open with. > > > > however, http is the =C2=AB substrate =C2=BB for everything on Internet= nowadays > and therefore one benefits of all services available through cloud > providers that are added every day. Therefore, by using http and that > offering, you can really scale up much more easily. > > I wholeheartedly agree! > > > Users of the HTTP/2 connection then still need to explicitly create new > streams for each request/reply pair if they want to avoid head-of-line > blocking. > > I don't think that's true. Even if the language supports a lower-level > interface, you wouldn't program to that directly. For example, in Node.js > you'd probably use something like http2-client > and in Rust you'd use reqwes= t > . > > > A concern I have with the cluster of stateless connectors is the > latency you'll introduce if the connector must do a balance check against= a > DB for each packet. This can be optimised to a point but in reality the > most efficient way to do this will be to ensure the stickiness at the loa= d > balancer routes packets from the same peer to the same connector. Not sur= e > this will be useful in a scenario where both peers are routing high volum= es > of packets (e.g. Coil and Strata). > > This is worth digging into more. A couple of thoughts: > > - I don't think we're anywhere near the throughput where we'd max out > Redis > - Connectors with very high volume and trust may not track balance > limits at all and may just settle up the amount after based on the pac= ket > log (I was envisioning the balance limits being optional) > > > Lastly, I don't think hosting an HTTP server should be treated as a > light requirement. I'd still like to be able to accept payment on a > connection that I establish to my ILSP (i.e. I am the client) and have so= me > third-party host my SPSP server. > > > - Practically speaking, which third party are you going to have > hosting your SPSP server? It's important to realize that anyone hostin= g > your SPSP server could steal money from you by changing the ILP addres= s and > shared secret they return > - I'm not saying the SPSP server and receiver *must* be the same > entity. However, even in the case you describe, I'd guess you'd be run= ning > that receiver on a server anyway so you could similarly accept HTTP > requests there > - Even if some power users might have separate SPSP servers and STREAM > receivers, I think most people will end up having that infrastructure > hosted by the same entity > > > Question: Why not transmit the ILP packets with the packet headers as > HTTP headers and the data payload as the body? In place byte replacement > for packet forwarding is not going to help much if you are framing and > de-framing the packets off the wire anyway. > > That was the direction I went with ILP3. > Stefan hates that idea so I'll let > you duke it out with him. > > In short, I think you could but it would mean somewhat needlessly > retransmitting the strings "ILP-Destination", "ILP-Amount", etc. Because = of > the way HTTP2 header compression works, you get most of the benefit when > the header keys *and* values are exactly the same each time (not true for > the expiry and condition). > > > I'm keen to get agreement on a basic JS interface that is agnostic to > the underlying wire protocol and then we can ship HTTP, WebSocket and any > other implementation (I'll try and implement your HTTP protocol in JS as > soon as it's stable). > > Don't we already have a JS interface (the plugin)? I don't see any value > in having a separate format for correlating requests and responses if we > are going to put the messages on a protocol that provides that correlatio= n. > It seems more useful for something like WebSockets or raw TLS sockets. As= I > said at the top of this email though, I would advocate against making > another WebSocket-based protocol since we already have one. > On Jan 23 2019, at 8:15 am, Marc Blanchet > wrote: > > > > On 23 Jan 2019, at 8:08, Adrian Hope-Bailie wrote: > > Hey Evan, > > Thanks for sharing. Some thoughts on your blog post, we can discuss > further > on the call: > > Many of the issues you highlight with BTP are not issues with > WebSockets > themselves but more with how we have used it. E.g. We should be > hosting > WebSocket servers that listen on a single port and path and then use a > proper auth during the handshake. If you do that, the subsequent > messages > sent over the underlying TCP connection are very light on framing and > meta-data. > > From my research WebSockets are one of the most efficient and well > standardised framing protocols available over a raw TCP socket. Binary > WebSocket frames have very little overhead and all of the session > context > is dealt with up front in the handshake. If you compare this to HTTP > (and > HTTP2) you have a lot of overhead in the headers (i.e. sent with every > message). > > Request/reply matching is a key feature requirement of bilateral > protocols > in ILP but I think we've overblown the complexity required in > implementations when you consider that an ILP request/reply pair > should be > very short lived and that by simply using TCP you already have many of > the > delivery guarantees you need. > > HTTP is only capable of request/reply matching because it only allows > a > request to be sent after a previous reply has been received on the > same > connection. i.e. You can only send requests serially. Any ILP > connections > using HTTP 1 or 1.1 will be very inefficient unless they open multiple > connections. > > HTTP/2 gets over this by mutiplexing the connection through streams. > But > under the hood this is just another framing protocol (like WebSockets) > and > giving those frames a stream id. Users of the HTTP/2 connection then > still > need to explicitly create new streams for each request/reply pair if > they > want to avoid head-of-line blocking. It's likely that the way this is > implemented in clients is likely to be highly optimised given the > popularity of HTTP/2, but given that streams have a complex state > machine > that the client must manage it seems unlikely that it can be optimised > further than a simple request/reply protocol using a correlation id. > (i.e. > simply adding a correlation id to the header of a packet and sending > in a > binary WebSocket frame is definitely more efficient in theory but in > practice will come down to implementations). > > > > however, http is the =C2=AB substrate =C2=BB for everything on Internet > nowadays and therefore one benefits of all services available through > cloud providers that are added every day. Therefore, by using http and > that offering, you can really scale up much more easily. > > my 2 cents, > > Marc. > > > The one aspect of WebSockets that may be hurting performance is the > masking > which means that payloads need to be pre-processed before being sent. > That > said, it's a very efficient algorithm so we'd need to test it to be > sure > it's an issue. > > A concern I have with the cluster of stateless connectors is the > latency > you'll introduce if the connector must do a balance check against a DB > for > each packet. This can be optimised to a point but in reality the most > efficient way to do this will be to ensure the stickiness at the load > balancer routes packets from the same peer to the same connector. Not > sure > this will be useful in a scenario where both peers are routing high > volumes > of packets (e.g. Coil and Strata). > > That said, I think decoupling the messaging from the clearing and > settlement (FYI: forwarding ILP packets isn't clearing, clearing > involves > reconciliation and netting, but that's a different issue) will be a > huge > improvement to the architecture. As I've mentioned before, this is the > common pattern in most payments. We've been discussing how we could > implement this in the reference connector and hope to do some > experiments > in this regard when we are back from the Mojaloop convening next week. > > Lastly, I don't think hosting an HTTP server should be treated as a > light > requirement. I'd still like to be able to accept payment on a > connection > that I establish to my ILSP (i.e. I am the client) and have some > third-party host my SPSP server. Assuming that the ILP receiver and > SPSP > server are the same thing is a mistake in my opinion and imposes > unnecessary limitations on the architecture. > > Question: Why not transmit the ILP packets with the packet headers as > HTTP > headers and the data payload as the body? In place byte replacement > for > packet forwarding is not going to help much if you are framing and > de-framing the packets off the wire anyway. > > All in, there are some great ideas in the blog but I don't think they > all > depend on changing the bilateral protocol (although I do still want to > propose some better options than BTP). > > I'm keen to get agreement on a basic JS interface that is agnostic to > the > underlying wire protocol and then we can ship HTTP, WebSocket and any > other > implementation (I'll try and implement your HTTP protocol in JS as > soon as > it's stable). > > Adrian > > > > > On Wed, 23 Jan 2019 at 01:11, Evan Schwartz wrote: > > I just wrote up this blog post describing my proposal for making > connectors stateless and switching bilateral communication to HTTP: > Thoughts > on Scaling Interledger Connectors > < > https://medium.com/@emschwartz/thoughts-on-scaling-interledger-connectors= -7e3cad0dab7f > > > : > < > https://medium.com/@emschwartz/thoughts-on-scaling-interledger-connectors= -7e3cad0dab7f > > > How I Learned to Stop Worrying and Love HTTP > < > https://medium.com/@emschwartz/thoughts-on-scaling-interledger-connectors= -7e3cad0dab7f > > > > Looking forward to hearing your thoughts and discussing on tomorrow's > call! > Evan > > On Jan 22 2019, at 2:17 pm, Adrian Hope-Bailie > > wrote: > > Hi all, > > Evan has been working on a new HTTP-based bilateral protocol which > he'll > be chatting about on the call tomorrow. Look out for a follow up mail > from > him with some details. > > As usual, if you have any other topics to bring to the call please > do! > > Join from PC, Mac, Linux, iOS or Android: https://zoom.us/s/377506344 > < > http://email.zoom.us/track/click/30854053/zoom.us?p=3DeyJzIjoiM0w2aVdDeVI= xQS1QMWFUY2dOWDNqRF9FXzg4IiwidiI6MSwicCI6IntcInVcIjozMDg1NDA1MyxcInZcIjoxLF= widXJsXCI6XCJodHRwczpcXFwvXFxcL3pvb20udXNcXFwvc1xcXC8zNzc1MDYzNDRcIixcImlkX= CI6XCI1NzY5ZmNlY2IyMTA0YjJlOGY5NTdjZDNkMzU3MDQxMlwiLFwidXJsX2lkc1wiOltcImY3= NTEyMWVkZGI2NDQ2YjA4YzI1ODAxNjdiMWQ5ODlhM2MyY2FmMTZcIl19In0 > > > > Or iPhone one-tap: > US: +14086380968,,377506344# or +16468769923,,377506344# > > Or Telephone: > Dial(for higher quality, dial a number based on your current > location): > US: +1 408 638 0968 <+1%20408%20638%200968> or +1 646 876 9923 > <+1%20646%20876%209923> or +1 669 900 6833 <+1%20669%20900%206833> > Meeting ID: 377 506 344 > International numbers available: https://zoom.us/u/cbrUAzE3W > < > http://email.zoom.us/track/click/30854053/zoom.us?p=3DeyJzIjoiTkhxZ1pWcWZ= EbDlWZy1VaTBYTnFqX2RwcUs0IiwidiI6MSwicCI6IntcInVcIjozMDg1NDA1MyxcInZcIjoxLF= widXJsXCI6XCJodHRwczpcXFwvXFxcL3pvb20udXNcXFwvdVxcXC9jYnJVQXpFM1dcIixcImlkX= CI6XCI1NzY5ZmNlY2IyMTA0YjJlOGY5NTdjZDNkMzU3MDQxMlwiLFwidXJsX2lkc1wiOltcImY3= NTEyMWVkZGI2NDQ2YjA4YzI1ODAxNjdiMWQ5ODlhM2MyY2FmMTZcIl19In0 > > > > _______________________________________________ > Ledger mailing list > Ledger@ietf.org > https://www.ietf.org/mailman/listinfo/ledger > > --000000000000d2cd170580243fd9 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
    An interesting project that could be undertaken if we have= a solid HTTP-based protocol would be a native module for nginx or a simila= r webserver that operates as a simple connector.

    It could simp= ly route packets based on an in-memory routing table that is updated by ano= ther process (which is also the target of peer.* packets)
    =
    On Wed, 23 Jan 2019 at 16:3= 6, Evan Schwartz <evan@ripple.com= > wrote:
    Most likely. No matter what messaging technologies we use, attackers will = figure out how to DDoS connectors and receivers. Another reason I like the = idea of using HTTP is that it puts us in the same boat as every website on = the internet and would make it easier for connectors to use off-the-shelf D= DoS protection tools or services.

    On Jan 23 2019, at 9:34 am, Jeff Martinez= <jmar42@gmail.com= > wrote:
    Will there be any issues wi= th cyber attacks such as DDoS attacks which can affect the stream?





    On Wed, Jan 23, 2019, 5:42 AM Evan Schwartz <evan@ripp= le.com> wrote:
    Here are a couple of poin= ts that I meant to emphasize in the piece:
    • I do not think= we need to (or are necessarily going to be able to) get rid of BTP. I thin= k BTP is fine for the client-to-server use case. However, it seems less app= ropriate for the server-to-server use case and I think something else is ne= eded for that specific case.
    • One part I really like abou= t the idea of using HTTP2 instead of WebSockets is that it would allow us t= o decouple the connection management logic from the application logic. Righ= t now, our actual implementations need to figure out if they have an open W= ebSocket with a specific peer and route to the correct socket. If we were u= sing HTTP with Keep-Alive or HTTP2, we'd be letting the client librarie= s figure out for us who to keep connections open with.

    <= div>> however, http is the =C2=AB=C2=A0substrate=C2=A0=C2=BB for ev= erything on Internet=C2=A0nowadays and therefore one benefits of all servic= es available through=C2=A0cloud providers that are added every day. Therefo= re, by using http and=C2=A0that offering, you can really scale up much more= easily.=C2=A0
    I wholeheartedly agree!
    > Users of the HTTP/2 connection then still need to ex= plicitly create new streams for each request/reply pair if they want to avo= id head-of-line blocking.

    I don't th= ink that's true. Even if the language supports a lower-level interface,= you wouldn't program to that directly. For example, in Node.js you'= ;d probably use something like http2-client and in Rust you'd use reqwest.

    > A concern I have wit= h the cluster of stateless connectors is the latency you'll introduce i= f the connector must do a balance check against a DB for each packet. This = can be optimised to a point but in reality the most efficient way to do thi= s will be to ensure the stickiness at the load balancer routes packets from= the same peer to the same connector. Not sure this will be useful in a sce= nario where both peers are routing high volumes of packets (e.g. Coil and S= trata).

    This is worth digging into more.= A couple of thoughts:
    • I don't think we're anywhe= re near the throughput where we'd max out Redis
    • Conn= ectors with very high volume and trust may not track balance limits at all = and may just settle up the amount after based on the packet log (I was envi= sioning the balance limits being optional)
    > = Lastly, I don't think hosting an HTTP server should be treated as a lig= ht requirement. I'd still like to be able to accept payment on a connec= tion that I establish to my ILSP (i.e. I am the client) and have some third= -party host my SPSP server.

    • Prac= tically speaking, which third party are you going to have hosting your SPSP= server? It's important to realize that anyone hosting your SPSP server= could steal money from you by changing the ILP address and shared secret t= hey return
    • I'm not saying the SPSP server and receiv= er must be the same entity. However, even in the case you describe= , I'd guess you'd be running that receiver on a server anyway so yo= u could similarly accept HTTP requests there
    • Even if som= e power users might have separate SPSP servers and STREAM receivers, I thin= k most people will end up having that infrastructure hosted by the same ent= ity
    > Question: Why not transmit the ILP packet= s with the packet headers as HTTP headers and the data payload as the body?= In place byte replacement for packet forwarding is not going to help much = if you are framing and de-framing the packets off the wire anyway.

    That was the direction I went with ILP3. Stefan hates that idea so I'll let you = duke it out with him.

    In short, I think you could but it woul= d mean somewhat needlessly retransmitting the strings "ILP-Destination= ", "ILP-Amount", etc. Because of the way HTTP2 header compre= ssion works, you get most of the benefit when the header keys and = values are exactly the same each time (not true for the expiry and conditio= n).

    > I'm keen to get agreement on a basic JS int= erface that is agnostic to the underlying wire protocol and then we can shi= p HTTP, WebSocket and any other implementation (I'll try and implement = your HTTP protocol in JS as soon as it's stable).<= /div>
    Don't we already have a JS interface (the plugin)? I don&= #39;t see any value in having a separate format for correlating requests an= d responses if we are going to put the messages on a protocol that provides= that correlation. It seems more useful for something like WebSockets or ra= w TLS sockets. As I said at the top of this email though, I would advocate = against making another WebSocket-based protocol since we already have one.<= /div>
    On Jan 23 2019, at 8:15 am, Marc Blanchet <marc.blanchet@viagenie.ca> wrote:
    =

    On 23 Jan 2019, at 8:08, Adrian Hope-Bailie wrote:

    Hey Evan,

    Thanks for sharing. Some thoughts on= your blog post, we can discuss
    further
    on the call:
    Many of the issues you highlight with BTP are not issues with
    WebSockets
    themselves but more with how we have used it.= E.g. We should be
    hosting
    WebSocket servers that liste= n on a single port and path and then use a
    proper auth during the= handshake. If you do that, the subsequent
    messages
    sen= t over the underlying TCP connection are very light on framing and
    meta-data.

    From my research WebSockets are one of the most = efficient and well
    standardised framing protocols available over = a raw TCP socket. Binary
    WebSocket frames have very little overhe= ad and all of the session
    context
    is dealt with up fron= t in the handshake. If you compare this to HTTP
    (and
    HT= TP2) you have a lot of overhead in the headers (i.e. sent with every
    <= div>message).

    Request/reply matching is a key feature require= ment of bilateral
    protocols
    in ILP but I think we'v= e overblown the complexity required in
    implementations when you c= onsider that an ILP request/reply pair
    should be
    very s= hort lived and that by simply using TCP you already have many of
    = the
    delivery guarantees you need.

    HTTP is only capa= ble of request/reply matching because it only allows
    a
    = request to be sent after a previous reply has been received on the
    same
    connection. i.e. You can only send requests serially. Any = ILP
    connections
    using HTTP 1 or 1.1 will be very ineffi= cient unless they open multiple
    connections.

    HTTP/2= gets over this by mutiplexing the connection through streams.
    Bu= t
    under the hood this is just another framing protocol (like WebS= ockets)
    and
    giving those frames a stream id. Users of t= he HTTP/2 connection then
    still
    need to explicitly crea= te new streams for each request/reply pair if
    they
    want= to avoid head-of-line blocking. It's likely that the way this is
    =
    implemented in clients is likely to be highly optimised given the
    popularity of HTTP/2, but given that streams have a complex state
    machine
    that the client must manage it seems unlikely that= it can be optimised
    further than a simple request/reply protocol= using a correlation id.
    (i.e.
    simply adding a correlat= ion id to the header of a packet and sending
    in a
    binar= y WebSocket frame is definitely more efficient in theory but in
    p= ractice will come down to implementations).


    = however, http is the =C2=AB=C2=A0substrate=C2=A0=C2=BB for everything on In= ternet
    nowadays and therefore one benefits of all services availa= ble through
    cloud providers that are added every day. Therefore, = by using http and
    that offering, you can really scale up much mor= e easily.

    my 2 cents,

    Marc.


    The one aspect of WebSockets that may be hurting performance is = the
    masking
    which means that payloads need to be pre-pr= ocessed before being sent.
    That
    said, it's a very e= fficient algorithm so we'd need to test it to be
    sure
    it's an issue.

    A concern I have with the cluster of st= ateless connectors is the
    latency
    you'll introduce = if the connector must do a balance check against a DB
    for
    each packet. This can be optimised to a point but in reality the most
    efficient way to do this will be to ensure the stickiness at the lo= ad
    balancer routes packets from the same peer to the same connect= or. Not
    sure
    this will be useful in a scenario where bo= th peers are routing high
    volumes
    of packets (e.g. Coil= and Strata).

    That said, I think decoupling the messaging fro= m the clearing and
    settlement (FYI: forwarding ILP packets isn= 9;t clearing, clearing
    involves
    reconciliation and nett= ing, but that's a different issue) will be a
    huge
    i= mprovement to the architecture. As I've mentioned before, this is the
    common pattern in most payments. We've been discussing how we = could
    implement this in the reference connector and hope to do so= me
    experiments
    in this regard when we are back from the= Mojaloop convening next week.

    Lastly, I don't think host= ing an HTTP server should be treated as a
    light
    require= ment. I'd still like to be able to accept payment on a
    connec= tion
    that I establish to my ILSP (i.e. I am the client) and have = some
    third-party host my SPSP server. Assuming that the ILP recei= ver and
    SPSP
    server are the same thing is a mistake in = my opinion and imposes
    unnecessary limitations on the architectur= e.

    Question: Why not transmit the ILP packets with the packet= headers as
    HTTP
    headers and the data payload as the bo= dy? In place byte replacement
    for
    packet forwarding is = not going to help much if you are framing and
    de-framing the pack= ets off the wire anyway.

    All in, there are some great ideas i= n the blog but I don't think they
    all
    depend on cha= nging the bilateral protocol (although I do still want to
    propose= some better options than BTP).

    I'm keen to get agreement= on a basic JS interface that is agnostic to
    the
    underl= ying wire protocol and then we can ship HTTP, WebSocket and any
    o= ther
    implementation (I'll try and implement your HTTP protoco= l in JS as
    soon as
    it's stable).

    Adri= an




    On Wed, 23 Jan 2019 at 01:11, Evan Schwartz &l= t;evan@ripple.com> wrote:

    I jus= t wrote up this blog post describing my proposal for making
    conne= ctors stateless and switching bilateral communication to HTTP:
    Th= oughts
    on Scaling Interledger Connectors

    On Jan 22 2019, at 2:17 pm, Adrian Hope-Bailie
    <adrian@hopebailie.com>
    wrote:=

    Hi all,

    Evan has been working on a new HTTP-ba= sed bilateral protocol which
    he'll
    be chatting abou= t on the call tomorrow. Look out for a follow up mail
    from
    <= div>him with some details.

    As usual, if you have any other to= pics to bring to the call please
    do!

    Join from PC, = Mac, Linux, iOS or Android: https://zoom.us/s/377506= 344

    Or iPhone one-tap:
    US: +14086380= 968,,377506344# or +16468769923,,377506344#

    Or Telephone:
    Dial(for higher quality, dial a number based on your current
    location):
    US: +1 408 638 0968 <+1%20408%20638%200968> o= r +1 646 876 9923
    <+1%20646%20876%209923> or +1 669 900 683= 3 <+1%20669%20900%206833>
    Meeting ID: 377 506 344
    International numbers available: https://zoom.us/u/cb= rUAzE3W

    ______________________________________= _________
    Ledger mailing list
    --000000000000d2cd170580243fd9-- From nobody Mon Jan 28 07:04:10 2019 Return-Path: X-Original-To: ledger@ietfa.amsl.com Delivered-To: ledger@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 764131200D7 for ; Mon, 28 Jan 2019 07:04:08 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.999 X-Spam-Level: X-Spam-Status: No, score=-1.999 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=ripple.com Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id kdoHHpoxF8gS for ; Mon, 28 Jan 2019 07:03:59 -0800 (PST) Received: from mail-qk1-x734.google.com (mail-qk1-x734.google.com [IPv6:2607:f8b0:4864:20::734]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 8586212894E for ; Mon, 28 Jan 2019 07:03:58 -0800 (PST) Received: by mail-qk1-x734.google.com with SMTP id q8so9553345qke.1 for ; Mon, 28 Jan 2019 07:03:58 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ripple.com; s=google; h=date:from:to:cc:message-id:in-reply-to:references:subject :mime-version; bh=dW1CymZ+Q9qqG94CMiZ/ikPd963Lv8X8hONeIbbmrJc=; b=oOBXOfyyUuyvkqjKT9dNpdENocO2dPDWBbgAajVNBSbdWjMwQqtoU1IxYQz0v6eejA RiXklqb6Am9+Fgkl+6BKgrm+O85tGNOZ5kax/JNOzb+8iDnORSGQtxDFMt2iw8C4fpLm y31rMObxwPRQat0QaLe+CqbI1eG3HK7Fh5g2U= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:message-id:in-reply-to :references:subject:mime-version; bh=dW1CymZ+Q9qqG94CMiZ/ikPd963Lv8X8hONeIbbmrJc=; b=TUxXhX3tyPVE5rmjaQjIl7g3auM0L/xHgsxrb7bmw3L6N4m1FhCqqWV1PbMcirPvoD RwGq7HNeDPWjZF1rXEGToNGX5U1zOoeOGkAaeYE2qynoRhny9WLEk/8sc49o8+FVz6TK A15WB9s+LXdLJos2nV177d+lHHVjCCUNGSAbu8rHnUpe3aDoYdow6mk/Ngqg9Zh1nLyP fPhMWgte71eLzT0+CwaUZrL58PJTBWYI+F97ngKEqtboRHZWvpHGoOG4NAnANddnRYzf R2eCM+KbDF+0DDQmyxnwUJObYL9lwqZJYWlG3WjhA8yOo5A8XL9Y0MhJeVLXV1aIoWN4 2V7Q== X-Gm-Message-State: AJcUukfUnNhQQKEi9WExeYY9+Bb9t2mUCsx5se8NIEqW4RZUGjGjtSF8 Ickhh7BXQ7RLUCkpzBOPeZgt5w== X-Google-Smtp-Source: ALg8bN6ewyXZKiFU84tOPfrIctN8f9VUJ+4M8t058u6mmjKbIloKLWZdVHdMX0QppLQC8n+L0FLz1w== X-Received: by 2002:a37:7b01:: with SMTP id w1mr19933981qkc.122.1548687837299; Mon, 28 Jan 2019 07:03:57 -0800 (PST) Received: from arch (inet-64-112-177-6.bos.netblazr.com. [64.112.177.6]) by smtp.gmail.com with ESMTPSA id c17sm110148535qtb.14.2019.01.28.07.03.56 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 28 Jan 2019 07:03:56 -0800 (PST) Date: Mon, 28 Jan 2019 10:03:55 -0500 From: Evan Schwartz To: Adrian Hope-Bailie Cc: Jeff Martinez , Marc Blanchet , Interledger Community Group , Interledger Mailing List - IETF , Hyperledger Quilt Mailing List Message-ID: <1548687723.local-847557c0-3d75-v1.5.5-b7939d38@getmailspring.com> In-Reply-To: References: X-Mailer: Mailspring MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="5c4f19db_174ab0c_2861" Archived-At: Subject: Re: [Ledger] AGENDA - Interledger Call - 23 January - 3pm UTC X-BeenThere: ledger@ietf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Discussion of interledger, originally a protocol stack for moving digital assets \(making payments\) between accounts operating on different payment networks or ledgers." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 28 Jan 2019 15:04:09 -0000 --5c4f19db_174ab0c_2861 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Content-Disposition: inline Agreed=21 One more thing, HTTP/3 came up on the community call and Cloudflare coinc= identally released a blog post (https://blog.cloudflare.com/http-3-from-r= oot-to-tip/) about it the following day. It's a bit of a long read but al= so gives a sense of how IET=46 standardization works, which may be intere= sting for those in this community as well. Best, Evan On Jan 23 2019, at 1:27 pm, Adrian Hope-Bailie = wrote: > An interesting project that could be undertaken if we have a solid HTTP= -based protocol would be a native module for nginx or a similar webserver= that operates as a simple connector. > > It could simply route packets based on an in-memory routing table that = is updated by another process (which is also the target of peer.* packets= ) > On Wed, 23 Jan 2019 at 16:36, Evan Schwartz wrote: > > Most likely. No matter what messaging technologies we use, attackers = will figure out how to DDoS connectors and receivers. Another reason I li= ke the idea of using HTTP is that it puts us in the same boat as every we= bsite on the internet and would make it easier for connectors to use off-= the-shelf DDoS protection tools or services. > > > > On Jan 23 2019, at 9:34 am, Jeff Martinez wrote: > > > Will there be any issues with cyber attacks such as DDoS attacks wh= ich can affect the stream=3F > > > > > > > > > > > > > > > > > > On Wed, Jan 23, 2019, 5:42 AM Evan Schwartz wrote: > > > > Here are a couple of points that I meant to emphasize in the piec= e: > > > > I do not think we need to (or are necessarily going to be able to= ) get rid of BTP. I think BTP is fine for the client-to-server use case. = However, it seems less appropriate for the server-to-server use case and = I think something else is needed for that specific case. > > > > > > > > One part I really like about the idea of using HTTP2 instead of W= ebSockets is that it would allow us to decouple the connection management= logic from the application logic. Right now, our actual implementations = need to figure out if they have an open WebSocket with a specific peer an= d route to the correct socket. If we were using HTTP with Keep-Alive or H= TTP2, we'd be letting the client libraries figure out for us who to keep = connections open with. > > > > > > > > > > > > > > > > > however, http is the =C2=AB substrate =C2=BB for everything on = Internet nowadays and therefore one benefits of all services available th= rough cloud providers that are added every day. Therefore, by using http = and that offering, you can really scale up much more easily. > > > > I wholeheartedly agree=21 > > > > > Users of the HTTP/2 connection then still need to explicitly cr= eate new streams for each request/reply pair if they want to avoid head-o= f-line blocking. > > > > I don't think that's true. Even if the language supports a lower-= level interface, you wouldn't program to that directly. =46or example, in= Node.js you'd probably use something like http2-client (https://www.npmj= s.com/package/http2-client) and in Rust you'd use reqwest (https://github= .com/seanmonstar/reqwest). > > > > > A concern I have with the cluster of stateless connectors is th= e latency you'll introduce if the connector must do a balance check again= st a DB for each packet. This can be optimised to a point but in reality = the most efficient way to do this will be to ensure the stickiness at the= load balancer routes packets from the same peer to the same connector. N= ot sure this will be useful in a scenario where both peers are routing hi= gh volumes of packets (e.g. Coil and Strata). > > > > This is worth digging into more. A couple of thoughts: > > > > I don't think we're anywhere near the throughput where we'd max o= ut Redis > > > > > > > > Connectors with very high volume and trust may not track balance = limits at all and may just settle up the amount after based on the packet= log (I was envisioning the balance limits being optional) > > > > > > > > > > > > > Lastly, I don't think hosting an HTTP server should be treated = as a light requirement. I'd still like to be able to accept payment on a = connection that I establish to my ILSP (i.e. I am the client) and have so= me third-party host my SPSP server. > > > > > > > > Practically speaking, which third party are you going to have hos= ting your SPSP server=3F It's important to realize that anyone hosting yo= ur SPSP server could steal money from you by changing the ILP address and= shared secret they return > > > > I'm not saying the SPSP server and receiver must be the same enti= ty. However, even in the case you describe, I'd guess you'd be running th= at receiver on a server anyway so you could similarly accept HTTP request= s there > > > > > > > > Even if some power users might have separate SPSP servers and STR= EAM receivers, I think most people will end up having that infrastructure= hosted by the same entity > > > > > > > > > > > > > Question: Why not transmit the ILP packets with the packet head= ers as HTTP headers and the data payload as the body=3F In place byte rep= lacement for packet forwarding is not going to help much if you are frami= ng and de-framing the packets off the wire anyway. > > > > > > > > That was the direction I went with ILP3. (https://github.com/emsc= hwartz/ilp3) Stefan hates that idea so I'll let you duke it out with him.= > > > > In short, I think you could but it would mean somewhat needlessly= retransmitting the strings =22ILP-Destination=22, =22ILP-Amount=22, etc.= Because of the way HTTP2 header compression works, you get most of the b= enefit when the header keys and values are exactly the same each time (no= t true for the expiry and condition). > > > > > I'm keen to get agreement on a basic JS interface that is agnos= tic to the underlying wire protocol and then we can ship HTTP, WebSocket = and any other implementation (I'll try and implement your HTTP protocol i= n JS as soon as it's stable). > > > > Don't we already have a JS interface (the plugin)=3F I don't see = any value in having a separate format for correlating requests and respon= ses if we are going to put the messages on a protocol that provides that = correlation. It seems more useful for something like WebSockets or raw TL= S sockets. As I said at the top of this email though, I would advocate ag= ainst making another WebSocket-based protocol since we already have one. > > > > On Jan 23 2019, at 8:15 am, Marc Blanchet wrote: > > > > > > > > > > > > > > > On 23 Jan 2019, at 8:08, Adrian Hope-Bailie wrote: > > > > > > Hey Evan, > > > > > > Thanks for sharing. Some thoughts on your blog post, we can d= iscuss > > > > > > further > > > > > > on the call: > > > > > > > > > > > > Many of the issues you highlight with BTP are not issues with= > > > > > > WebSockets > > > > > > themselves but more with how we have used it. E.g. We should = be > > > > > > hosting > > > > > > WebSocket servers that listen on a single port and path and t= hen use a > > > > > > proper auth during the handshake. If you do that, the subsequ= ent > > > > > > messages > > > > > > sent over the underlying TCP connection are very light on fra= ming and > > > > > > meta-data. > > > > > > > > > > > > =46rom my research WebSockets are one of the most efficient a= nd well > > > > > > standardised framing protocols available over a raw TCP socke= t. Binary > > > > > > WebSocket frames have very little overhead and all of the ses= sion > > > > > > context > > > > > > is dealt with up front in the handshake. If you compare this = to HTTP > > > > > > (and > > > > > > HTTP2) you have a lot of overhead in the headers (i.e. sent w= ith every > > > > > > message). > > > > > > > > > > > > Request/reply matching is a key feature requirement of bilate= ral > > > > > > protocols > > > > > > in ILP but I think we've overblown the complexity required in= > > > > > > implementations when you consider that an ILP request/reply p= air > > > > > > should be > > > > > > very short lived and that by simply using TCP you already hav= e many of > > > > > > the > > > > > > delivery guarantees you need. > > > > > > > > > > > > HTTP is only capable of request/reply matching because it onl= y allows > > > > > > a > > > > > > request to be sent after a previous reply has been received o= n the > > > > > > same > > > > > > connection. i.e. You can only send requests serially. Any ILP= > > > > > > connections > > > > > > using HTTP 1 or 1.1 will be very inefficient unless they open= multiple > > > > > > connections. > > > > > > > > > > > > HTTP/2 gets over this by mutiplexing the connection through s= treams. > > > > > > But > > > > > > under the hood this is just another framing protocol (like We= bSockets) > > > > > > and > > > > > > giving those frames a stream id. Users of the HTTP/2 connecti= on then > > > > > > still > > > > > > need to explicitly create new streams for each request/reply = pair if > > > > > > they > > > > > > want to avoid head-of-line blocking. It's likely that the way= this is > > > > > > implemented in clients is likely to be highly optimised given= the > > > > > > popularity of HTTP/2, but given that streams have a complex s= tate > > > > > > machine > > > > > > that the client must manage it seems unlikely that it can be = optimised > > > > > > further than a simple request/reply protocol using a correlat= ion id. > > > > > > (i.e. > > > > > > simply adding a correlation id to the header of a packet and = sending > > > > > > in a > > > > > > binary WebSocket frame is definitely more efficient in theory= but in > > > > > > practice will come down to implementations). > > > > > > > > > > > > > > > > > > > > however, http is the =C2=AB substrate =C2=BB for everything on = Internet > > > > > nowadays and therefore one benefits of all services available t= hrough > > > > > cloud providers that are added every day. Therefore, by using h= ttp and > > > > > that offering, you can really scale up much more easily. > > > > > > > > > > my 2 cents, > > > > > Marc. > > > > > > > > > > > > The one aspect of WebSockets that may be hurting performance = is the > > > > > > masking > > > > > > which means that payloads need to be pre-processed before bei= ng sent. > > > > > > That > > > > > > said, it's a very efficient algorithm so we'd need to test it= to be > > > > > > sure > > > > > > it's an issue. > > > > > > > > > > > > A concern I have with the cluster of stateless connectors is = the > > > > > > latency > > > > > > you'll introduce if the connector must do a balance check aga= inst a DB > > > > > > for > > > > > > each packet. This can be optimised to a point but in reality = the most > > > > > > efficient way to do this will be to ensure the stickiness at = the load > > > > > > balancer routes packets from the same peer to the same connec= tor. Not > > > > > > sure > > > > > > this will be useful in a scenario where both peers are routin= g high > > > > > > volumes > > > > > > of packets (e.g. Coil and Strata). > > > > > > > > > > > > That said, I think decoupling the messaging from the clearing= and > > > > > > settlement (=46YI: forwarding ILP packets isn't clearing, cle= aring > > > > > > involves > > > > > > reconciliation and netting, but that's a different issue) wil= l be a > > > > > > huge > > > > > > improvement to the architecture. As I've mentioned before, th= is is the > > > > > > common pattern in most payments. We've been discussing how we= could > > > > > > implement this in the reference connector and hope to do some= > > > > > > experiments > > > > > > in this regard when we are back from the Mojaloop convening n= ext week. > > > > > > > > > > > > Lastly, I don't think hosting an HTTP server should be treate= d as a > > > > > > light > > > > > > requirement. I'd still like to be able to accept payment on a= > > > > > > connection > > > > > > that I establish to my ILSP (i.e. I am the client) and have s= ome > > > > > > third-party host my SPSP server. Assuming that the ILP receiv= er and > > > > > > SPSP > > > > > > server are the same thing is a mistake in my opinion and impo= ses > > > > > > unnecessary limitations on the architecture. > > > > > > > > > > > > Question: Why not transmit the ILP packets with the packet he= aders as > > > > > > HTTP > > > > > > headers and the data payload as the body=3F In place byte rep= lacement > > > > > > for > > > > > > packet forwarding is not going to help much if you are framin= g and > > > > > > de-framing the packets off the wire anyway. > > > > > > > > > > > > All in, there are some great ideas in the blog but I don't th= ink they > > > > > > all > > > > > > depend on changing the bilateral protocol (although I do stil= l want to > > > > > > propose some better options than BTP). > > > > > > > > > > > > I'm keen to get agreement on a basic JS interface that is agn= ostic to > > > > > > the > > > > > > underlying wire protocol and then we can ship HTTP, WebSocket= and any > > > > > > other > > > > > > implementation (I'll try and implement your HTTP protocol in = JS as > > > > > > soon as > > > > > > it's stable). > > > > > > > > > > > > Adrian > > > > > > > > > > > > > > > > > > > > > > > > On Wed, 23 Jan 2019 at 01:11, Evan Schwartz wrote: > > > > > > > I just wrote up this blog post describing my proposal for m= aking > > > > > > > connectors stateless and switching bilateral communication = to HTTP: > > > > > > > Thoughts > > > > > > > on Scaling Interledger Connectors > > > > > > > > > > > > > > : > > > > > > > > > > > > > > How I Learned to Stop Worrying and Love HTTP > > > > > > > > > > > > > > > > > > > > > Looking forward to hearing your thoughts and discussing on = tomorrow's > > > > > > > call=21 > > > > > > > Evan > > > > > > > > > > > > > > On Jan 22 2019, at 2:17 pm, Adrian Hope-Bailie > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > Hi all, > > > > > > > Evan has been working on a new HTTP-based bilateral protoco= l which > > > > > > > he'll > > > > > > > be chatting about on the call tomorrow. Look out for a foll= ow up mail > > > > > > > from > > > > > > > him with some details. > > > > > > > > > > > > > > As usual, if you have any other topics to bring to the call= please > > > > > > > do=21 > > > > > > > > > > > > > > Join from PC, Mac, Linux, iOS or Android: https://zoom.us/s= /377506344 > > > > > > > > > > > > > > > > > > > > > Or iPhone one-tap: > > > > > > > US: +14086380968,,377506344=23 or +16468769923,,377506344=23= > > > > > > > > > > > > > > Or Telephone: > > > > > > > Dial(for higher quality, dial a number based on your curren= t > > > > > > > location): > > > > > > > US: +1 408 638 0968 <+1%20408%20638%200968> or +1 646 876 9= 923 > > > > > > > <+1%20646%20876%209923> or +1 669 900 6833 <+1%20669%20900%= 206833> > > > > > > > Meeting ID: 377 506 344 > > > > > > > International numbers available: https://zoom.us/u/cbrUAzE3= W > > > > > > > > > > > > > > > > > > > > > =5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F= =5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F= =5F=5F > > > > > > > Ledger mailing list > > > > > > > Ledger=40ietf.org (mailto:Ledger=40ietf.org) > > > > > > > https://www.ietf.org/mailman/listinfo/ledger > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > --5c4f19db_174ab0c_2861 Content-Type: text/html; charset="utf-8" Content-Transfer-Encoding: quoted-printable Content-Disposition: inline
    Agreed=21

    One more thing, HTTP/3 came up on the commun= ity call and Cloudflare coincidentally released a blog post about it the follow= ing day. It's a bit of a long read but also gives a sense of how IET=46 s= tandardization works, which may be interesting for those in this communit= y as well.

    Best,
    Evan

    On Jan 23 2019, at 1:27 pm, Adrian Hope-Bail= ie <adrian=40hopebailie.com> wrote:
    An interesting project that could be undertaken if we have a solid HTTP-= based protocol would be a native module for nginx or a similar webserver = that operates as a simple connector.

    It could simply r= oute packets based on an in-memory routing table that is updated by anoth= er process (which is also the target of peer.* packets)
    =
    On Wed, 23 Jan 2019 at 16:36, Evan Schwartz <evan=40ripple.com> wrote:
    Most likely= . No matter what messaging technologies we use, attackers will figure out= how to DDoS connectors and receivers. Another reason I like the idea of = using HTTP is that it puts us in the same boat as every website on the in= ternet and would make it easier for connectors to use off-the-shelf DDoS = protection tools or services.

    On Jan 23 2019, at 9:34 am, J= eff Martinez <jmar42=40gmail.com> wrote:
    Will there be any issues with cyber attacks such as DD= oS attacks which can affect the stream=3F





    On Wed, Jan 23= , 2019, 5:42 AM Evan Schwartz <evan=40ripple.com> wrote:<= /div>
    Here are a couple of points that I meant to e= mphasize in the piece:
    • I do not think we need to (or ar= e necessarily going to be able to) get rid of BTP. I think BTP is fine fo= r the client-to-server use case. However, it seems less appropriate for t= he server-to-server use case and I think something else is needed for tha= t specific case.
    • One part I really like about the idea= of using HTTP2 instead of WebSockets is that it would allow us to decoup= le the connection management logic from the application logic. Right now,= our actual implementations need to figure out if they have an open WebSo= cket with a specific peer and route to the correct socket. If we were usi= ng HTTP with Keep-Alive or HTTP2, we'd be letting the client libraries fi= gure out for us who to keep connections open with.

    > however, http is the =C2=AB substra= te =C2=BB for everything on Internet nowadays and therefore one= benefits of all services available through cloud providers that are= added every day. Therefore, by using http and that offering, you ca= n really scale up much more easily. 

    <= div>I wholeheartedly agree=21

    > Users of the HTTP/2 connection then still need to explicitly create new = streams for each request/reply pair if they want to avoid head-of-line bl= ocking.

    I don't think that's true. Eve= n if the language supports a lower-level interface, you wouldn't program = to that directly. =46or example, in Node.js you'd probably use something = like http2-client and in Rus= t you'd use reqwest.

    &= gt; A concern I have with the cluster of statel= ess connectors is the latency you'll introduce if the connector must do a= balance check against a DB for each packet. This can be optimised to a p= oint but in reality the most efficient way to do this will be to ensure t= he stickiness at the load balancer routes packets from the same peer to t= he same connector. Not sure this will be useful in a scenario where both = peers are routing high volumes of packets (e.g. Coil and Strata).<= /font>

    This is worth digging into more. A couple of = thoughts:
    • I don't think we're anywhere near the through= put where we'd max out Redis
    • Connectors with very high= volume and trust may not track balance limits at all and may just settle= up the amount after based on the packet log (I was envisioning the balan= ce limits being optional)
    > Lastly, I don't think hosting an HTTP server should be treated as a ligh= t requirement. I'd still like to be able to accept payment on a connectio= n that I establish to my ILSP (i.e. I am the client) and have some third-= party host my SPSP server.

    • Pra= ctically speaking, which third party are you going to have hosting your S= PSP server=3F It's important to realize that anyone hosting your SPSP ser= ver could steal money from you by changing the ILP address and shared sec= ret they return
    • I'm not saying the SPSP server and rec= eiver must be the same entity. However, even in the case you des= cribe, I'd guess you'd be running that receiver on a server anyway so you= could similarly accept HTTP requests there
    • Even if so= me power users might have separate SPSP servers and STREAM receivers, I t= hink most people will end up having that infrastructure hosted by the sam= e entity
    > Question: Why n= ot transmit the ILP packets with the packet headers as HTTP headers and t= he data payload as the body=3F In place byte replacement for packet forwa= rding is not going to help much if you are framing and de-framing the pac= kets off the wire anyway.

    That was the= direction I went with ILP3. Stefan hates = that idea so I'll let you duke it out with him.

    In short, I= think you could but it would mean somewhat needlessly retransmitting the= strings =22ILP-Destination=22, =22ILP-Amount=22, etc. Because of the way= HTTP2 header compression works, you get most of the benefit when the hea= der keys and values are exactly the same each time (not true for= the expiry and condition).

    > = I'm keen to get agreement on a basic JS interface that is agnostic to the= underlying wire protocol and then we can ship HTTP, WebSocket and any ot= her implementation (I'll try and implement your HTTP protocol in JS as so= on as it's stable).

    Don't we already h= ave a JS interface (the plugin)=3F I don't see any value in having a sepa= rate format for correlating requests and responses if we are going to put= the messages on a protocol that provides that correlation. It seems more= useful for something like WebSockets or raw TLS sockets. As I said at th= e top of this email though, I would advocate against making another WebSo= cket-based protocol since we already have one.
    On Jan 23 2019, = at 8:15 am, Marc Blanchet <marc.blanchet=40= viagenie.ca> wrote:


    On 23 Jan 2= 019, at 8:08, Adrian Hope-Bailie wrote:

    Hey Eva= n,

    Thanks for sharing. Some thoughts on your blog post, we = can discuss
    further
    on the call:

    Many o= f the issues you highlight with BTP are not issues with
    WebSock= ets
    themselves but more with how we have used it. E.g. We shoul= d be
    hosting
    WebSocket servers that listen on a singl= e port and path and then use a
    proper auth during the handshake= . If you do that, the subsequent
    messages
    sent over t= he underlying TCP connection are very light on framing and
    meta= -data.

    =46rom my research WebSockets are one of the most ef= ficient and well
    standardised framing protocols available over = a raw TCP socket. Binary
    WebSocket frames have very little over= head and all of the session
    context
    is dealt with up = front in the handshake. If you compare this to HTTP
    (and
    <= div>HTTP2) you have a lot of overhead in the headers (i.e. sent with ever= y
    message).

    Request/reply matching is a key featu= re requirement of bilateral
    protocols
    in ILP but I th= ink we've overblown the complexity required in
    implementations = when you consider that an ILP request/reply pair
    should be
    very short lived and that by simply using TCP you already have many= of
    the
    delivery guarantees you need.

    H= TTP is only capable of request/reply matching because it only allows
    a
    request to be sent after a previous reply has been rece= ived on the
    same
    connection. i.e. You can only send r= equests serially. Any ILP
    connections
    using HTTP 1 or= 1.1 will be very inefficient unless they open multiple
    connect= ions.

    HTTP/2 gets over this by mutiplexing the connection t= hrough streams.
    But
    under the hood this is just anoth= er framing protocol (like WebSockets)
    and
    giving thos= e frames a stream id. Users of the HTTP/2 connection then
    still=
    need to explicitly create new streams for each request/reply p= air if
    they
    want to avoid head-of-line blocking. It's= likely that the way this is
    implemented in clients is likely t= o be highly optimised given the
    popularity of HTTP/2, but given= that streams have a complex state
    machine
    that the c= lient must manage it seems unlikely that it can be optimised
    fu= rther than a simple request/reply protocol using a correlation id.
    <= div>(i.e.
    simply adding a correlation id to the header of a pac= ket and sending
    in a
    binary WebSocket frame is defini= tely more efficient in theory but in
    practice will come down to= implementations).


    however, http is the =C2= =AB substrate =C2=BB for everything on Internet
    nowad= ays and therefore one benefits of all services available through
    cloud providers that are added every day. Therefore, by using http and<= /div>
    that offering, you can really scale up much more easily.
    <= br>
    my 2 cents,

    Marc.


    The= one aspect of WebSockets that may be hurting performance is the
    masking
    which means that payloads need to be pre-processed be= fore being sent.
    That
    said, it's a very efficient alg= orithm so we'd need to test it to be
    sure
    it's an iss= ue.

    A concern I have with the cluster of stateless connecto= rs is the
    latency
    you'll introduce if the connector m= ust do a balance check against a DB
    for
    each packet. = This can be optimised to a point but in reality the most
    effici= ent way to do this will be to ensure the stickiness at the load
    balancer routes packets from the same peer to the same connector. Not
    sure
    this will be useful in a scenario where both peers= are routing high
    volumes
    of packets (e.g. Coil and S= trata).

    That said, I think decoupling the messaging from th= e clearing and
    settlement (=46YI: forwarding ILP packets isn't = clearing, clearing
    involves
    reconciliation and nettin= g, but that's a different issue) will be a
    huge
    impro= vement to the architecture. As I've mentioned before, this is the
    common pattern in most payments. We've been discussing how we could
    implement this in the reference connector and hope to do some
    experiments
    in this regard when we are back from the Moj= aloop convening next week.

    Lastly, I don't think hosting an= HTTP server should be treated as a
    light
    requirement= . I'd still like to be able to accept payment on a
    connection
    that I establish to my ILSP (i.e. I am the client) and have some=
    third-party host my SPSP server. Assuming that the ILP receive= r and
    SPSP
    server are the same thing is a mistake in = my opinion and imposes
    unnecessary limitations on the architect= ure.

    Question: Why not transmit the ILP packets with the pa= cket headers as
    HTTP
    headers and the data payload as = the body=3F In place byte replacement
    for
    packet forw= arding is not going to help much if you are framing and
    de-fram= ing the packets off the wire anyway.

    All in, there are some= great ideas in the blog but I don't think they
    all
    d= epend on changing the bilateral protocol (although I do still want to
    propose some better options than BTP).

    I'm keen to g= et agreement on a basic JS interface that is agnostic to
    the
    underlying wire protocol and then we can ship HTTP, WebSocket and= any
    other
    implementation (I'll try and implement you= r HTTP protocol in JS as
    soon as
    it's stable).
    <= br>
    Adrian




    On Wed, 23 Jan 2019 at 01:11, Ev= an Schwartz <evan=40ripple.com> wrote:

    I just wrote up this blog post describing my proposal for making=
    connectors stateless and switching bilateral communication to = HTTP:
    Thoughts
    on Scaling Interledger Connectors
    :
    How= I Learned to Stop Worrying and Love HTTP
    <https://medium.com/=40emschw= artz/thoughts-on-scaling-interledger-connectors-7e3cad0dab7f>
    Looking forward to hearing your thoughts and discussing on tomo= rrow's
    call=21
    Evan

    On Jan 22 2019, at = 2:17 pm, Adrian Hope-Bailie
    wrote:

    Hi all,

    = Evan has been working on a new HTTP-based bilateral protocol which
    <= div>he'll
    be chatting about on the call tomorrow. Look out for = a follow up mail
    from
    him with some details.
    As usual, if you have any other topics to bring to the call please<= /div>
    do=21

    Join from PC, Mac, Linux, iOS or Android: <= a href=3D=22https://zoom.us/s/377506344=22 title=3D=22https://zoom.us/s/3= 77506344=22>https://zoom.us/s/377506344

    Or iPhone one-tap:
    US: +14086380968,,377506344=23 or +164687= 69923,,377506344=23

    Or Telephone:
    Dial(for higher= quality, dial a number based on your current
    location):
    <= div>US: +1 408 638 0968 <+1%20408%20638%200968> or +1 646 876 9923<= /div>
    <+1%20646%20876%209923> or +1 669 900 6833 <+1%20669%2= 0900%206833>
    Meeting ID: 377 506 344
    International= numbers available: https://zoom.us/u/cbrUAzE3W
    = --5c4f19db_174ab0c_2861--