Subject: [Storagesync] CORS headers on storage sync APIs - a requirement?

From: Michiel de Jong <[email-masked]>
Date: Mon, 9 Nov 2015 12:06:10 +0100

Hi!

I wanted to draw your attention to the situation where a web app, rather
than an OS-native app, wants to sync data with a user's storage.

In this case, it's necessary that the API of the storage server be served
with CORS headers. [1] The following storage sync APIs already do this:

* Dropbox
* Google Drive (including YouTube)
* Kinto (a Mozilla project) [2]
* remoteStorage (an IETF I-D of which I'm a contributor) [3]
* Hoodie http://hood.ie/

I work on data sync for Firefox OS [4] - a mobile OS where every
application is a client-side web app, so CORS headers are important there.

[1] http://enable-cors.org/
[2] http://kinto.readthedocs.org/en/latest/
[3] https://remotestorage.io/
[4] https://wiki.mozilla.org/Firefox_OS_Data_Sync

Cheers,
Michiel.

---

From: Linhui Sun <[email-masked]>
Date: Mon, 9 Nov 2015 20:37:40 +0800

> application is a client-side web app, so CORS headers are important there.
I think we should also consider the sync among a web app, a native app and
user's storage since we cannot force users to use a web app or a native
app. However this should be discussed in the scope of the ISS work.

> _______________________________________________
> Storagesync mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/storagesync
>
> Regards,
Linhui

---

From: Michiel de Jong <[email-masked]>
Date: Mon, 9 Nov 2015 14:02:38 +0100

On Mon, Nov 9, 2015 at 1:37 PM, Linhui Sun <[email-masked]> wrote:

> user's storage since we cannot force users to use a web app or a native app.
>

ok, so then CORS headers are a must (otherwise you can only use OS-native
apps).

> However this should be discussed in the scope of the ISS work.
>
>>

---

From: Linhui Sun <[email-masked]>
Date: Tue, 10 Nov 2015 18:54:18 +0800

2015-11-09 21:02 GMT+08:00 Michiel de Jong <[email-masked]>:

>>> In this case, it's necessary that the API of the storage server be
>>> served with CORS headers. [1] The following storage sync APIs already do
>>> this:
>>>
>>> * Dropbox
>> I think we should also consider the sync among a web app, a native app
>> and user's storage since we cannot force users to use a web app or a native
>> app.
>>
>
Web sync is important when we have no access to our own laptops or
computers. It may also play an important role in the mobile devices. But it
is a frequently happened scenario that other devices or subscribers uses a
native app. How would this (CORS header) work for native apps or a hybrid
combination of native and web apps? IMO, the native apps also need the CORS
header. Or not?

Regards,
Linhui

---

From: Michiel de Jong <[email-masked]>
Date: Tue, 10 Nov 2015 12:55:15 +0100

No, for web apps CORS headers are necessary, but for native apps it doesn't
make any difference if they are present or not.

Cheers,
Michiel.

On Tue, Nov 10, 2015 at 11:54 AM, Linhui Sun <[email-masked]> wrote:

>>>> I wanted to draw your attention to the situation where a web app,
>>>> rather than an OS-native app, wants to sync data with a user's storage.
>>>>

---

From: Linhui Sun <[email-masked]>
Date: Tue, 10 Nov 2015 20:11:31 +0800

2015-11-10 19:55 GMT+08:00 Michiel de Jong <[email-masked]>:

> No, for web apps CORS headers are necessary, but for native apps it
> doesn't make any difference if they are present or not.
>
I think I must miss something about the CORS, could you explain it a little
more in detail why the CORS does not make any difference in the native
apps. IMO, the native apps are still using HTTP connection and may also
encounter a cross-domain access problem.

Thanks,
Linhui

>>> ok, so then CORS headers are a must (otherwise you can only use
>>> OS-native apps).
>>>

---

From: Michiel de Jong <[email-masked]>
Date: Tue, 10 Nov 2015 13:29:14 +0100

OK! "In the beginning" ;) there was HTTP. Then XMLHttpRequest was invented
(also known as XHR or AJAX, later superseded by XHR2, and by the fetch API
in the latest browser versions). Since XHR allows any website to make any
HTTP request from the JavaScript that runs in your browser, the same-origin
policy was applied to it (a similar policy already existed for Cookies and
other things, and the same thing was also done for the HTTP client that was
added to Flash at the time).

This created a situation where a HTTP client is present in the browser's
sandbox, for use by JavaScript code that runs in the web page, but it is
restricted to the same origin as the website this JavaScript is part of (or
is included from).

Web developers got more creative, and invented JSONP as a work-around, to
effectively do HTTP requests to other origins.

JSONP was a dirty hack and had its limitations, but since it was recognized
that its use cases were in themselves valid (sometimes depending on which
resource is being fetched you want to have a HTTP client running inside
JavaScript on a web page, without the same-origin restriction), and that's
why CORS was invented.

Instead of removing the overly simplistic and restrictive same-origin
policy for XHR, and making it more flexible, CORS headers are added by the
cross-origin resource, to overwrite the strict same-origin policies which
are still the default behavior of HTTP clients that run inside browsers.
It's a case of Deny-Then-Allow (where Deny is the default blanket same
origin policy, and Allow are the explicit exceptions defined through CORS).

HTTP clients that run outside browsers (in native apps, or on the command
line, for instance), never had same-origin policies in the first place! In
fact they don't even have a well-defined origin. So that's why they just
ignore the CORS headers on API responses. CORS headers remove a limitation
which these other HTTP clients never had.

HTH!

Cheers,
Michiel.

On Tue, Nov 10, 2015 at 1:11 PM, Linhui Sun <[email-masked]> wrote:

> I think I must miss something about the CORS, could you explain it a
> little more in detail why the CORS does not make any difference in the
> native apps. IMO, the native apps are still using HTTP connection and may
> also encounter a cross-domain access problem.
>
> Thanks,
> Linhui
>
>>
>>
>> Cheers,
>> Michiel.
>>

---

From: Linhui Sun <[email-masked]>
Date: Tue, 10 Nov 2015 21:29:54 +0800

Cool and got it! Many thanks for the detailed explanation! Thus we will
have no problem when users use a combination of web apps and native apps
(with the help of CORS).

In addition, I had a look at the wiki of Firefox OS Data Sync and got few
comments. In the "The product" section, you mentioned that you want to
achieve a "user choice". Does that mean you want to implement a general
third-party app that can connect to various storage services? If so, this
is only part of the so called "user choice". IMO, the "user choice" should
also include the concept of "interoperability" (sync among different
storage services). Since a further result of "user choice" is to
synchronize files among different services. Do you think this is also
needed?

And in section 2.2.4 (i.e. "File sharing"), have you ever considered the
scenario that Alice could also modify and update the file shared by Bob?

Thanks,
Linhui

2015-11-10 20:29 GMT+08:00 Michiel de Jong <[email-masked]>:

> JSONP was a dirty hack and had its limitations, but since it was
> recognized that its use cases were in themselves valid (sometimes depending
> on which resource is being fetched you want to have a HTTP client running
> inside JavaScript on a web page, without the same-origin restriction), and
> that's why CORS was invented.
>
>>>>>> I think we should also consider the sync among a web app, a native
>>>>>> app and user's storage since we cannot force users to use a web app or a
>>>>>> native app.
>>>>>>
>>>>>

---

From: Michiel de Jong <[email-masked]>
Date: Tue, 10 Nov 2015 18:22:39 +0100

> synchronize files among different services. Do you think this is also

Do you mean server-to-server? I think client-to-(any)-server would already
be a big win. :) The current landscape for many internet users is that
they're not really aware where their data is synced to, but it mainly
depends on the device they buy: a Windows Phone will sync to Microsoft, an
Android phone will sync to Google, and an iPhone will sync to Apple. The
user can choose their storage provider only by buying a new device. That's
like buying a new car just so you can drive on a different road. :)

> scenario that Alice could also modify and update the file shared by Bob?

Yes, but I must say it's a pretty complicated thing to get working if Alice
does not have her own account at Bob's storage provider. So if Bob uses
Dropbox and Alice has a Dropbox account, or Bob uses Google Drive and Alice
has a Google account, then this will work. I agree with you that this is a
point where the way the internet currently works for Bob and Alice can be
improved. I'm not aware of anybody within Mozilla working on that
currently, if that was your question. But there's been some work on this
between Cozy cloud and myself earlier this year, see
https://tools.ietf.org/html/draft-dejong-decentralized-sharing.

>> OK! "In the beginning" ;) there was HTTP. Then XMLHttpRequest was
>> invented (also known as XHR or AJAX, later superseded by XHR2, and by the
>> fetch API in the latest browser versions). Since XHR allows any website to
>> make any HTTP request from the JavaScript that runs in your browser, the
>> same-origin policy was applied to it (a similar policy already existed for
>> Cookies and other things, and the same thing was also done for the HTTP
>> client that was added to Flash at the time).
>>

---

From: Linhui Sun <[email-masked]>
Date: Wed, 11 Nov 2015 11:03:41 +0800

> like buying a new car just so you can drive on a different road. :)
I'm not saying the server-to-server sync, since it may be involved in some
authentication and security issues among different services. What we are
trying to do in this ISS work is to achieve the interoperability through
standardizing the sync protocol. And currently, if I got a iPhone, I could
still use the GoogleDrive through installing a GoogleDrive app. I think a
more appropriate analogy is that we can drive our cars on different roads
but we have to pay for them. If we have the "interoperability", our cars
could drive on different roads without any restriction. And I think that is
what the client-to-any-server should be : )

> Yes, but I must say it's a pretty complicated thing to get working if
> Alice does not have her own account at Bob's storage provider. So if Bob
> uses Dropbox and Alice has a Dropbox account, or Bob uses Google Drive and
> Alice has a Google account, then this will work. I agree with you that this
> is a point where the way the internet currently works for Bob and Alice can
> be improved. I'm not aware of anybody within Mozilla working on that
 The current situation is caused by various proprietary sync protocols and
that is what "interoperability" should also resolve (though it is hard).
IMO, the Internet-based storage services should not be blocked by different
vendors. We have discussed this in the BOF, we got some positive feedbacks
but it is also upset that some vendors express that they have no interest.
Anyway, we think this is still worth doing. I think the draft you mentioned
here is a good start, the use case described in the Introduction is what we
desire. I will read this draft and maybe you could post it to this ML to
request for more reviews : )

Regards,
Linhui