Subject: [apps-discuss] Unified User-Agent String (draft-karcz-uuas)

From: Mateusz Karcz <[email-masked]>
Date: Tue, 11 Nov 2014 02:28:20 +0100

Welcome.

After two years of thinking and two nights of working I've finally written the second revision of my I-D. I changed construction of the draft, simplified proposed UUAS structure, made a research about security and added some other, minor changes. I've committed it as draft-karcz-uuas-01.

I would like to see Your comments about it. Could You check out this draft and clarify me a way of developing it?

Sincerely,
Mateusz Karcz

---

From: Karl Dubost <[email-masked]>
Date: Tue, 11 Nov 2014 11:11:18 +0900

Mateusz,
Interesting. What was the initial motivation for the draft?

Below some comments.

> After two years of thinking and two nights of working I've finally written the second revision of my I-D. I changed construction of the draft, simplified proposed UUAS structure, made a research ab…

And the full link 
https://tools.ietf.org/html/draft-karcz-uuas-01

In https://tools.ietf.org/html/draft-karcz-uuas-01#section-2
2.  Use of the User-Agent strings

You are writing,

[…]
   However, in modern browsers these limitations greatly decreased and
   "user agent spoofing" is now unnecessary.  Unfortunately, there are a
   lot of websites still discriminating particular web browsers.
[…]

I really wish it was true, given it is 80% of what I do all day long ;) aka discovering, analyzing, and contacting Web sites with a kind of UA sniffing (be client side or server side). There are a load of issues which I'm not happy about but are still a reality. User Agent sniffing is a future fail strategy, but is widely used not only because of the disparity in between browsers.

Here are some of the use cases that UA sniffing has been used for. We collected these during a survey this year from Web developers, Bizdev people and UA databases companies.
http://www.otsukare.info/2014/03/31/ua-detection-use-cases

There are other headers which have been popping up 
http://mobiforge.com/research-analysis/x-device-user-agent-header-appearing-requests

Also have you seen the draft 
https://tools.ietf.org/html/draft-brown-device-stock-ua-00
The background behind this draft.
https://dev.opera.com/blog/introducing-device-stock-ua/

Another source you might want to read
http://www.nczonline.net/blog/2010/01/12/history-of-the-user-agent-string/

-- 
Karl Dubost 🐄
http://www.la-grange.net/karl/

---

From: Mateusz Karcz <[email-masked]>
Date: Tue, 11 Nov 2014 03:31:57 +0100

My intention was to propose a way of simplification, clarification and standardization of complex User-Agent string syntax.

I've faced these publications before (except the survey). The problem is: These headers are still experimental and work-in-progress. However, by their definition they would just "inherit" characteristics of UUAS.

A word about UA sniffing, yeah, I should precise, what do I mean by "unnecessary". I mean, by definition my "web browser UUAS" would be applied to new and standards-compliant web browsers.

Użytkownik "Karl Dubost" <[email-masked]> napisał(a): 
> Temat: Re: [apps-discuss] Unified User-Agent String (draft-karcz-uuas)
> Data: 2014-11-11 3:11
> Nadawca: "Karl Dubost" <[email-masked]>
> Adresat: "Mateusz Karcz" <[email-masked]>; 
> 
> Mateusz,
> Le 11 nov. 2014 à 10:28, Mateusz Karcz  a écrit :

---

From: Karl Dubost <[email-masked]>
Date: Tue, 11 Nov 2014 12:25:33 +0900

Mateusz,

Le 11 nov. 2014 à 11:31, Mateusz Karcz <[email-masked]> a écrit :
> I mean, by definition my "web browser UUAS" would be applied to new and standards-compliant web browsers.

This is part of the issue at stake and how your effort could be successful or fail.

Mozilla has taken a step forward by simplifying Firefox OS and Firefox Android user agent strings. [1] 

Firefox OS: "Mozilla/5.0 (Mobile; rv:30.0) Gecko/30.0 Firefox/30.0"
Firefox Android: "Mozilla/5.0 (Android; Mobile; rv:30.0) Gecko/30.0 Firefox/30.0"

It creates Web Compatibility issues. And it's what the Web Compatibility activity [2][3] is trying to solve. Basically the issue is not about having a cleaner UA, but more about the Web itself. We can't reboot the Web, there is a huge legacy of sites which stop working when the UA is not shaped like scripts are expecting them. 

What am I saying is that you will be able to simplify the UA by making the UA useless and adding features to the Web platform that will answer some of the needs of Web developers. It's really unfortunate but not that much solutions apart of that. These 3 blogs will give you the extent of the issues. 

https://miketaylr.com/posts
http://whatcouldbewrong.com/
http://www.otsukare.info/

Another issue is related to the capabilities of the device itself. When the browser version is the same but the device is different. We still lack a good and simple way to advertise some of the capabilities. We became better with feature detections, it helped a lot to remove some burden. But still not enough. In the Mobile world, the practices of phone Operators is also a big issue: Device X == Promotion/Benefits X == UA Sniffing

[1]: https://developer.mozilla.org/en-US/docs/Web/HTTP/Gecko_user_agent_string_reference#Firefox_OS
[2]: https://webcompat.com/
[3]: https://wiki.mozilla.org/Compatibility/

---

From: Mateusz Karcz <[email-masked]>
Date: Tue, 11 Nov 2014 05:21:32 +0100

One of main goal on which I was working was to maintain existing solutions. Believe me or not, but UAs of Firefox, IE11, Chrome, Safari, Opera and Konqeror are 100% valid web browser UUAS. Firefox OS and Firefox Mobile are now 100% valid regular UUAS, but it can be made "web browser UUAS just by changing 1 rule in ABNF definition. I thought a lot before designing this draft and compared a lot of recent UAs. And one another thing: This memo is not intended to make existing software useless. It is intended to make an advice for present and future developers, how to generate good UAs preserving behavior of legacy websites.

A word about devices: if Device-UA becomes a standard, it will behave as today. And that will also be UUAS-valid. Why? There is a point in backwards compatibility, so I preserved it. "Standard string" is almost copy-paste from httpbis. "Regular string" just adds obligatory platform signature. "Web browser string" is designed in conformance with W3C Recomendation on client software identification. 

There will be no need to re-create the Web!

Użytkownik "Karl Dubost" <[email-masked]> napisał(a): 
> Data: 2014-11-11 4:25
> Le 11 nov. 2014 à 11:31, Mateusz Karcz  a écrit :

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Tue, 11 Nov 2014 11:18:00 -1000

On Mon, Nov 10, 2014 at 6:21 PM, Mateusz Karcz <[email-masked]>
wrote:

> One of main goal on which I was working was to maintain existing
> solutions. Believe me or not, but UAs of Firefox, IE11, Chrome, Safari,
> Opera and Konqeror are 100% valid web browser UUAS. Firefox OS and Firefox
> Mobile are now 100% valid regular UUAS, but it can be made "web browser
> UUAS just by changing 1 rule in ABNF definition. I thought a lot before
> designing this draft and compared a lot of recent UAs. And one another
> thing: This memo is not intended to make existing software useless. It is
> intended to make an advice for present and future developers, how to
> generate good UAs preserving behavior of legacy websites.
> [...]
>

It's been suggested that you might find support in HTTPBIS since it is more
related to work they're doing.  Have you proposed your draft to that
working group?

-MSK

---

From: Mateusz Karcz <[email-masked]>
Date: Tue, 11 Nov 2014 22:51:04 +0100

No, not yet. While I was submitting my I-D two years ago RFC Editor (I don't really remember who exactly) claimed, that I should look for help in AppsAWG or W3C, so I came here. But now I think it would be a good idea to write on HTTPBis.

Użytkownik "Murray S. Kucherawy" <[email-masked]> napisał(a): 
> Data: 2014-11-11 22:18
> Nadawca: "Murray S. Kucherawy" <[email-masked]>
> On Mon, Nov 10, 2014 at 6:21 PM, Mateusz Karcz <[email-masked]> wrote:One of main goal on which I was working was to maintain existing solutions. Believe me or not, but UAs of Firefox, IE11, Chrome, Safari, Opera and Konqeror are 100% valid web browser UUAS. Firefox OS and Firefox Mobile are now 100% valid regular UUAS, but it can be made "web browser UUAS just by changing 1 rule in ABNF definition. I thought a lot before designing this draft and compared a lot of recent UAs. And one another thing: This memo is not intended to make existing software useless. It is intended to make an advice for present and future developers, how to generate good UAs preserving behavior of legacy websites.
> [...]It's been suggested that you might find support in HTTPBIS since it is more related to work they're doing.  Have you proposed your draft to that working group?-MSK
>

---

From: Mateusz Karcz <[email-masked]>
Date: Wed, 12 Nov 2014 18:08:52 +0100

I've checked out and (please correct me if I'm wrong) and if I want to write anything to HTTPBis, I have to join W3C, what is not very simple for me. However, I truly understand that proposing my draft there would be more appropriate.
-M.Karcz

Użytkownik "Murray S. Kucherawy" <[email-masked]> napisał(a): 
> It's been suggested that you might find support in HTTPBIS since it is more related to work they're doing.  Have you proposed your draft to that working group?-MSK

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Wed, 12 Nov 2014 07:48:45 -1000

I can't imagine that's correct.  Mark Nottingham, care to comment?

Simply posting to [email-masked] should be all you need to do.

-MSK

On Wed, Nov 12, 2014 at 7:08 AM, Mateusz Karcz <[email-masked]>
wrote:

> I've checked out and (please correct me if I'm wrong) and if I want to
> write anything to HTTPBis, I have to join W3C, what is not very simple for
> me. However, I truly understand that proposing my draft there would be more
> appropriate.
> -M.Karcz
>
> > It's been suggested that you might find support in HTTPBIS since it is
> more related to work they're doing.  Have you proposed your draft to that
> working group?-MSK
>

---

From: Julian Reschke <[email-masked]>
Date: Wed, 12 Nov 2014 19:19:09 +0100

> I've checked out and (please correct me if I'm wrong) and if I want to write anything to HTTPBis, I have to join W3C, what is not very simple for me. However, I truly understand that proposing my d…

HTTPbis is an IETF Working Group. The reason for your confusion might be 
that we are using a W3C mailing list (mainly for historical reasons).

Best regards, Julian

---

From: Barry Leiba <[email-masked]>
Date: Wed, 12 Nov 2014 13:10:17 -1000

> Simply posting to [email-masked] should be all you need to do.

That's not the address for the httpbis working group.  It's
<[email-masked]> -- as Julian said, on the W3C servers, for
historical reasons.  But there's no W3C membership required, of
course.

Barry