Subject: [Slim] Issue 43: How to know the modality of a language indication?

From: Bernard Aboba <[email-masked]>
Date: Fri, 13 Oct 2017 13:46:44 -0700

Issue 43 ( https://trac.ietf.org/trac/slim/ticket/43 ) results from a
review comment that said that a simple way is required to decide if a
language tag is a sign language or a written or spoken language.

Some applications scan the IANA language registry at startup for the word
"Sign" in the tag description:
https://www.iana.org/assignments/language-subtag-registry/language-subtag-registry

Currently, there are 319 language subtags that include "Sign Language" in
their description.

Given the current layout of the language subtag registry, it is not clear
to me that there is an easier way to determine which tags represent sign
languages.  Nor is it within the SLIM WG charter to develop a modification
to the language subtag registry to address this concern.

So I am wondering whether we might resolve this with a Note outlining the
problem but not offering a solution.

---

From: Brian Rosen <[email-masked]>
Date: Fri, 13 Oct 2017 16:58:02 -0400

Sounds good to me.

Brian

> On Oct 13, 2017, at 4:46 PM, Bernard Aboba <[email-masked]> wrote:
> 
> Issue 43 ( https://trac.ietf.org/trac/slim/ticket/43 <https://trac.ietf.org/trac/slim/ticket/43> ) results from a review comment that said that a simple way is required to decide if a language tag is a sign language or a written or spoken language.
> 
> Some applications scan the IANA language registry at startup for the word "Sign" in the tag description:
> https://www.iana.org/assignments/language-subtag-registry/language-subtag-registry <https://www.iana.org/assignments/language-subtag-registry/language-subtag-registry>
> 
> Currently, there are 319 language subtags that include "Sign Language" in their description. 
> 
> Given the current layout of the language subtag registry, it is not clear to me that there is an easier way to determine which tags represent sign languages.  Nor is it within the SLIM WG charter to develop a modification to the language subtag registry to address this concern. 
> 
> So I am wondering whether we might resolve this with a Note outlining the problem but not offering a solution. 
> 
> _______________________________________________
> SLIM mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/slim

---

From: Randall Gellens <[email-masked]>
Date: Fri, 13 Oct 2017 19:25:59 -0700

At 1:46 PM -0700 10/13/17, Bernard Aboba wrote:

>  Issue 43 ( 
> <https://trac.ietf.org/trac/slim/ticket/43>https://trac.ietf.org/trac/slim/ticket/43 
> ) results from a review comment that said that a simple way is 
> required to decide if a language tag is a sign language or a 
> written or spoken language.
>
> <https://www.iana.org/assignments/language-subtag-registry/language-subtag-registry>https://www.iana.org/assignments/language-subtag-registry/language-subtag-registry
>
>
>  Given the current layout of the language subtag registry, it is not 
> clear to me that there is an easier way to determine which tags 
> represent sign languages.  Nor is it within the SLIM WG charter to 
> develop a modification to the language subtag registry to address 
> this concern. 
>
>  So I am wondering whether we might resolve this with a Note 
> outlining the problem but not offering a solution. 

I think the wording in -14 addresses the comment by accepting Dale's 
suggestion that, rather than know non-signed tags, it's the use of 
the exact same tag in both an audio and a video stream that is the 
indicator.  That both tightens up the technical issue and simplifies 
it greatly.

The only other instance where we might add such a note would be in 5.4:

5.4.  Undefined Combinations

    With the exception of the case mentioned in Section 5.2 (an audio
    stream in parallel with a video stream with the exact same (spoken)
    language tag), the behavior when specifying a non-signed language tag
    for a video media stream, or a signed language tag for an audio or
    text media stream, is not defined.

We could add your suggested note to 5.4.

-- 
Randall Gellens
Opinions are personal;    facts are suspect;    I speak for myself only
-------------- Randomly selected tag: ---------------
Leisure for men of business, and business for men of leisure would
cure many complaints.                         --Esther L.S. Thrale

---

From: Gunnar Hellström <[email-masked]>
Date: Sat, 14 Oct 2017 10:21:35 +0200

Den 2017-10-14 kl. 04:25, skrev Randall Gellens:
>> required to decide if a language tag is a sign language or a written 
>> or spoken language.
>>
> suggestion that, rather than know non-signed tags, it's the use of the 
> exact same tag in both an audio and a video stream that is the 
<GH>We can replace 5.4 with a more explicit section guiding applications 
to how to make the deduction simple. So, instead of a note, I suggest 
that we replace 5.4 with:

5.4 Relations between media and modality
There is no easy way to deduct the intended modality from a language 
tag. Other specifications may introduce specific notations for modality 
used in a media or in relation to a language tag. Applications not 
implementing such specific notations may use the following simple 
deductions.
- A language tag in audio media is supposed to indicate spoken modality.
- A language tag in text media is supposed to indicate  written modality.
- A language tag in video media is supposed to indicate visual sign 
language modality except for the case when it is supposed to indicate a 
view of a speaking person mentioned in section 5.2 characterized by the 
exact same language tag also appearing in an audio media specification.
- A language tag in media where the modality is obvious or specified for 
the media subtype definition is supposed to indicate that modality.
- A language tag in other media descriptions than above has undefined 
modality.
---------------------------------------------------------------------------------------------------------------------

My note:  by this we currently consciously exclude the following use and 
I am ok with that:
-text in mp4 video
-audio in mp4 video ( or is that only allowed in application/mp4 ??)
-any modality in message media
-most application media, however some may have explicit descriptions in 
subtype specifications.

The exception with a view of a speaker stands out as very odd now, 
requiring comparison of language tags used in different media 
descriptions, and requiring simultaneous use of language in two 
different media that is otherwise out of scope for this draft. It was 
introduced while I still hoped that we could introduce other 
dependencies between language use in different media.  It is not the 
most urgent media/language combination to specify. It is also handled in 
draft-hellstrom-slim-modality-grouping. So, assuming that we can get 
progress on that draft, we could clean up the current draft by deleting 
the exception. I suggest that we delete the exception.

/Gunnar

-- 
-----------------------------------------
Gunnar Hellström
Omnitor
[email-masked]
+46 708 204 288

---

From: Randall Gellens <[email-masked]>
Date: Sat, 14 Oct 2017 04:56:02 -0700

At 10:21 AM +0200 10/14/17, Gunnar Hellström wrote:

>>> ) results from a review comment that said 
>>> that a simple way is required to decide if a 
>>>   Some applications scan the IANA language 
>>> registry at startup for the word "Sign" in 
>>> the tag description:
>>>
>>> 
>>>   Currently, there are 319 language subtags 
>>> that include "Sign Language" in their 
>>> description.
>>>   Given the current layout of the language 
>>> subtag registry, it is not clear to me that 
>>> there is an easier way to determine which 
>>> tags represent sign languages.  Nor is it 
>>> within the SLIM WG charter to develop a 
>>> modification to the language subtag registry 
>>> to address this concern.
>>>   So I am wondering whether we might resolve 
>>> this with a Note outlining the problem but 
>>> not offering a solution.
>>
>>  I think the wording in -14 addresses the 
>> comment by accepting Dale's suggestion that, 
>> rather than know non-signed tags, it's the use 
>> of the exact same tag in both an audio and a 
>> video stream that is the indicator.  That both 
>> tightens up the technical issue and simplifies 
>> it greatly.
>>
>  <GH>We can replace 5.4 with a more explicit 
> section guiding applications to how to make the 
> deduction simple. So, instead of a note, I 
> suggest that we replace 5.4 with:
>
>  There is no easy way to deduct the intended 
> modality from a language tag. Other 
> specifications may introduce specific notations 
> for modality used in a media or in relation to 
> a language tag. Applications not implementing 
> such specific notations may use the following 
> simple deductions.
>  - A language tag in video media is supposed to 
> indicate visual sign language modality except 
> for the case when it is supposed to indicate a 
> view of a speaking person mentioned in section 
> 5.2 characterized by the exact same language 
> tag also appearing in an audio media 
> specification.
>  - A language tag in media where the modality is 
> obvious or specified for the media subtype 
> definition is supposed to indicate that 
> modality.
>  - A language tag in other media descriptions 
> than above has undefined modality.
> 

The suggested text makes a very different point than what's there now:

    the behavior when specifying a non-signed language tag
    for a video media stream, or a signed language tag for an audio or
    text media stream, is not defined in this document.

    The problem of knowing which language tags are signed and which are
    not is out of scope of this document.

An implementation could, for example, have a 
table (even a static table that's fixed) of 
language tags it knows to be for signed 
languages.  It could treat other tags as for 
non-signed languages.  This would be an imperfect 
approach (yielding incorrect results if a new 
signed language tag is introduced, e.g.), but 
might be good enough.

>  My note:  by this we currently consciously 
> exclude the following use and I am ok with that:
>  -text in mp4 video
>  -most application media, however some may have 
> explicit descriptions in subtype specifications.
>
>  The exception with a view of a speaker stands 
> out as very odd now, requiring comparison of 
> language tags used in different media 
> descriptions, and requiring simultaneous use of 
> language in two different media that is 
> otherwise out of scope for this draft. It was 
> introduced while I still hoped that we could 
> introduce other dependencies between language 
> use in different media.  It is not the most 
> urgent media/language combination to specify. 
> It is also handled in 
> draft-hellstrom-slim-modality-grouping. So, 
> assuming that we can get progress on that 
> draft, we could clean up the current draft by 
> deleting the exception. I suggest that we 
> delete the exception.

I like the suggestion to delete the exception.

The exception is in 5.2:

    Note that while signed language tags are used with a video stream to
    indicate sign language, a video stream in parallel with an audio
    stream, both using the exact same (spoken) language tag, indicates a
    request for a supplemental video stream to see the speaker.

I agree that deleting this exception makes for a 
simpler and less complex draft.

-------------- Randomly selected tag: ---------------
    The highlight of the annual Computer Bowl occurred when Bill Gates,
who was a judge, posed the following question to the contestants:
    "What contest, held via Usenet, is dedicated to examples of weird,
obscure, bizarre, and really bad programming?"
    After a moment of silence, Jean-Louis Gassee (ex-honcho at Apple)
hit his buzzer and answered "Windows."
                                          --Recounted by Adam C. Engst

---

From: Bernard Aboba <[email-masked]>
Date: Sat, 14 Oct 2017 11:03:31 -0700

Gunnar said:

"Applications not implementing such specific notations may use the
following simple deductions.

[BA] Even a tag with "Sign Language" in the description??

[BA] If the tag has "Sign Language" in the description, can this document
really say that?

[BA] It seems like an over-reach to say that a spoken language tag in video
media should instead be interpreted as a request for Sign Language.  If
this were done, would it always be clear which Sign Language was intended?
And could we really assume that both sides, if negotiating a spoken
language tag in video media, were really indicating the desire to sign?  It
seems like this could easily result interoperability failure.

modality."

On Sat, Oct 14, 2017 at 1:21 AM, Gunnar Hellström <
[email-masked]> wrote:

>>  Issue 43 ( <https://trac.ietf.org/trac/slim/ticket/43>https://trac.ietf
>>> .org/trac/slim/ticket/43 ) results from a review comment that said that
>>> a simple way is required to decide if a language tag is a sign language or
>>> a written or spoken language.
>>>
>>> <https://www.iana.org/assignments/language-subtag-registry/
>>> language-subtag-registry>https://www.iana.org/assignments/
>>> language-subtag-registry/language-subtag-registry
>>>
>>>
>>> clear to me that there is an easier way to determine which tags represent
>>> sign languages.  Nor is it within the SLIM WG charter to develop a
>>> modification to the language subtag registry to address this concern.
>> exact same tag in both an audio and a video stream that is the indicator.
>> That both tightens up the technical issue and simplifies it greatly.
>>
> to how to make the deduction simple. So, instead of a note, I suggest that
> we replace 5.4 with:
>
> There is no easy way to deduct the intended modality from a language tag.
> Other specifications may introduce specific notations for modality used in
> a media or in relation to a language tag. Applications not implementing
> such specific notations may use the following simple deductions.
> ------------------------------------------------------------
> ---------------------------------------------------------
>
> My note:  by this we currently consciously exclude the following use and I
> am ok with that:
> -text in mp4 video
> requiring comparison of language tags used in different media descriptions,
> and requiring simultaneous use of language in two different media that is
> otherwise out of scope for this draft. It was introduced while I still
> hoped that we could introduce other dependencies between language use in
> different media.  It is not the most urgent media/language combination to
> specify. It is also handled in draft-hellstrom-slim-modality-grouping.
> So, assuming that we can get progress on that draft, we could clean up the
> current draft by deleting the exception. I suggest that we delete the
> exception.
>
> /Gunnar
>
>
>
> --

---

From: Gunnar Hellström <[email-masked]>
Date: Sat, 14 Oct 2017 22:22:03 +0200

> - A language tag in audio media is supposed to indicate spoken modality.
<GH>This proposal was intended to express approximately the same as the 
current 5.4, but in a more strict way, being easy for applications to 
use and allowing for extensions.
We have now (in version -16 reduced the defined modalities to sign 
language in video, written in text and spoken in audio. That is 
simplification enough for the review comment to be satisfied.
It seems too complicated to get agreement about this rewording so we can 
drop the proposal. The already done rewording in 5.4 in version -16 to 
say that other use of language tags is not defined in this document 
opens well for other work to add new valid media/modality/language tag 
combinations.

I continue answering your questions anyway.
> [BA] Even a tag with "Sign Language" in the description??
<GH>Yes. The receiving application would trust the sending application 
and know that a language tag in audio is not a sign language tag. If it 
was anyway, a match would be very unlikely.

> [BA] If the tag has "Sign Language" in the description, can this 
> document really say that?
<GH>The idea is to have a simple rule to meet the needs indicated by the 
review comment.
> language modality except for the case when it is supposed to indicate 
> a view of a speaking person mentioned in section 5.2 characterized by 
> the exact same language tag also appearing in an audio media 
> specification.
>
> [BA] It seems like an over-reach to say that a spoken language tag in 
> video media should instead be interpreted as a request for Sign 
> Language.  If this were done, would it always be clear which Sign 
> Language was intended?  And could we really assume that both sides, if 
> negotiating a spoken language tag in video media, were really 
> indicating the desire to sign?  It seems like this could easily result 
> interoperability failure.
<GH>Yes it would result in interoperability failure. A match would be 
very unlikely. But the sentence means that  the sending application 
should only put sign language tags in the video description and the 
receiving application could trust that the tags in video descriptions 
are sign language tags. ( except for the exception that we are now 
prepared to drop if all agree. (already gone in version -16))
>
> - A language tag in media where the modality is obvious or specified 
> for the media subtype definition is supposed to indicate that modality.
<GH>these two were maybe the more important additions plus the 
mentioning in the beginning of a possibility to add indication of 
modality to a media specification or a language specification by further 
work.

/Gunnar

> On Sat, Oct 14, 2017 at 1:21 AM, Gunnar Hellström 
> <[email-masked] <mailto:[email-masked]>> wrote:
>
>              Issue 43 ( <https://trac.ietf.org/trac/slim/ticket/43
>             <https://trac.ietf.org/trac/slim/ticket/43>>https://trac.ietf.org/trac/slim/ticket/43
>             <https://trac.ietf.org/trac/slim/ticket/43> ) results from
>             a review comment that said that a simple way is required
>             to decide if a language tag is a sign language or a
>              Some applications scan the IANA language registry at
>             startup for the word "Sign" in the tag description:
>
>             <https://www.iana.org/assignments/language-subtag-registry/language-subtag-registry
>             <https://www.iana.org/assignments/language-subtag-registry/language-subtag-registry>>https://www.iana.org/assignments/language-subtag-registry/language-subtag-registry
>             <https://www.iana.org/assignments/language-subtag-registry/language-subtag-registry>
>
>
>
>              Given the current layout of the language subtag registry,
>             it is not clear to me that there is an easier way to
>             determine which tags represent sign languages.  Nor is it
>             within the SLIM WG charter to develop a modification to
>             the language subtag registry to address this concern.
>              So I am wondering whether we might resolve this with a
>             Note outlining the problem but not offering a solution.
>
>
>         I think the wording in -14 addresses the comment by accepting
>         Dale's suggestion that, rather than know non-signed tags, it's
>         the use of the exact same tag in both an audio and a video
>         stream that is the indicator. That both tightens up the
>         technical issue and simplifies it greatly.
>
>     <GH>We can replace 5.4 with a more explicit section guiding
>     applications to how to make the deduction simple. So, instead of a
>     note, I suggest that we replace 5.4 with:
>
>     There is no easy way to deduct the intended modality from a
>     language tag. Other specifications may introduce specific
>     notations for modality used in a media or in relation to a
>     language tag. Applications not implementing such specific
>     notations may use the following simple deductions.
>     - A language tag in video media is supposed to indicate visual
>     sign language modality except for the case when it is supposed to
>     indicate a view of a speaking person mentioned in section 5.2
>     characterized by the exact same language tag also appearing in an
>     audio media specification.
>     - A language tag in media where the modality is obvious or
>     specified for the media subtype definition is supposed to indicate
>     that modality.
>     - A language tag in other media descriptions than above has
>     undefined modality.
>     My note:  by this we currently consciously exclude the following
>     use and I am ok with that:
>     -text in mp4 video
>     -most application media, however some may have explicit
>     descriptions in subtype specifications.
>
>     different media that is otherwise out of scope for this draft. It
>     was introduced while I still hoped that we could introduce other
>     dependencies between language use in different media.  It is not
>     the most urgent media/language combination to specify. It is also
>     handled in draft-hellstrom-slim-modality-grouping. So, assuming
>     that we can get progress on that draft, we could clean up the
>     current draft by deleting the exception. I suggest that we delete
>     the exception.
>
>     /Gunnar
>
>
>
>     -- 
>     [email-masked] <mailto:[email-masked]>
>     +46 708 204 288 <tel:%2B46%20708%20204%20288>
>
>
>     [email-masked] <mailto:[email-masked]>
>     <https://www.ietf.org/mailman/listinfo/slim>
>
>
>
>

---

From: Paul Kyzivat <[email-masked]>
Date: Sat, 14 Oct 2017 19:19:55 -0400

On 10/14/17 2:03 PM, Bernard Aboba wrote:
> negotiating a spoken language tag in video media, were really indicating 
> the desire to sign?  It seems like this could easily result 

IMO the right way to indicate that two (or more) media streams are 
conveying alternative representations of the same language content is by 
grouping them with a new grouping attribute. That can tie together an 
audio with a video and/or text. A language tag for sign language on the 
video stream then clarifies to the recipient that it is sign language. 
The grouping attribute by itself can indicate that these streams are 
conveying language.

(IIRC I suggested something along these lines a long time ago.)

	Thanks,
	Paul

---

From: Gunnar Hellström <[email-masked]>
Date: Sun, 15 Oct 2017 08:24:17 +0200

Paul,
Den 2017-10-15 kl. 01:19, skrev Paul Kyzivat:
>> Language was intended?  And could we really assume that both sides, 
>> if negotiating a spoken language tag in video media, were really 
>> indicating the desire to sign?  It seems like this could easily 
>> result interoperability failure.
>
> conveying alternative representations of the same language content is 
> by grouping them with a new grouping attribute. That can tie together 
> an audio with a video and/or text. A language tag for sign language on 
> the video stream then clarifies to the recipient that it is sign 
> language. The grouping attribute by itself can indicate that these 
> streams are conveying language.
<GH>Yes, and that is proposed in 
draft-hellstrom-slim-modality-grouping    with two kinds of grouping: 
One kind of grouping to tell that two or more languages in different 
streams are alternatives with the same content and a priority order is 
assigned to them to guide the selection of which one to use during the 
call. The other kind of grouping telling that two or more languages in 
different streams are desired together with the same language content 
but different modalities ( such as the use for captioned telephony with 
the same content provided in both speech and text, or sign language 
interpretation where you see the interpreter,  or possibly spoken 
language interpretation with the languages provided in different audio 
streams ). I hope that that draft can be progressed. I see it as a 
needed complement to the pure language indications per media.

The discussion in this thread is more about how an application would 
easily know that e.g. "ase" is a sign language and "en" is a spoken (or 
written) language, and also a discussion about what kinds of languages 
are allowed and indicated by default in each media type. It was not at 
all about falsely using language tags in the wrong media type as Bernard 
understood my wording. It was rather a limitation to what modalities are 
used in each media type and how to know the modality with cases that are 
not evident, e.g. "application" and "message" media types.

Right now we have returned to a very simple rule: we define only use of 
spoken language in audio media, written language in text media and sign 
language in video media.
We have discussed other use, such as a view of a speaking person in 
video, text overlay on video, a sign language notation in text media, 
written language in message media, written language in WebRTC data 
channels, sign written and spoken in bucket media maybe declared as 
application media. We do not define these cases. They are just not 
defined, not forbidden. They may be defined in the future.

My proposed wording in section 5.4 got too many misunderstandings so I 
gave up with it. I think we can live with 5.4 as it is in version -16.

Thanks,
Gunnar

---

From: Paul Kyzivat <[email-masked]>
Date: Sun, 15 Oct 2017 13:13:18 -0400

> needed complement to the pure language indications per media.

Oh, sorry. I did read that draft but forgot about it.

> not evident, e.g. "application" and "message" media types.

What do you mean by "know"? Is it for the *UA* software to know, or for 
the human user of the UA to know? Presumably a human user that cares 
will understand this if presented with the information in some way. But 
typically this isn't presented to the user.

For the software to know must mean that it will behave differently for a 
tag that represents a sign language than for one that represents a 
spoken or written language. What is it that it will do differently?

	Thanks,
	Paul

---

From: Bernard Aboba <[email-masked]>
Date: Sun, 15 Oct 2017 10:49:14 -0700

Paul said:

"For the software to know must mean that it will behave differently for a
tag that represents a sign language than for one that represents a spoken
or written language. What is it that it will do differently?"

[BA] In terms of behavior based on the signed/non-signed distinction, in
-17 the only reference appears to be in Section 5.4, stating that certain
combinations are not defined in the document (but that definition of those
combinations was out of scope):

5.4 <https://tools.ietf.org/html/draft-ietf-slim-negotiating-human-language-17#section-5.4>.
Undefined Combinations

   The behavior when specifying a non-signed language tag for a video
   media stream, or a signed language tag for an audio or text media
   stream, is not defined in this document.

On Sun, Oct 15, 2017 at 10:13 AM, Paul Kyzivat <[email-masked]>
wrote:

>>>> video media should instead be interpreted as a request for Sign Language.
>>>> If this were done, would it always be clear which Sign Language was
>>>> intended?  And could we really assume that both sides, if negotiating a
>>>> spoken language tag in video media, were really indicating the desire to
>>>> sign?  It seems like this could easily result interoperability failure.
>>>>
>>>
>>> grouping them with a new grouping attribute. That can tie together an audio
>>> with a video and/or text. A language tag for sign language on the video
>>> stream then clarifies to the recipient that it is sign language. The
>>> grouping attribute by itself can indicate that these streams are conveying
>>> language.
>>>
>> <GH>Yes, and that is proposed in draft-hellstrom-slim-modality-grouping
>> with two kinds of grouping: One kind of grouping to tell that two or more
>> languages in different streams are alternatives with the same content and a
>> priority order is assigned to them to guide the selection of which one to
>> use during the call. The other kind of grouping telling that two or more
>> languages in different streams are desired together with the same language
>> content but different modalities ( such as the use for captioned telephony
>> with the same content provided in both speech and text, or sign language
>> interpretation where you see the interpreter,  or possibly spoken language
>> interpretation with the languages provided in different audio streams ). I
>> hope that that draft can be progressed. I see it as a needed complement to
>> the pure language indications per media.
>>
>
>> written) language, and also a discussion about what kinds of languages are
>> allowed and indicated by default in each media type. It was not at all
>> about falsely using language tags in the wrong media type as Bernard
> the human user of the UA to know? Presumably a human user that cares will
> understand this if presented with the information in some way. But
> tag that represents a sign language than for one that represents a spoken
> or written language. What is it that it will do differently?
>
>         Thanks,
>         Paul
>
>
>> application media. We do not define these cases. They are just not defined,
>> not forbidden. They may be defined in the future.
>>

---

From: Paul Kyzivat <[email-masked]>
Date: Sun, 15 Oct 2017 15:27:40 -0400

On 10/15/17 1:49 PM, Bernard Aboba wrote:
> Paul said:
> 
> "For the software to know must mean that it will behave differently for 
> a tag that represents a sign language than for one that represents a 
> spoken or written language. What is it that it will do differently?"
> 
> -17 the only reference appears to be in Section 5.4, stating that 
> certain combinations are not defined in the document (but that 
> definition of those combinations was out of scope):

I'm asking whether this is a distinction without a difference. I'm not 
asking whether this makes a difference in the *protocol*, but whether in 
the end it benefits the participants in the call in any way. For instance:

- does it help the UA to decide how to alert the callee, so that the
   callee can better decide whether to accept the call or instruct the
   UA about how to handle the call?

- does it allow the UA to make a decision whether to accept the media?

- can the UA use this information to change how to render the media?

And if there is something like this, will the UA be able to do this 
generically based on whether the media is sign language or not, or will 
the UA need to already understand *specific* sign language tags?

E.g., A UA serving a deaf person might automatically introduce a sign 
language interpreter into an incoming audio-only call. If the incoming 
call has both audio and video then the video *might* be for conveying 
sign language, or not. If not then the UA will still want to bring in a 
sign language interpreter. But is knowing the call generically contains 
sign language sufficient to decide against bringing in an interpreter? 
Or must that depend on it being a sign language that the user can use? 
If the UA is configured for all the specific sign languages that the 
user can deal with then there is no need to recognize other sign 
languages generically.

	Thanks,
	Paul

>       5.4
>       <https://tools.ietf.org/html/draft-ietf-slim-negotiating-human-language-17#section-5.4>.
>       Undefined Combinations
> 
> 
> 
> On Sun, Oct 15, 2017 at 10:13 AM, Paul Kyzivat <[email-masked] 
> <mailto:[email-masked]>> wrote:
> 
>                 "Applications not implementing such specific notations
>                 may use the following simple deductions.
> 
>                 [BA] If the tag has "Sign Language" in the description,
>                 can this document really say that?
> 
>                 - A language tag in video media is supposed to indicate
>                 visual sign language modality except for the case when
>                 it is supposed to indicate a view of a speaking person
>                 mentioned in section 5.2 characterized by the exact same
>                 language tag also appearing in an audio media specification.
> 
>                 [BA] It seems like an over-reach to say that a spoken
>                 language tag in video media should instead be
>                 interpreted as a request for Sign Language.  If this
>                 were done, would it always be clear which Sign Language
>                 was intended?  And could we really assume that both
>                 sides, if negotiating a spoken language tag in video
>                 media, were really indicating the desire to sign?  It
>             IMO the right way to indicate that two (or more) media
>             streams are conveying alternative representations of the
>             same language content is by grouping them with a new
>             grouping attribute. That can tie together an audio with a
>             video and/or text. A language tag for sign language on the
>             video stream then clarifies to the recipient that it is sign
>             language. The grouping attribute by itself can indicate that
>             these streams are conveying language.
> 
>         draft-hellstrom-slim-modality-grouping    with two kinds of
>         grouping: One kind of grouping to tell that two or more
>         languages in different streams are alternatives with the same
>         content and a priority order is assigned to them to guide the
>         selection of which one to use during the call. The other kind of
>         grouping telling that two or more languages in different streams
>         are desired together with the same language content but
>         different modalities ( such as the use for captioned telephony
>         with the same content provided in both speech and text, or sign
>         language interpretation where you see the interpreter,  or
>         possibly spoken language interpretation with the languages
>         provided in different audio streams ). I hope that that draft
>         can be progressed. I see it as a needed complement to the pure
>         language indications per media.
> 
> 
>         The discussion in this thread is more about how an application
>         would easily know that e.g. "ase" is a sign language and "en" is
>         a spoken (or written) language, and also a discussion about what
>         kinds of languages are allowed and indicated by default in each
>         media type. It was not at all about falsely using language tags
>         in the wrong media type as Bernard understood my wording. It was
>         rather a limitation to what modalities are used in each media
>         type and how to know the modality with cases that are not
>         evident, e.g. "application" and "message" media types.
> 
> 
>     What do you mean by "know"? Is it for the *UA* software to know, or
>     for the human user of the UA to know? Presumably a human user that
>     cares will understand this if presented with the information in some
>     way. But typically this isn't presented to the user.
> 
>     For the software to know must mean that it will behave differently
>     for a tag that represents a sign language than for one that
>     represents a spoken or written language. What is it that it will do
>     differently?
> 
>              Thanks,
>              Paul
> 
> 
>         Right now we have returned to a very simple rule: we define only
>         use of spoken language in audio media, written language in text
>         media and sign language in video media.
>         We have discussed other use, such as a view of a speaking person
>         in video, text overlay on video, a sign language notation in
>         text media, written language in message media, written language
>         in WebRTC data channels, sign written and spoken in bucket media
>         maybe declared as application media. We do not define these
>         cases. They are just not defined, not forbidden. They may be
>         defined in the future.
> 
>         My proposed wording in section 5.4 got too many
>         misunderstandings so I gave up with it. I think we can live with
>         5.4 as it is in version -16.
> 
>         Thanks,
>         Gunnar
> 
> 
>

---

From: Gunnar Hellström <[email-masked]>
Date: Sun, 15 Oct 2017 23:22:08 +0200

Den 2017-10-15 kl. 21:27, skrev Paul Kyzivat:
>> "For the software to know must mean that it will behave differently 
>> differently?"
>> [BA] In terms of behavior based on the signed/non-signed distinction, 
>> in -17 the only reference appears to be in Section 5.4, stating that 
> asking whether this makes a difference in the *protocol*, but whether 
> in the end it benefits the participants in the call in any way. 
<GH>Good point, I was on my way to make a similar comment earlier today. 
The difference it makes for applications to "know" what modality a 
language tag represents in its used position seems to be only for 
imagined functions that are out of scope for the protocol specification.
> For instance:
<GH>Yes, for a regular human user -to-user call, the result of the 
negotiation must be presented to the participants, so that they can 
start the call with a language and modality that is agreed.
That presentation could be exactly the description from the language tag 
registry, and then no "knowledge" is needed from the application. But it 
is more likely that the application has its own string for presentation 
of the negotiated language and modality. So that will be presented. But 
it is still found by a table lookup between language tag and string for 
a language name, so no real knowledge is needed.
We have said many times that the way the application tells the user the 
result of the negotiation is out of scope for the draft, but it is good 
to discuss and know that it can be done.
A similar mechanism is also needed for configuration of the user's 
language preference profile further discussed below.
> - does it allow the UA to make a decision whether to accept the media?
<GH>No, the media should be accepted regardless of the result of the 
language negotiation.
> - can the UA use this information to change how to render the media?
<GH>Yes, for the specialized text notation of sign language we have 
discussed but currently placed out of scope, a very special rendering 
application is needed. The modality would be recognized by a script 
subtag to a sign language tag used in text media. However, I think that 
would be best to also use it with a specific text subtype, so that the 
rendering can be controlled by invocation of a "codec" for that rendering.
> generically based on whether the media is sign language or not, or 
> will the UA need to already understand *specific* sign language tags?
<GH>Applications will need to have localized versions of the names for 
the different sign languages and also for spoken languages and written 
languages, to be used in setting of preferences and announcing the 
results of the negotiation. It might be overkill to have such localized 
names for all languages in the IANA language registry, so it will need 
to be able to handle localized names of a subset och the registry. With 
good design however, this is just an automatic translation between a 
language tag and a corresponding name, so it does in fact not require 
any "knowledge" of what modality is used with each language tag.
The application can ask for the configuration:
"Which languages do you want to offer to send in video"
"Which languages do you want to offer to send in text"
"Which languages do you want to offer to send in audio"
"Which languages do you want to be prepared to receive in video"
"Which languages do you want to be prepared to receive in text"
"Which languages do you want to be prepared to receive in audio"

And for each question provide a list of language names to select from. 
When the selection is made, the corresponding language tag is placed in 
the profile for negotiation.

If the application provides the whole IANA language registry to the user 
for each question, then there is a possibility that the user by mistake 
selects a language that requires another modality than the question was 
about. If the application shall limit the lists provided for each 
question, then it will need a kind of knowledge about which language 
tags suit each modality (and media)

> sign language, or not. If not then the UA will still want to bring in 
> a sign language interpreter. But is knowing the call generically 
> contains sign language sufficient to decide against bringing in an 
> interpreter? Or must that depend on it being a sign language that the 
> user can use? If the UA is configured for all the specific sign 
> languages that the user can deal with then there is no need to 
> recognize other sign languages generically.
<GH>We are talking about specific language tags here and knowing what 
modality they are used for. The user needs to specify which sign 
languages they prefer to use. The callee application can be made to look 
for gaps between what the caller offers and what the callee can accept, 
and from that deduct which type and languages for a conversion that is 
needed, and invoke that as a relay service. That invocation can be made 
completely table driven and have corresponding translation profiles for 
available relay services. But it is more likely that it is done by 
having some knowledge about which languages are sign languages and which 
are spoken languages and sending the call to the relay service to try to 
sort out if they can handle the translation.
So, the answer is - no, the application does not really have any 
knowledge about which modality a language tag represents in its used 
position. If the user selects to indicate very rare language tag 
indications for a media, then a match will just become very unlikely.

Where does this discussion take us? Should we modify section 5.4 again?

Thanks
Gunnar

---

From: Paul Kyzivat <[email-masked]>
Date: Sun, 15 Oct 2017 17:58:06 -0400

> start the call with a language and modality that is agreed.

*Today* we don't do this. We leave it for the end users to negotiate the 
language they will use by other means - typically in-band by informal 
negotiation. Most UAs don't have any provision to provide the specified 
language to the end user.

This of course will also work for deaf users with sign language in the 
absence of any interpreters in the call. When it doesn't work is when 
there is a deaf user and a hearing user, and video isn't present at both 
ends. But in that case there isn't much hope.

> a language name, so no real knowledge is needed.

Clearly presenting the raw language tags won't be useful to most users. 
So some sort of translation is needed. Such a hypothetical translation 
table could also have properties, such as whether the language is a sign 
language. Hence this doesn't seem to be a problem we need to solve.

> rendering can be controlled by invocation of a "codec" for that rendering.

But won't the rendering also be in some way language specific? If so, 
the device will need to be configured for the specific languages it 
supports. Hence again a generic mechanism isn't needed.

> any "knowledge" of what modality is used with each language tag.

My earlier comment applies to the above.

> the profile for negotiation.

Hence, there need not be any *generic* mechanism, since this is based on 
configuration.

> Where does this discussion take us? Should we modify section 5.4 again?

Frankly, I see no need for section 5.4.

	Thanks,
	Paul

---

From: Gunnar Hellström <[email-masked]>
Date: Mon, 16 Oct 2017 00:36:58 +0200

Den 2017-10-15 kl. 23:58, skrev Paul Kyzivat:
>>>> [BA] In terms of behavior based on the signed/non-signed 
>>>> distinction, in -17 the only reference appears to be in Section 
>>>> 5.4, stating that certain combinations are not defined in the 
>>>> document (but that definition of those combinations was out of scope):
>>>
>>> I'm asking whether this is a distinction without a difference. I'm 
>>> not asking whether this makes a difference in the *protocol*, but 
>>> whether in the end it benefits the participants in the call in any way. 
>> <GH>Good point, I was on my way to make a similar comment earlier 
>> today. The difference it makes for applications to "know" what 
>> modality a language tag represents in its used position seems to be 
>> only for imagined functions that are out of scope for the protocol 
>> specification.
>>> For instance:
>>>
> *Today* we don't do this. We leave it for the end users to negotiate 
> the language they will use by other means - typically in-band by 
> informal negotiation. Most UAs don't have any provision to provide the 
> specified language to the end user.
<GH>Yes, and the main purpose of this work is to improve on that. - To 
make sure that the users are informed so they can start the call in an 
appropriate language. And even invoke support if needed.
> there is a deaf user and a hearing user, and video isn't present at 
> both ends. But in that case there isn't much hope.
>
>> That presentation could be exactly the description from the language 
>> tag registry, and then no "knowledge" is needed from the application. 
>> But it is more likely that the application has its own string for 
>> presentation of the negotiated language and modality. So that will be 
>> presented. But it is still found by a table lookup between language 
>> tag and string for a language name, so no real knowledge is needed.
>
> Clearly presenting the raw language tags won't be useful to most 
> users. So some sort of translation is needed. Such a hypothetical 
> translation table could also have properties, such as whether the 
> language is a sign language. Hence this doesn't seem to be a problem 
> we need to solve.
<GH>Right, we provide the information in language tag format to the 
devices, and how the devices collect configuration information from the 
users and provide negotiation result to the users is not specified in 
this draft.
>
>> We have said many times that the way the application tells the user 
>> the result of the negotiation is out of scope for the draft, but it 
>> is good to discuss and know that it can be done.
>> subtag to a sign language tag used in text media. However, I think 
>> that would be best to also use it with a specific text subtype, so 
>> that the rendering can be controlled by invocation of a "codec" for 
>> that rendering.
>
<GH>I got lost here in the reasoning. We were looking for reasons for 
the device to do things differently for different modalities and 
therefore know modality from language tag and its position in media 
description.
We have the unsolved case of a language tag for a spoken or written 
language when used in video media. Since these cases use the same 
language tag we do not know what is meant if it is provided in video 
media description. It can mean a view of a speaker or it can mean text 
overlay on video, e.g. with mp4 coding with text component together with 
video (if I remember the use of the subtypes right). This problem is why 
we for a while had a complex rule about just one case that we allowed.

Rendering will be modality specific. Text embedded in video requires 
other rendering than the view of a speaking person. All ways we have 
proposed to indicate the difference between these two modalities have 
been rejected.
>> <GH>Applications will need to have localized versions of the names 
>> for the different sign languages and also for spoken languages and 
>> written languages, to be used in setting of preferences and 
>> announcing the results of the negotiation. It might be overkill to 
>> have such localized names for all languages in the IANA language 
>> registry, so it will need to be able to handle localized names of a 
>> subset och the registry. With good design however, this is just an 
>> automatic translation between a language tag and a corresponding 
>> name, so it does in fact not require any "knowledge" of what modality 
>> is used with each language tag.
>
>> And for each question provide a list of language names to select 
>> from. When the selection is made, the corresponding language tag is 
>> placed in the profile for negotiation.
>
> Hence, there need not be any *generic* mechanism, since this is based 
> on configuration.
>
>> If the application provides the whole IANA language registry to the 
>> user for each question, then there is a possibility that the user by 
>> mistake selects a language that requires another modality than the 
>> question was about. If the application shall limit the lists provided 
>> for each question, then it will need a kind of knowledge about which 
>> language tags suit each modality (and media)
>>
>>
>>>
>>> E.g., A UA serving a deaf person might automatically introduce a 
>>> sign language interpreter into an incoming audio-only call. If the 
>>> incoming call has both audio and video then the video *might* be for 
>>> conveying sign language, or not. If not then the UA will still want 
>>> to bring in a sign language interpreter. But is knowing the call 
>>> generically contains sign language sufficient to decide against 
>>> bringing in an interpreter? Or must that depend on it being a sign 
>>> language that the user can use? If the UA is configured for all the 
>>> specific sign languages that the user can deal with then there is no 
>>> need to recognize other sign languages generically.
>> languages they prefer to use. The callee application can be made to 
>> look for gaps between what the caller offers and what the callee can 
>> accept, and from that deduct which type and languages for a 
>> conversion that is needed, and invoke that as a relay service. That 
>> invocation can be made completely table driven and have corresponding 
>> translation profiles for available relay services. But it is more 
>> likely that it is done by having some knowledge about which languages 
>> are sign languages and which are spoken languages and sending the 
>> call to the relay service to try to sort out if they can handle the 
>> translation.
>>>
>>>
<GH>This discussion at least changes the need for section 5.4 dramatically.
What we still might need to say is that we have no agreed way to 
differentiate between a view of a speaking person and text embedded in 
video by the defined notation. And we could warn for using unusual 
language tag - media combinations that will rarely be matched.
>>>> On Sun, Oct 15, 2017 at 10:13 AM, Paul Kyzivat 
>>>> <[email-masked] <mailto:[email-masked]>> wrote:
>>>>
>>>>                 - A language tag in audio media is supposed to 
>>>> indicate
>>>>                 spoken modality.
>>>>
>>>> indicate

---

From: Bernard Aboba <[email-masked]>
Date: Sun, 15 Oct 2017 16:21:34 -0700

Paul said:

""- can the UA use this information to change how to render the media?"

[BA]  If the video is used for signing, an application might infer an
encoder preference for frame rate over resolution (e.g. in WebRTC,
RTCRtpParameters.degradationPreference = "maintain-framerate" )

See:
https://rawgit.com/w3c/webrtc-pc/master/webrtc.html#dom-rtcrtpparameters-degradationpreference

On Sun, Oct 15, 2017 at 2:22 PM, Gunnar Hellström <
[email-masked]> wrote:

>>> a tag that represents a sign language than for one that represents a spoken
>> the end it benefits the participants in the call in any way.
>>
> The difference it makes for applications to "know" what modality a language
> tag represents in its used position seems to be only for imagined functions
> that are out of scope for the protocol specification.
>
>> For instance:
>>
> negotiation must be presented to the participants, so that they can start
> the call with a language and modality that is agreed.
> registry, and then no "knowledge" is needed from the application. But it is
> more likely that the application has its own string for presentation of the
> negotiated language and modality. So that will be presented. But it is
> still found by a table lookup between language tag and string for a
> language name, so no real knowledge is needed.
> result of the negotiation is out of scope for the draft, but it is good to
> discuss and know that it can be done.
> application is needed. The modality would be recognized by a script subtag
> to a sign language tag used in text media. However, I think that would be
> best to also use it with a specific text subtype, so that the rendering can
> be controlled by invocation of a "codec" for that rendering.
>
>>
>> generically based on whether the media is sign language or not, or will the
>> UA need to already understand *specific* sign language tags?
>>
> <GH>Applications will need to have localized versions of the names for the
> different sign languages and also for spoken languages and written
> languages, to be used in setting of preferences and announcing the results
> of the negotiation. It might be overkill to have such localized names for
> all languages in the IANA language registry, so it will need to be able to
> handle localized names of a subset och the registry. With good design
> however, this is just an automatic translation between a language tag and a
> corresponding name, so it does in fact not require any "knowledge" of what
> modality is used with each language tag.
> When the selection is made, the corresponding language tag is placed in the
> profile for negotiation.
>
> about. If the application shall limit the lists provided for each question,
> then it will need a kind of knowledge about which language tags suit each
> modality (and media)
>
>
>
>> language interpreter into an incoming audio-only call. If the incoming call
>> has both audio and video then the video *might* be for conveying sign
>> language, or not. If not then the UA will still want to bring in a sign
>> language interpreter. But is knowing the call generically contains sign
>> language sufficient to decide against bringing in an interpreter? Or must
>> that depend on it being a sign language that the user can use? If the UA is
>> configured for all the specific sign languages that the user can deal with
>> then there is no need to recognize other sign languages generically.
>>
> modality they are used for. The user needs to specify which sign languages
> they prefer to use. The callee application can be made to look for gaps
> between what the caller offers and what the callee can accept, and from
> that deduct which type and languages for a conversion that is needed, and
> invoke that as a relay service. That invocation can be made completely
> table driven and have corresponding translation profiles for available
> relay services. But it is more likely that it is done by having some
> knowledge about which languages are sign languages and which are spoken
> languages and sending the call to the relay service to try to sort out if
> they can handle the translation.
>
>>
>>

---

From: Randall Gellens <[email-masked]>
Date: Sun, 15 Oct 2017 18:09:40 -0700

>  Where does this discussion take us? Should we modify section 5.4 again?

No.  That should be for follow-on work.

-------------- Randomly selected tag: ---------------
The First Amendment is often inconvenient. But that is besides the
point.  Inconvenience does not absolve the government of its
obligation to tolerate speech. --Justice Anthony Kennedy, in 91-155

---

From: Gunnar Hellström <[email-masked]>
Date: Tue, 17 Oct 2017 00:21:01 +0200

> RTCRtpParameters.degradationPreference = "maintain-framerate" )
<GH>Right, that is a valid example of how real "knowledge" of the 
modality can be used by the application.

And, as a response on issue #43,

A simple way is to say

Video media descriptions shall only contain sign language tags
Audio media descriptions shall only contain language tags for spoken 
language
Text media descriptions shall only contain language tags for written 
language
Use of other media descriptions such as message and application with 
language indications require other specifications on how to assess the 
modality for non-signed languages.

The current 5.4 does not mention our main problem with the language 
tags, that there is no difference on them if we mean use for spoken 
language or written language. We should have made better efforts to 
solve that problem long ago, but we have not.

5.4 can be modified to specify the simple limited case and the problems 
that block us from specifying other cases:

      5.4
      <https://tools.ietf.org/html/draft-ietf-slim-negotiating-human-language-17#section-5.4>.
      Media and modality Combination problems

    The problem of indicating a language tag for the view of a speaking person in a video stream is out of scope for this document.

    The problem of indicating a language tag for use of written language coded as a component in a video stream is out of scope for this document.

    The use of language tags for negotiation of languages in other media than audio, video and text is not defined in this document.

    The problem of knowing which language tags are signed and which are not can be deducted
    from the IANA language tag registry. How this is done is out of scope of this document.

--------------------------------------------------------

But if we want to allow more cases, we need to consider the following 
complications:

1. to assess if a language represents a Sign Language, the application 
can look for the word "sign" in the description in the IANA language 
registry or a copy thereof as Randall already indicated.

2. For written languages used as a text component in a video stream, it 
is possible to code this for languages requiring a script subtag, but 
not for languages with suppressed script subtags

3. We have also discussed proposals for how to code written language in 
video stream for languages not requiring a script subtag, but not got 
acceptance for our proposals. So we need to say that that is currently 
undefined.

4. We also discussed how to code a view of a speaking person in video 
and said that that could be done by using the "definitively not written" 
script subtag on a non-signed language tag in video. But that was not 
appreciated by the language experts. Another option was to not allow 
written language overlayed on video, and that is the lately used option. 
( up to version -16 or so)

5. For talking and hearing audio media, we only have that case for 
language-tags in Audio. So that is easy to code and assess.

6. For written language in text media, a check can be made about if 
"sign" is part of the language tag description, and if not, it is a 
written language.

7. For signed language in text media, a check can be made about if 
"sign" is part of the language tag description, and if it is, it is a 
signed language in text notation. (extremely unusual)

8. For use with language tags in other media than audio, video and text, 
there is a need for a description on how to assess the modality, 
especially for non-signed languages before it is used.

We can construct a section 5.4 to describe this situation, but I doubt 
that it is worth the effort.

> On Sun, Oct 15, 2017 at 2:22 PM, Gunnar Hellström 
>             "For the software to know must mean that it will behave
>             differently for a tag that represents a sign language than
>             for one that represents a spoken or written language. What
>             is it that it will do differently?"
>
>             distinction, in -17 the only reference appears to be in
>             Section 5.4, stating that certain combinations are not
>             defined in the document (but that definition of those
>         I'm asking whether this is a distinction without a difference.
>         I'm not asking whether this makes a difference in the
>         *protocol*, but whether in the end it benefits the
>         participants in the call in any way.
>
>     modality a language tag represents in its used position seems to
>     be only for imagined functions that are out of scope for the
>     protocol specification.
>
>         For instance:
>
>     negotiation must be presented to the participants, so that they
>     can start the call with a language and modality that is agreed.
>     That presentation could be exactly the description from the
>     language tag registry, and then no "knowledge" is needed from the
>     application. But it is more likely that the application has its
>     own string for presentation of the negotiated language and
>     modality. So that will be presented. But it is still found by a
>     table lookup between language tag and string for a language name,
>     so no real knowledge is needed.
>     We have said many times that the way the application tells the
>     user the result of the negotiation is out of scope for the draft,
>     but it is good to discuss and know that it can be done.
>     <GH>No, the media should be accepted regardless of the result of
>     the language negotiation.
>
>
>     <GH>Yes, for the specialized text notation of sign language we
>     have discussed but currently placed out of scope, a very special
>     rendering application is needed. The modality would be recognized
>     by a script subtag to a sign language tag used in text media.
>     However, I think that would be best to also use it with a specific
>     text subtype, so that the rendering can be controlled by
>     invocation of a "codec" for that rendering.
>
>
>         And if there is something like this, will the UA be able to do
>         this generically based on whether the media is sign language
>         or not, or will the UA need to already understand *specific*
>         sign language tags?
>
>     registry, so it will need to be able to handle localized names of
>     a subset och the registry. With good design however, this is just
>     an automatic translation between a language tag and a
>     corresponding name, so it does in fact not require any "knowledge"
>     of what modality is used with each language tag.
>     from. When the selection is made, the corresponding language tag
>     is placed in the profile for negotiation.
>
>     If the application provides the whole IANA language registry to
>     the user for each question, then there is a possibility that the
>     user by mistake selects a language that requires another modality
>     than the question was about. If the application shall limit the
>     lists provided for each question, then it will need a kind of
>     knowledge about which language tags suit each modality (and media)
>
>
>
>         E.g., A UA serving a deaf person might automatically introduce
>         a sign language interpreter into an incoming audio-only call.
>         If the incoming call has both audio and video then the video
>         *might* be for conveying sign language, or not. If not then
>         the UA will still want to bring in a sign language
>         interpreter. But is knowing the call generically contains sign
>         language sufficient to decide against bringing in an
>         interpreter? Or must that depend on it being a sign language
>         that the user can use? If the UA is configured for all the
>         specific sign languages that the user can deal with then there
>         is no need to recognize other sign languages generically.
>
>     <GH>We are talking about specific language tags here and knowing
>     what modality they are used for. The user needs to specify which
>     sign languages they prefer to use. The callee application can be
>     made to look for gaps between what the caller offers and what the
>     callee can accept, and from that deduct which type and languages
>     for a conversion that is needed, and invoke that as a relay
>     service. That invocation can be made completely table driven and
>     have corresponding translation profiles for available relay
>     services. But it is more likely that it is done by having some
>     knowledge about which languages are sign languages and which are
>     spoken languages and sending the call to the relay service to try
>     to sort out if they can handle the translation.
>
>
>
>     knowledge about which modality a language tag represents in its
>     used position. If the user selects to indicate very rare language
>     tag indications for a media, then a match will just become very
>     unlikely.
>
>             <https://tools.ietf.org/html/draft-ietf-slim-negotiating-human-language-17#section-5.4
>             <https://tools.ietf.org/html/draft-ietf-slim-negotiating-human-language-17#section-5.4>>.
>                   Undefined Combinations
>
>
>
>             <[email-masked] <mailto:[email-masked]>
>             <mailto:[email-masked]
>             <mailto:[email-masked]>>> wrote:
>
>                             - A language tag in audio media is
>             supposed to indicate
>                             spoken modality.
>
>                             seems like this could easily result
>             interoperability
>                             failure.
>
>
>             [email-masked] <mailto:[email-masked]> <mailto:[email-masked]
>             <mailto:[email-masked]>>
>                         <https://www.ietf.org/mailman/listinfo/slim
>             <https://www.ietf.org/mailman/listinfo/slim>>
>
>
>
>             [email-masked] <mailto:[email-masked]> <mailto:[email-masked]
>             <mailto:[email-masked]>>
>                 <https://www.ietf.org/mailman/listinfo/slim
>             <https://www.ietf.org/mailman/listinfo/slim>>
>
>
>
>
>     --

---

From: Gunnar Hellström <[email-masked]>
Date: Tue, 17 Oct 2017 11:02:51 +0200

An even more general way to express what section 5.4 tries to say is:

------------------------------------------------------------------------------------------------------------------------------------------

5.4 Combinations of Language tags and Media descriptions

The combination of Language tags and other information in the media 
descriptions should be made so that the intended modality can be 
concluded by the negotiating parties.

----------------------------------------------------------------------------------------

That makes us not need to investigate what is possible today, and what 
further attributes or coding rules may be added in the future.

We have a risk that implementers start using some insufficient coding, 
that can cause interop issues. But instead we do not need to limit valid 
use that we just have not thought about by saying that specific 
combinations are out of scope or not defined. It is up to implementers 
to check that the combinations they use result in unambiguous modality.

And it opens for use of possible new attributes e.g. a=modality:spoken  
or a=modality:written  etc, to complement the undefined case when a 
non-signed language tag without script subtag is used in video media, 
and also for explaining any use of m=application or m=message media in 
interactive communication.

It does not really answer Issue #43 by explaining HOW to assess the 
modality easily, but it requires the implementers to make sure that it 
is possible.

And deducting the intended modality is the key to successful neotiation 
and communication.

Do you think this would be clear enough, or do we need to go into what 
clear cases we have?

Gunnar

Den 2017-10-17 kl. 00:21, skrev Gunnar Hellström:
> 5.4 can be modified to specify the simple limited case and the 
> problems that block us from specifying other cases:
>
>
>       5.4
> 2. For written languages used as a text component in a video stream, 
> it is possible to code this for languages requiring a script subtag, 
> but not for languages with suppressed script subtags
>
> 3. We have also discussed proposals for how to code written language 
> in video stream for languages not requiring a script subtag, but not 
> got acceptance for our proposals. So we need to say that that is 
> currently undefined.
>
> and said that that could be done by using the "definitively not 
> written" script subtag on a non-signed language tag in video. But that 
> was not appreciated by the language experts. Another option was to not 
> allow written language overlayed on video, and that is the lately used 
> option. ( up to version -16 or so)
>
> 8. For use with language tags in other media than audio, video and 
> text, there is a need for a description on how to assess the modality, 
>>             differently for a tag that represents a sign language
>>             than for one that represents a spoken or written
>>             language. What is it that it will do differently?"
>>
>>         I'm asking whether this is a distinction without a
>>         difference. I'm not asking whether this makes a difference in
>>         the *protocol*, but whether in the end it benefits the
>>     <GH>Yes, for a regular human user -to-user call, the result of
>>     the negotiation must be presented to the participants, so that
>>     they can start the call with a language and modality that is agreed.
>>     A similar mechanism is also needed for configuration of the
>>     user's language preference profile further discussed below.
>>
>>
>>     However, I think that would be best to also use it with a
>>     specific text subtype, so that the rendering can be controlled by
>>         And if there is something like this, will the UA be able to
>>         do this generically based on whether the media is sign
>>         language or not, or will the UA need to already understand
>>         *specific* sign language tags?
>>
>>     <GH>Applications will need to have localized versions of the
>>     names for the different sign languages and also for spoken
>>     languages and written languages, to be used in setting of
>>     preferences and announcing the results of the negotiation. It
>>     might be overkill to have such localized names for all languages
>>     in the IANA language registry, so it will need to be able to
>>     handle localized names of a subset och the registry. With good
>>     design however, this is just an automatic translation between a
>>     language tag and a corresponding name, so it does in fact not
>>     require any "knowledge" of what modality is used with each
>>     language tag.
>>         E.g., A UA serving a deaf person might automatically
>>         introduce a sign language interpreter into an incoming
>>         audio-only call. If the incoming call has both audio and
>>         video then the video *might* be for conveying sign language,
>>         or not. If not then the UA will still want to bring in a sign
>>         language interpreter. But is knowing the call generically
>>         contains sign language sufficient to decide against bringing
>>         in an interpreter? Or must that depend on it being a sign
>>         language that the user can use? If the UA is configured for
>>         all the specific sign languages that the user can deal with
>>             <mailto:[email-masked] <mailto:[email-masked]>>
>>             <mailto:[email-masked] <mailto:[email-masked]>>

---

From: Gunnar Hellström <[email-masked]>
Date: Mon, 23 Oct 2017 23:17:29 +0200

Issue #43 is the only issue we have left now. I do not want to see the 
discussion stop again until we have a solution on it that seems acceptable.

Section 5.4 seems to be a good place to handle Issue #43.

Currently, section 5.4 is more aimed at limiting what kind of coding for 
languages and modalities are acceptable.

Some viewpoints said that such limitations are not needed and that 5.4 
can be deleted.

I think we can do something in between these extremes. We can introduce 
explanations for what is required from an acceptable coding of a 
combination of media, languages, directions, and other parameters and 
explain basic ways to assess what is the resulting modality, and also 
explain that the more common media and language combinations that are 
used, the higher chance there is for a match. Thus unusual combinations 
are discouraged but not forbidden as long as the modality can be 
assessed from them. They can be used in specific applications.

I might continue tomorrow with wording proposal for the reasoning above, 
hoping that we can close issue #43 and the discussions around 5.4 soon.

/Gunnar

Den 2017-10-17 kl. 11:02, skrev Gunnar Hellström:
> that can cause interop issues. But instead we do not need to limit 
> valid use that we just have not thought about by saying that specific 
> And it opens for use of possible new attributes e.g. 
> a=modality:spoken  or a=modality:written  etc, to complement the 
> undefined case when a non-signed language tag without script subtag is 
> used in video media, and also for explaining any use of m=application 
> or m=message media in interactive communication.
>
> And deducting the intended modality is the key to successful 
> neotiation and communication.
>
>> language indications require other specifications on how to assess 
>> the modality for non-signed languages.
>>
>> 1. to assess if a language represents a Sign Language, the 
>> application can look for the word "sign" in the description in the  
>> IANA language registry or a copy thereof as Randall already indicated.
>>
>> written" script subtag on a non-signed language tag in video. But 
>> that was not appreciated by the language experts. Another option was 
>> to not allow written language overlayed on video, and that is the 
>> lately used option. ( up to version -16 or so)
>>
>> text, there is a need for a description on how to assess the 
>> modality, especially for non-signed languages before it is used.
>>
>>
>> We can construct a section 5.4 to describe this situation, but I 
>> doubt that it is worth the effort.
>>
>>
>>>
>>> See: 
>>>         difference. I'm not asking whether this makes a difference
>>>         in the *protocol*, but whether in the end it benefits the
>>>     <GH>Good point, I was on my way to make a similar comment
>>>     earlier today. The difference it makes for applications to
>>>     "know" what modality a language tag represents in its used
>>>     position seems to be only for imagined functions that are out of
>>>     scope for the protocol specification.
>>>
>>>         For instance:
>>>
>>>     language tag registry, and then no "knowledge" is needed from
>>>     the application. But it is more likely that the application has
>>>     its own string for presentation of the negotiated language and
>>>     table lookup between language tag and string for a language
>>>     name, so no real knowledge is needed.
>>>     user the result of the negotiation is out of scope for the
>>>     draft, but it is good to discuss and know that it can be done.
>>>     rendering application is needed. The modality would be
>>>     recognized by a script subtag to a sign language tag used in
>>>     text media. However, I think that would be best to also use it
>>>     with a specific text subtype, so that the rendering can be
>>>     controlled by invocation of a "codec" for that rendering.
>>>
>>>
>>>     user by mistake selects a language that requires another
>>>     modality than the question was about. If the application shall
>>>     limit the lists provided for each question, then it will need a
>>>     kind of knowledge about which language tags suit each modality
>>>     (and media)
>>>
>>>
>>>
>>>         or not. If not then the UA will still want to bring in a
>>>         sign language interpreter. But is knowing the call
>>>         generically contains sign language sufficient to decide
>>>         against bringing in an interpreter? Or must that depend on
>>>         it being a sign language that the user can use? If the UA is
>>>         configured for all the specific sign languages that the user
>>>         can deal with then there is no need to recognize other sign
>>>         languages generically.
>>>
>>>     made to look for gaps between what the caller offers and what
>>>     the callee can accept, and from that deduct which type and
>>>     languages for a conversion that is needed, and invoke that as a
>>>     relay service. That invocation can be made completely table
>>>     driven and have corresponding translation profiles for available
>>>     relay services. But it is more likely that it is done by having
>>>     some knowledge about which languages are sign languages and
>>>     which are spoken languages and sending the call to the relay
>>>     service to try to sort out if they can handle the translation.
>>>
>>>
>>>
>>>     used position. If the user selects to indicate very rare
>>>     language tag indications for a media, then a match will just
>>>     become very unlikely.
>>>

---

From: Bernard Aboba <[email-masked]>
Date: Mon, 23 Oct 2017 18:51:28 -0700

Thanks for suggesting a way forward, Gunnar. I too would like to get Issue
43 resolved so we can move forward in the process.
Please send your thoughts to the mailing list (preferrably before the
October 30 submission deadline so we can spin a new draft version).

In thinking about the issue, a question Paul asked has stuck in my mind:
What difference does it/should it make?

Let us presume that the user agents are configured to signal the language
preferences.

In a pure peer-to-peer case (me calling you, no intermediaries), I
configure my UA to indicate a preference for Mandarin on the text modality,
French sign language on the audio modality and Swahili on the video
modality.

You (knowing my background and weird sense of humour) after being notified
of my preference, realize I am kidding and agree to accept the call,
knowing that whatever language preference you indicate, we will most likely
communicate in English since I do not speak Mandarin or Swahili and am
barely conversant in French, let along French sign language.

Would this scenario have worked out better with rules that mandated that my
odd choices for audio and video languages be labelled "undefined"? I think
not.

In a scenario where the call is between me and a call center (not a PSAP)
my flippant UA configuration might result in the call being rejected due to
a lack of Mandarin, Swahili or French sign language resources within the
call center.  But it's not clear that labelling my odd choices as
"undefined" should play a role in that decision.  For example, if the call
center did have someone who spoke Swahili, and connected me to them
(perhaps under the theory that my declared preference might indicate an
ability to lip-read Swahili), this might have improved the chance of
communication had my UA configuration been based on genuine expertise
rather than a warped sense of humour.

In other words,it is not clear to me how Section 5.4's discussion of scope
improves or clarifies the situation in any way - and there is some
possibility that it could cause problems.

On Mon, Oct 23, 2017 at 2:17 PM, Gunnar Hellström <
[email-masked]> wrote:

> Some viewpoints said that such limitations are not needed and that 5.4 can
> be deleted.
>
> explain that the more common media and language combinations that are used,
> the higher chance there is for a match. Thus unusual combinations are
> discouraged but not forbidden as long as the modality can be assessed from
> them. They can be used in specific applications.
>
> ------------------
> descriptions should be made so that the intended modality can be concluded
> by the negotiating parties.
>
>
> ----------------------------
>
> combinations are out of scope or not defined. It is up to implementers to
> check that the combinations they use result in unambiguous modality.
>
> non-signed language tag without script subtag is used in video media, and
> also for explaining any use of m=application or m=message media in
> modality easily, but it requires the implementers to make sure that it is
> possible.
>
> <GH>Right, that is a valid example of how real "knowledge" of the modality
> can be used by the application.
>
>
> The current 5.4 does not mention our main problem with the language tags,
> that there is no difference on them if we mean use for spoken language or
> written language. We should have made better efforts to solve that problem
> long ago, but we have not.
>
> 5.4 <https://tools.ietf.org/html/draft-ietf-slim-negotiating-human-language-17#section-5.4>.  Media and modality Combination problems
>
>
> 1. to assess if a language represents a Sign Language, the application can
> look for the word "sign" in the description in the  IANA language registry
> or a copy thereof as Randall already indicated.
>
> 2. For written languages used as a text component in a video stream, it is
> possible to code this for languages requiring a script subtag, but not for
> languages with suppressed script subtags
>
> 4. We also discussed how to code a view of a speaking person in video and
> said that that could be done by using the "definitively not written" script
> subtag on a non-signed language tag in video. But that was not appreciated
> by the language experts. Another option was to not allow written language
> overlayed on video, and that is the lately used option. ( up to version -16
> or so)
>
> 6. For written language in text media, a check can be made about if "sign"
> is part of the language tag description, and if not, it is a written
> language.
>
> 7. For signed language in text media, a check can be made about if "sign"
> is part of the language tag description, and if it is, it is a signed
> language in text notation. (extremely unusual)
>
> there is a need for a description on how to assess the modality, especially
> for non-signed languages before it is used.
>
>
> See:  https://rawgit.com/w3c/webrtc-pc/master/webrtc.html#
> dom-rtcrtpparameters-degradationpreference
>
>>>> certain combinations are not defined in the document (but that definition
>>>> of those combinations was out of scope):
>>>>
>>>
> [email-masked]
> +46 708 204 288
>
>
> --
> [email-masked]
> +46 708 204 288
>
>
>
> SLIM mailing [email-masked]://www.ietf.org/mailman/listinfo/slim
>
>
> --
> [email-masked]
> +46 708 204 288
>
>

---

From: Gunnar Hellström <[email-masked]>
Date: Tue, 24 Oct 2017 16:51:36 +0200

Bernard and all,

Yes, I agree with your reasoning. I also understand how we ended up in a 
limiting section 5.4. We were mainly thinking of a globally 
interoperable multimedia communication system, and wanted high chance 
for language and modality match between callers and callees. Limiting 
the choices then seems to increase the opportunities for match. But the 
RFCs can be used in much smaller application areas, where it can be of 
value to differentiate between some less common combinations of media 
and languages that by the current wording of 5.4 maybe would be 
discouraged from using the mechanism, while it instead could do it by an 
internal agreement of which media - language combinations are relevant 
and possibly what extra information will be used to distinguish between 
otherwise ambiguous combinations. (as a tag for written or spoken 
language in video that may need further information to be decided.)

So, yet another proposal for Issue #43 and section 5.4, much more 
informative and less restrictive than the previous version.

-----Old text----

      5.4 Undefined Combinations

-----New text------------
5.4 Media, Language and Modality indications

The combination of Language tags and other information in the media 
descriptions should be composed so that the intended modality can be 
concluded by the negotiating parties. For general use, with best 
opportunity for finding matching languages, it is recommended to use the 
most apparent combinations of language tags and media: sign language 
tags in video media, spoken language tags for audio media and written 
language tags for text media. The examples in this specification are all 
from this set of three obvious language/media/modality combinations.

The following explains some factors in combining language tags, media 
types and other media description information to identify intended 
language modalities.

A specific sign language can be identified by its existence in the IANA 
registry of language subtags according to BCP 47 [RFC5646] , and finding 
that the language subtag is found at least in two entries in the 
registry, once with the Type field "language" and once with the Type 
field "extlang" combined with the Prefix field value "sgn".

A generic identification of sign language competence or preference, 
without specifying exactly which sign language, can be indicated by use 
of the value "sgn" in the language tag of the corresponding "hlang" 
attribute.

Sign language communication in the usually used visual modality is most 
often conveyed in a "video" media stream. Application specific use may 
appear in other media, such as "message" and "application". Certain 
textual notation modalities of sign language may appear in the "text" 
media stream.

A specific spoken or written language can be identified by finding that 
the language subtag exsists in the IANA registry of language subtags 
according to  BCP 47 [RFC5646] with the Type field "language" and no 
entry for the language subtag exists with the value "sgn" in the Prefix 
field of any entry with type field value "extlang".  The spoken modality 
is usually conveyed in an "audio" media stream. The written modality in 
real-time is usually conveyed in a "text" media stream.

Use of a language subtag for a written or spoken language in other media 
streams than  "text" or "audio" require further indications or 
application agreements for identification of the modality. A number of 
such further indications are available and new ones may be added by 
further work. Use of written modality in another media stream than 
"text", may be discriminated by use of a script subtag in the language 
tag, where that is appropriate. Use for sending of a visual view of a 
speaking person may be indicated by the value "speaker" in an SDP 
Content attribute according to RFC 4796 [RFC4796] in a "video" media 
stream or another media carrying video (e.g. "message" or "application").

Use of written modality in another media stream than "text", may, for 
cases when the script subtag is suppressed, be discriminated by use of 
any other appropriate notation or application agreement. An appropriate 
notation may be use of a media subtype specific to the intended modality.
-------------------------------------------------------------------End 
of new text---------------------------------------

Gunnar

Den 2017-10-24 kl. 03:51, skrev Bernard Aboba:
> Thanks for suggesting a way forward, Gunnar. I too would like to get 
> Issue 43 resolved so we can move forward in the process.
> In thinking about the issue, a question Paul asked has stuck in my 
> mind:  What difference does it/should it make?
>
> Let us presume that the user agents are configured to signal the 
> language preferences.
>
> configure my UA to indicate a preference for Mandarin on the text 
> modality, French sign language on the audio modality and Swahili on 
> the video modality.
>
> You (knowing my background and weird sense of humour) after being 
> notified of my preference, realize I am kidding and agree to accept 
> the call, knowing that whatever language preference you indicate, we 
> will most likely communicate in English since I do not speak Mandarin 
> or Swahili and am barely conversant in French, let along French sign 
> language.
>
> Would this scenario have worked out better with rules that mandated 
> that my odd choices for audio and video languages be labelled 
> "undefined"? I think not.
>
> In a scenario where the call is between me and a call center (not a 
> PSAP) my flippant UA configuration might result in the call being 
> rejected due to a lack of Mandarin, Swahili or French sign language 
> resources within the call center.  But it's not clear that labelling 
> my odd choices as "undefined" should play a role in that decision.  
> For example, if the call center did have someone who spoke Swahili, 
> and connected me to them (perhaps under the theory that my declared 
> preference might indicate an ability to lip-read Swahili), this might 
> have improved the chance of communication had my UA configuration been 
> based on genuine expertise rather than a warped sense of humour.
>
> In other words,it is not clear to me how Section 5.4's discussion of 
> scope improves or clarifies the situation in any way - and there is 
> some possibility that it could cause problems.
>
> On Mon, Oct 23, 2017 at 2:17 PM, Gunnar Hellström 
>     Issue #43 is the only issue we have left now. I do not want to see
>     the discussion stop again until we have a solution on it that
>     seems acceptable.
>
>     Currently, section 5.4 is more aimed at limiting what kind of
>     coding for languages and modalities are acceptable.
>
>     Some viewpoints said that such limitations are not needed and that
>     5.4 can be deleted.
>
>     I think we can do something in between these extremes. We can
>     introduce explanations for what is required from an acceptable
>     coding of a combination of media, languages, directions, and other
>     parameters and explain basic ways to assess what is the resulting
>     modality, and also explain that the more common media and language
>     combinations that are used, the higher chance there is for a
>     match. Thus unusual combinations are discouraged but not forbidden
>     as long as the modality can be assessed from them. They can be
>     used in specific applications.
>
>     I might continue tomorrow with wording proposal for the reasoning
>     above, hoping that we can close issue #43 and the discussions
>     around 5.4 soon.
>
>     /Gunnar
>
>
>>     The combination of Language tags and other information in the
>>     media descriptions should be made so that the intended modality
>>     can be concluded by the negotiating parties.
>>
>>
>>     That makes us not need to investigate what is possible today, and
>>     what further attributes or coding rules may be added in the future.
>>
>>     We have a risk that implementers start using some insufficient
>>     coding, that can cause interop issues. But instead we do not need
>>     to limit valid use that we just have not thought about by saying
>>     that specific combinations are out of scope or not defined. It is
>>     up to implementers to check that the combinations they use result
>>     in unambiguous modality.
>>
>>     undefined case when a non-signed language tag without script
>>     subtag is used in video media, and also for explaining any use of
>>     m=application or m=message media in interactive communication.
>>
>>     It does not really answer Issue #43 by explaining HOW to assess
>>     the modality easily, but it requires the implementers to make
>>     sure that it is possible.
>>
>>     Do you think this would be clear enough, or do we need to go into
>>     what clear cases we have?
>>
>>     Gunnar
>>
>>
>>
>>>>     [BA] If the video is used for signing, an application might
>>>>     infer an encoder preference for frame rate over resolution
>>>>     (e.g. in WebRTC, RTCRtpParameters.degradationPreference =
>>>>     "maintain-framerate" )
>>>     <GH>Right, that is a valid example of how real "knowledge" of
>>>     the modality can be used by the application.
>>>
>>>
>>>     Audio media descriptions shall only contain language tags for
>>>     spoken language
>>>     Text media descriptions shall only contain language tags for
>>>     written language
>>>     Use of other media descriptions such as message and application
>>>     with language indications require other specifications on how to
>>>     assess the modality for non-signed languages.
>>>
>>>     The current 5.4 does not mention our main problem with the
>>>     language tags, that there is no difference on them if we mean
>>>     use for spoken language or written language. We should have made
>>>     better efforts to solve that problem long ago, but we have not.
>>>
>>>     But if we want to allow more cases, we need to consider the
>>>     following complications:
>>>
>>>
>>>     application can look for the word "sign" in the description in
>>>     the  IANA language registry or a copy thereof as Randall already
>>>     indicated.
>>>
>>>     2. For written languages used as a text component in a video
>>>     stream, it is possible to code this for languages requiring a
>>>     script subtag, but not for languages with suppressed script subtags
>>>
>>>     3. We have also discussed proposals for how to code written
>>>     language in video stream for languages not requiring a script
>>>     subtag, but not got acceptance for our proposals. So we need to
>>>     say that that is currently undefined.
>>>
>>>     4. We also discussed how to code a view of a speaking person in
>>>     video and said that that could be done by using the
>>>     "definitively not written" script subtag on a non-signed
>>>     language tag in video. But that was not appreciated by the
>>>     language experts. Another option was to not allow written
>>>     language overlayed on video, and that is the lately used option.
>>>     5. For talking and hearing audio media, we only have that case
>>>     for language-tags in Audio. So that is easy to code and assess.
>>>
>>>     6. For written language in text media, a check can be made about
>>>     if "sign" is part of the language tag description, and if not,
>>>     it is a written language.
>>>
>>>     7. For signed language in text media, a check can be made about
>>>     if "sign" is part of the language tag description, and if it is,
>>>     it is a signed language in text notation. (extremely unusual)
>>>
>>>     8. For use with language tags in other media than audio, video
>>>     and text, there is a need for a description on how to assess the
>>>>     <https://rawgit.com/w3c/webrtc-pc/master/webrtc.html#dom-rtcrtpparameters-degradationpreference>
>>>>
>>>>     <[email-masked]
>>>>     <mailto:[email-masked]>> wrote:
>>>>
>>>>                 "For the software to know must mean that it will
>>>>                 behave differently for a tag that represents a sign
>>>>                 language than for one that represents a spoken or
>>>>                 written language. What is it that it will do
>>>>                 differently?"
>>>>
>>>>                 [BA] In terms of behavior based on the
>>>>                 signed/non-signed distinction, in -17 the only
>>>>                 reference appears to be in Section 5.4, stating
>>>>                 that certain combinations are not defined in the
>>>>             difference. I'm not asking whether this makes a
>>>>             difference in the *protocol*, but whether in the end it
>>>>             benefits the participants in the call in any way.
>>>>
>>>>         position seems to be only for imagined functions that are
>>>>         out of scope for the protocol specification.
>>>>
>>>>             For instance:
>>>>
>>>>         <GH>Yes, for a regular human user -to-user call, the result
>>>>         of the negotiation must be presented to the participants,
>>>>         so that they can start the call with a language and
>>>>         modality that is agreed.
>>>>         language tag registry, and then no "knowledge" is needed
>>>>         from the application. But it is more likely that the
>>>>         application has its own string for presentation of the
>>>>         negotiated language and modality. So that will be
>>>>         presented. But it is still found by a table lookup between
>>>>         language tag and string for a language name, so no real
>>>>         knowledge is needed.
>>>>         We have said many times that the way the application tells
>>>>         the user the result of the negotiation is out of scope for
>>>>         the draft, but it is good to discuss and know that it can
>>>>         be done.
>>>>         <GH>No, the media should be accepted regardless of the
>>>>         result of the language negotiation.
>>>>
>>>>
>>>>         <GH>Yes, for the specialized text notation of sign language
>>>>         we have discussed but currently placed out of scope, a very
>>>>         special rendering application is needed. The modality would
>>>>         be recognized by a script subtag to a sign language tag
>>>>         used in text media. However, I think that would be best to
>>>>         also use it with a specific text subtype, so that the
>>>>             And if there is something like this, will the UA be
>>>>             able to do this generically based on whether the media
>>>>             is sign language or not, or will the UA need to already
>>>>             understand *specific* sign language tags?
>>>>
>>>>         <GH>Applications will need to have localized versions of
>>>>         the names for the different sign languages and also for
>>>>         spoken languages and written languages, to be used in
>>>>         setting of preferences and announcing the results of the
>>>>         negotiation. It might be overkill to have such localized
>>>>         names for all languages in the IANA language registry, so
>>>>         it will need to be able to handle localized names of a
>>>>         subset och the registry. With good design however, this is
>>>>         just an automatic translation between a language tag and a
>>>>         corresponding name, so it does in fact not require any
>>>>         "knowledge" of what modality is used with each language tag.
>>>>         And for each question provide a list of language names to
>>>>         select from. When the selection is made, the corresponding
>>>>         language tag is placed in the profile for negotiation.
>>>>
>>>>         If the application provides the whole IANA language
>>>>         registry to the user for each question, then there is a
>>>>         possibility that the user by mistake selects a language
>>>>         that requires another modality than the question was about.
>>>>         If the application shall limit the lists provided for each
>>>>         question, then it will need a kind of knowledge about which
>>>>             audio-only call. If the incoming call has both audio
>>>>             and video then the video *might* be for conveying sign
>>>>             language, or not. If not then the UA will still want to
>>>>             bring in a sign language interpreter. But is knowing
>>>>             the call generically contains sign language sufficient
>>>>             to decide against bringing in an interpreter? Or must
>>>>             that depend on it being a sign language that the user
>>>>             can use? If the UA is configured for all the specific
>>>>             sign languages that the user can deal with then there
>>>>         <GH>We are talking about specific language tags here and
>>>>         knowing what modality they are used for. The user needs to
>>>>         specify which sign languages they prefer to use. The callee
>>>>         application can be made to look for gaps between what the
>>>>         caller offers and what the callee can accept, and from that
>>>>         deduct which type and languages for a conversion that is
>>>>         needed, and invoke that as a relay service. That invocation
>>>>         can be made completely table driven and have corresponding
>>>>         translation profiles for available relay services. But it
>>>>         is more likely that it is done by having some knowledge
>>>>         about which languages are sign languages and which are
>>>>         spoken languages and sending the call to the relay service
>>>>         to try to sort out if they can handle the translation.
>>>>
>>>>
>>>>
>>>>         So, the answer is - no, the application does not really
>>>>         have any knowledge about which modality a language tag
>>>>         represents in its used position. If the user selects to
>>>>         indicate very rare language tag indications for a media,
>>>>         then a match will just become very unlikely.
>>>>
>>>>         <mailto:[email-masked]>