Subject: [Dime] Diameter Extensibility & M-Bit Usage

From: <[email-masked]>
Date: Mon, 11 Feb 2008 17:07:42 +0200

Hi all, 

as part of finishing the Diameter Base protocol-bis document we ran into
the M-bit issue. Victor, the RFC3588bis draft editor, incorporated a
discussion about the M bit into the latest version of the Diameter
Design Guidelines document that lead me to believe that there is a
problem. 

So, here is the text from Section 4.1 of RFC 3588:
"
The 'M' Bit, known as the Mandatory bit, indicates whether support of
the AVP is required. If an AVP with the 'M' bit set is received by a
Diameter client, server, proxy, or translation agent and either the AVP
or its value is unrecognized, the message MUST be rejected. 
"

Also note the extensibility rules; by adding an AVP with the M bit set
to an existing application requires a new application id to be
allocated. 

Let us, for the moment, focus only on the usage of the M bit for the AVP
(rather than the values carried within an AVP). 

There is no direct dependency between the ABNF and the usage of the M
bit but one can certainly say that a specification indicating within the
ABNF that a certain AVP must at least occur 1 (or multiple) times
requires the implementation to understand it. Hence, in this case the
usage of the M bit is redundant. 

For the same application a couple of AVPs might be optional in the ABNF
but the definition of the AVP flag table may allow the M bit to be set.
This allows certain deployments to have a dynamic "capability
indication" via probing. Having the AVP optional in the ABNF and at the
same time require the M bit to be set just implies that the AVP has to
be understood (= implemented) but not necessarily used. This implements
the classical: mandatory to implement - optional to use behavior. Hence,
for a newly defined Diameter application it is fine to indicate all AVPs
as mandatory to implement (if this is the intention of the authors). An
author of a specification would therefore have to think of what the
minimum to implement functionality of a given specification has to be. 

For certain use cases, the M bit merely serves the purpose of
determining the support for certain features. When optional AVPs are
available then there are essentially two points in time when to agree on
a specific functionality: 

A) At Configuration Time: 

Two operators get together and agree to support a specific
functionality. Based on the configuration of the stack certain AVPs are
sent whereas others' aren't. We have seen this approach happening a lot
(also at the interop). It is obviously more time consuming. 

B) At Run Time: 

This is the sort of zero-configuration approach where both figure out
what they support during the protocol run. 

With the deployments we see today we are largely talking about (A)
rather than (B). Nevertheless, Diameter provides functionality for (B)
in the Diameter Base protocol and it is certainly useful and powerful.
Unfortunately, this functionality is provided only in a very basic form
since there is little feedback from the receiving end (and intermediary)
when certain functionality is not supported (unless one incrementially
probes to determine what went wrong). 

More important is the usage of the M bit when it comes to the
extensibility story. When a new application is extended by new AVPs then
one could require support of the new functionality from the other end
during run-time. When the ABNF is changed substantially then there is
clearly a need for defining a new Diameter application, like it is
necessary to register a new XML schema when you change it (rather than
extending it). When you, however, only make use of the extension points
to add new functionality then there is also with XML no need to change
the parent schema(s). It seems to be interesting to re-use the same
concepts in Diameter as well. 
 
Other protocols have similar concepts. For example, consider SIP that
defines the supported / required feature. See Section 8.1.1.9 and
Section 8.2.2.3 of RFC 3261. In some sense this refers to the
"end-to-end capability" mechanism used in some of our documents and that
is being used also in RADIUS in some places. Unfortunately, there has
never been an attempt to define such a feature consistently so that it
could be used by many Diameter extensions in a generic way. Note that
the concept in SIP refers to "features" (an abstract functionality)
rather than a specific header field or even a specific value in a header
field. Corresponding error codes are defined in the SIP specification as
well along with this mechanism. 

Example: SIP Location Conveyance --
http://tools.ietf.org/html/draft-ietf-sip-location-conveyance-09 --
specifies an option tag "geolocation" for usage with the Require,
Supported and Unsupported headers. The document defines how to carry
location information. Obviously, with the processing of location
information many things can go wrong and could be misinterpreted by the
receiving party. Hence, there are a large number of specific error codes
that can be used. 

I am not sure what todo about the extensiblity rule regarding an AVP
with the M bit set being added to an existing application that requires
a new Diameter application to be defined. When comparing this to SIP one
could argue that the application ID corresponds to the option tag.
Unfortunately, it is not quite the same since the application ID is used
in Diameter also for routing (but not in SIP) and hence it would
correspond better to a Require header combined with a Proxy-Require
header. Comparing this approach with XML and Relax NG schemas one could
also say that defining new AVPs in a specification is like defining a
new XML / Relax NG schema and these schemas are typically registered as
well. The comparison is, however, also not fully correct since most of
the XML usage I have been using in the IETF was an end-to-end matter and
hence the problem with intermediaries did not appear. Furthermore, XML
schemas do not allow individual attributes or elements to be registered
unless they appear in separate schemas. AVPs are, however, registered
separately. Merging of several applications in a single Diameter message
is, however, not possible. This has often been the concern when new
Diameter applications are defined since new Diameter applications lead
to additional message exchanges and cannot be piggy-backed on top of
other applications without making a design decision to incorporate the
functionality of another Diameter application. When you want to develop
them independently then you might encounter an interesting extensibility
story since you have very monolithic applications.  

Regarding the end-to-end capability negotation there is room to learn
something from work done in other areas on extensiblity and maybe we
don't need to re-invent the wheel every time we define another
extension. Currently, capabilities are described using different levels
of granularity. For example: 

In
http://www.ietf.org/internet-drafts/draft-ietf-dime-mip6-integrated-07.t
xt we use the MIP6-Feature-Vector that is used to convey information
from the NAS to the AAAH server. It is a bit map, as shown in Section
4.7.5. The same AVP is used in the reverse direction by the AAAH to
indicate the authorization decision (e.g., whether the local domain is
authorized to allocate a HA or not) in addition to implemented
functionality that could be used for a specific protocol run.

As such, this functionality is more than the functionality used in SIP
with the Require, Supported etc. headers. 

One problem I see is that the initial goal of routing with the help of
Diameter application was to simplify progressing at Diameter proxies.
Instead of having to parse through the entire message in order to figure
out whether there is something interesting in there for them to process
they just had to inspect the Diameter application. Unforuntely, this
hasn't really worked too well since there is no such functionality as
the SIP Proxy Require that would delegate the decision to set a certain
mode from the specification designer to the person that deploys it. The
same problem already appeared with the usage of the application ID and
the way of doing accounting. Hence, many decisions should be really
configuration time or design time decisions rather than design time
decision. 

It is kind of difficult to make a good decision on the M-bit and on the
larger extensibility story given that
* Diameter routing depends also on the Application ID
* the M-bit indicates that intermediaries and other end points have to
support a certain functionality
  Hence, the M-bit has various functionality mixed into a single piece
of information. 
* the usage of "MAY/SHOULD use the M-bit" in the AVP flag table isn't
really considered in the extensibility story for Diameter
* adding functionality to resolve some of these problems isn't so easy
given that these are pretty core protocol features (hint: backward
compatibility).

I don't see an easy way to solve some of these issues. I wonder whether
anyone of you has an opinion about this rather complicated subject. 

Ciao
Hannes

PS: It might be good to start with a list of things we want and do not
want: 

* We clearly don't want to define new applications with every tiny
extension. Consider the following example. Think about the Diameter QoS
attribute draft. Imagine that we would have to define an application.
This would mean to define it to run it in the following environments: 
   - Diameter EAP + Diameter QoS attributes 
   - Diameter NASREQ + Diameter QoS attributes
   - Diameter CC + Diameter QoS attributes 
   - Diameter Mobile IP + Diameter QoS attributes
   - etc. 

* We don't want to decide during design time whether intermediaries
should be involved.
The previous bullet shows the dilemma. The application ID is used to
tell intermediate nodes to participate in a particular exchange. On the
other hand we would like to have an easy way for network operators to
configure their nodes to process certain extensions. One way is
obviously to have these intermediaries to look through the entire
Diameter message for the presence (or absence) of certain AVPs. 

* It would help protocol designers to have an e2e capability indication
mechanism. 
With the way how people extend Diameter applications already today I
don't see a way to avoid having such a capability. The question is only
whether we want to define a generic mechanism (even though it is a bit
late). 
_______________________________________________
DiME mailing list
[email-masked]
http://www.ietf.org/mailman/listinfo/dime

---

From: "Asveren, Tolga" <[email-masked]>
Date: Mon, 11 Feb 2008 13:37:19 -0500

Hi Hannes,

Please see below for comments/questions.

Thanks,
Tolga

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On Behalf
Of
> [email-masked]
> Sent: Monday, February 11, 2008 10:08 AM
> To: [email-masked]
> Subject: [Dime] Diameter Extensibility & M-Bit Usage
> 
> Hi all,
> 
[TOLGA]IMHO whether the AVP is actually used in the messages at all is
orthogonal to whether support for it needs to be present (and whether it
would force a new applicationId to be defined). The latter is determined
whether the M-bit is set for the new AVP.
> probes to determine what went wrong).
[TOLGA]Are we talking about "Capability Exchange" when we refer to
"Diameter Base protocol mechanism to determine support"? If so, it is
only a hop-by-hop mechanism. I think it could be used to propagate
information about supported applications end-to-end if the
intermediaries are intelligent enough to update what they support(but
then there are relay agents which advertise support for all
applications) (and the mechanism lack the capability to signal support
for specific application per realm).
 
> well along with this mechanism.
[TOLGA]I guess your worry is whether combinations of supported
extensions should be enumerated in advance by a standards organization
(defining a new ApplicationId for each of them) or should be signaled as
listed exclusively by endpoints (maybe via an AVP).
> header. Comparing this approach with XML and Relax NG schemas one
could
[TOLGA]I am not sure whether it would really correspond to
Require/Require-Proxy header. The field containing the information used
for routing is ApplicationId, just the content of the field changes.
Being able to route based on this information is part of the base
protocol.
> functionality that could be used for a specific protocol run.
[TOLGA]I think, the way Diameter base protocol stands right now, there
is little room to negotiate capabilities outside of as indicated by
ApplicationId. Anything other than that (e.g. with an AVP) would be
negotiating optional capabilities, whose lack of support shouldn't cause
the session to be terminated (because routing logic is unaware of these
optional capabilities and could route initial messages always to a
non-supporting endpoint)
> Diameter application was to simplify progressing at Diameter proxies.
[TOLGA]Not only simplification. Routing based only on ApplicationId does
not require software updates on intermediaries as well.
> configuration time or design time decisions rather than design time
[TOLGA]Again, I think your real worry is whether combination of
capabilities should be enumerated in advance by SDOs; and I share your
concern.
> really considered in the extensibility story for Diameter
[TOLGA]IMHO
i- If there is new AVP where M-bit may/should/must be set ==> new
ApplicationId
ii- If for an existing AVP, M-bit setting changes so that it was
previously not possible to set it and now it is ==> new ApplicationId
>    - Diameter Mobile IP + Diameter QoS attributes
[TOLGA]Yes, I think all this should not require a new ApplicationId (but
it currently does).
> whether we want to define a generic mechanism (even though it is a bit
[TOLGA]Currently the only way is to define a new ApplicationId. I don't
think being late should be an issue here if a new mechanism is designed
carefully:
i- Should not cause existing nodes to break
ii- Should allow the endpoints to determine, whether the message was
routed by making use of this new mechanism
iii- Should not require intermediaries which are capable of routing
according to the new mechanism to be upgraded to support new
extensions/applications.
iv- Scope of changes signaled by extensions should be
well-defined/limited

---

From: Hannes Tschofenig <[email-masked]>
Date: Thu, 14 Feb 2008 14:54:29 +0200

Hi Tolga,

Asveren, Tolga wrote:
>> as part of finishing the Diameter Base protocol-bis document we ran
>>     
> into
>   
>> Diameter client, server, proxy, or translation agent and either the
>>     
> AVP
>   
>> Let us, for the moment, focus only on the usage of the M bit for the
>>     
> AVP
>   
>> bit but one can certainly say that a specification indicating within
>>     
> the
>   
>> For the same application a couple of AVPs might be optional in the
>>     
> ABNF
>   
>> but the definition of the AVP flag table may allow the M bit to be
>>     
> set.
>   
>> indication" via probing. Having the AVP optional in the ABNF and at
>>     
> the
>   
>> be understood (= implemented) but not necessarily used. This
>>     
> implements
>   
>> the classical: mandatory to implement - optional to use behavior.
>>     
> Hence,
>   
>> for a newly defined Diameter application it is fine to indicate all
>>     
> AVPs
>   
>> as mandatory to implement (if this is the intention of the authors).
>>     
> An
>   

I should introduce some terminology here:

Required AVP: AVP that is listed in the ABNF with {}.
Mandatory AVP: AVP that has the M-bit flag set.

Hence, in a case of required AVP then it is obviously mandatory as well 
(regardless of the setting of the M-flag).

Hence, with the current extensibility rules you have a problem:

* When you have AVPs with the M-bit set in the "MUST" column then it 
just means
   "MUST implement/MAY use". According to the extensibility rules a 
mandatory AVP would require a new application.

* When you have AVPs with the M-bit set in the "MUST NOT" column then 
this is purely optional.

* The semantic of the "SHOULD" and "MAY" column is not really well 
described.

>> available then there are essentially two points in time when to agree
>>     
> on
>   
>> functionality. Based on the configuration of the stack certain AVPs
>>     
> are
>   
>> sent whereas others' aren't. We have seen this approach happening a
>>     
> lot
>   
>> Unfortunately, this functionality is provided only in a very basic
>>     
> form
>   
>> since there is little feedback from the receiving end (and
>>     
> intermediary)
>   
No I am not talking about the Diameter base exchange capability 
exchange. I should have used a different term.
This is about an E2E exchange built into applications by some protocol 
designers.

>> extensibility story. When a new application is extended by new AVPs
>>     
> then
>   
>> extending it). When you, however, only make use of the extension
>>     
> points
>   
>> "end-to-end capability" mechanism used in some of our documents and
>>     
> that
>   
>> rather than a specific header field or even a specific value in a
>>     
> header
>   
>> field. Corresponding error codes are defined in the SIP specification
>>     
> as
>   

I would like to have protocol designers to have a better understanding 
of the tradeoffs of certain design decisions.
Creating monolithic applications is fine when intended that way. 
However, it took us a long time to figure out how to extend Diameter for 
our purpose; I could imagine that many other SDOs have similar problems.

>> information many things can go wrong and could be misinterpreted by
>>     
> the
>   
>> receiving party. Hence, there are a large number of specific error
>>     
> codes
>   
>> that can be used.
>>
>> with the M bit set being added to an existing application that
>>     
> requires
>   
>> a new Diameter application to be defined. When comparing this to SIP
>>     
> one
>   
>> Unfortunately, it is not quite the same since the application ID is
>>     
> used
>   
>> header. Comparing this approach with XML and Relax NG schemas one
>>     
> could
The problem with the Application ID is that many people do not want to 
allocate a new Application ID with every tiny extension.
The main reason is based on the fact that you essentially require 
separate exchanges for different Diameter applications.

Furthermore, the usage of M-bit is (when you look at existing Diameter 
applications) almost random.
Hence, it cannot seriously be used by a proxy to determine the behavior.

>> new XML / Relax NG schema and these schemas are typically registered
>>     
> as
>   
>> the XML usage I have been using in the IETF was an end-to-end matter
>>     
> and
>   
>> schemas do not allow individual attributes or elements to be
>>     
> registered
>   
>> separately. Merging of several applications in a single Diameter
>>     
> message
>   
>> functionality of another Diameter application. When you want to
>>     
> develop
>   
>> them independently then you might encounter an interesting
>>     
> extensibility
>   
>> extension. Currently, capabilities are described using different
>>     
> levels
>   

I agree with your observation. Backwards compatibility is the challenge.

> not require software updates on intermediaries as well.

Well. The software update comes when you want todo something meaningful 
based on the intercepted Diameter application messages.

>> Instead of having to parse through the entire message in order to
>>     
> figure
>   
>> out whether there is something interesting in there for them to
>>     
> process
>   
>> the SIP Proxy Require that would delegate the decision to set a
>>     
> certain
>   
>> mode from the specification designer to the person that deploys it.
>>     
> The
>   
>> It is kind of difficult to make a good decision on the M-bit and on
>>     
> the
>   

The problem is that this practice has not been followed within the IETF 
and also not by other SDOs.
What should we do with all these documents?

>> I don't see an easy way to solve some of these issues. I wonder
>>     
> whether
>   
>> extension. Consider the following example. Think about the Diameter
>>     
> QoS
>   
But the application ID is used to tell intermediaries to inspect and 
progress the messages.
Without a new application ID the proxy would have to parse through the 
entire message.
Essentially we would then move to a more flat architecture, like RADIUS 
has, and the
proxy would have to look at more information than just the Application ID.

>> tell intermediate nodes to participate in a particular exchange. On
>>     
> the
>   
>> * It would help protocol designers to have an e2e capability
>>     
> indication
>   
>> mechanism.
>> don't see a way to avoid having such a capability. The question is
>>     
> only
>   
There are different nodes in Diameter and some of them are not really 
deployed, such as Diameter proxies.

> iv- Scope of changes signaled by extensions should be
Good design goals.

Ciao
Hannes