Subject: [Ecrit] deployment (was R:5)

From: Ted Hardie <[email-masked]>
Date: Tue, 12 Jul 2005 16:28:17 -0700

I think Henning is right that we may have multiple models of deployment in mind, and that both Andy and Henning are right in saying that we can't mandate or assume too much about the deployment.

But I think there are a few questions that we need to have some explicit statements
about in  order to avoid optimizing for the wrong cases.  We've all agreed that the
mapping step may occur at multiple points in the call flow.  For the case where the
mapping step occurs on a caller's behalf at a SIP proxy server (or similar for other
contact protocols), I believe we can agree that the proxy server probably won't be 
talking to a mapping server associated with the caller's access provider unless the 
access provider is also the voice services provider.  Instead, I believe, the proxy 
server will talk either to a mapping server maintained by its own organization or 
to the servers of the organization that is the source of the mapping data.

    So here's my assumption one:  the source of the mapping data runs
    one or more mapping servers, and these are available for querying.
    This is necessary for cache-fill replication, and it makes sense
    for local mapping servers that have to mix data from multiple
    sources.  It also provides a "default" server to query when a local
    mapping server isn't available (either because it isn't provided or
    beause it's down during the emergency).  It isn't strictly necessary
    from a protocol standpoint, since other servers can serve replicated
    data derived from other replication mechanisms, but I think we
    should optimize for the case where this is true.

In the proxy server case, a voice service provider's mapping server will likely
need to mix data derived from multiple mapping sources.  This will be true whenever
the organization's service boundaries cross those of mapping sources (certainly
true for multi-national and multi-State U.S. service providers, and likely true at
even smaller units).   Note that there is no protocol requirement that it get
that data using the same query protocol used by the caller; it's very likely
to be more efficient to get it by other means, either in an extension to the
protocol or through some other data transfer protocol.  Put another way,
data replication is a systemic requirement, not a requirement for the
query protocol itself.

For the case in which the mapping step occurs prior to the call flow being initiated,
the caller may contact a mapping server associated with the voice services provider,
one associated with the source of the mapping data, one associated with an
access provider, or one associated with some other party (in some countries, for
example, the government may run local mapping servers on behalf of the
public good).  The caller can, of course, only contact a mapping server that it
knows about and is reachable.

    Here's my assumption two:   callers will have multiple ways to derive
    knowledge about available mapping servers.  They may have one
    configured by their voice services provider.  They may learn one
    via DHCP from an access provider.  They may be able to locate the
    mapping server from something like slp if they are querying in advance
    of need.  As part of this assumption, I assume that any mapping server
    they reach can provide a referral to the correct server to query if it
    does not have the data locally; alternatively, it may query on behalf
    of the client and return the data.

For both cases, I believe we need to optimize for the situation in which the
caller has no data at all about emergency response URIs.  There are other cases
where the data is stale or not known to correspond to the current location, but
we're optimizing for the case where the caller needs something to start with.

I also believe we're talking about data where freshness is measured in large numbers
of minutes.  I believe if we go down to seconds (even in large numbers), we run the
risk of the call not completing before the information is stale; that's bad.  If we measure
in minutes we can handle the common case (reasonably long-lived), but also handle
the case where it needs to be shortened in order to handle a transition.  In the
DNS, for example, the common case is TTLs on the order of three days, but these are
shortened prior to changes to short times (e.g. 15 minutes), so that the extant data
ages out of cacches and is refreshed with long-lived data once the transition occurs.
A mechanism to allow pre-fetch of data before it is "live" is even better here,
since it replicates the data without waiting until it is known stale.  This reinforces
the point that the mechanism to query for current state and the mechanism for
keep server replicas up to date do not need to be the same, since this is protocol
element not interesting to the caller's mapping query interface.

I realize that many elements of this data do not change over very long periods (years,
as I understand it, for one common case), but I don't believe that means we should
measure in days or weeks.  If reasonably possible, we need to get fresh data to the
user and or server of replicated date.  That means taking a minor
hit on network traffic to increase the possibility the data is good; it's a tradeoff
but I think its the right one.

On another "taking a hit" front, I think we have to rule out any requirement that
the system provide a mechanism for winnowing data sets; if a replicated server
has to have full data sets available in order to provide the data it needs, it
may need to take the hit in terms of data storage and indexing.   
The number of potential database views is, after all, practically infinite, and 
burdening the authoritative source with the ability to generate them is onerous 
whatever the transfer mechanism may be.

There are clearly other assumptions we could tease apart here, but I thought I'd start
from these and get others' reactions.
           		 regards,
            			    Ted Hardie

_______________________________________________
Ecrit mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ecrit

---

From: Henning Schulzrinne <[email-masked]>
Date: Tue, 12 Jul 2005 22:51:24 -0400

Ted Hardie wrote:

> services provider.  Instead, I believe, the proxy server will talk
> either to a mapping server maintained by its own organization or to
> the servers of the organization that is the source of the mapping
> data.

I would add that the corporate outbound proxy could also be talking to 
the mapping service maintained by another vendor, such as a voice 
service provider offering enterprise-level services, that is itself not 
the (authoritative) source of the mapping data.

If we assume a model where authoritative data is distributed, it is 
unlikely that that proxy server will directly talk to such a server - 
simply because it has no clue who that server might be.

I'm probably missing what you mean by "source", but assume for a moment 
that mapping data is maintained by each country, just to keep the number 
of layers down. If a proxy needs to find out who provides rescue 
services for 123 Main Street, Ottawa, Canada, the client doesn't know 
the "source" of that data (the Canadian authority). It somehow needs to 
find that first-contact server that knows who handles Canada. One model 
is the DNS "root" model - start at the well-known (logical) top and 
traverse the tree.

> 
> So here's my assumption one:  the source of the mapping data runs one
> or more mapping servers, and these are available for querying. This
> is necessary for cache-fill replication, and it makes sense for local
> mapping servers that have to mix data from multiple sources.  It also
> provides a "default" server to query when a local mapping server
> isn't available (either because it isn't provided or beause it's down
> during the emergency).  It isn't strictly necessary from a protocol
> standpoint, since other servers can serve replicated data derived
> from other replication mechanisms, but I think we should optimize for
> the case where this is true.

My problem here is that there is no single source of mapping data, 
unless you consider a model where an organization, through some 
unspecified means, aggregates information for the whole world and makes 
it appear that it knows mappings for every street and location in the world.

> In the proxy server case, a voice service provider's mapping server
> will likely need to mix data derived from multiple mapping sources.
> This will be true whenever the organization's service boundaries
> cross those of mapping sources (certainly true for multi-national and
> multi-State U.S. service providers, and likely true at even smaller
> units).   Note that there is no protocol requirement that it get that
> data using the same query protocol used by the caller; it's very
> likely to be more efficient to get it by other means, either in an
> extension to the protocol or through some other data transfer
> protocol.  Put another way, data replication is a systemic
> requirement, not a requirement for the query protocol itself.

I suppose one could design DNS so that the client-resolver query 
protocol differs from the server-server protocol. As far as I can tell, 
the requirements for the two are close enough that I see no advantage, 
but lots of structural disadvantages, of doing so.

> 
> For the case in which the mapping step occurs prior to the call flow
> being initiated, the caller may contact a mapping server associated
> with the voice services provider, one associated with the source of
> the mapping data, one associated with an access provider, or one
> associated with some other party (in some countries, for example, the
> government may run local mapping servers on behalf of the public
> good).  The caller can, of course, only contact a mapping server that
> it knows about and is reachable.

The problem is that each such mapping server would have to know the 
whole world, thus essentially becoming a giant cache.

> configured by their voice services provider.  They may learn one via
> DHCP from an access provider.  They may be able to locate the mapping
> server from something like slp if they are querying in advance of
> need.  As part of this assumption, I assume that any mapping server 
> they reach can provide a referral to the correct server to query if
> it does not have the data locally; alternatively, it may query on
> behalf of the client and return the data.

I think we agree on that assumption. There clearly should be multiple 
entry-ways into the maze of mapping servers.

> I also believe we're talking about data where freshness is measured
> in large numbers of minutes.  I believe if we go down to seconds
> (even in large numbers), we run the risk of the call not completing
> before the information is stale; that's bad.  If we measure in

Agreed. As far as I can tell, load balancing between different PSAPs 
should be done at a different layer, such as the DNS lookup that occurs 
when the mapping protocol returns. There are well-known techniques, 
already specified in likely signaling protocols, that offer this 
capability.

> minutes we can handle the common case (reasonably long-lived), but
> also handle the case where it needs to be shortened in order to
> handle a transition.  In the DNS, for example, the common case is
> TTLs on the order of three days, but these are shortened prior to
> changes to short times (e.g. 15 minutes), so that the extant data 
> ages out of cacches and is refreshed with long-lived data once the
> transition occurs. A mechanism to allow pre-fetch of data before it
> is "live" is even better here, since it replicates the data without
> waiting until it is known stale.  This reinforces the point that the
> mechanism to query for current state and the mechanism for keep
> server replicas up to date do not need to be the same, since this is
> protocol element not interesting to the caller's mapping query
> interface.

I suspect you could design a different protocol, but I have a hard time 
seeing how this would allow us to build thousands, if not millions, of 
per-client-organization mapping servers, as you posit earlier. In other 
words, we would not be solving the real problem of having a deployable 
system without immediately turning around and designing that other protocol.

> 
> I realize that many elements of this data do not change over very
> long periods (years, as I understand it, for one common case), but I
> don't believe that means we should measure in days or weeks.  If
> reasonably possible, we need to get fresh data to the user and or
> server of replicated date.  That means taking a minor hit on network
> traffic to increase the possibility the data is good; it's a tradeoff

My perception is that the main changes are due to two factors: new 
subdivisions get build (frequent in some parts of the US, impossible in 
the town I live in), adding new entries, or service boundaries change 
(appears to be rare and is probably something where both old and new 
will work for a while). In any event, as long as we agree on the rough 
timescale, a factor of ten up isn't going to make much difference.

Adding new entries obviously does not impact cache freshness, just hit 
ratios.

There may well be a trade-off which is a bit different from other lookup 
applications, namely timeliness: If I have a result cached and can't get 
the new result within (say) a second, I'm probably better off routing to 
the old result rather than waiting. Total response time, from dialing to 
pickup by a human, has an SLA of less than ten seconds, after all, and 
you don't want to "burn" eight of those confirming, with 99% 
probability, the result you already had eight seconds earlier.

> 
> On another "taking a hit" front, I think we have to rule out any
> requirement that the system provide a mechanism for winnowing data
> sets; if a replicated server has to have full data sets available in
> order to provide the data it needs, it may need to take the hit in
> terms of data storage and indexing. The number of potential database
> views is, after all, practically infinite, and burdening the
> authoritative source with the ability to generate them is onerous 

A replica of an authoritative source definitely needs to have the full 
set for the region that it is authoritative for (not the whole world) - 
otherwise it isn't a replica. However, local non-authoritative copies 
don't need anything close to the full set to be useful in reducing the 
average response time under both routine and adverse network conditions.

> 
> There are clearly other assumptions we could tease apart here, but I
> thought I'd start from these and get others' reactions. regards, Ted
> Hardie
> 
> _______________________________________________ Ecrit mailing list 
> [email-masked] https://www1.ietf.org/mailman/listinfo/ecrit

---

From: Ted Hardie <[email-masked]>
Date: Wed, 13 Jul 2005 12:38:16 -0700

At 10:51 PM -0400 7/12/05, Henning Schulzrinne wrote:
>I would add that the corporate outbound proxy could also be talking to the mapping service maintained by another vendor, such as a voice service provider offering enterprise-level services, that is itself not the (authoritative) source of the mapping data.

It makes sense that people might outsource this, but I want to clarify if
you that service will be delivered with a box on the corporate network or
not.  That is, does Example corp contract with Invalid VSP for service and
get a local copy of the needed mapping service fed by some form of
data replication by Invalid VSP?  or does Example corp just point its proxies
to Invalid's mapping servers?

I think the "deployment advice" document wordage I've been suggesting go
with this might want to recommend something here....

>If we assume a model where authoritative data is distributed, it is unlikely that that proxy server will directly talk to such a server - simply because it has no clue who that server might be.

To clarify, distributed by locality, right?  Some agency in Campbell, Santa Clara County,
the state of California or the U.S is authoritative for where I sit in Campbell, CA and
a different one is authoritative when I'm in New Orleans, LA at my brother's house.

(We're not talking about, it other words, one source being authoritative for one *kind*
of data and one for another")

>I'm probably missing what you mean by "source", but assume for a moment that mapping data is maintained by each country, just to keep the number of layers down. If a proxy needs to find out who provides rescue services for 123 Main Street, Ottawa, Canada, the client doesn't know the "source" of that data (the Canadian authority). It somehow needs to find that first-contact server that knows who handles Canada. One model is the DNS "root" model - start at the well-known (logical) top and traverse the tree.
>

I've been thinking of it in registry terms.  There is a registry of mappings for a
specific locality that is authoritative for that locality.  That registry serves
data on its information by those making queries to it or by data replication
to other servers interested in serving queries for its locality.  The mapping
servers which are not registries have data derived from one or more registries
and make it available using the same query protocol.  Either the registry or
a non-registry mapping server may provide a referral to another registry
if it does not have appropriate data; it may do that by pointing to the
top of a hierarchy or by more direct referral (the latter would be more
efficient, if it maintains mappings below the top).

>My problem here is that there is no single source of mapping data, unless you consider a model where an organization, through some unspecified means, aggregates information for the whole world and makes it appear that it knows mappings for every street and location in the world.

I didn't mean "the source of all mapping data", I meant the "source of mapping
data for a specific area".  Sorry that that was not clear.  Aggregating all known
information should not be required for correct operation of the protocol, even
if there were a tree-walk way to accomplish it.

>I suppose one could design DNS so that the client-resolver query protocol differs from the server-server protocol. As far as I can tell, the requirements for the two are close enough that I see no advantage, but lots of structural disadvantages, of doing so.
>

Di you mean "DNS" above, of the mapping protocol?  DNS does have a different
query protocol from a server to server protocol; the server to server protocol
(AXFR) is designed to retrieve zonefuls of data, rather than using an opportunistic
cache fill.  A caching resolver can, of course, also use a standard query to fill
its cache.  But it's interesting to look at the motivation for the two transfer
mechanisms.  Using AXFR is a good way to get closely related information (the
data in a single zone/level of the DNS hierarchy); opportunistic cache fill
based on demand is a good way to get data based on usage patterns when
those patterns don't relate to a single zone.  Since you want the mapping
servers in different parts of the network topology to be able to serve all of
the data associated with the localities which they serve, I think the zone
transfer model (or full replication model, in non-DNS terms) makes more sense.
Even if a mapping server handles data from multiple registries, it can likely
handle the full data transfer pretty easily.

>The problem is that each such mapping server would have to know the whole world, thus essentially becoming a giant cache.
>

See above clarification.

>> knowledge about available mapping servers.  They may have one configured by their voice services provider.  They may learn one via
>>need.  As part of this assumption, I assume that any mapping server they reach can provide a referral to the correct server to query if
>I think we agree on that assumption. There clearly should be multiple entry-ways into the maze of mapping servers.
>
>Agreed. As far as I can tell, load balancing between different PSAPs should be done at a different layer, such as the DNS lookup that occurs when the mapping protocol returns. There are well-known techniques, already specified in likely signaling protocols, that offer this capability.
>
>>changes to short times (e.g. 15 minutes), so that the extant data ages out of cacches and is refreshed with long-lived data once the
>I suspect you could design a different protocol, but I have a hard time seeing how this would allow us to build thousands, if not millions, of per-client-organization mapping servers, as you posit earlier. In other words, we would not be solving the real problem of having a deployable system without immediately turning around and designing that other protocol.

Perhaps other protocol is too strong and "other mechanism" would be better,
but the point is that you don't have to use many, many individual queries
to get the data; you can use a more tailored mechanism for cases where
you want replication of all of locality's data.

>My perception is that the main changes are due to two factors: new subdivisions get build (frequent in some parts of the US, impossible in the town I live in), adding new entries, or service boundaries change (appears to be rare and is probably something where both old and new will work for a while). In any event, as long as we agree on the rough timescale, a factor of ten up isn't going to make much difference.
>
>Adding new entries obviously does not impact cache freshness, just hit ratios.
>
>There may well be a trade-off which is a bit different from other lookup applications, namely timeliness: If I have a result cached and can't get the new result within (say) a second, I'm probably better off routing to the old result rather than waiting. Total response time, from dialing to pickup by a human, has an SLA of less than ten seconds, after all, and you don't want to "burn" eight of those confirming, with 99% probability, the result you already had eight seconds earlier.

I agree that if you have some result, routing to it as soon as possible makes sense.
That was why I orginally stated that we were designing for the case where there
was no data available to the caller.

>>authoritative source with the ability to generate them is onerous whatever the transfer mechanism may be.
>
>A replica of an authoritative source definitely needs to have the full set for the region that it is authoritative for (not the whole world) - otherwise it isn't a replica. However, local non-authoritative copies don't need anything close to the full set to be useful in reducing the average response time under both routine and adverse network conditions.
>

I think this goes back to the global set misunderstanding before, but just in
case, let me elaborate.  You mentioned the case of a company with 20 franchises.
Let's assume that company wants to maintain a mapping server that covers
the locations of those 20 franchises as part of  its VoIP infrastructure.  Those
20 franchises are in 3 towns and two states, and there are a total of 3 registries
which have the authoritative mapping data covering the areas of the franchises.
I'm saying that it we should not make a systemic requirement that the
company's mapping server get custom data replication service from the authoritative
registries.  If it gets the data via data replication, it may well get data
it doesn't need, but that's the right trade off (versus having the responsible
registries generate a custom view (in database terms) for the query).

		thanks for your response,
					Ted

---

From: Henning Schulzrinne <[email-masked]>
Date: Wed, 13 Jul 2005 16:26:38 -0400

I think this discussion would benefit from a whiteboard or at least  
some slide diagrams, as I have the nagging suspicion that terminology  
and assumption either hide or amplify disagreements. There are only a  
relatively small number of tools in our big-picture design toolbox  
for distributed directory/mapping/lookup protocols, after all.

[snipping a lot]

> to Invalid's mapping servers?

Several options: Example corp could have its own mapping server and  
gets some subset of the data (where subset obviously includes "the  
whole world" as one option), either as-needed or ahead of time, i.e.,  
either push or pull.

To avoid misunderstandings: as-needed does not necessarily imply  
"when the user dials 9-1-1", but could be "when the user gets a new  
address/location and needs to verify that this is valid".

Clearly, pointing at the VSP is another option.

> with this might want to recommend something here....

I think it will be hard to recommend any specific solution - this  
will be driven by costs and other considerations. (If Example Corp  
pays per lookup, it might want to keep data locally to avoid paying  
for each validation, for example.)

I don't think it is particularly hard to support both push and pull  
models; as a community, we've done this type of stuff for 20+ years,  
on 8-bit hardware.

>
>
>> If we assume a model where authoritative data is distributed, it  
>> is unlikely that that proxy server will directly talk to such a  
>> server - simply because it has no clue who that server might be.
>>
>

Yes, distributed by locality.

> of data and one for another")

Not sure what you mean by "kind of data", but it is perfectly  
possible, as has been discussed here, that there's one hierarchy for  
marine rescue (probably not much coverage for the Alps) and another  
one for mountain rescue (may not be populated in Holland), managed by  
different (government) entities in terms of mapping data.

> efficient, if it maintains mappings below the top).

That sounds close to my mental model, but we probably need to start  
working on a common terminology to avoid needlessly wasting time on  
non-existing disagreements.

I'm trying to minimize the number of entities involved and unify them  
as much as possible, just to keep the interface count down. In my  
view, we have two kinds of entities:

- Authoritative: They contain a known-correct set of mappings  
directly managed by the data owner, but this set of mappings may not  
actually point to an emergency URI, but rather to another mapping  
server. The set of such servers can be large (e.g., one per PSAP in  
the US).

For example, if there's a US server, it might have 51 entries (both  
in civic and as polygons) outlining the states and DC, each with a  
references such as "ecrit:someserver.ca" (for CA). This leads to the  
standard referral model.

Alternatively, in the recursion model, the US server will make the  
query itself, returning an emergency URI to the querier. (The querier  
could obviously be another ECRIT server, e.g., the one operated by  
the United Nations for the whole world, if that's how the system ends  
up being built.)

- Non-authoritative: The server contains bindings (polygons, civic  
tuples) that it has learned from somewhere else, to speed up requests  
and to avoid that every request has to go to the "world root" and  
down again. The set of such servers can be very large - e.g., every  
enterprise could have one.

Learning can happen by push or pull, i.e., explicitly by getting the  
data from somewhere or by doing recursive queries and keeping the  
result for future use.

Again, this is not exactly new - LDAP, DNS and other directory-like  
systems all work pretty much like that. (Even SIP shares some aspect  
of this model, as it is partially a binding management protocol.)

As a first contact, a client needs to find a server that has bindings  
for the area it cares about. There is only a finite number of ways  
that I have seen that this can be done, essentially boiling down to  
"tall tree" and "forest" models. Again, a picture would help here.

> if there were a tree-walk way to accomplish it.

Glad we agree on that. I think we kind of have this in the  
requirements list, but this needs to be captured very clearly. I  
suspect that there are a number of people, particularly outside  
ECRIT, who will hear directory and think "BigEvilEntity-run white  
pages listing each PSAP in the world" (and hear the black helicopters  
buzzing in the background if BigEvilEntity = UN.)

> handle the full data transfer pretty easily.

See above. I think we need both models for scaling and avoiding the  
"go to root for each query" problem. I agree with you that these have  
different applications (full vs. partial/opportunistic replication).

> you want replication of all of locality's data.

I agree that just using a bunch of queries for replication is not  
likely to be efficient or appropriate, so it will at least be a  
different protocol operation.

> was no data available to the caller.

Maybe there's another requirement lurking here, as this differs from  
standard mapping/lookup protocols:

"If caching is used, servers should return these cached results if  
they cannot obtain a current mapping in a timely fashion."

If you can design a protocol without caching, this requirement  
obviously wouldn't apply to you, but then we might need to agree on  
something like

"It should be avoided that each query has to visit the same set of  
global "root" servers."

Motivation: reliability, single (logical) point of failure and juicy  
target for evildoers.

> registries generate a custom view (in database terms) for the query).

These servers can either get copies of the whole listing (as you  
suggest) or can learn the necessary mappings. The latter might work  
better for widely-distributed organizations. (For example, I believe  
JetBlue has its reservation agents work from home, anywhere in the US  
and, who knows, in India.)

Learning the mappings ahead of time isn't hard - the requirement for  
validation means that any VoIP client (or their proxy stand-in) with  
a new address will have to make a pre-need call to ECRIT in any event.