Subject: RE: WG Last Call on the LDAPv3 Directory Replication I-D - applications doing locking

From: "Ed Reed" <[email-masked]>
Date: Tue, 20 Feb 2001 19:01:27 -0700

This anecdote is not intended to make any important points, but
instead to merely relate a particularly - how shall I say it? - perverse
example of the law of unintended consequences.
==============================
>>> "Paul Leach" <[email-masked]> 02/19/01 06:50PM >>>

> -----Original Message-----
> From: John Strassner [mailto:[email-masked]] 
> Sent: Monday, February 19, 2001 10:27 AM
> To: [email-masked] 
> Cc: Kurt D. Zeilenga; [email-masked] 
> Subject: RE: WG Last Call on the LDAPv3 Directory Replication I-D
> 
> <js>
> Not true. First, applications can certainly build locking and other 
> mechanisms to isolate operations affecting an update.

Huh? The application instances could be operating on different replicas,
so will never even see the "locking" operations of the other. They will
proceed as if they were isolated, and then their updates will clash
laters. When the attributes being modified overlap partially but not
completely, neither one of them will get a consistent set.

[eer]
Worse than that - we've seen applications using client libraries that were
"overly agressive" when it came to load balancing across replicas, and
sometimes consecutive operations from the same application thread
would wind up going against different directory servers.  Pathological,
I'll admit, but just an example of the kind of trouble you can get into.

In this case, the client library (not an LDAP one, but the native client
for NDS at the time) handled all connection setup, authentication, cacheing
and teardown for applications to "preserve the location independence"
of the data in the directory.  A worthy objective, but one that is not
meaningfull when that client issued "create user" to one directory and
"create file system directory for user" to a file server on a different
machine.  The second create would fail until the first create propagated
via directory synchronization to the second server...utterly mysterious
to the poor application developer who just wanted to create a home 
directory for the user.  

Note that in this example there were NOT two directory operations
issued by the application, but rather a directory operation (the first
create of the user) and a file service operation whose side effect
was to check the validity of the user who would own the file directory
to be created.  The file system's resource ownership integrity logic
issued a directory operation (what's the user id of this user?) before
the user creation had occured at the directory replica coresident
with the file system (to which the file system was told to talk when
needing directory information like user ids).

The details of why the problem occured are wrapped up in the 
specific implementation details of NDS and the NetWare file service,
but they illustrate these points:

race conditions are made worse by loose consistincy replication
race conditions may occur as a result of side effects, not just by
   the primary actions of a single application nor even by the
   random operations of two independent operators
to help avoid race conditions, try to make sequential operations
   against a single server, at least until changes propagate to
   the other servers you would use (and use client libraries
   that either follow that behaviour for all the applications 
   operations and for the actions that will likely generate
   third party side-effects)

In this instance, the directory was not "at fault", but rather the
overly-helpful client libraries, optimized to spread queries against
multiple servers, didn't provide the degree of connection management
needed by the application who had "special" needs.

Library developers take heed!  (Actually, all the LDAP libraries
I know leave connection management to the application, so they're
not likely to suffer this same problem, but the potential effects of
side effects in the chain of events is a problem).

[/eer]

=================
Ed Reed
Reed-Matthews, Inc.
+1 801 796 7065
http://www.Reed-Matthews.COM

---

From: Jerry Combs <[email-masked]>
Date: Tue, 20 Feb 2001 19:55:48 -0800

This makes my point that only the commit to a specific replica should be
atomic. Anything beyond that must be the responsibility of the application.

JC