Subject: RE: Operations crossing replication contexts

From: "John McMeeking" <[email-masked]>
Date: Mon, 1 Oct 2001 14:06:27 -0500

I'll defer to Steven on this.  I think these may affect the URP document,
but  even if they don't, we ought to define how to handle these cases, or
define constraints/expected behavior, somewhere in LDUP.

My original concern with respect to URP was the relationship of L&F to
replication contexts.  URP seems to assume there is a single L&F; for
example, 4.1 describes a pre-allocated UID for the Lost&Found entry.  The
information model implies there could be a single L&F for each replication
context.  On further reflection, it may not make any difference.  It may
only be necessary that a server maintain some association between an entry
in L&F and the replication context it came from.  Each replication
primitive is associated with a replication context.  If LDUP uses a
pre-allocated UID for L&F during a replication session, the consumer server
could have a single L&F, or multiple L&F.  In the case of a single L&F, the
consumer could "tag" the entry with a replication context.  In the case of
separate L&F for each replication context, the consumer places the entry in
the proper L&F.  In either case, the replication session need only flow the
pre-allocated L&F UID and the replication context.

The move scenario that Steven brought up complicates this analysis.  There
might be two replication contexts associated with an operation.  URP works
fine for handling the replication conflicts (example below), but we may not
have enough information for administrative repair to work properly.
Specifically, when an adminstrator performs repair via LDAP client
operations, under what replication context does that operation get
replicated?  This applies to any operation that involves multiple
replication contexts.

A brief example of the move scenario, with:

2 replication contexts: ou=engineering and ou=manufacting;
3 servers:
   server 1 contains both contexts
   server 2 contains ou=engineering
   server 3 contains ou=manufacturing
   server 4 contains both contexts
move cn=coyote from engineering to manufacturing (do this at server 1)

LDUP replicates a MoveEntryPrimitive.  What replication context is the
primitive associated with?  Ignoring that for a moment...

On server 2, a glue entry is created in L&F to represent ou=manufacturing,
and cn=coyote moved under that glue entry.

On server 3, a glue entry is created for cn=coyote, and it is placed under
ou=manufacturing.  Figures - last time I read this I thought that glue
entries were only present in L&F.  If I followed this correctly, glue
entries can show up elsewhere.

On server 4, everything works fine.

URP handled this in a reasonable way.  Maybe we think the outcome is fine;
if so, we should say this somewhere (model?).  If we don't think the
original client operation should have been allowed, that is a change (but
not to URP).  If we think that cn=coyote should have been created on server
3 (rather than a glue entry),then there is work to be done to a few
documents; this probably requires change to the protocol, but possibly not
to URP.

John  McMeeking
OS/400 Directory Services
(507)253-4596 (T/L) 553-4596

                    "John Strassner"                                                                              
                    <john.strassner@intel       To:     John McMeeking/Rochester/IBM@IBMUS, <[email-masked]>   
                    liden.com>                  cc:                                                               
                                                Subject:     RE: Operations crossing replication contexts         
                    09/29/2001 03:55 PM                                                                           
                    Please respond to                                                                             
                    john.strassner                                                                                

Hmmm, just when I was about to call to see if all of the comments were
finished on URP, up pops this message. So I have a quick question to John,
Steven, and others that are interested in this thread: is it your aim to
resolve this question as part of the URP Last Call, or is this something
that you think affects other documents than URP?

thanks and kind regards,
John

-----Original Message-----
From: [email-masked]
[mailto:[email-masked]]On Behalf Of John McMeeking
Sent: Friday, September 28, 2001 6:52 AM
To: [email-masked]
Subject: Operations crossing replication contexts

Thought I'd move a discussion between Steven Legg and myself into the
public eye...

This particular discussion is about how to deal with operations that cross
replication contexts:
- move an entry/subtree from one replication context to another
- delete/rename entries that ancesters of replication contexts (mentioned
at March IETF meeting)

It hasn't progressed to the point of proposing concrete solutions just
yet.

With respect to moving entries between replication contexts, we note that
this has to be initiated on a server that has contains updateable replicas
of both contexts, else the server would fail the client request.  However,
others servers may have replicas of only one of the contexts.  Perhaps
such
an action shouldn't be allowed. Perhaps it should be allowed only if the
initial server can detect (based on replicaSubebtries) that both contexts
are present on all relevant servers.  Or it requires a bit more invention.

With respect to the second point, this has to do with how a server knows
what the boundaries of a replication context are, then how the server(s)
should behave.  An engineering group was formed at the March IETF meeting.
I couldn't attend the first get together, and I am not aware that anything
happened afterwards.  An example of the situation:
- renaming the ancester of a relication context has the effect the
renaming
the child replication context.  The server where the ancester is located
may not be aware that the child context even exists.  Should this be
allowed?  How would replication handle it?

John  McMeeking
IBM

----- Forwarded by John McMeeking/Rochester/IBM on 09/28/2001 08:31 AM
-----

                    "Steven Legg"
                    <steven.legg@adac       To:     John
McMeeking/Rochester/IBM@IBMUS
                    el.com.au>              cc:
                                            Subject:     RE: Single lost &
found (RE: WG Last Call on URP
                    09/28/2001 12:01         I-D)
                    AM
                    Please respond to
                    steven.legg

John,

Apologies for taking my time responding.

> Pondering this a bit more...
>
> The problem of cross replication context operations could be
> rather crudely
> solved by stating that LDUP does not support moving entries
> into or out of
> areas of replication.  You must delete entries from the source, and
> recreate them in the target.  Hopefully an implementation
> would provide
> some tools to make that exercise simpler.

I'm not sure requiring delete/insert for moves between replication
contexts
actually solves more problems than it creates.

Moves by delete/insert are something I want to avoid, in part because
it makes moving subtrees expensive and dangerous. Expensive because
every entry in the subtree has to be deleted and reinserted, and
dangerous because other updates to entries in the subtree could get
lost or corrupted. Recovery from system crashes in the middle of a
subtree move also has to be dealt with.

Since the propagation of updates between servers is not instantaneous,
we would still have to deal with the possibility of a clash between
an already propagating move update and a propagating administrative
action to repartition the replication area. Propagation delays and
conflicting changes also make it possible for a server to perform
what looks to it like an intra-context move but which to another server
appears to be an inter-context move. This can happen even if both
servers have full knowledge of the replication context boundaries.

I also have concerns with any solution where consumer DSAs making
secondary shadowing agreements can hamstring the operations of the
original supplier DSA, i.e. by creating additional replication context
boundaries across which the original supplier can no longer do simple
moves. In X.500 replication (which we implement) a supplier DSA doesn't
need to have any knowledge of the secondary replication agreements
a consumer might make. This is a good thing for scalability in large
distributed environments.

My preferred approach is to have a go at solving the problems with
entries moving in and out scope for partial replicas, and apply that
solution to entries moving in and out of replication contexts, since
the problem spaces appear to be related.

I haven't been devoting much time to thinking about replication in recent
times but an LDUP implementation is on the agenda for our next release
so I will be spending more time on it. In the last couple of days I've
been following up on the idea of tagging entries with the replication
areas in which they appear. This approach is looking promising.
I'll let you and the rest of the working group know what I discover.

>
> That still leaves the question of how a server knows when an operation
> crosses such boundaries.  I asked that question at the March
> IETF meeting
> in Minneapolis.  An engineering group was formed (to include LDAP-EXT
> folks), and far as I can tell, never met.  I've had various
> thoughts on the
> matter.

Except for the effects of propagation delays I don't see a serious problem
in deciding if a move crosses a boundary, but maybe we're not talking
about the same thing. A server can only action a move operation if it
masters both the old and new superior entries. If it masters those
entries then it will have the necessary information about the replication
context of each. A new superior entry in a child replication context
held in the same server presents no problem. If the child replication
context is primarily held in a different server then the superior server
either doesn't master the new superior so it will disallow the move, or
it replicates the subordinate server's replication context in which
case it has the necessary information.

> -  One idea was that each server participated in a
> domain-like entity,where
> the definition of the domain resides in LDAP as a special area of
> replication.  Each server then can has knowledge of all the
> replication
> contexts and all the servers.  Alternately, x.500 glue
> entries might be
> used.  My sense is that this type of approach implies more distributed
> directory function than the LDUP members want.
> - At a minimum, it seems that each server needs to know the
> bounds of the
> replicas it hosts, that is the names of subordinate
> replication contexts.
> I envision this being done through something like an
> operational attribute
> of a replication context.  When a child area of replication
> is created, the
> "childReplicationContext" attribute of the parent replication
> context is
> updated, and that change replicated to other replicas of the parent.
> Deleting a child replication context may require
> administrative action, as
> I don't see how any single server could know that it held the last
> remaining copy of the child area of replication (unless this
> happens on a
> server that holds both the parent and child replication contexts).  I
> suppose creating a child area of replication might require
> administrative
> action too, unless we require that the child be created on a
> server that
> also holds the parent.  And as I think this through more, I
> suppose that
> some sort of glue entry would do the trick here (possibly
> with a reference
> to some server containing that replication context).
>
> Either way, it implies that something holds knowledge of the entire
> replication topology, and that a tool hide as much of the nastiness as
> possible.
>
> Any thoughts?  And I suppose I really ought to put this
> discussion on the
> LDUP mailing list.  Do you agree?

It wouldn't hurt.

Regards,
Steven

#### smime.p7s has been removed from this note on October 01 2001 by John
McMeeking

---

From: "Steven Legg" <[email-masked]>
Date: Wed, 3 Oct 2001 19:03:25 +1100

John,

> that you think affects other documents than URP?

The question of immediate relevance to the last call on URP is whether
there is a single Lost & Found per server, or per replication context.
However there is a more general problem that the current LDUP documents
don't adequately address partial replication. I regard two or more
replication contexts held in the same server as an example of partial
replication since moving an entry between replication contexts is
essentially the same problem as a modification altering whether an entry
appears in a sparse replica. We don't have a documented solution in
either case.

If the current LDUP specification can only be effectively implemented
for servers holding one replication context and participating in no
partial replication agreements then the question of whether there is
one L&F per server or one L&F per replication context is rather academic
since the total is one in both cases.

The choice before us is to either:

(1) Issue a first round of LDUP RFCs that address the particular case
of only one replication context per server and no partial replication
agreements. We can issue a second round of RFCs that updates the first
set once we have a general solution for partial replication. Apart
from the amendments discussed during the last call period, URP is ready
for such a first round.

OR

(2) Not issue any RFCs until we have a solution for partial replication.

It is likely that a general solution for partial replication will have
some impact on URP. I don't envision any changes to the existing
procedures for processing replication primitives, but there might need
to be some extra primitives, and the procedures for applying user updates
will have to address changes that span multiple areas of replication.

Recently I've been thinking seriously again about LDUP partial replication.
In order to give an indication of the sort of impact partial replication
will make, here is a summary of some changes that would need to be made
to the LDUP architecture to apply the most promising ideas I've been
kicking around.

(i) Hierarchically identified replication areas (RAs). RAs form
subset/superset relationships. Servers enter into agreements to
replicate specific identified RAs. The RA has to exist before the
replication agreement is created.

(ii) An update vector per RA instead of per server. This gets around
severe restrictions on topology imposed by the Update Vector and
facilitates expedited changes.

(iii) Tagging replication primitives with the identifier of the RA
where the original client update was performed and restricting the
flow of replication primitives from servers holding subset RAs to
servers holding superset RAs.

(iv) Tagging each entry with the RAs it currently appears in
(it could be an operational attribute). Entries moving in and
out of scope of an RA are explicitly indicated by modifying the
tags. This is where extra replication primitives might come in.

At the moment I not sure if a have a foolproof way for a server
holding a subset RA to get all the relevant content for an entry
that has just moved into the scope of the subset RA. Glue entries
in sparse replicas also complicate the picture. I'm looking at
both a "push" model where the server holding the superset RA sends
the entry contents it *thinks* the server holding the subset RA
*might* need, and a "pull" model where the server holding the
subset RA asks the server holding the superset RA for the information
it believes is missing. Either method implies extensions to the
replication protocol.

When I think I have a workable solution I'll write up the details
and post to the list.

Regards,
Steven

---

From: Christopher Apple <[email-masked]>
Date: Tue, 9 Oct 2001 14:45:21 -0400

Sounds good. Please revise and publish the URP spec based
on all comments but those relevant to this thread and we'll
consider this WG Last Call of URP over officially - it was
a while ago actually, but discussions kept going several days
past the actual deadline...

However, until we know specifically what the impact
could be on the URP document, I don't plan to recommend
that the Apps ADs review it for IESG consideration until
after the subsequent postings relevant to this thread.

And then, we may have to do another revision of the URP
document with a subsequent WG Last Call based on any changes
that the WG feels need be made to it.

If after discussing the additional postings that you
plan to make the WG indicates that no additional changes
need to be made to URP, John and I could make a recommendation
to the Apps ADs at that time to review the document without
further changes or another WG Last Call.

Chris Apple
Program Manager - Integration Services
United Messaging Inc.
<http://www.unitedmessaging.com>
<mailto:[email-masked]> 
(V) +1 610 425 2860

-----Original Message-----
From: Steven Legg [mailto:[email-masked]]
Sent: Monday, October 08, 2001 12:48 AM
To: 'Christopher Apple'
Cc: [email-masked]
Subject: RE: Operations crossing replication contexts

Chris,

Christopher Apple wrote:
> Strongly agree also. I can't see any real benefit to the folks
> who actually have to use LDUP implementations to have to cope
> with two ways of handling LDAPv3 replication depending on
> how many contexts per server they need.
> 
> Steven: when could you have a revised I-D posted to the list
> plus some bullets indicating changes implied for other documents?

I can submit a revised URP draft with the changes already agreed
during the last call period within a week or so, but for the changes
required to properly support partial replication, I think the best
approach is for me to write up the architectural details of what
I've been working on and post that to the list. I can do that in
parts over the next month. Once everyone has had a chance to think
about it we can decide what amendments need to be made to which
documents, including further changes to URP.

Regards,
Steven

> 
> Chris Apple
> Program Manager - Integration Services
> United Messaging Inc.
> <http://www.unitedmessaging.com>
> <mailto:[email-masked]> 
> (V) +1 610 425 2860
> 
> 
> From: Mark Smith [mailto:[email-masked]]
> Sent: Wednesday, October 03, 2001 9:14 AM
> To: Timothy Hahn
> Cc: [email-masked]
> Subject: Re: Operations crossing replication contexts
> 
> 
> 
> Timothy Hahn wrote:
> > 
> > Steven,
> > 
> > I really feel that LDUP should address the issues involved 
> in a single
> > server handling multiple replication contexts.  If we don't 
> I think we
> > would be miss the mark by providing a replication model 
> that would not
> > meet the needs of LDAP directory deployments.
> > 
> > I agree that providing this implies changes to all the replication
> > drafts - with exception, perhaps, of the requirements.  And 
> since this
> > probably has implications on the LDUP update protocol as well as the
> > information model, it seems to me we should take the time 
> to build it
> > right now rather than have to build a whole new "version" to support
> > mulitple replication contexts in a single server.
> 
> I strongly agree.  A directory server that supports only one 
> replication
> context, presumably for one contiguous subtree for the DIT, 
> will not be
> usable in many situations.
> 
> -- 
> Mark Smith
> Netscape
>