Subject: Re: [decade] draft-zhu-decade-additional-requirements

From: Richard Alimi <[email-masked]>
Date: Mon, 3 Jan 2011 12:06:06 -0800

Hi Xiao,

Thank you for the draft.  Some comments on the requirements:

Request Redirects:

This would provide some additional freedom for the storage provider.
I think it is reasonable since it doesn't necessitate additional
complexity for the DECADE server, but allows it if desired. However,
note that it may complicate some of the other components of the
design. In particular, if there is a per-user quota for storage, is
the user made aware of the quota at each in-network storage (if there
is no shared storage between them)?  Resource sharing policies may be
impacted (e.g., if a bandwidth sharing weight of 1 may mean something
different at DECADE server A than it does at DECADE server B).  At
this point it may be best to just note these so they are documented,
but since the specification of the resource sharing policies are still
being contemplated for the basic case of a single server it may be
premature to even try and solve them for the case supporting
redirection.

Multi-connection:

I'm not quite clear on the intention of the requirement.  My
understanding is that you wish the client to be able to have multiple
connections open spanning multiple DECADE servers. Is that correct?

If so, do we need this stated as a requirement or the protocol? Or is
this a policy that would be allowed/disallowed by the storage
provider?

Data Distribution:

I'm also confused about the intent of this requirement.  This
statement has me somewhat worried: "The distribution is transparent to
the users as they interact with the in-network storage as a single
logical system." Does this mean that you are proposing for DECADE
servers to assume the responsibility for deciding how data is to be
distributed throughout the network, including the path of the data,
which servers act as intermediate caches, content expiration policies,
etc?  If this is true, I think it is missing one of the major points
of DECADE. In particular, these decisions are application-dependent
and are not implemented within DECADE. Instead, DECADE provides the
basic capabilities and the functionality described above are
implemented by the applications using DECADE.

Or, am I misinterpreting the intent of the requirement?

Service Assurance:

It seems problematic to include "assurance" in a DECADE.  Shouldn't
these instead be part of SLAs with a storage provider?  Why should
they be DECADE's responsibility?

Regarding "resource reservation", are you referring to an actual
reservation (which might be problematic -- see above) or do you mean
that the resource share should able to be specified at the time the
connection opens and be assumed to be the same for the duration of the
connection?

Regarding Dynamic Feedback, is this orthogonal to the storage
protocol?  It seems like what you want here is a generic "status"
service saying how loaded a server is?

Data classification:

Would it be better to instead have the client specify properties of
the content instead of place it into ? See
www.ietf.org/proceedings/78/slides/nfsv4-0.pdf for an example of this
approach for NFS.

I think a problem with classifying applications is that it assumes
that applications fit one of the supplied classifications. What if it
may fit multiple classifications? or none?  Another problem is it
assumes that servers assume based on indirect and assumed information
that they know what to do with a particular piece of content. Why not
have an application specify its desires directly?

Small Objects:

What is the new requirement here?  Why is the existing requirement in
draft-ietf-decade-reqs-00 insufficient?

This also has me a bit worried:
"And the in-network storage has the limited storage capacity, with the
arrival of user requests and data distribution requirements, the data
stored in the in-network storage will be replaced if the storage
reaches the limit and data arrivals continually."

How does the server know what the proper replacement strategy is for
an application? I think DECADE's philosophy is that applications
should decide this. A simple way to do this is explicit deletion by an
application, but perhaps a more efficient (yet more complex) solution
is for an application to specify the replacement policy to the server.

Removal of Expired Records:

"However, the two scenarios did not mention how to handle the old
resource if the user distributes the new resource which is an updated
copy of the old resource."

Why does this need to be handled in the requirements?  Are you trying
to draw the distinction between immediate deletion of content and lazy
deletion?

Thanks!
Rich

On Mon, Dec 27, 2010 at 12:57 AM, 朱潇 <[email-masked]> wrote:
> Dear all,
>
> There is a slightly updated version of the decade additional requirements
> draft.
> https://datatracker.ietf.org/doc/draft-zhu-decade-additional-requirements/
>
> Jin Peng, Yunfei Zhang and me are expecting to have a discussion with all of
> you.
>
> Any comments are appreciated!
>
> A new version of I-D, draft-zhu-decade-additional-requirements-00.txt has been successfully submitted by Xiao Zhu and posted to the IETF repository.
>
> Filename:	 draft-zhu-decade-additional-requirements
> Revision:	 00
> Title:		 Additional protocol requirements on DECADE
> Creation_date:	 2010-12-27
> WG ID:		 Independent Submission
> Number_of_pages: 10
>
> Abstract:
> The DECoupled Application Data Enroute(DECADE)working group is
> specifying standardized interfaces for accessing in-network storage
> from applications to store, retrieve and manage data. The main object
> is to provide a framework that is useful to the applications,
> including P2P applications and other data-oriented applications,
> possibly related applications that can benefit from accessing in-
> network storage. This memo focuses on some requirements such as
> request redirecting and so on which are on the central of mobility,
> wireless network environment and CDN application. We present these in
> this memo as additional requirements that should be considered for
> the DECADE architecture and protocol specifications.
>
>
>
> The IETF Secretariat.
>
> --
> Best wishes,
>
> Beijing University of Posts & Telecommunications (BUPT)
> Zhu Xiao  ( 朱潇 )
> E-mail: [email-masked]
> mobile:+86 134-8881-9004
>
> _______________________________________________
> decade mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/decade
>
>

---

From: 朱潇 <[email-masked]>
Date: Tue, 11 Jan 2011 10:45:38 +0800

hi, Richard, sorry for so late response because of be busy with other
projects.

some discussion see inline:D marked by red.

On Tue, Jan 4, 2011 at 4:06 AM, Richard Alimi <[email-masked]> wrote:

> actually, you mean two points, right?
1. whether or not to be ware of the content at each in-network storage and
of course resource sharing policies can be impact in the request
redirection. your suggestion"just to note these so they are documented" will
be ok, or DECADE server list with some parameters will be return for user to
select the appropriate DECADE server, which can avoid the different weighted
of the same parameter in server decision. but the parameter used in select
the best DECADE server will be known for DECADE servers, in which other
complexities will be added. anyway, all the solution are
the implementation issue, which, i believe, does not impact the necessity of
the requirement.
2. you mean "the resource sharing policies are still being considered in a
single server", so it may be premature to support redirection.  the
architecture and protocol will be badly impacted if the requirements change.
so the designing can be  taken  step by step, but the requirements should be
overall considered.

> yep, your understanding is right, the scenarios were just described in
draft as "soft handover in wireless environment and delete operation in
multi-servers". under these consideration, the authentication
and authorization need to be taken each time when setup connection with a
new DECADE server, or just be taken only once during  the service?

> you mean DECADE provides the basic capabilities, so can you give
some specify explanations on DECADE capabilities in supporting data
distribution?

Service Assurance:
> thats right, actually, the flow control mechanism was under discussion in
writing the draft at first. what about your opinion in this requirements?

> actually, the key in "the data classification and small objects " is
whether does it or not in P2P application? i did not find data
classification was adopted in P2P application so far, let alone DECADE need
to classify the data used in all kinds of application.

> i mean the meaning of delete operation and how to handle the expired
records need to be clarify in requirements.

-- 
Best wishes,

---

From: Richard Alimi <[email-masked]>
Date: Mon, 10 Jan 2011 22:17:38 -0800

Hi Xiao,

On Mon, Jan 10, 2011 at 6:45 PM, 朱潇 <[email-masked]> wrote:
>
> hi, Richard, sorry for so late response because of be busy with other projects.
> 1. whether or not to be ware of the content at each in-network storage and of course resource sharing policies can be impact in the request redirection. your suggestion"just to note these so they are documented" will be ok, or DECADE server list with some parameters will be return for user to select the appropriate DECADE server, which can avoid the different weighted of the same parameter in server decision. but the parameter used in select the best DECADE server will be known for DECADE servers, in which other complexities will be added. anyway, all the solution are the implementation issue, which, i believe, does not impact the necessity of the requirement.

In general, I'd agree that the decision about where to redirect would
be an implementation issue.

>
> 2. you mean "the resource sharing policies are still being considered in a single server", so it may be premature to support redirection.  the architecture and protocol will be badly impacted if the requirements change. so the designing can be  taken  step by step, but the requirements should be overall considered.

I said that it is probably premature to consider how resource sharing
policies works across servers (or even if we need to say anything
about it) since we have not determined how to specify them (or rather,
how little we need to specify in protocol) for a single server. I did
not mean to imply that redirection could not be introduced as a
requirement.

> yep, your understanding is right, the scenarios were just described in draft as "soft handover in wireless environment and delete operation in multi-servers". under these consideration, the authentication and authorization need to be taken each time when setup connection with a new DECADE server, or just be taken only once during  the service?

The DECADE server should need to do some sort of check on each new
connection, regardless of whether the user has or previously had a
connection open to a different DECADE server, right?  Note that the
requirements don't indicate how authentication or authorization is
done, and allow a server to make optimizations if it is enforcing the
same permissions.

Can you indicate where the existing authorization-related requirements
(in particular, 4.1.6.3 of draft-ietf-decade-reqs-00) do not suffice
for the use case you are thinking of?

> you mean DECADE provides the basic capabilities, so can you give some specify explanations on DECADE capabilities in supporting data distribution?
>>

The problem statement gives a couple of scenarios. The "Integration
Examples" presentation from IETF 79 also has more details of an
on-going effort at Yale.

> thats right, actually, the flow control mechanism was under discussion in writing the draft at first. what about your opinion in this requirements?
>

I'm not sure what the typical way of providing such "load status"
information for IETF protocols, but my inclination is that it should
not be in the DECADE protocol itself.  Maybe someone else with a
longer history in IETF could jump in here :)

> actually, the key in "the data classification and small objects " is whether does it or not in P2P application? i did not find data classification was adopted in P2P application so far, let alone DECADE need to classify the data used in all kinds of application.
>

So, do you agree that it is problematic to try and classify each type
of application and try to specify the typical workload for each class?

My point was that I don't think its necessary to do the classification
at all.  It is more extensible (and directly useful) for a server to
just give it direct hints about what would be preferable.

> i mean the meaning of delete operation and how to handle the expired records need to be clarify in requirements.

My inclination is that "deleted" means that other requests the object
of the same name should result an error, until another object with the
same name is stored.

I think that the time the data is removed from disk (or memory) should
be up to the implementation. A storage provider might still have as
part of some agreement that deleted data will be removed within X
days/hours/minutes/whatever.

If people in the WG think it is necessary, we could have a requirement
that says the protocol should support an argument indicating immediate
deletion, but it is not clear that we need this.

Rich

---

From: 朱潇 <[email-masked]>
Date: Thu, 13 Jan 2011 10:00:03 +0800

---------- Forwarded message ----------
From: 朱潇 <[email-masked]>
Date: 2011/1/13
Subject: Re: [decade] draft-zhu-decade-additional-requirements
To: Richard Alimi <[email-masked]>

hi, Richard

see inline:D

2011/1/11 Richard Alimi <[email-masked]>

Hi Xiao,
> > 1. whether or not to be ware of the content at each in-network storage
> and of course resource sharing policies can be impact in the request
> complexities will be added. anyway, all the solution are the implementation
> issue, which, i believe, does not impact the necessity of the requirement.
>
> > 2. you mean "the resource sharing policies are still being considered in
> a single server", so it may be premature to support redirection.  the

> multi-servers". under these consideration, the authentication and
> authorization need to be taken each time when setup connection with a new
> DECADE server, or just be taken only once during  the service?
>
> as considering the service continuity, the next serving  DECADE server
> should know the progress of the service, how does they know? so if the
> service proceeding information should be known by the next serving server,
> or different servers need to coordinate and schedule each other to fulfill
> the user request, i believe the the 4.1.6.3 of the draft-ietf-decade-req-00
> cannot satisfy the req. what do you think about?
>
> >
> >
> >>
> >> Data Distribution:
> >>
> > you mean DECADE provides the basic capabilities, so can you give some
> specify explanations on DECADE capabilities in supporting data distribution?
> >>
>
> okay, thx for your information, i will lookup and refer, actually, the
> content publisher described in problem statement remind me of  the
> protocol requirements which i did not find in draft-ietf-decade-reqs-00 to
> support content publish. or i miss some points?
>
> >> Service Assurance:
> >>
> so can i understand that "load status" is kind of necessity  information
> for DECADE Server, but where and who collects the information are
> remain discussion?
>
> >>
> >> Data classification:
> >>
> yep, i believe it may be a little difficult, but worthy doing. specially
> for improving the resource utilization within a single server and reducing
> the response time for client. what about others opinion?
>
> >>
> okay, under the scenario "client uploads the new version, and did not
> specify how to handle the old version", did DECADE server delete the older
> version (or just label it unavailable, anyway, it is implementation issue )
> or not ? in another word, just replace the older version with the new
> version with the same name? how to handle the older version which is
> transferring from server to client?
>
   agree with you.

> >> > A new version of I-D, draft-zhu-decade-additional-requirements-00.txt
> has been successfully submitted by Xiao Zhu and posted to the IETF
> repository.
> >> >

-- 
Best wishes,

-- 
Best wishes,

---

From: 朱潇 <[email-masked]>
Date: Mon, 17 Jan 2011 10:01:55 +0800

hi, Richard:

thanks for your reply:D
additional discussion see inline:D

2011/1/14 Richard Alimi <[email-masked]>

> HI Xiao,
>
> 2011/1/12 朱潇 <[email-masked]>:
> > hi, Richard
> > see inline:D
> >
> > should know the progress of the service, how does they know?
>
> By progress of the service, do you mean current user state (e.g.,
> quota, recent resource usage history, permissions, etc)?
> yes, and include data delivery proceeding.
> > so if the
> Note that the protocol that is covered here is the client-server
> protocol. Some of the same protocol may be useful between DECADE
> servers (especially in different administrative domains) for storing
> and retrieving data, but that does not mean that there can't be
> additional protocol(s) (not specified here) that are used between
> DECADE servers as well.  For example, if DECADE servers within an
> administrative domain can certainly have their own mechanism to share
> such information.  If such capabilities were included in the DECADE
> protocol (e.g., a need to do this between administrative domains),
> that sounds like lots more complexity that we need at this point.
> but the access between in-network storage also is included in IAP described
> in problem statement.  i mean why not put this kind of function in DECADE
> since the IAP is defined just like that?
> That said, it sounds to me like what is described in 4.1.6.3 would be
> sufficient (from the perspective of the client-server protocol,
> anyways) to implement what you're describing. If not, what
> specifically is missing and what use-case does it not meet?
> so you mean the information you mentioned above, just like current user
> state (e.g.,
> quota, recent resource usage history, permissions, etc) was already
> included in DECADE requirement? what about the data delivery proceeding
> information? can you specify the chapter for me ? thanks?
> >> >
> >> >
> >> >>
> >> >> Data Distribution:
> >> >>
> Which requirements do you think are missing?
>
> >> >> Service Assurance:
> >> >>
> >> > thats right, actually, the flow control mechanism was under discussion
> >> > in writing the draft at first. what about your opinion in this
> requirements?
> >> >
> >>
> The storage provider is free to collect the information wherever and
> however they wish.  The DECADE server implementation could additional
> export whatever status information it wishes. I don't think the DECADE
> protocol needs to be concerned with that.
>
> Note that certain error conditions (e.g., overload, insufficient
> resources) may be signaled by a DECADE server (there are already have
> requirements for those).  If you are referring to status for a user's
> resources, there is already a requirement for that too.
>
> I'm just not convinced that the DECADE protocol needs to export its
> current load status to clients.
> actually i am not sure which elements in DECADE needs the load
> status,(DECADE client or network storage if the network storage needs to
> redirect the request or both?).
>

> And the requirement draft currently seems describe the overload condition
> as the event triggering. do you think if it is necessary to periodically
> reporting of the DECADE network status to client for its network storage
> selection?
>

> and the requirement draft just describe DECADE which is busy to serve other
> requests must be able to reject requests, not consider how to handle the
> user request after being rejected?
> >> >>
> >> >> Data classification:
> >> >>
> Can you justify why giving classifications (e.g., "I'm a live
> streaming application") would be better than giving specific hints
> (e.g., "please don't bother persistent these objects to disk")?
> i mean data in different kind of operation mode and attribute should have
> different user patterns and storage rules, which may be help to improve the
> resource utilization and reduce the response time for request, what are you
> understanding?
> > >>
> > version with the same name?
>
> In this case, I would think the "write" of the new object should be
> rejected, or if desired, we could have an overwrite option.  This
> could be added to the requirements if it helps to clarify.
> yep, no matter which solution is chosen, let the understanding unanimously
> :D
> > how to handle the older version which is
> I think it would be cleaner to ask the server to continue sending an
> object once it has been started, but ultimately this would be the
> decision of the server's implementation I think.  Maybe a "SHOULD"
> requirement.  This could be added if desired.
>
> Thank you for these questions.  It helps design the requirements more
> cleanly if there are specific scenarios in front of us :)
> just discussion together:D and also thanks for your points to help me
> understanding more!
> Rich
>
> >> >> > Jin Peng, Yunfei Zhang and me are expecting to have a discussion
> with
> >> >> > all of
> >> >> > you.
> >> >> >
> >> >> > A new version of I-D,
> draft-zhu-decade-additional-requirements-00.txt

-- 
Best wishes,