Subject: Re: [decade] URIs for DECADE -- Named Information URI Scheme

From: Hongqiang Liu <[email-masked]>
Date: Tue, 15 Nov 2011 17:11:34 -0500

Hi Dirk

I have just read these two documents. I think it is a good design.
I have some issues to throw here:
	-- Grouping. We have discussed about the objects grouping in DECADE-Arch design. A group might be only a property. I think this property is one of the query item with "?", e.g. group?movie-1. But I do not think it is clean because the group here is not a query.
	-- Batching. It is not clear that how to perform batch request with this scheme. If a request has multiple objects, we might need to specify their authority, algorithm and other properties together or separately. I think batch requests might be important for delay sensitive applications such as live streaming.

Thank you very much

Harry Liu

On Oct 26, 2011, at 9:06 AM, Dirk Kutscher wrote:

> Dear all,
> 
> We have updated the Named Information URI scheme that we presented at IETF-81.
> 
> It turned out that the WEBSEC folks (Phillip Hallam-Baker and colleagues) had started a parallel effort on a very similar approach, and -- fortunately -- we have been able to combine our efforts and have advanced the spec so that it would be useful for DECADE, WEBSEC and potentially other applications.
> 
> We still think that DECADE could benefit most at this time, so we submitted this (as an individual submission) to DECADE now.
> 
> We found a, IMO, nice way to have a concise base specification without excluding extensions for future application requirements. Basically, there is an extension mechanism that allows applications to specify additional parameters.
> 
> To make this manageable, we have split the specification into two pieces:
> 
> The base spec:
> 
> http://tools.ietf.org/html/draft-farrell-decade-ni-00
> 
> This document defines a URI-based name form that identifies a named
> object via hash-based binding.  The URI name form defined is intended
> for use in applications that need to uniquely identify resources in a
> location-independent way such as accessing in-network storage
> (DECADE), information-centric networking and more generally.  The
> format is designed to support a strong link to the referenced object
> such that the referenced object may be authenticated to the same
> degree as the reference to it.
> 
> The base spec has a set of useful general features (in addition to the actual syntax definition), such as processing rules (testing for equality), and an algorithm that can be used to map NI URIs to HTTP(S) automatically -- which I believe could be useful for DECADE, too. The base spec also defines the fundamentals of the extension mechanism.
> 
> 
> A spec that defines a set of extension parameters and their semantics:
> 
> http://tools.ietf.org/html/draft-hallambaker-decade-ni-params-00
> This document specifies some optional algorithms and parameters that
> may be used in the query string of ni URIs.
> 
> One of the parameters that we have introduced here would allow you to specify additional locators for resources -- again, that is expected to be useful for DECADE.
> 
> Looking forward, we envision that DECADE protocols may need additional parameters. Investigating this and specifying the parameters could be done in DECADE (should we decide to re-charter for that).
> 
> Please note that these two are individual submissions only -- i.e., merely proposals that the group of authors are offering at this point.
> 
> Nevertheless, we would be happy for any feedback.
> 
> Best regards,
> 
> Dirk
> 
> 
> 
> 
> _______________________________________________
> decade mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/decade

---

From: Dirk Kutscher <[email-masked]>
Date: Wed, 16 Nov 2011 03:30:47 +0000

Hi Harry,

Thank you very much for your comments.

> 	-- Grouping. We have discussed about the objects grouping in
> DECADE-Arch design. A group might be only a property. I think this property
> is one of the query item with "?", e.g. group?movie-1. But I do not think it is
> clean because the group here is not a query.

Yes, my general thinking is that grouping needs some discussion, i.e., to what degree DECADE needs a grouping concept.

I can imagine that *applications* would benefit from grouping, but we need to decide whether this needs to reflected on the protocol / naming level.

For example, if you want to group objects (say chunks) into an application context you could employ index files (think torrent files) and reference the individual chunks by names -- but the names would not need to expose any grouping information. 

I could imagine that it may be interesting to have the publisher / user information exposed in DECADE names -- the NI scheme has an optional authority element for this:

ni://example.com/sha-256;B_K97zTtFuOhug27fke4_Zgc4Myz4b_lZNgsQjy6fkc

> 	-- Batching. It is not clear that how to perform batch request with this
> scheme. If a request has multiple objects, we might need to specify their
> authority, algorithm and other properties together or separately. I think
> batch requests might be important for delay sensitive applications such as
> live streaming.

Interesting point.

In general, I'd say that being able to do batch requests rather depends on the protocol. I am not sure about the additional costs that would be incurred by having to specify authority, algorithm etc.

If we are working on the basis that each object is individually named, then you have to specify the name.

For the protocol, I think we ought to look into this. For example, for an HTTP-based DECADE protocol, we might want to make HTTP Pipelining mandatory.

Thanks again,

Dirk

> > It turned out that the WEBSEC folks (Phillip Hallam-Baker and colleagues)
> had started a parallel effort on a very similar approach, and -- fortunately --
> we have been able to combine our efforts and have advanced the spec so
> that it would be useful for DECADE, WEBSEC and potentially other
> applications.
> >
> > We still think that DECADE could benefit most at this time, so we submitted
> this (as an individual submission) to DECADE now.
> >
> > We found a, IMO, nice way to have a concise base specification without
> excluding extensions for future application requirements. Basically, there is
> an extension mechanism that allows applications to specify additional
> parameters.
> >
> > The base spec has a set of useful general features (in addition to the actual
> syntax definition), such as processing rules (testing for equality), and an
> algorithm that can be used to map NI URIs to HTTP(S) automatically -- which I
> believe could be useful for DECADE, too. The base spec also defines the
> fundamentals of the extension mechanism.
> >
> >
> > One of the parameters that we have introduced here would allow you to
> specify additional locators for resources -- again, that is expected to be useful
> for DECADE.
> >
> > Looking forward, we envision that DECADE protocols may need additional
> parameters. Investigating this and specifying the parameters could be done
> in DECADE (should we decide to re-charter for that).
> >
> > Please note that these two are individual submissions only -- i.e., merely
> proposals that the group of authors are offering at this point.
> >

---

From: Phillip Hallam-Baker <[email-masked]>
Date: Tue, 15 Nov 2011 23:11:16 -0500

On Tue, Nov 15, 2011 at 10:30 PM, Dirk Kutscher <[email-masked]>wrote:

> Yes, my general thinking is that grouping needs some discussion, i.e., to
> what degree DECADE needs a grouping concept.
>
> I can imagine that *applications* would benefit from grouping, but we need
> to decide whether this needs to reflected on the protocol / naming level.
>
> For example, if you want to group objects (say chunks) into an application
> context you could employ index files (think torrent files) and reference
> the individual chunks by names -- but the names would not need to expose
> any grouping information.
>
> I could imagine that it may be interesting to have the publisher / user
> information exposed in DECADE names -- the NI scheme has an optional
> authority element for this:
>

There are really two separate sets of technical requirements. The first
being when we are dealing with static data and the second when the data is
dynamic and the link is to a concept rather than a specific sequence of
bytes.

Lets leave the second set of requirements on the stack for the time being.
At the end of the road every request has to be satisfied by either an error
response or some sequence of bytes...

In the first case we can adopt a layered approach. The outer digest links
to a file that contains a sequence of digests for the inner digests or to
the root element in a Merkle tree or similar scheme.

In this case the only thing that is different is that the content type is
now 'list of digests'.

Another cute trick that has been proposed in the past that we could use if
someone can be bothered to go through the IP search is that the chain of
digests can be embedded in the content in such a way that the verifier can
check the data before reading the whole stream.

For example.

Digest0 = H ( Block 0 + Digest1)
Digest1 = H (Block 1 + Digest2 )
..
Digestn = H (Block n)

Stream =  Digest0 + Block0 + Digest1 + Block1 + ... + Digestn + .. + Blockn

This digest can only be calculated by reading in the whole stream because
Digest0 depends on every byte that follows. But Digest 0 can be verified by
reading only up to Digest1.

Alternatively, this type of option could be considered to be an encoding
variation or even a signature option.

-- 
Website: http://hallambaker.com/

---

From: Harry <[email-masked]>
Date: Thu, 17 Nov 2011 01:58:35 -0500

Hi Dirk

Thank you for your replies. Please see my comments inline.

> For example, if you want to group objects (say chunks) into an application context you could employ index files (think torrent files) and reference the individual chunks by names -- but the names w…

It is a good idea but I am thinking about how to maintain the meta files. These indexing files can also be very large and updated frequently. 

> For the protocol, I think we ought to look into this. For example, for an HTTP-based DECADE protocol, we might want to make HTTP Pipelining mandatory.

I remember that Apple is using play lists. Shall we do a similar thing?

---

From: Harry <[email-masked]>
Date: Thu, 17 Nov 2011 02:22:26 -0500

On Nov 15, 2011, at 11:11 PM, Phillip Hallam-Baker wrote:

> 
> 
> On Tue, Nov 15, 2011 at 10:30 PM, Dirk Kutscher <[email-masked]> wrote:
> Hi Harry,
> 
> There are really two separate sets of technical requirements. The first being when we are dealing with static data and the second when the data is dynamic and the link is to a concept rather than a specific sequence of bytes.
> 
> Lets leave the second set of requirements on the stack for the time being. At the end of the road every request has to be satisfied by either an error response or some sequence of bytes...
> 
> 
> In the first case we can adopt a layered approach. The outer digest links to a file that contains a sequence of digests for the inner digests or to the root element in a Merkle tree or similar scheme. 
> 
> In this case the only thing that is different is that the content type is now 'list of digests'.
> 
> 
> Another cute trick that has been proposed in the past that we could use if someone can be bothered to go through the IP search is that the chain of digests can be embedded in the content in such a way that the verifier can check the data before reading the whole stream.
> 
> For example.
> 
> This digest can only be calculated by reading in the whole stream because Digest0 depends on every byte that follows. But Digest 0 can be verified by reading only up to Digest1. 
> 

This is a good solution for streaming. But I think grouping is still needed for object aggregation (for purpose such as content routing).

> 
> Alternatively, this type of option could be considered to be an encoding variation or even a signature option.
> 
> 
> --