Subject: [Spasm] Change of Algorithms: Digest Algorithms

From: Jim Schaad <[email-masked]>
Date: Sun, 7 Aug 2016 19:34:53 -0700

This message looks at Digest Algorithms.  Please restrict your discussion to
this set of algorithms.  I will be sending out messages on other algorithms
over time.

Note:  Digest Algorithms are used for computing the digest of the content.
These are not the "same" as the digest algorithms that are used as part of
signature algorithms.  There may be things one wants to try and keep
similar, as we suggest that you use the same digest algorithm, but they are
not the same things.

Current Document for S/MIME 3.1

MUST: SHA-256
SHOULD-: SHA-1, MD5(for backwards compatibility)

Proposed for S/MIME 3.5

MUST: SHA-256
Historic: SHA-1, MD5

Questions:

1.  There is only one MUST and no path forward.  Is this a problem that
needs to be dealt with?  EdDSA25519 uses SHA-512 under the covers, does that
mean we should make SHA-512 at least a SHOULD here?

2.  Is there anybody who wants to standup and argue for adopting in any of
the SHA-3 algorithms?

Jim

---

From: "Salz, Rich" <[email-masked]>
Date: Mon, 8 Aug 2016 14:07:52 +0000

> 1.  There is only one MUST and no path forward.  Is this a problem that needs
> to be dealt with?  EdDSA25519 uses SHA-512 under the covers, does that

I'd like to see *something* as a should.    I have a slight preference for SHA512, but could live with SHA3 if others feel strongly.

---

From: Russ Housley <[email-masked]>
Date: Mon, 8 Aug 2016 10:49:14 -0400

I would like to see SHA-384 on the SHOULD list.

If we are going to have EdDSA25519 on the MUST list for signatures, then SHA-512 belongs on the MUST list for digest algorithms.  I’m also fine with both of these being SHOULD+.

Russ

On Aug 7, 2016, at 10:34 PM, Jim Schaad <[email-masked]> wrote:

> _______________________________________________
> Spasm mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/spasm

---

From: Gilles Van Assche <[email-masked]>
Date: Mon, 8 Aug 2016 18:51:18 +0200

Hi,

Why not consider SHAKE? It has good performance and has been publicly
evaluated.

Kind regards,
Gilles

---

From: Jim Schaad <[email-masked]>
Date: Mon, 8 Aug 2016 20:06:48 -0700

Given that SHAKE is really the same as SHA3, that would be part of the last
question that I asked.  SHAKE is not considered to be a hash algorithm, but
is instead an XOF (Extendable Output Function).  These do not have the quite
the same security claims as digest algorithms due to the arbitrary sized
output.  (I will modify the previous statement as saying this is my
understanding and may have nothing to do with reality.)

SHAKE256(M, d) is almost the same as SHA3-256.  (The only difference is the
unique algorithm marker).

Jim

> -----Original Message-----
> From: Spasm [mailto:[email-masked]] On Behalf Of Gilles Van Assche
> Sent: Monday, August 08, 2016 9:51 AM
> To: [email-masked]
> Subject: Re: [Spasm] Change of Algorithms: Digest Algorithms
> 
> Hi,
> 
> > This message looks at Digest Algorithms.  Please restrict your
> > discussion to this set of algorithms.  I will be sending out messages
> > on other algorithms over time.
> >
> > These are not the "same" as the digest algorithms that are used as
> > part of signature algorithms.  There may be things one wants to try
> > and keep similar, as we suggest that you use the same digest
> > algorithm, but they are not the same things.
> >
> >
> > 1.  There is only one MUST and no path forward.  Is this a problem
> > that needs to be dealt with?  EdDSA25519 uses SHA-512 under the
> > covers, does that mean we should make SHA-512 at least a SHOULD here?
> >
> > 2.  Is there anybody who wants to standup and argue for adopting in
> > any of the SHA-3 algorithms?
> >
> > Jim
> >
> >
> >
>

---

From: Gilles Van Assche <[email-masked]>
Date: Tue, 9 Aug 2016 14:17:02 +0200

On 09/08/16 05:07, Jim Schaad wrote:
> Given that SHAKE is really the same as SHA3, that would be part of the
> last question that I asked. SHAKE is not considered to be a hash
> algorithm, but is instead an XOF (Extendable Output Function).

Indeed. However, when restricted to a particular output length, the XOF
becomes a traditional hash function.

> These do not have the quite the same security claims as digest
> algorithms due to the arbitrary sized output. (I will modify the
> previous statement as saying this is my understanding and may have
> nothing to do with reality.)
>
> SHAKE256(M, d) is almost the same as SHA3-256. (The only difference is
> the unique algorithm marker).

The reason I mention SHAKE as opposed to the SHA3-* hash functions is
that SHAKE128 and SHAKE256 are faster for a given security strength.
I.e., SHAKE128 is faster than SHA3-256 for 128-bit security, and
SHAKE256 is faster than SHA3-512 for 256-bit security.

Kind regards,
Gilles

---

From: Sean Leonard <[email-masked]>
Date: Tue, 9 Aug 2016 16:44:35 -0700

> I'd like to see *something* as a should.    I have a slight preference for SHA512, but could live with SHA3 if others feel strongly.

I think SHA-3 should be SHOULD-.

Sean

---

From: Jim Schaad <[email-masked]>
Date: Tue, 9 Aug 2016 18:07:03 -0700

Why not just make it a MAY in that case or not mention it?

SHOULD-    This term means the same as SHOULD. However, the authors expect
that a requirement marked as SHOULD- will be demoted to a MAY in a future
version of this document.

Are you really saying it is going to go away in future versions?

jim

> From: Spasm [mailto:[email-masked]] On Behalf Of Sean Leonard
> Sent: Tuesday, August 09, 2016 4:45 PM
> To: [email-masked]
> > I'd like to see *something* as a should.    I have a slight preference
for
> SHA512, but could live with SHA3 if others feel strongly.
>

---

From: Sean Leonard <[email-masked]>
Date: Tue, 9 Aug 2016 21:16:15 -0700

whoops, my bad. I guess I meant SHOULD- to mean "not really SHOULD right 
now, but it's going to be SHOULD soon", which I guess would be like a 
MAY+ or SHOULD↑↑.

But there are no MAY+ or SHOULD↑↑ keywords. So, I suppose, just 
"SHOULD"? Unless you want to propose a new term.

Sean

---

From: Sean Turner <[email-masked]>
Date: Wed, 10 Aug 2016 12:14:53 -0400

I’m half expecting a lmgtfy link, but is SHAKE available in OpenSSL or on Widows or Macs?  Having folks generate code for a new content type to support AEAD algorithms is one thing and maybe a pretty big thing at that.  Being the first (correct me if I’m wrong here) IETF protocol to have a SHOULD SHAKE requirement seems like a requirement that’s going to get ignored.  If implementers don’t respond with a resounding chorus of "oh yeah! we were going to do that in our next release of product X" then I think that we should leave driving the SHAKE adoption train to some other more widely deployed/used protocol for now and S/MIME can piggy back off of it later.

spt

> On Aug 08, 2016, at 12:51, Gilles Van Assche <[email-masked]> wrote:
> 
> Hi,
>

---

From: Yoav Nir <[email-masked]>
Date: Wed, 10 Aug 2016 20:58:41 +0300

> On 10 Aug 2016, at 7:14 PM, Sean Turner <[email-masked]> wrote:
> 

Not in OpenSSL. AFAICT not in Windows or Mac library. You can find code online if you want to, though.

Looking at other places in the IETF, nobody’s rushing to write a “SHA-3 and its use in xxx” documents. In fact, there are none. Compare and contrast with SHA-2 documents when that algorithm was new or with ChaCha20/Poly1305 or Curve25519 documents more recently. 

The only document I’ve seen that uses SHA-3 is the EdDSA document where for Curve448 SHAKE256 is used internally.

So in a word, the IETF response to SHA-3 is “meh”

Yoav

---

From: Jim Schaad <[email-masked]>
Date: Wed, 10 Aug 2016 12:10:51 -0700

> From: Spasm [mailto:[email-masked]] On Behalf Of Yoav Nir
> Sent: Wednesday, August 10, 2016 10:59 AM
> To: Sean Turner <[email-masked]>
> Cc: SPASM <[email-masked]>
> > I’m half expecting a lmgtfy link, but is SHAKE available in OpenSSL or on
> Widows or Macs?  Having folks generate code for a new content type to support
> AEAD algorithms is one thing and maybe a pretty big thing at that.  Being the
> first (correct me if I’m wrong here) IETF protocol to have a SHOULD SHAKE
> requirement seems like a requirement that’s going to get ignored.  If
> implementers don’t respond with a resounding chorus of "oh yeah! we were
> going to do that in our next release of product X" then I think that we should
> leave driving the SHAKE adoption train to some other more widely
> deployed/used protocol for now and S/MIME can piggy back off of it later.
> 
> Looking at other places in the IETF, nobody’s rushing to write a “SHA-3 and its
> use in xxx” documents. In fact, there are none. Compare and contrast with SHA-
> 2 documents when that algorithm was new or with ChaCha20/Poly1305 or
> Curve25519 documents more recently.
> 

This is the only place that I have seen it yet.  There is also an issue of having OIDs created.  They exist for the SHA3 algorithms, but the level of OIDs needed to treat SHAKE as a hash algorithm don't exist the last time I went on a search.

Jim

---

From: Sean Turner <[email-masked]>
Date: Wed, 10 Aug 2016 17:32:40 -0400

> On Aug 10, 2016, at 15:10, Jim Schaad <[email-masked]> wrote:
> 
> 
> 

I knew I had seen some oids (if we need them):

http://csrc.nist.gov/groups/ST/crypto_apps_infra/csor/algorithms.html

spt

---

From: Sean Turner <[email-masked]>
Date: Wed, 10 Aug 2016 17:48:01 -0400

> On Aug 10, 2016, at 13:58, Yoav Nir <[email-masked]> wrote:
> 
> 

There's also a handful of validated products too:
http://csrc.nist.gov/groups/STM/cavp/documents/sha3/sha3val.html

But, yeah I'm still thinking SHAKE shouldn't be a SHOULD.

spt

---

From: Jim Schaad <[email-masked]>
Date: Wed, 10 Aug 2016 15:07:46 -0700

From: Spasm [mailto:[email-masked]] On Behalf Of Sean Turner
Sent: Wednesday, August 10, 2016 2:33 PM
To: Jim Schaad <[email-masked]>
Cc: SPASM <[email-masked]>; Yoav Nir <[email-masked]>
Subject: Re: [Spasm] Change of Algorithms: Digest Algorithms

On Aug 10, 2016, at 15:10, Jim Schaad <[email-masked] <mailto:[email-masked]> > wrote:

-----Original Message-----

To: Sean Turner <[email-masked] <mailto:[email-masked]> >

Cc: SPASM <[email-masked] <mailto:[email-masked]> >

 

On 10 Aug 2016, at 7:14 PM, Sean Turner <[email-masked] <mailto:[email-masked]> > wrote:

Not in OpenSSL. AFAICT not in Windows or Mac library. You can find code online

if you want to, though.

The only document I’ve seen that uses SHA-3 is the EdDSA document where for

Curve448 SHAKE256 is used internally.

Jim

Yoav

Yes – that gives you the OIDs for doing SHA3.  But if you are doing SHAKE, where are you encoding the length of the output?  That is what I meant by the OIDs for SHAKE do not exist.

Jim

spt

---

From: Gilles Van Assche <[email-masked]>
Date: Thu, 11 Aug 2016 12:05:01 +0200

It still surprises me that people stick to SHA-2, a family of algorithms
designed by NSA behind closed doors, while SHA-3
- comes with a design rationale;
- received many years of public scrutiny before it became a standard;
- has a thicker safety margin;
- has software performances similar to SHA-2 (or better performance at a
comparable safety margin);
- offers much better performance in many cases (e.g., hardware,
side-channel protections);
- excels at flexibility with simpler and more efficient solutions for
mask generating functions, PRF, MAC, PRNG, etc.

Kind regards,
Gilles

On 10/08/16 23:48, Sean Turner wrote:
>
> On Aug 10, 2016, at 13:58, Yoav Nir <[email-masked]
> <mailto:[email-masked]>> wrote:
>
>>
>>> On 10 Aug 2016, at 7:14 PM, Sean Turner <[email-masked]
>>> <mailto:[email-masked]>> wrote:
>>>
>>> I’m half expecting a lmgtfy link, but is SHAKE available in OpenSSL
>>> or on Widows or Macs?  Having folks generate code for a new content
>>> type to support AEAD algorithms is one thing and maybe a pretty big
>>> thing at that.  Being the first (correct me if I’m wrong here) IETF
>>> protocol to have a SHOULD SHAKE requirement seems like a requirement
>>> that’s going to get ignored.  If implementers don’t respond with a
>>> resounding chorus of "oh yeah! we were going to do that in our next
>>> release of product X" then I think that we should leave driving the
>>> SHAKE adoption train to some other more widely deployed/used
>>> protocol for now and S/MIME can piggy back off of it later.
>>
>> Looking at other places in the IETF, nobody’s rushing to write a
>> “SHA-3 and its use in xxx” documents. In fact, there are none.
>> Compare and contrast with SHA-2 documents when that algorithm was new
>> or with ChaCha20/Poly1305 or Curve25519 documents more recently.
>>
>> The only document I’ve seen that uses SHA-3 is the EdDSA document
>> where for Curve448 SHAKE256 is used internally.
>>

---

From: "Dang, Quynh (Fed)" <[email-masked]>
Date: Thu, 11 Aug 2016 10:24:33 +0000

Hi Jim,

I can get SHAKE128/256 and SHAKE256/512 (256-bit and 512-bit hash outputs respectively) assigned with new OIDs for you and the working group.

One of the beauties of the SHAKEs is that one can use a SHAKE  (1) to do full-domain hash RSA signatures or  (2) as a MGF very efficiently because the SHAKEs are one-pass construction.

SHAKEs are also very efficient PRFs, MACs and KDFs because they are one-pass construction; not like HMAC and other PRFs.  One can also have authenticated ciphers from the Keccak permuation inside the SHAKES and more.

Regards,
Quynh.

From: Spasm <[email-masked]<mailto:[email-masked]>> on behalf of Jim Schaad <[email-masked]<mailto:[email-masked]>>
Date: Wednesday, August 10, 2016 at 6:07 PM
To: 'Sean Turner' <[email-masked]<mailto:[email-masked]>>
Cc: 'SPASM' <[email-masked]<mailto:[email-masked]>>, 'Yoav Nir' <[email-masked]<mailto:[email-masked]>>
Subject: Re: [Spasm] Change of Algorithms: Digest Algorithms

From: Spasm [mailto:[email-masked]] On Behalf Of Sean Turner
Sent: Wednesday, August 10, 2016 2:33 PM
To: Jim Schaad <[email-masked]<mailto:[email-masked]>>
Cc: SPASM <[email-masked]<mailto:[email-masked]>>; Yoav Nir <[email-masked]<mailto:[email-masked]>>
Subject: Re: [Spasm] Change of Algorithms: Digest Algorithms

On Aug 10, 2016, at 15:10, Jim Schaad <[email-masked]<mailto:[email-masked]>> wrote:

-----Original Message-----
From: Spasm [mailto:[email-masked]] On Behalf Of Yoav Nir
Sent: Wednesday, August 10, 2016 10:59 AM
To: Sean Turner <[email-masked]<mailto:[email-masked]>>
Cc: SPASM <[email-masked]<mailto:[email-masked]>>
Subject: Re: [Spasm] Change of Algorithms: Digest Algorithms

On 10 Aug 2016, at 7:14 PM, Sean Turner <[email-masked]<mailto:[email-masked]>> wrote:

I'm half expecting a lmgtfy link, but is SHAKE available in OpenSSL or on
Widows or Macs?  Having folks generate code for a new content type to support
AEAD algorithms is one thing and maybe a pretty big thing at that.  Being the
first (correct me if I'm wrong here) IETF protocol to have a SHOULD SHAKE
requirement seems like a requirement that's going to get ignored.  If
implementers don't respond with a resounding chorus of "oh yeah! we were
going to do that in our next release of product X" then I think that we should
leave driving the SHAKE adoption train to some other more widely
deployed/used protocol for now and S/MIME can piggy back off of it later.

Looking at other places in the IETF, nobody's rushing to write a "SHA-3 and its
use in xxx" documents. In fact, there are none. Compare and contrast with SHA-
2 documents when that algorithm was new or with ChaCha20/Poly1305 or
Curve25519 documents more recently.

The only document I've seen that uses SHA-3 is the EdDSA document where for
Curve448 SHAKE256 is used internally.

Jim

So in a word, the IETF response to SHA-3 is "meh"

Yoav

Yes - that gives you the OIDs for doing SHA3.  But if you are doing SHAKE, where are you encoding the length of the output?  That is what I meant by the OIDs for SHAKE do not exist.

Jim

spt

---

From: Russ Housley <[email-masked]>
Date: Thu, 11 Aug 2016 10:10:29 -0400

Gilles:

I recognize some of the these advantages you point out.  last time I looked, SHA3 is less widely implemented and slower than SHA2.  You say it is faster.  Can you point me to the measurements?

Russ

---

From: Russ Housley <[email-masked]>
Date: Thu, 11 Aug 2016 10:13:27 -0400

Sean:

Please see RFC 7696 for the definitions being used here:

      SHOULD+  This term means the same as SHOULD.  However, it is
               likely that an algorithm marked as SHOULD+ will be
               promoted to a MUST in the future.

      SHOULD-  This term means the same as SHOULD.  However, it is
               likely that an algorithm marked as SHOULD- will be
               deprecated to a MAY or worse in the future.

      MUST-    This term means the same as MUST.  However, it is
               expected that an algorithm marked as MUST- will be
               downgraded in the future.  Although the status of the
               algorithm will be determined at a later time, it is
               reasonable to expect that a the status of a MUST-
               algorithm will remain at least a SHOULD or a SHOULD-.

Russ

On Aug 10, 2016, at 12:16 AM, Sean Leonard <[email-masked]> wrote:

> whoops, my bad. I guess I meant SHOULD- to mean "not really SHOULD right now, but it's going to be SHOULD soon", which I guess would be like a MAY+ or SHOULD↑↑.
> 
> But there are no MAY+ or SHOULD↑↑ keywords. So, I suppose, just "SHOULD"? Unless you want to propose a new term.
> 
> Sean
>

---

From: Michael Richardson <[email-masked]>
Date: Thu, 11 Aug 2016 10:32:10 -0400

Gilles Van Assche <[email-masked]> wrote:
    > It still surprises me that people stick to SHA-2, a family of
    > algorithms designed by NSA behind closed doors, while SHA-3 - comes

I tend to agree.

--
]               Never tell me the odds!                 | ipv6 mesh networks [
]   Michael Richardson, Sandelman Software Works        | network architect  [
]     [email-masked]  http://www.sandelman.ca/        |   ruby on rails    [

---

From: Sean Leonard <[email-masked]>
Date: Thu, 11 Aug 2016 15:42:55 -0700

Yes.

As I pointed out, I was going for "MAY+", which is not actually a term 
that exists (yet). Not sure if a MAY+ should be added to the 
definitions, or just leave it alone. My preferences are:
#1 MAY+
#2 SHOULD
#3 (omit)

As Gilles Van Assche mentioned, SHA-3 has a lot of design advantages. 
Given that SHA-1 is waning, it would be better to get ahead of the 
curve. But this industry has a tendency to be very reactionary: "don't 
add new stuff if the old stuff is not obviously broken". It took 11+ 
years to get SHA-1 just starting to be phased out. Just something to 
think about.

Sean

---

From: Gilles Van Assche <[email-masked]>
Date: Fri, 12 Aug 2016 10:06:06 +0200

Hi Russ,

For hardware performance, you can have a look at the reports made during
the SHA-3 competition (including comparisons with SHA-2), e.g., [1] [2].

For software performance, eBASH contains benchmarks on several machines.
You may also want to run "KeccakTests --speed" with the Keccak code
package [3], which contains some more recent implementations. (It is on
our ever-growing to-do list to submit those to eBASH.)

E.g., on Sandy Bridge [4]:
SHAKE128 (keccakc256): 8.81 c/b
SHAKE256 (keccakc512): 10.87 c/b
SHA-512: 10.94 c/b
SHA-256: 14.12 c/b

E.g., on Skylake [5]:
SHA-512: 5.08 c/b
SHAKE128: 7.01 c/b
SHA-256: 7.62 c/b
SHAKE256: 8.64 c/b

Kind regards,
Gilles

[1]
http://csrc.nist.gov/groups/ST/hash/sha-3/Round3/March2012/documents/papers/GURKAYNAK_paper.pdf
[2] http://eprint.iacr.org/2012/368.pdf
[3] https://github.com/gvanas/KeccakCodePackage
[4] http://bench.cr.yp.to/results-hash.html#amd64-h6sandy
[5] http://bench.cr.yp.to/results-hash.html#amd64-skylake