Subject: [Isms] Evaluation status

From: "David B Harrington" <[email-masked]>
Date: Fri, 7 Jan 2005 13:41:05 -0500

Hi,

To the WG chairs and the evaluation team members:

Can we get a status report on the evaluation effort?
I'm not asking for status of the evaluation, but of the evaluation
team's effort.

Has the team met? 
How recently?
Are all members still committed to doing the evaluation asap (although
obviously not by WG-specified deadline)?
Have all team members read the three proposals yet?
Has the team established evaluation guidelines?
Can we know what the evaluation guidelines are?
Has the team started analyzing the three proposals against the
guidelines yet?
Has the team decided how they will report the results of the
evaluation?
Has the team started to document their results yet?
Does the team have an estimated date for completion of their report?

As already pointed out, we have a tight charter deadline, but all work
is completely suspended while waiting for the results from the
evaluation team.
Is the evaluation team making any progress?
Should we be looking for a new evaluation team? 

Thanks,
David Harrington
[email-masked]

_______________________________________________
Isms mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/isms

---

From: Juergen Quittek <[email-masked]>
Date: Mon, 10 Jan 2005 14:05:57 +0100

Dear all,

I wish you all a happy new year!

Wes, Eric, and David, thank you for the call for more information.
Indeed it was too quiet on this list over the holiday break
and I can understand that many people want to know about the
status of the evaluation process.  I apologize for not
clarifying things earlier.

The most important news is that our shepherding AD granted a
deadline extension until March for the WG decision on the
solution to focus on.  You will find this reflected on our
charter web page.

The evaluation team has met and discussed all proposals.
The team will produce an Internet draft reflecting the
discussion and giving a recommendation to the WG.
There is already a version of this draft, but the essential
section, the recommendation, is still to be done.

One of the problems the team is facing is that the different
proposals cover different aspects of the overall problem.

I hope that the team will be able to sketch the recommendation
in January and have the draft ready for the cut-off date on
February 14.

Thanks,

    Juergen
-- 
Juergen Quittek        [email-masked]       Tel: +49 6221 90511-15
NEC Europe Ltd.,       Network Laboratories        Fax: +49 6221 90511-55
Kurfuersten-Anlage 36, 69115 Heidelberg, Germany   http://www.netlab.nec.de

---

From: Wes Hardaker <[email-masked]>
Date: Mon, 10 Jan 2005 08:20:56 -0800

>>>>> On Mon, 10 Jan 2005 14:05:57 +0100, Juergen Quittek <[email-masked]> said:

Juergen> The most important news is that our shepherding AD granted a
Juergen> deadline extension until March for the WG decision on the
Juergen> solution to focus on.  You will find this reflected on our
Juergen> charter web page.

I actually don't find this a big change.  The original November date
was really just an early deadline that tried to emphasize the urgency
of the one in march which was the real cut-off: decide something or
shut down.  By simply aligning the should-be-decided deadline with the
must-be-decided deadline you haven't really changed anything.  The end
result is still the same: if you wait until Feb 14th to let the
members of the group know what 4-6 people have decided, I have strong
doubts that there is enough time between that point and the
MUST-have-consensus deadline to handle the resulting discussions.

-- 
Wes Hardaker
Sparta

---

From: "David B Harrington" <[email-masked]>
Date: Mon, 10 Jan 2005 12:16:49 -0500

I agree. I don't feel this is a significant change.
The question posed was essentially, is the evaluation team making real
progress?
The answer "we have permission to push back an interim deadline"
doesn't answer the question asked.

I think the more important points are that:
1) The evaluation team has met and discussed all proposals.

It is good to know that the team has been active, although this
doesn't give us much info on how active. They met at IETF61. It would
be nice if they and the chairs chose to share info, like how many
times have they met and had useful discussions. We are looking for
assurance that the team is seriously working on comparing and
discussing the proposals, and that the recommendations they provide
will be well-researched. 

They apparently have not conferred with the authors of the proposals
for any clarifications yet, and that seriously concerns me. I do not
feel comfortable that we will have meaningful comparisons completed
before the March IETF meeting. Please comfort me.

2) The team will produce an Internet draft reflecting the
discussion and giving a recommendation to the WG.

OK. But do we get to know what they are using for evaluation
guidelines, or do we have to wait until feb14 to find out how the
evaluation is being done? If we don't even get to know what guidelines
are being used before the final publication deadline for the March
meeting, we certainly will not have time to do much if the WG
disagrees with the guidelines. Eric proposed some guidelines to meet
his security needs; Dave Perkins proposed some application guidelines;
Wes published a table of attributes for various potential solutions.
Are any or all of these proposed guidelines being taken into account
in defining the team's guidelines? Are all team members using the same
guidelines, or each using their own guidelines?

I just want to know what the team's guidelines are so we are not
blind-sided if the evaluation team doesn't consider the issues that
the WG agrees need to be considered. The secrecy combined with the
tight deadline is stressful; I'd like to know if there is something
the rest of the WG or the authors of the proposals could be doing to
make the evaluation process more effective/efficient.

3) There is already a version of this draft, but the essential
section, the recommendation, is still to be done.

Understood. Having an empty template for an internet-draft isn't very
reassuring, however. Does the initial version indicate what factors
are being considered in the analysis - e.g. an abstraction of the
guidelines being used?

4) One of the problems the team is facing is that the different
proposals cover different aspects of the overall problem.

So is there anything we can do to help make it easier to complete the
evaluation? 
Can I clarify anything in the TLS proposal, or provide additional
specifications to address specfic aspects that are not currently
covered in the proposal, or provide research services such as
providing references to sections of other documents to help the team
quickly compare the proposals?

dbh 

> -----Original Message-----
> From: Wes Hardaker [mailto:[email-masked]] 
> Sent: Monday, January 10, 2005 11:21 AM
> To: Juergen Quittek
> Cc: [email-masked]; [email-masked]
> Subject: Re: [Isms] Evaluation status
>