Subject: RE: [NTDP] Purpose of NTDP

From: "Craig Brown \(crbrown\)" <[email-masked]>
Date: Tue, 1 Aug 2006 16:27:29 +0800

Loki,

I can see the areas of controlling testing device and gathering data.
Can you expand the description of the other areas ?

Cheers,
Craig 

> -----Original Message-----
> From: Loki Jorgenson [mailto:[email-masked]] 
> Sent: Tuesday, August 01, 2006 2:30 AM
> To: Craig Brown (crbrown)
> Cc: [email-masked]
> Subject: RE: [NTDP] Purpose of NTDP
> 
> Craig - I would request that we expand the scope sufficiently 
> to include the complete cycle required to support 
> auto-diagnostic processes.  In my view, the ability to gather 
> information about the network, analyze it and then direct it 
> is essential and missing from any other charter.
> 
> Something like
>    o control testing devices
>    o gather data
>    o analyze data
>    o generate outcome
>    o remediate/provision by issuing state or configuration requests
> 
> All of the above except that last appears within scope so 
> far.  Some sort of explicit relationship to 
> Netconf-or-similar might be sufficient.
> However, the linkage between "outcome" and "requests" should 
> at least be supported if not defiend - possibly that looks 
> like some sort of API to a network device management system.
> 
> I would like to address any implicit assumption that the 
> output of such an automatically managed system of testing 
> devices would be consumed by a human.
> 
> Loki Jorgenson
> Chief Scientist
> Apparent Networks
> 
> e   [email-masked]
> t   604 433 2333 ext 105
> f   604 433 2311
> m   604 250-4642
> w   www.ApparentNetworks.com
> 
> -----Original Message-----
> From: Craig Brown (crbrown) [mailto:[email-masked]]
> Sent: Sunday, July 30, 2006 7:12 PM
> To: [email-masked]
> Subject: [NTDP] Purpose of NTDP
> 
> 
> It was raised at the last IETF meeting that this alias has 
> been lacking
> a statement of purpose. The objective of this alias is to discuss and
> propose a solution for how we manage a network testing devices. It can
> be considered that these devices are similar to any device in the
> network and from some perspectives that's true. The intent 
> therefore is
> to further identify the unique areas of these devices and define how
> that part is managed. 
> 
> A proposed charter ....
> 

_______________________________________________
NTDP mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ntdp

---

From: "Loki Jorgenson" <[email-masked]>
Date: Tue, 1 Aug 2006 10:01:42 -0700

Craig - let us suppose that "controlling testing devices" is related to
some methodology for detecting network conditions (possibly distinct
from the testing itself).  The testing system is applied reactively in
response to the detection of some conditions.  For example, a VoIP
management system sampling RTCP-XR data from IP phones might discover a
general degredation and send out an alarm.

Today, that is typically a user calling to complain.  However, with
continuous monitoring, app-aware network devices, adaptive applications,
etc. it is more the likely that either the "testing devices" themselves,
or some other other automated external stimulus, triggers a "controlling
of test devices" and directs them to a specific task.  Some call this
"proactive" simply because such a system responds to degraded conditions
before the end user does, potentially resulting in a break-fix before
the user is aware.

While we are barely beyond reactivity in the sense of the user
complaint, it is clear that the technologies are moving toward such a
proactive stance and we need to be able to provide a framework for
"controlling testing devices" that accepts triggers or other feeds from
non-human sources.  This might look like a simple API of
trigger/response but it should at least preserve the relationship
between the stimulus, the reaction, and the outcome.

In directing the testing devices, there should be a set of constraints
that define the nature and extent of the gathering of data.  How long?
Until what is satisified?  Indefinitely until there is an "off" trigger?

Finally, a specific set of constraints on data gathering are met (e.g.
enough data), then an analysis process should be associated with the
data.  Once again, the relationship between the initial conditions and
the analysis should be preserved.

Once the analysis is complete, some conclusion or action may be
associated with it.  Typically, that amounts to a particular signal, a
threshold against which it is compared, and an alarm sent to a human (or
at least a management system).  This is very anthro-centric insofar as
it assumes that a human (or possibly an NMS) will interpret the outcomes
of the analysis and determine appropriate actions to take.

A device/configuration management system might be invoked to remediate
or provision against a particular outcome of the analysis, with yet
another "controlling of testing devices" activity subsequently triggered
to validate the remediation/provisioning action.  Once a framework is
capable of 
   o preserving the association between 
		conditions/trigger --> testing --> analysis -->
remediation --> validation
   o associating condition sets with analyses, analysis outcomes with
remediation, remediation with validation actions
   o relating triggers/testing/analysis over time
   o  running without human intervention
then it has the potential to support fully autonomic operation.

The specifics of those associations would be what I hope we would
discuss and develop on this list.  I anticipate that the differences
between methodologies, data set content, analyses, etc. and the
challenges of associating dissimilar data/analyses will be substantial.
This parallels work undertaken by the Internet2 End-to-End project (e.g.
perfSONAR) attempting to abstract a range of well-defined testing
technologies sufficiently that they can be integrated into a single
framework.

On the other hand, it may be that the "controlling test
devices/gathering data" aspect of this larger framework is all that we
are interested in with NTDP.  It would be a layer that simply accepts
testing requests of a certain form, executes and returns the data.  The
analysis and other levels of behavior might be found in a separate
system/layer.  That would be less interesting (unless there were some
well-defined descriptions for those layers that we can reference).

Hope that helps to clarify my point, as requested.

Loki

-----Original Message-----
From: Craig Brown (crbrown) [mailto:[email-masked]] 
Sent: Tuesday, August 01, 2006 1:27 AM
To: Loki Jorgenson
Cc: [email-masked]
Subject: RE: [NTDP] Purpose of NTDP

Loki,

Cheers,
Craig