Subject: [scim] source code for compliance tests

From: Dale Olds <[email-masked]>
Date: Tue, 18 Sep 2012 22:48:11 -0700

I apologize to the wg that this is not directly related to the scim 
specs, but I find that compliance tests are very useful to clarify the 
specs. The scim WG (as represented at simplecloud.info) has taken the 
the bold and quite useful step to publish compliance tests. I don't 
expect the tests determine some PM's view of marketing compliance -- I'd 
just like to know that the implementers of the spec agree about what the 
spec means.

Is the source code for the tests available?

--Dale

---

From: Erik Wahlström <[email-masked]>
Date: Wed, 19 Sep 2012 07:03:40 +0000

Hi Dale,

Great news that you find it useful. We would love some feedback on it. It's still in beta and we have plans to add and improve existing tests.

Code can currently be found here: http://code.google.com/p/scimproxy/

/ Erik

On Sep 19, 2012, at 7:48 AM, Dale Olds wrote:

I apologize to the wg that this is not directly related to the scim specs, but I find that compliance tests are very useful to clarify the specs. The scim WG (as represented at simplecloud.info<http://simplecloud.info>) has taken the the bold and quite useful step to publish compliance tests. I don't expect the tests determine some PM's view of marketing compliance -- I'd just like to know that the implementers of the spec agree about what the spec means.

--Dale
_______________________________________________
scim mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/scim

---

From: Hannes Tschofenig <[email-masked]>
Date: Wed, 19 Sep 2012 10:04:30 +0300

Hi Dale, 

I agree that interoperability tests are quite important but, as you said, it would be very helpful to know the details of these tests. 
In addition there is the challenge that these tests (regardless of whether they are done online or not) are a bit misleading at this point in time when the work on the protocol is still at an early stage. It may lead someone in the false believe that their code is interoperable although that only means interoperability with a work-in-progress specification that may change any minute. 

Ciao
Hannes

PS: I had worked on test cases in a different IETF working group, called DIME, and published them as IETF drafts before we used them for the interop-events. Here is one example: 
http://tools.ietf.org/html/draft-fajardo-dime-base-test-suite-02

Maybe the group can create something similar. 

On Sep 19, 2012, at 8:48 AM, Dale Olds wrote:

> I apologize to the wg that this is not directly related to the scim specs, but I find that compliance tests are very useful to clarify the specs. The scim WG (as represented at simplecloud.info) has taken the the bold and quite useful step to publish compliance tests. I don't expect the tests determine some PM's view of marketing compliance -- I'd just like to know that the implementers of the spec agree about what the spec means.
> 
> [email-masked]

---

From: Erik Wahlström <[email-masked]>
Date: Wed, 19 Sep 2012 07:25:13 +0000

Hi,

That site tests the 1.1 specification that earlier was developed under the Open Web Foundation before the IETF work started. I think it's very important to separate the two (hence the no email to this list when tests went live). We should probably also make that very clear on the simplecloud site that tests has nothing to do with the work done here.

+1 on that a document describing the tests for the work done here would be good.

/ Erik

---

From: Leif Johansson <[email-masked]>
Date: Wed, 19 Sep 2012 09:47:06 +0200

On 09/19/2012 09:04 AM, Hannes Tschofenig wrote:
> Hi Dale,
> 
> I agree that interoperability tests are quite important but, as you
> said, it would be very helpful to know the details of these tests.
>  In addition there is the challenge that these tests (regardless of
> whether they are done online or not) are a bit misleading at this
> point in time when the work on the protocol is still at an early
> stage. It may lead someone in the false believe that their code is
> interoperable although that only means interoperability with a
> work-in-progress specification that may change any minute.
> 

Agreed. Keep tracking the speck but please note to all your users
that the spec is very much a moving target at this point.

	Cheers Leif