From nobody Wed Jun 24 19:15:18 2020 Return-Path: X-Original-To: tools-arch@ietfa.amsl.com Delivered-To: tools-arch@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id A68F53A1235; Wed, 24 Jun 2020 19:15:16 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.899 X-Spam-Level: X-Spam-Status: No, score=-1.899 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id aJi0A-0Vzpt9; Wed, 24 Jun 2020 19:15:15 -0700 (PDT) Received: from jays-mbp.localdomain (unknown [158.140.230.105]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPSA id 8848E3A1234; Wed, 24 Jun 2020 19:15:14 -0700 (PDT) From: Jay Daley Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Mime-Version: 1.0 (Mac OS X Mail 13.4 \(3608.80.23.2.2\)) Message-Id: Date: Thu, 25 Jun 2020 14:15:12 +1200 To: tools-arch@ietf.org, tools-discuss@ietf.org X-Mailer: Apple Mail (2.3608.80.23.2.2) Archived-At: Subject: [Tools-arch] Draft RFP SoW for "Review of the current landscape of IETF document processing tools" X-BeenThere: tools-arch@ietf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Tools Architecture and Strategy Team List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 25 Jun 2020 02:15:17 -0000 In support of the work of the Tools Architecture and Strategy Team = (TAS), an RFP will be issued for a "Review of the current landscape of = IETF document processing tools". A draft SoW is below and I welcome = your comments/feedback on it. Jay # Overview The IETF seeks a contractor to review the current landscape of tools = that have been specifically designed to be used by the IETF community to = work with IETF documents (I-Ds and RFCs) and to build a dataset about = these tools, including technical details, maintenance processes, = licensing and others, and further to determine what parts of the = document production, management and usage lifecycle each tool supports. # Deliverables ## Dataset This data is to be presented in a table using simple scales and = classifications in addition to any supporting narrative. For example, = for licensing, a simple classification may include "GPL v3, Proprietary, = BSD-like" and more. The data to be collected is: 1. A full catalog of tools, including all the relevant tools at = https://tools.ietf.org and other tools that are not listed there but are = known to be in use. The contractor will be responsible for drawing up = the list, which will then be reviewed, possibly iterated and approved. 2. Details of the maintainer(s) of each tool 3. The licensing of the tool (simple classification) 4. The technical environment each tool is intended to work in (OS, = packages, containers, etc) 5. How each tool is maintained, including - where the source code is located - how issues are raised (simple classification) - the level of version control and release management in use (on a = simple scale) - how comprehensive the test coverage is (on a simple scale)=20 - the form of testing framework in use (simple classification) 6. A subjective assessment of the current maintained state of the tool = (on a simple scale) 6. A subjective assessment of the suitability of each tool (on a simple = scale), based on 3, 4, 5 and 6 above. 7. What stages of the document lifecycle (see below) each tools covers. ## Document lifecycle As part of this work, the contractor will need to construct a model of = the IETF document lifecycle including document production (authoring, = review, format conversion, etc), document access (download tools, search = tools, etc) and document usage (extracting certain parts, etc). This is = only to cover the work carried out by the IETF community and not the = work of the RFC Production Centre. There are no templates for this model and it is expected that there will = be two or three review and and change cycles. ## Coverage map The information collected above is additionally to be presented in some = form of map showing each tool and each stage, the coverage of each stage = in the lifecycle and the subjective assessment of the suitability of = each tool. --=20 Jay Daley IETF Executive Director jay@ietf.org From nobody Wed Jun 24 20:19:44 2020 Return-Path: X-Original-To: tools-arch@ietfa.amsl.com Delivered-To: tools-arch@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 399FD3A124D; Wed, 24 Jun 2020 20:19:43 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -2.098 X-Spam-Level: X-Spam-Status: No, score=-2.098 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=lowentropy.net header.b=U1w9pGMy; dkim=pass (2048-bit key) header.d=messagingengine.com header.b=SEQuVLuW Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Kc-nRE5A2sCE; Wed, 24 Jun 2020 20:19:41 -0700 (PDT) Received: from wout5-smtp.messagingengine.com (wout5-smtp.messagingengine.com [64.147.123.21]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 6D6783A124C; Wed, 24 Jun 2020 20:19:41 -0700 (PDT) Received: from compute2.internal (compute2.nyi.internal [10.202.2.42]) by mailout.west.internal (Postfix) with ESMTP id 5E5BBAA2; Wed, 24 Jun 2020 23:19:40 -0400 (EDT) Received: from imap2 ([10.202.2.52]) by compute2.internal (MEProxy); Wed, 24 Jun 2020 23:19:40 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=lowentropy.net; h=mime-version:message-id:in-reply-to:references:date:from:to :subject:content-type; s=fm2; bh=+Zz2Jd8VfzKaWD+7BWVUHBbMmqu0CKG 60TGgV/F59IQ=; b=U1w9pGMy8Ycj1VlXgn7wqbXU7QocNcTY4dZIR2nW5HjDt4H 2Jr3bboOey/mNy6As+lUI7gGvyTwbH/e8egnupZVJEhGVGu/+PyCVW1ueCFNp75D RwNpGyR+yZKNkY2xKVWslk/tbB2EiASZQ8foEU584x0+nw+OZR2P/Rtlb7A5dw6J 9clTPmriByK8eNdt5T7Qtls13UicrvIj5CFo4SA2YeX+2GKYn8UEdPDM731QGbey ePA8zBi4pgs3hmYH8GX9RIES685DJqsCEEpx7aokv/UxLxoYFEq3i9F0KCzTfPpu DaWruQAxVvQD/QHh4NR2tr5PXUnRTv6gm54Pnyw== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm3; bh=+Zz2Jd 8VfzKaWD+7BWVUHBbMmqu0CKG60TGgV/F59IQ=; b=SEQuVLuWF3MA0lpTSTKWHZ yJFQ2jiFq5iVWsyNWxd3DtGA8WKGoSRkm0Iw4vtVXsscSTB3cJZZi0oXq45srwkx jxeuIxNzh+QV+N7WIb87rbVkKhQ+3q4dXVOf6wv+CKP0qdA2stYcmjVFoB6PTOvl BxvVC2VGL5TWh+hYRAZfbm5ZcTXlJ5BEscwf0PJccdRDCWmHX1ThiC5wC6bQc2L/ CJAVXeIzicDnviwoqlVDmBykoRZs9xJDNn7y2gpwTBq/6QsPJUc4AMPK6/JacPp4 jUwdxeMF1whb927jPYQyiUrKym34L4zg4B1mOY1H4zfz4F59BYmwP/xdTrTfKy7Q == X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduhedrudekkedgieelucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucenucfjughrpefofgggkfgjfhffhffvufgtsehttd ertderreejnecuhfhrohhmpedfofgrrhhtihhnucfvhhhomhhsohhnfdcuoehmtheslhho figvnhhtrhhophihrdhnvghtqeenucggtffrrghtthgvrhhnpeelfedugfelgeeigfeuve eludejjeeutdelgedvteehueevhfdvhefggeeuffeivdenucffohhmrghinhepihgvthhf rdhorhhgnecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomh epmhhtsehlohifvghnthhrohhphidrnhgvth X-ME-Proxy: Received: by mailuser.nyi.internal (Postfix, from userid 501) id 6CD45E00A8; Wed, 24 Jun 2020 23:19:39 -0400 (EDT) X-Mailer: MessagingEngine.com Webmail Interface User-Agent: Cyrus-JMAP/3.3.0-dev0-543-gda70334-fm-20200618.004-gda703345 Mime-Version: 1.0 Message-Id: <0eff2634-5263-44ba-80c7-6e05c19b34cb@www.fastmail.com> In-Reply-To: References: Date: Thu, 25 Jun 2020 13:19:19 +1000 From: "Martin Thomson" To: "Jay Daley" , tools-arch@ietf.org, tools-discuss@ietf.org Content-Type: text/plain Archived-At: Subject: Re: [Tools-arch] =?utf-8?q?Draft_RFP_SoW_for_=22Review_of_the_curren?= =?utf-8?q?t_landscape_of_IETF_document_processing_tools=22?= X-BeenThere: tools-arch@ietf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Tools Architecture and Strategy Team List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 25 Jun 2020 03:19:43 -0000 Thanks Jay, I realize that while I might provide some of this list, I am completely unaware of the extent of it. But this seems achievable in a relatively short time-scale. I might prefer to see more details on how the contractor manages the work product. Specifically, I think that this should be a relatively open process during which tool maintainers can see the assessment and make factual corrections. A public document or repository might be a good way to ensure that contributions of this sort are easy - and getting those contributions might greatly simplify the process. Is your intent to include the set of tools employed by the RPC in this? I think that would be good, but it appears that you intentionally chose "used by the IETF community". Nits: Two items are numbered 6. "and and" in "two or three review and and change cycles" On Thu, Jun 25, 2020, at 12:15, Jay Daley wrote: > In support of the work of the Tools Architecture and Strategy Team > (TAS), an RFP will be issued for a "Review of the current landscape of > IETF document processing tools". A draft SoW is below and I welcome > your comments/feedback on it. > > Jay > > > # Overview > The IETF seeks a contractor to review the current landscape of tools > that have been specifically designed to be used by the IETF community > to work with IETF documents (I-Ds and RFCs) and to build a dataset > about these tools, including technical details, maintenance processes, > licensing and others, and further to determine what parts of the > document production, management and usage lifecycle each tool supports. > > # Deliverables > > ## Dataset > This data is to be presented in a table using simple scales and > classifications in addition to any supporting narrative. For example, > for licensing, a simple classification may include "GPL v3, > Proprietary, BSD-like" and more. > > The data to be collected is: > > 1. A full catalog of tools, including all the relevant tools at > https://tools.ietf.org and other tools that are not listed there but > are known to be in use. The contractor will be responsible for > drawing up the list, which will then be reviewed, possibly iterated and > approved. > > 2. Details of the maintainer(s) of each tool > > 3. The licensing of the tool (simple classification) > > 4. The technical environment each tool is intended to work in (OS, > packages, containers, etc) > > 5. How each tool is maintained, including > - where the source code is located > - how issues are raised (simple classification) > - the level of version control and release management in use (on a simple scale) > - how comprehensive the test coverage is (on a simple scale) > - the form of testing framework in use (simple classification) > > 6. A subjective assessment of the current maintained state of the tool > (on a simple scale) > > 6. A subjective assessment of the suitability of each tool (on a > simple scale), based on 3, 4, 5 and 6 above. > > 7. What stages of the document lifecycle (see below) each tools covers. > > ## Document lifecycle > As part of this work, the contractor will need to construct a model of > the IETF document lifecycle including document production (authoring, > review, format conversion, etc), document access (download tools, > search tools, etc) and document usage (extracting certain parts, etc). > This is only to cover the work carried out by the IETF community and > not the work of the RFC Production Centre. > > There are no templates for this model and it is expected that there > will be two or three review and and change cycles. > > ## Coverage map > The information collected above is additionally to be presented in some > form of map showing each tool and each stage, the coverage of each > stage in the lifecycle and the subjective assessment of the suitability > of each tool. > > > > > -- > Jay Daley > IETF Executive Director > jay@ietf.org > > -- > Tools-arch mailing list > Tools-arch@ietf.org > https://www.ietf.org/mailman/listinfo/tools-arch > From nobody Wed Jun 24 20:59:40 2020 Return-Path: X-Original-To: tools-arch@ietfa.amsl.com Delivered-To: tools-arch@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 56B503A1268; Wed, 24 Jun 2020 20:59:38 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.898 X-Spam-Level: X-Spam-Status: No, score=-1.898 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HTML_MESSAGE=0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ek6FA2mY-Rm2; Wed, 24 Jun 2020 20:59:36 -0700 (PDT) Received: from jays-mbp.localdomain (unknown [158.140.230.105]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPSA id 8D94E3A1266; Wed, 24 Jun 2020 20:59:35 -0700 (PDT) From: Jay Daley Message-Id: <0167CABC-5497-44FA-A60B-380E79AAB5CF@ietf.org> Content-Type: multipart/alternative; boundary="Apple-Mail=_E34DFA2C-E87D-423F-83FE-1AF69D8361E0" Mime-Version: 1.0 (Mac OS X Mail 13.4 \(3608.80.23.2.2\)) Date: Thu, 25 Jun 2020 15:59:33 +1200 In-Reply-To: <0eff2634-5263-44ba-80c7-6e05c19b34cb@www.fastmail.com> Cc: tools-arch@ietf.org, tools-discuss@ietf.org To: Martin Thomson References: <0eff2634-5263-44ba-80c7-6e05c19b34cb@www.fastmail.com> X-Mailer: Apple Mail (2.3608.80.23.2.2) Archived-At: Subject: Re: [Tools-arch] Draft RFP SoW for "Review of the current landscape of IETF document processing tools" X-BeenThere: tools-arch@ietf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Tools Architecture and Strategy Team List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 25 Jun 2020 03:59:38 -0000 --Apple-Mail=_E34DFA2C-E87D-423F-83FE-1AF69D8361E0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 > On 25/06/2020, at 3:19 PM, Martin Thomson wrote: >=20 > Thanks Jay, >=20 > I realize that while I might provide some of this list, I am = completely unaware of the extent of it. But this seems achievable in a = relatively short time-scale. >=20 > I might prefer to see more details on how the contractor manages the = work product. Specifically, I think that this should be a relatively = open process during which tool maintainers can see the assessment and = make factual corrections. That=E2=80=99s a good point yes - I will add that. > A public document or repository might be a good way to ensure that = contributions of this sort are easy - and getting those contributions = might greatly simplify the process. I think if we limit it to tool maintainers contributing then that will = work well >=20 > Is your intent to include the set of tools employed by the RPC in = this? I think that would be good, but it appears that you intentionally = chose "used by the IETF community". My intent was to exclude tools that the community is not free to use. I = think the main tool the RPC use is xml2rfc and that is free for the = community to use and so I would expect that to be included. =20 Jay >=20 > Nits: >=20 > Two items are numbered 6. >=20 > "and and" in "two or three review and and change cycles" >=20 > On Thu, Jun 25, 2020, at 12:15, Jay Daley wrote: >> In support of the work of the Tools Architecture and Strategy Team=20 >> (TAS), an RFP will be issued for a "Review of the current landscape = of=20 >> IETF document processing tools". A draft SoW is below and I welcome=20= >> your comments/feedback on it. >>=20 >> Jay >>=20 >>=20 >> # Overview >> The IETF seeks a contractor to review the current landscape of tools=20= >> that have been specifically designed to be used by the IETF community=20= >> to work with IETF documents (I-Ds and RFCs) and to build a dataset=20 >> about these tools, including technical details, maintenance = processes,=20 >> licensing and others, and further to determine what parts of the=20 >> document production, management and usage lifecycle each tool = supports. >>=20 >> # Deliverables >>=20 >> ## Dataset >> This data is to be presented in a table using simple scales and=20 >> classifications in addition to any supporting narrative. For = example,=20 >> for licensing, a simple classification may include "GPL v3,=20 >> Proprietary, BSD-like" and more. >>=20 >> The data to be collected is: >>=20 >> 1. A full catalog of tools, including all the relevant tools at=20 >> https://tools.ietf.org and other tools that are not listed there but=20= >> are known to be in use. The contractor will be responsible for=20 >> drawing up the list, which will then be reviewed, possibly iterated = and=20 >> approved. >>=20 >> 2. Details of the maintainer(s) of each tool >>=20 >> 3. The licensing of the tool (simple classification) >>=20 >> 4. The technical environment each tool is intended to work in (OS,=20= >> packages, containers, etc) >>=20 >> 5. How each tool is maintained, including >> - where the source code is located >> - how issues are raised (simple classification) >> - the level of version control and release management in use (on a = simple scale) >> - how comprehensive the test coverage is (on a simple scale)=20 >> - the form of testing framework in use (simple classification) >>=20 >> 6. A subjective assessment of the current maintained state of the = tool=20 >> (on a simple scale) >>=20 >> 6. A subjective assessment of the suitability of each tool (on a=20 >> simple scale), based on 3, 4, 5 and 6 above. >>=20 >> 7. What stages of the document lifecycle (see below) each tools = covers. >>=20 >> ## Document lifecycle >> As part of this work, the contractor will need to construct a model = of=20 >> the IETF document lifecycle including document production (authoring,=20= >> review, format conversion, etc), document access (download tools,=20 >> search tools, etc) and document usage (extracting certain parts, = etc). =20 >> This is only to cover the work carried out by the IETF community and=20= >> not the work of the RFC Production Centre. >>=20 >> There are no templates for this model and it is expected that there=20= >> will be two or three review and and change cycles. >>=20 >> ## Coverage map >> The information collected above is additionally to be presented in = some=20 >> form of map showing each tool and each stage, the coverage of each=20 >> stage in the lifecycle and the subjective assessment of the = suitability=20 >> of each tool. >>=20 >>=20 >>=20 >>=20 >> --=20 >> Jay Daley >> IETF Executive Director >> jay@ietf.org >>=20 >> --=20 >> Tools-arch mailing list >> Tools-arch@ietf.org >> https://www.ietf.org/mailman/listinfo/tools-arch = >>=20 >=20 > --=20 > Tools-arch mailing list > Tools-arch@ietf.org > https://www.ietf.org/mailman/listinfo/tools-arch = --=20 Jay Daley IETF Executive Director jay@ietf.org --Apple-Mail=_E34DFA2C-E87D-423F-83FE-1AF69D8361E0 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=utf-8

On 25/06/2020, at 3:19 PM, Martin Thomson <mt@lowentropy.net> = wrote:

Thanks Jay,

I realize that while I might = provide some of this list, I am completely unaware of the extent of it. =  But this seems achievable in a relatively short = time-scale.

I might = prefer to see more details on how the contractor manages the work = product.  Specifically, I think that this should be a relatively = open process during which tool maintainers can see the assessment and = make factual corrections.

That=E2=80=99s a good point yes - I will add = that.

 A = public document or repository might be a good way to ensure that = contributions of this sort are easy - and getting those contributions = might greatly simplify the process.

I think if we limit it to tool maintainers = contributing then that will work well


Is your intent to include the set of tools employed by the = RPC in this?  I think that would be good, but it appears that you = intentionally chose "used by the IETF community".

My intent = was to exclude tools that the community is not free to use.  I = think the main tool the RPC use is xml2rfc and that is free for the = community to use and so I would expect that to be included. =  

Jay


Nits:

Two items are numbered = 6.

"and and" in = "two or three review and and change cycles"

On Thu, Jun 25, 2020, at 12:15, = Jay Daley wrote:
In support of the work of the Tools = Architecture and Strategy Team 
(TAS), an = RFP will be issued for a "Review of the current landscape of 
IETF = document processing tools".  A draft SoW is below and I = welcome 
your comments/feedback on it.

Jay


# Overview
The IETF seeks a contractor to review the current landscape = of tools 
that have been specifically designed to be used by the IETF = community 
to work with IETF documents (I-Ds and RFCs) and to build a = dataset 
about these tools, including technical details, maintenance = processes, 
licensing and others, and further to determine what parts of = the 
document production, management and usage lifecycle each tool = supports.

# Deliverables

## Dataset
This data is to be presented in a = table using simple scales and 
classifications in addition to any supporting narrative. =  For example, 
for licensing, a simple classification may include "GPL = v3, 
Proprietary, BSD-like" and more.

The data to be collected is:

1. =  A full catalog of tools, including all the relevant tools at 
https://tools.ietf.org = and other tools that are not listed there but 
are known to = be in use.   The contractor will be responsible for 
drawing up = the list, which will then be reviewed, possibly iterated and 
approved.

2.  Details of the maintainer(s) of each = tool

3.  The licensing of the tool = (simple classification)

4.  The = technical environment each tool is intended to work in (OS, 
packages, = containers, etc)

5.  How each tool is = maintained, including
- where the source code is = located
- how issues are raised (simple classification)
- the level of version control and release management in use = (on a simple scale)
- how comprehensive the test coverage = is (on a simple scale) 
- the form = of testing framework in use (simple classification)

6.  A subjective assessment of the current maintained = state of the tool 
(on a simple scale)

6.  A = subjective assessment of the suitability of each tool (on a 
simple = scale), based on 3, 4, 5 and 6 above.

7. =  What stages of the document lifecycle (see below) each tools = covers.

## Document lifecycle
As part of this work, the contractor will need to construct a = model of 
the IETF document lifecycle including document production = (authoring, 
review, format conversion, etc), document access (download = tools, 
search tools, etc) and document usage (extracting certain = parts, etc).  
This is only to cover the work carried = out by the IETF community and 
not the work = of the RFC Production Centre.

There are no = templates for this model and it is expected that there 
will be two = or three review and and change cycles.

## = Coverage map
The information collected above is = additionally to be presented in some 
form of map = showing each tool and each stage, the coverage of each 
stage in the = lifecycle and the subjective assessment of the suitability 
of each = tool.




-- 
Jay Daley
IETF Executive Director
jay@ietf.org

-- 
Tools-arch = mailing list
Tools-arch@ietf.org
https://www.ietf.org/mailman/listinfo/tools-arch


-- Tools-arch mailing = list
Tools-arch@ietf.org
https://www.ietf.org/mailman/listinfo/tools-arch

-- 
Jay Daley
IETF = Executive Director
jay@ietf.org

= --Apple-Mail=_E34DFA2C-E87D-423F-83FE-1AF69D8361E0-- From nobody Wed Jun 24 21:20:46 2020 Return-Path: X-Original-To: tools-arch@ietfa.amsl.com Delivered-To: tools-arch@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 3442A3A1274 for ; Wed, 24 Jun 2020 21:20:45 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.851 X-Spam-Level: X-Spam-Status: No, score=-1.851 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, HEADER_FROM_DIFFERENT_DOMAINS=0.249, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1536-bit key) header.d=iecc.com header.b=ZKeHLl2B; dkim=pass (1536-bit key) header.d=taugh.com header.b=SwznJDR7 Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id yrDlO9M8nvfr for ; Wed, 24 Jun 2020 21:20:43 -0700 (PDT) Received: from gal.iecc.com (gal.iecc.com [IPv6:2001:470:1f07:1126:0:43:6f73:7461]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 58C6B3A1272 for ; Wed, 24 Jun 2020 21:20:43 -0700 (PDT) Received: (qmail 35646 invoked from network); 25 Jun 2020 04:20:42 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=simple; d=iecc.com; h=date:message-id:from:to:cc:subject:in-reply-to:mime-version:content-type:content-transfer-encoding; s=8b3c.5ef4261a.k2006; bh=OczpihkEnLD7sK2+XXL2Jew/akhxDAZD/zSasR9H8Og=; b=ZKeHLl2BKcXwL5Osq50hvd4lfFc0e9XQPtsZBDSlJcVSDLOdbEcQtCWLgmJZW2J00LWD1YSwPWjT6vgLrP1VWunLesECA72nOR5UqcDUH8fmos5t42TVMu6nigUbEcXyA1tow1sH0negyJNVrtwijKUz6DqQfiXCAfQmKx8Ni0nOkvWDAqVuRRUOYzVYNLBfVfgEz/unhYo2ZRRpkvQaUWjevQo7v1NBG9Uhj4N7EkVa+KkQVNpFNgkm2RYJ+YKk DKIM-Signature: v=1; a=rsa-sha256; c=simple; d=taugh.com; h=date:message-id:from:to:cc:subject:in-reply-to:mime-version:content-type:content-transfer-encoding; s=8b3c.5ef4261a.k2006; bh=OczpihkEnLD7sK2+XXL2Jew/akhxDAZD/zSasR9H8Og=; b=SwznJDR7m3F4KQCswu0mwQp38uB0aQLcPzJP359b7jRhPc/vGmaUlDs7LkdtZccZ+CP3q/Vr02paz4D3SP2nx0C7p+uYLI9kPYZpy+yU88Spv8jMMWR/099e2f3Lecjc4jjydPBDDwo+EC43knkMZOC/VOQyGLXWwR+tFc34uIs6U9EE8LWNdLdfV+b0IzPNSysrekMnkSxDWGxj1LEvPtNkgLHOI1fWj3DGLn+hWWDJ76re1P+1sHggS2zglMlH Received: from ary.qy ([IPv6:2001:470:1f07:1126::78:696d:6170]) by imap.iecc.com ([IPv6:2001:470:1f07:1126::78:696d:6170]) with ESMTP via TCP6; 25 Jun 2020 04:20:41 -0000 Received: by ary.qy (Postfix, from userid 501) id B18561BACFA2; Thu, 25 Jun 2020 00:20:41 -0400 (EDT) Date: 25 Jun 2020 00:20:41 -0400 Message-Id: <20200625042041.B18561BACFA2@ary.qy> From: "John Levine" To: tools-arch@ietf.org Cc: jay@ietf.org In-Reply-To: <0167CABC-5497-44FA-A60B-380E79AAB5CF@ietf.org> Organization: Taughannock Networks X-Headerized: yes Mime-Version: 1.0 Content-type: text/plain; charset=utf-8 Content-transfer-encoding: 8bit Archived-At: Subject: Re: [Tools-arch] Draft RFP SoW for "Review of the current landscape of IETF document processing tools" X-BeenThere: tools-arch@ietf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Tools Architecture and Strategy Team List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 25 Jun 2020 04:20:45 -0000 In article <0167CABC-5497-44FA-A60B-380E79AAB5CF@ietf.org> you write: >> Is your intent to include the set of tools employed by the RPC in this? I think that would be good, but it appears that you >intentionally chose "used by the IETF community". > >My intent was to exclude tools that the community is not free to use. I think the main tool the RPC use is xml2rfc and that is >free for the community to use and so I would expect that to be included. The RPC use some commercial stuff such as the PDF tools from Callas. It would be good to at least mention them since otherwise there'd be mysterious gaps in how a document gets from A to B. They also have a lot of custom web and command line stuff to manage the publication process, e.g., wha document is in what state, what's missinf refernces, and other odds and ends like the DOI uploader I wrote. Some of that belongs to AMS, some (like the stuff I wrote) to the IETF. R's, John From nobody Thu Jun 25 19:23:12 2020 Return-Path: X-Original-To: tools-arch@ietfa.amsl.com Delivered-To: tools-arch@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id B598E3A10E7; Thu, 25 Jun 2020 19:23:10 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.898 X-Spam-Level: X-Spam-Status: No, score=-1.898 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HTML_MESSAGE=0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id mLlfWrmwOhCi; Thu, 25 Jun 2020 19:23:09 -0700 (PDT) Received: from jays-mbp.localdomain (unknown [158.140.230.105]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPSA id 48F903A1074; Thu, 25 Jun 2020 19:23:08 -0700 (PDT) From: Jay Daley Content-Type: multipart/alternative; boundary="Apple-Mail=_42C7F47E-FD83-4CD4-AFEE-065B645BD888" Mime-Version: 1.0 (Mac OS X Mail 13.4 \(3608.80.23.2.2\)) Date: Fri, 26 Jun 2020 14:23:06 +1200 References: To: tools-arch@ietf.org, tools-discuss@ietf.org In-Reply-To: Message-Id: <34D4C033-2BC0-43ED-BFA0-34699EF3EB52@ietf.org> X-Mailer: Apple Mail (2.3608.80.23.2.2) Archived-At: Subject: Re: [Tools-arch] Draft RFP SoW for "Review of the current landscape of IETF document processing tools" X-BeenThere: tools-arch@ietf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Tools Architecture and Strategy Team List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Jun 2020 02:23:11 -0000 --Apple-Mail=_42C7F47E-FD83-4CD4-AFEE-065B645BD888 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 Thanks for all of your comments. A revised version of the SoW is below, = which will go to RFP unless I hear anything else. Thanks to Russ for pointing out that these lists may include people who = wish to bid for this RFP. and so please note that if this applies to you = then you should either not provide any comment on the SoW, or declare = that when providing comment.=20 Jay =E2=80=94=E2=80=94=20 # SoW - Review of the current landscape of IETF document processing = tools ## Overview The IETF seeks a contractor to review the current landscape of tools = that have been specifically designed to be used by the IETF community to = work with IETF documents (I-Ds and RFCs) and to build a dataset about = these tools, including technical details, maintenance processes, = licensing and others, and further to determine what parts of the = document production, management and usage lifecycle each tool supports. ## Deliverables ### Deliverable 1 - Dataset A dataset about the tools containing a combination of text, simple = scales, simple classifications and supporting narrative. For example, = for licensing, a simple classification may include "GPL v3, Proprietary, = BSD-like" and more. The dataset is to be provided as raw data in CSV format. The data to be collected for the dataset is: 1. A full catalog of tools, including all the relevant tools at = https://tools.ietf.org and other tools that are not listed there but are = known to be in use.=20 2. Details of the maintainer(s) of each tool 3. The licensing of the tool (simple classification) 4. The technical environment each tool is intended to work in (OS, = packages, containers, etc) 5. How each tool is maintained, including 1. where the source code is located 2. how issues are raised (simple classification) 3. the level of version control and release management in use (on a = simple scale) 4. how comprehensive the test coverage is (on a simple scale)=20 5. the form of testing framework in use (simple classification) 6. A subjective assessment of the current maintained state of the tool = (on a simple scale) 7. A subjective assessment of the suitability of each tool (on a simple = scale), based on 3, 4, 5 and 6 above. 8. What stages of the document lifecycle (see below) each tool covers. ### Deliverable 2 - Document lifecycle A model of the IETF document lifecycle including document production = (authoring, review, format conversion, etc), document access (download = tools, search tools, etc) and document usage (extracting certain parts, = etc). This is only to cover the work carried out by the IETF community = and not the work of the RFC Production Centre. There are no templates for this model nor any existing work to build = upon and so it is expected that this will be entirely new.=20 ### Deliverable 3 - Coverage map Some form of coverage map/infographic showing each tool and each stage = of the document lifecycle, the coverage of each stage in the lifecycle = and the subjective assessment of the suitability of each tool. This = coverage map should allow at-a-glance understanding of the coverage of = each stage and each tool including stages covered by multiple tools and = stages not covered by any tool. This coverage map is to be supplied in any common vector format. ## Requirements ### Selecting the tools to assess The contractor will be responsible for drawing up the list of tools to = be included for approval by the IETF LLC. ### Review and transparency The IETF is committed to transparency and community engagement and so = requires this work to be carried out as follows: 1. All three deliverables are to be developed using a public repository = so that interested parties can see the work as it unfolds. 2. The initial list of tools proposed by the contractor will go out for = community comment and potentially revised before the final version is = presented for approval. 3. The maintainers of the tools are to be given every opportunity to = recommend changes to the data and the assessments within (though the = contractor retains the final say in what data they choose to include in = the deliverable), which means: 1. Any issues or pull requests raised by maintainers on the = repository are to be considered and responded to. 2. Once the initial version of the dataset is complete, it will go = out for community comment and potentially revised. ### Work of the RFC Production Center The RFC Production Center (RPC) is solely responsible for two steps in = the document lifecycle: 1. The publication of the RFC, which uses a number of internal tools to = manage references, assign DOIs and more.=20 2. The conversion of the XML RFC into the long term ar PDF/A format for = long term archive, which uses commercial software.=20 The tools used solely for the two processes above are not within scope = for this RFP but these steps should appear in the document lifecycle and = coverage map. ## Additional Details None. =E2=80=94=E2=80=94=20 --=20 Jay Daley IETF Executive Director jay@ietf.org --Apple-Mail=_42C7F47E-FD83-4CD4-AFEE-065B645BD888 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=utf-8
Thanks for all of your comments.  A revised version of = the SoW is below, which will go to RFP unless I hear anything = else.

Thanks = to Russ for pointing out that these lists may include people who wish to = bid for this RFP. and so please note that if this applies to you then = you should either not provide any comment on the SoW, or declare that = when providing comment. 

Jay

=E2=80=94=E2=80=94 
# SoW - Review of the current landscape of = IETF document processing tools

## Overview

The IETF seeks a contractor to review = the current landscape of tools that have been specifically designed to = be used by the IETF community to work with IETF documents (I-Ds and = RFCs) and to build a dataset about these tools, including technical = details, maintenance processes, licensing and others, and further to = determine what parts of the document production, management and usage = lifecycle each tool supports.


## = Deliverables

###= Deliverable 1 - Dataset

A dataset about the tools containing a combination of text, = simple scales, simple classifications and supporting narrative. =  For example, for licensing, a simple classification may include = "GPL v3, Proprietary, BSD-like" and more.

The dataset is to be provided as raw = data in CSV format.

The data to be collected for the dataset is:

1. A full catalog of = tools, including all the relevant tools at https://tools.ietf.org = and other tools that are not listed there but are known to be in = use. 
2. Details of the maintainer(s) of each = tool
3. The licensing of the tool (simple = classification)
4. The technical environment each = tool is intended to work in (OS, packages, containers, etc)
5. How each tool is maintained, including
    1. where the source code is located
    2. how issues are raised (simple = classification)
    3. the level of = version control and release management in use (on a simple = scale)
    4. how comprehensive the test = coverage is (on a simple scale) 
    = 5. the form of testing framework in use (simple = classification)
6. A subjective assessment of the = current maintained state of the tool (on a simple scale)
7. A subjective assessment of the suitability of each tool = (on a simple scale), based on 3, 4, 5 and 6 above.
8.= What stages of the document lifecycle (see below) each tool = covers.


### Deliverable 2 -  Document = lifecycle

A = model of the IETF document lifecycle including document production = (authoring, review, format conversion, etc), document access (download = tools, search tools, etc) and document usage (extracting certain parts, = etc).  This is only to cover the work carried out by the IETF = community and not the work of the RFC Production Centre.

There are no templates = for this model nor any existing work to build upon and so it is expected = that this will be entirely new. 


### = Deliverable 3 - Coverage map

Some form of coverage map/infographic = showing each tool and each stage of the document lifecycle, the coverage = of each stage in the lifecycle and the subjective assessment of the = suitability of each tool.  This coverage map should allow = at-a-glance understanding of the coverage of each stage and each tool = including stages covered by multiple tools and stages not covered by any = tool.

This = coverage map is to be supplied in any common vector format.


## Requirements

### Selecting the tools to assess

The contractor will be responsible for = drawing up the list of tools to be included for approval by the IETF = LLC.


### Review and transparency

The IETF is committed to = transparency and community engagement and so requires this work to be = carried out as follows:

1. All three deliverables are to be developed using a public = repository so that interested parties can see the work as it = unfolds.
2. The initial list of tools proposed by = the contractor will go out for community comment and potentially revised = before the final version is presented for approval.
3. The maintainers of the tools are to be given every = opportunity to recommend changes to the data and the assessments within = (though the contractor retains the final say in what data they choose to = include in the deliverable), which means:
  =   1. Any issues or pull requests raised by maintainers on the = repository are to be considered and responded to.
    2. Once the initial version of the dataset is = complete, it will go out for community comment and potentially = revised.


### Work of the RFC Production = Center

The RFC = Production Center (RPC) is solely responsible for two steps in the = document lifecycle:

1. The publication of the RFC, which uses a number of = internal tools to manage references, assign DOIs and = more. 
2. The conversion of the XML RFC into = the long term ar PDF/A format for long term archive, which uses = commercial software. 

The tools used solely for the two processes above are not = within scope for this RFP but these steps should appear in the document = lifecycle and coverage map.


## Additional = Details

None.
=E2=80=94=E2=80=94 
=


-- 
Jay Daley
IETF = Executive Director
jay@ietf.org

= --Apple-Mail=_42C7F47E-FD83-4CD4-AFEE-065B645BD888-- From nobody Thu Jun 25 19:29:29 2020 Return-Path: X-Original-To: tools-arch@ietfa.amsl.com Delivered-To: tools-arch@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id C355D3A10CB; Thu, 25 Jun 2020 19:29:24 -0700 (PDT) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -1.898 X-Spam-Level: X-Spam-Status: No, score=-1.898 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HTML_MESSAGE=0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Z53rAD32P2DZ; Thu, 25 Jun 2020 19:29:23 -0700 (PDT) Received: from jays-mbp.localdomain (unknown [158.140.230.105]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPSA id 954F83A10F6; Thu, 25 Jun 2020 19:29:17 -0700 (PDT) From: Jay Daley Content-Type: multipart/alternative; boundary="Apple-Mail=_6511DE79-4053-446A-B6A9-5E5BB0047196" Mime-Version: 1.0 (Mac OS X Mail 13.4 \(3608.80.23.2.2\)) Date: Fri, 26 Jun 2020 14:29:15 +1200 References: <34D4C033-2BC0-43ED-BFA0-34699EF3EB52@ietf.org> To: tools-arch@ietf.org, tools-discuss@ietf.org In-Reply-To: <34D4C033-2BC0-43ED-BFA0-34699EF3EB52@ietf.org> Message-Id: <6886246A-8B94-4E59-8808-6EA8FCB79691@ietf.org> X-Mailer: Apple Mail (2.3608.80.23.2.2) Archived-At: Subject: Re: [Tools-arch] [Tools-discuss] Draft RFP SoW for "Review of the current landscape of IETF document processing tools" X-BeenThere: tools-arch@ietf.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Tools Architecture and Strategy Team List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Jun 2020 02:29:25 -0000 --Apple-Mail=_6511DE79-4053-446A-B6A9-5E5BB0047196 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 Sorry, some minor changes, please use this version and please remember = that if you wish to bid for the RFP then you should either not provide = any comment on the SoW, or declare that when providing comment.=20 Jay =E2=80=94=E2=80=94=20 # SoW - Review of the current landscape of IETF document processing = tools ## Overview The IETF seeks a contractor to review the current landscape of tools = that have been specifically designed to be used by the IETF community to = work with IETF documents (I-Ds and RFCs) and to build a dataset about = these tools, including technical details, maintenance processes, = licensing and others, and further to determine what parts of the = document production, management and usage lifecycle each tool supports. ## Deliverables ### Deliverable 1 - Dataset A dataset about the tools containing a combination of text, simple = scales, simple classifications and supporting narrative. For example, = for licensing, a simple classification may include "GPL v3, Proprietary, = BSD-like" and more. The dataset is to be provided as raw data in CSV format. The data to be collected for the dataset is: 1. A full catalog of tools, including all the relevant tools at = https://tools.ietf.org and other tools that are not listed there but are = known to be in use.=20 2. Details of the maintainer(s) of each tool 3. The licensing of the tool (simple classification) 4. The technical environment each tool is intended to work in (OS, = packages, containers, etc) 5. How each tool is maintained, including 1. where the source code is located 2. how issues are raised (simple classification) 3. the level of version control and release management in use (on a = simple scale) 4. how comprehensive the test coverage is (on a simple scale)=20 5. the form of testing framework in use (simple classification) 6. A subjective assessment of the current maintained state of the tool = (on a simple scale) 7. A subjective assessment of the suitability of each tool (on a simple = scale), based on 3, 4, 5 and 6 above. 8. What stages of the document lifecycle (see below) each tool covers. ### Deliverable 2 - Document lifecycle A model of the IETF document lifecycle including document production = (authoring, review, format conversion, etc), document access (download = tools, search tools, etc) and document usage (extracting certain parts, = etc). This is only to cover the work carried out by the IETF community = and not the work of the RFC Production Centre. There are no templates for this model nor any existing work to build = upon and so it is expected that this will be entirely new.=20 ### Deliverable 3 - Coverage map Some form of coverage map/infographic showing each tool and each stage = of the document lifecycle, the coverage of each stage in the lifecycle = and the subjective assessment of the suitability of each tool. This = coverage map should allow at-a-glance understanding of the coverage of = each stage and each tool including stages covered by multiple tools and = stages not covered by any tool. This coverage map is to be supplied in any common vector format. ## Requirements ### Selecting the tools to assess The contractor will be responsible for drawing up the list of tools to = be included for approval by the IETF LLC. ### Review and transparency The IETF is committed to transparency and community engagement and so = requires this work to be carried out as follows: 1. All three deliverables are to be developed using a public repository = so that interested parties can see the work as it unfolds. 2. The initial list of tools proposed by the contractor will go out for = community comment and the contractor will need to consider the feedback = and potentially revise the list before the final version is presented = for approval. 3. The maintainers of the tools are to be given every opportunity to = recommend changes to the data and the assessments within (though the = contractor retains the final say in what data they choose to include in = the deliverable), which means: 1. Any issues or pull requests raised by maintainers on the = repository are to be considered and responded to. 2. Once the initial version of the dataset is complete, it will go = out for community comment and the contractor will need to consider the = feedback and potentially revise the dataset before the final version is = delivered. 4. The initial versions of the document lifecycle and coverage map will = also go out for community feedback and the contractor will need to = consider the feedback and potentially revise these before the final = versions are delivered. ### Work of the RFC Production Center The RFC Production Center (RPC) is solely responsible for two steps in = the document lifecycle: 1. The publication of the RFC, which uses a number of internal tools to = manage references, assign DOIs and more.=20 2. The conversion of the XML RFC into the long term ar PDF/A format for = long term archive, which uses commercial software.=20 The tools used solely for the two processes above are not within scope = for this RFP but these steps should appear in the document lifecycle and = coverage map. ## Additional Details None. =E2=80=94=E2=80=94=20 --=20 Jay Daley IETF Executive Director jay@ietf.org --Apple-Mail=_6511DE79-4053-446A-B6A9-5E5BB0047196 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=utf-8 Sorry, some minor changes, please use this version and please = remember that if you wish to bid for the RFP then you should either not = provide any comment on the SoW, or declare that when providing = comment. 

Jay

=E2=80=94=E2=80=94 
#= SoW - Review of the current landscape of IETF document processing = tools

## = Overview

The = IETF seeks a contractor to review the current landscape of tools that = have been specifically designed to be used by the IETF community to work = with IETF documents (I-Ds and RFCs) and to build a dataset about these = tools, including technical details, maintenance processes, licensing and = others, and further to determine what parts of the document production, = management and usage lifecycle each tool supports.


## Deliverables

### Deliverable 1 - Dataset

A dataset about the tools containing a = combination of text, simple scales, simple classifications and = supporting narrative.  For example, for licensing, a simple = classification may include "GPL v3, Proprietary, BSD-like" and = more.

The = dataset is to be provided as raw data in CSV format.

The data to be collected = for the dataset is:

1. A full catalog of tools, including all the relevant tools = at https://tools.ietf.org and other tools that are not = listed there but are known to be in use. 
2. = Details of the maintainer(s) of each tool
3. The = licensing of the tool (simple classification)
4. = The technical environment each tool is intended to work in (OS, = packages, containers, etc)
5. How each tool is = maintained, including
    1. where the = source code is located
    2. how issues = are raised (simple classification)
    3. = the level of version control and release management in use (on a simple = scale)
    4. how comprehensive the test = coverage is (on a simple scale) 
    = 5. the form of testing framework in use (simple = classification)
6. A subjective assessment of the = current maintained state of the tool (on a simple scale)
7. A subjective assessment of the suitability of each tool = (on a simple scale), based on 3, 4, 5 and 6 above.
8.= What stages of the document lifecycle (see below) each tool = covers.


### Deliverable 2 -  Document = lifecycle

A = model of the IETF document lifecycle including document production = (authoring, review, format conversion, etc), document access (download = tools, search tools, etc) and document usage (extracting certain parts, = etc).  This is only to cover the work carried out by the IETF = community and not the work of the RFC Production Centre.

There are no templates = for this model nor any existing work to build upon and so it is expected = that this will be entirely new. 


### = Deliverable 3 - Coverage map

Some form of coverage map/infographic = showing each tool and each stage of the document lifecycle, the coverage = of each stage in the lifecycle and the subjective assessment of the = suitability of each tool.  This coverage map should allow = at-a-glance understanding of the coverage of each stage and each tool = including stages covered by multiple tools and stages not covered by any = tool.

This = coverage map is to be supplied in any common vector format.


## Requirements

### Selecting the tools to assess

The contractor will be responsible for = drawing up the list of tools to be included for approval by the IETF = LLC.


### Review and transparency

The IETF is committed to = transparency and community engagement and so requires this work to be = carried out as follows:

1. All three deliverables are to be developed using a public = repository so that interested parties can see the work as it = unfolds.
2. The initial list of tools proposed by = the contractor will go out for community comment and the contractor will = need to consider the feedback and potentially revise the list before the = final version is presented for approval.
3. The = maintainers of the tools are to be given every opportunity to recommend = changes to the data and the assessments within (though the contractor = retains the final say in what data they choose to include in the = deliverable), which means:
    1. Any = issues or pull requests raised by maintainers on the repository are to = be considered and responded to.
    2. = Once the initial version of the dataset is complete, it will go out for = community comment and the contractor will need to consider the feedback = and potentially revise the dataset before the final version is = delivered.
4. The initial versions of the document = lifecycle and coverage map will also go out for community feedback and = the contractor will need to consider the feedback and potentially revise = these before the final versions are delivered.


### = Work of the RFC Production Center

The RFC Production Center (RPC) is = solely responsible for two steps in the document lifecycle:

1. The publication of = the RFC, which uses a number of internal tools to manage references, = assign DOIs and more. 
2. The conversion of = the XML RFC into the long term ar PDF/A format for long term archive, = which uses commercial software. 

The tools used solely for the two = processes above are not within scope for this RFP but these steps should = appear in the document lifecycle and coverage map.


## Additional Details

None.

=E2=80=94=E2=80=94 


-- 
Jay Daley
IETF = Executive Director
jay@ietf.org

= --Apple-Mail=_6511DE79-4053-446A-B6A9-5E5BB0047196--