Subject: [Tools-arch] Recommendation 6: Architectural model for clients

From: Jay Daley <[email-masked]>
Date: Fri, 9 Apr 2021 13:12:25 +1200

=  Recommendation

Recommendation 6:  This survey has identified four broad categories of client that are in use: Editor, Build System, Web Service and Command Line Tool; and four broad categories of functionality provided by the tools: Editing, Boilerplate, Validation and Format Conversion.  An architectural model is needed that sits above and incorporates these categories of clients and functionality, and determines how new and existing services/tools integrate, easily and efficiently. 

=  Commentary

Respondents have been clear that they use the tools from a variety of clients and the current de facto interoperability model is the filesystem.  

	• Editors directly invoke tools locally by pushing the edited content to a temporary file and then reading in the results

	• Build systems invoke multiple tools sequentially passing the output from one to another using a filesystem

	• Using a web service as an interactive UI to a tool with the document provided as input

	• Direct invocation front the command line on a local document.

If these categories of clients are explicitly recognised and put on an equal footing then this leads to a big question:

Should our tools development adopt a modern paradigm for supporting a variety of clients doing the same thing, which is a single backend API with different front ends all calling that same API?

If that is to be the case then it is worthwhile considering standardising aspects of those APIs so that new tools authors do not have to start from scratch and so that clients do not need to support too many different APIs.  Such standardisation might include:

	• The expected action of the tools
		• Editing - alter the existing document/fragment
		• Boilerplate - insert into an existing document/fragment
		• Validation - make no changes to the existing document/fragment but provide a set of annotations of it
		• Format conversion - provide an entirely new document/fragment
	• Fragment identification, selection, annotation etc.

This could come under a single architectural model such as::

        Clients                            API Services      
                                                            
┌──────────────────────┐             ┌──────────────────────┐
│                      │             │                      │
│                      │             │   ┌──────────────┐   │
│                      ├────────────▶│   │   Extract    │   │
│                      │             │   │   Elements   │   │
│   ┌──────────────┐   │             │   └──────────────┘   │
│   │    Editor    │   │             │   ┌──────────────┐   │
│   │              │   │             │   │   Insert     │   │
│   └──────────────┘   ├────────────▶│   │   Custom or  │   │
│   ┌──────────────┐   │             │   │ Boilerplate  │   │
│   │    Build     │   │             │   └──────────────┘   │
│   │    System    │   │             │   ┌──────────────┐   │
│   └──────────────┘   │             │   │   Validate   │   │
│   ┌──────────────┐   ├────────────▶│   │   Full or    │   │
│   │     Web      │   │             │   │   Partial    │   │
│   │   Service    │   │             │   └──────────────┘   │
│   └──────────────┘   │             │   ┌──────────────┐   │
│   ┌──────────────┐   │             │   │   Convert    │   │
│   │   Command    │   ├────────────▶│   │   Between    │   │
│   │  Line Tool   │   │             │   │   Formats    │   │
│   └──────────────┘   │             │   └──────────────┘   │
│                      │             │   ┌──────────────┐   │
│                      ├────────────▶│   │  Datatracker │   │
│                      │             │   │  Submission  │   │
│                      │             │   └──────────────┘   │
│                      │             │                      │
└──────────────────────┘             └──────────────────────┘

-- 
Jay Daley
IETF Executive Director
[email-masked]

---

From: Mark Nottingham <[email-masked]>
Date: Fri, 9 Apr 2021 14:37:07 +1000

Hi Jay,

> On 9 Apr 2021, at 11:12 am, Jay Daley <[email-masked]> wrote:
> 
> =  Recommendation
> 

I'm not sure 'architectural model' is the right terminology here. What we're seeing is different modes of access to the tools -- I think our goal is to assure that all tools a) are able to be used in each of the modes, and b) get as much leverage/common use out of the provided interfaces as possible (by standardising formats / options / etc.).

> Should our tools development adopt a modern paradigm for supporting a variety of clients doing the same thing, which is a single backend API with different front ends all calling that same API?

I don't understand the use of 'API' here. Are you suggesting that there is some service that runs in the network centrally to provide all tool functions? Or is this an API specific to one programming language?

--
Mark Nottingham   https://www.mnot.net/

---

From: Jay Daley <[email-masked]>
Date: Fri, 9 Apr 2021 17:10:46 +1200

> On 9/04/2021, at 4:37 PM, Mark Nottingham <[email-masked]> wrote:
> 
> Hi Jay,
> 

I’m happy to use your phrase ‘modes of access’ as that’s better for understanding one part of this. By architectural model I meant the overall model that documents both these modes and how those modes of access are supported, which is where your next question touches on. 
> I don't understand the use of 'API' here. Are you suggesting that there is some service that runs in the network centrally to provide all tool functions? Or is this an API specific to one programmi…

The former, which I personally think is the right way to go about it, with various clients to those central APIs to support the different modes above.  However, I know that Robert for one has a different view. 

Jay

-- 
Jay Daley
IETF Executive Director

---

From: Mark Nottingham <[email-masked]>
Date: Fri, 9 Apr 2021 15:14:53 +1000

> On 9 Apr 2021, at 3:10 pm, Jay Daley <[email-masked]> wrote:
> 

I think I agree with Robert here, but I'd like to understand what's driving towards centralisation. If there's some 'source of truth' necessary, an API might make sense; however, my current mental model is that the tools should operate on well-documented formats, which can live in a variety of places (e.g., data tracker, my local disk, GitHub, a server somewhere).

Cheers,

---

From: "Martin Thomson" <[email-masked]>
Date: Mon, 19 Apr 2021 11:17:18 +1000

On Fri, Apr 9, 2021, at 15:14, Mark Nottingham wrote:
> I think I agree with Robert here, but I'd like to understand what's 
> driving towards centralisation. If there's some 'source of truth' 
> necessary, an API might make sense; however, my current mental model is 
> that the tools should operate on well-documented formats, which can 
> live in a variety of places (e.g., data tracker, my local disk, GitHub, 
> a server somewhere).

I like that model, but I want to ensure that we all understand what that entails.  Right now, we have a bunch of tools that are generally deployed as command-line utilities.  In general, these take a file (or files) and a bunch of options.  They produce a status code and output that might be directed to a file.

We ship these in a variety of ways.  I get xml2rfc using pip.  I get kramdown-rfc2629 using gem.  I get idnits and rfcdiff by downloading a tarball from tools.ietf.org.  Other people don't bother with the getting part, they just use a web page.  Others cases (rfcdiff for instance) effectively use scripted remote access to underlying web resources.

That is largely good.  We provide flexibility, but the overall complexity is low.  A user that climbs the complexity curve from web page to remote access to command-line has a few minor hurdles to clear that we might help them with by documenting each tool better, but the overall shape of the system seems right to me.

We manage these tools in different ways too.  Most are individually maintained, and so are subject to whatever the individual maintainer prefers (GitHub more often than not).

As far as centralization goes, what I would like to see is a deliberate decision to sponsor tools that will get additional resources.  What that might mean is greater consistency of support (more updates, x.ietf.org hosting, help with fixing bugs), greater consistency of interface (at different levels), greater functionality (by paying for features to be added), or other things we might collectively value.

Sponsorship might come with some modest conditions.  For instance, if we believe that a particular tool is valuable enough to sponsor we might decide that sponsorship is conditional on: it being written in a language better suited to maintenance, or hosted somewhere more conducive to community input, or distributed using a more reliable platform, or that it gets a better test harness that can be automated (note: I don't know if any of these things are concretely problems; that would be something that would need to be assessed for each tool).

---

From: "John Levine" <[email-masked]>
Date: 19 Apr 2021 14:04:07 -0400

It appears that Martin Thomson  <[email-masked]> said:
>I like that model, but I want to ensure that we all understand what that entails.  Right now, we have a bunch of tools that are generally
>deployed as command-line utilities.  In general, these take a file (or files) and a bunch of options.  They produce a status code and output
>that might be directed to a file.
>
>We ship these in a variety of ways.  I get xml2rfc using pip.  I get kramdown-rfc2629 using gem.  I get idnits and rfcdiff by downloading a
>tarball from tools.ietf.org.  Other people don't bother with the getting part, they just use a web page.  Others cases (rfcdiff for instance)
>effectively use scripted remote access to underlying web resources.
>
>That is largely good.  We provide flexibility, but the overall complexity is low.

I cannot agree. We have a bunch of tools each of which work
differently, and are very difficult to install if you don't happen to
be an experienced Linux or Unix user. Installing tarballs on Windows
machines is nearly impossible, and even on Linux or Macs I've had
people tell me they gave up trying to install xml2rfc because
something failed mysteriously, probably incompatible library versions
or colliding fonts. In some cases there are web versions, but in too
many the web versions are out of date or don't provide all of the
features of the command line version.

We need to shrink the number of ways that the tools work to a small
enough set that we can describe and maintain them.

R's,
John

---

From: Mark Nottingham <[email-masked]>
Date: Tue, 20 Apr 2021 12:14:06 +1000

> On 20 Apr 2021, at 4:04 am, John Levine <[email-masked]> wrote:
> 
> or colliding fonts.

Commercial and Open Source Python applications are distributed as binaries or as standalone distributions all the time successfully, and are marketed to much less technical audiences than the IETF.

> In some cases there are web versions, but in too

That seems like a problem that can be easily solved with some operational discipline.

> enough set that we can describe and maintain them.

Nothing above leads me to believe that we need to fundamentally revisit how our tools relate to each other. 

Cheers,

---

From: "John R Levine" <[email-masked]>
Date: 19 Apr 2021 22:32:26 -0400

>> differently, ...

> Commercial and Open Source Python applications are distributed as 
> binaries or as standalone distributions all the time successfully, and 
> are marketed to much less technical audiences than the IETF.

Sure, but that requires a much more sophisticated and disciplined software 
development environment than we have.  I'll let the various volunteers 
chime in here and say how hard they think it'd be to ship versions of 
their stuff for, say, Windows, Mac, and some version of Linux.

> Nothing above leads me to believe that we need to fundamentally revisit how our tools relate to each other.

If you want standalone distributions for multiple platforms we sure do.

A large part of the appeal of Jay's proposal that we move to APIs and 
front ends is that the main program that exposes the API only has to work 
in one environment, and we can have relatively small generic front ends 
for each platform and for a web service.  AWS does this with a command 
line front end that drives all of their various services.

Regards,
John Levine, [email-masked], Taughannock Networks, Trumansburg NY
Please consider the environment before reading this e-mail. https://jl.ly

---

From: Mark Nottingham <[email-masked]>
Date: Tue, 20 Apr 2021 12:38:53 +1000

On 20 Apr 2021, at 12:32 pm, John R Levine <[email-masked]> wrote:
> Sure, but that requires a much more sophisticated and disciplined software development environment than we have.  I'll let the various volunteers chime in here and say how hard they think it'd be to ship versions of their stuff for, say, Windows, Mac, and some version of Linux.
> 
> A large part of the appeal of Jay's proposal that we move to APIs and front ends is that the main program that exposes the API only has to work in one environment, and we can have relatively small generic front ends for each platform and for a web service.  AWS does this with a command line front end that drives all of their various services.

If we can't even maintain a 'sophisticated and disciplined software development environment', I have no idea how we're going to make such services available -- especially when the IETF's current services often struggle to be available and performant. Also, the *point* of AWS's services are that they're in the cloud; I still don't have a good answer as to why transforming some XML into HTML needs to be.

Let me be clear -- moving to a service-oriented architecture has bedevilled the industry for about two decades now; it is not simple, and it is not somehow magically easier than shipping software. Teams in many companies are still trying to figure out how to do it best, and they have substantially greater resources available than we do.

This group was convened to get expert input on the tools. Mine is to strongly urge you (Jay) not to pursue this path.

---

From: "John R Levine" <[email-masked]>
Date: 20 Apr 2021 11:10:28 -0400

> If we can't even maintain a 'sophisticated and disciplined software 
> development environment', I have no idea how we're going to make such 
> services available -- especially when the IETF's current services often 
> struggle to be available and performant. Also, the *point* of AWS's 
> services are that they're in the cloud; I still don't have a good answer 
> as to why transforming some XML into HTML needs to be.

I'm now completely confused about what you want us to do.  Assuming we 
agree that we want to have tools that work reliably on the platforms that 
people are likely to use (which seems to be Windows, Mac, and linux) how 
do we get there?

Regards,

---

From: Mark Nottingham <[email-masked]>
Date: Fri, 23 Apr 2021 11:32:39 +1000

> On 21 Apr 2021, at 1:10 am, John R Levine <[email-masked]> wrote:
> 
> I'm now completely confused about what you want us to do.  Assuming we agree that we want to have tools that work reliably on the platforms that people are likely to use (which seems to be Windows, Mac, and linux) how do we get there?

By improving the tools we have -- including how they're delivered, documented and supported.

---

From: "John R Levine" <[email-masked]>
Date: 22 Apr 2021 22:23:24 -0400

> By improving the tools we have -- including how they're delivered, documented and supported.

I was hoping for something more concrete.

For example, should we be able to provide each tool as a native 
installable package on the major platforms?  If so, do you have an 
estimate of how much work that would involve?

(Try installing xml2rfc on Windows to see how painful the current setup 
is.)

Regards,

---

From: Mark Nottingham <[email-masked]>
Date: Mon, 26 Apr 2021 12:07:07 +1000

> On 23 Apr 2021, at 12:23 pm, John R Levine <[email-masked]> wrote:
> 
> For example, should we be able to provide each tool as a native installable package on the major platforms?  If so, do you have an estimate of how much work that would involve?

My role here, as I understand it, is not to provide estimates or detailed plans -- it's to provide high-level input to the architecture (note: architecture) of our tools stack. Jay, if I misunderstand that, please correct me.

> (Try installing xml2rfc on Windows to see how painful the current setup is.)

Well, about 20 seconds on DuckDuckGo found:
  
"""
“Freezing” your code is creating a single-file executable file to distribute to end-users, that contains all of your application code as well as the Python interpreter.

Applications such as ‘Dropbox’, ‘Eve Online’, ‘Civilization IV’, and BitTorrent clients do this.

The advantage of distributing this way is that your application will “just work”, even if the user doesn’t already have the required version of Python (or any) installed. On Windows, and even on many Linux distributions and OS X, the right version of Python will not already be installed.
""" -- <https://docs.python-guide.org/shipping/freezing/>

There are other approaches if that isn't suitable, e.g.:
  https://cyrille.rossant.net/create-a-standalone-windows-installer-for-your-python-application/

---

From: "John R Levine" <[email-masked]>
Date: 27 Apr 2021 17:50:47 -0400

On Sun, 25 Apr 2021, Mark Nottingham wrote:
>>> By improving the tools we have -- including how they're delivered, documented and supported. ...

> “Freezing” your code is creating a single-file executable file to distribute to end-users, ...

Well, OK, we could probably do that give or take the detail that frozen 
executables need for the user's machine to have a C compiler installed to 
use some of its libraries, and it would bundle a bunch of other people's 
libraries with licenses incompatible with ours, and the PDF libraries and 
fonts are enormous, over a gigabyte.

But that misses the point -- once you've done that, you have a copy of 
xml2rfc on your Windows machine or maybe your Mac you can run from a 
command line to render a file of XML into HTML or PDF or text.  But the 
Windows users I know don't run stuff from the command line, and that 
doesn't even matter because they don't have any way to prepare XML to 
render.

What they actually do is to use Joe Touch's MS Word template that lets 
them type up a text draft, with boilerplate and page headers and such. 
When they submit it, the RPC exports it as text and manually adds all of 
the XML coding.  Considering that the authors surely know what is a header 
and what is a paragraph or list as they create their drafts and probably 
taggged much of it with Word styles, I'd say that's a pretty egregious 
tool failure.  The authors I've talked to feel rather strongly that they 
like Word and are not interested in switching.

In AUTH48 we're just as weak -- author reviews and corrections are tracked 
entirely by hand from email exchanges. In principle authors are supposed 
to review the XML, in practice they rarely do, just some of the rendered 
versions, and since we don't have an XML diff, even if you submitted XML, 
it's hard to tell what changed in the final version.

We have tools that work OK for a subset of authors who work in particular 
ways, e.g, write in markdown or in Emacs with XML mode, and can write 
makefiles to do the rendering.  For everyone else, there's way too much 
manual work and it's way too hard to proofread.

Regards,
John Levine, [email-masked], Taughannock Networks, Trumansburg NY
"I dropped the toothpaste", said Tom, crestfallenly.

---

From: "Martin Thomson" <[email-masked]>
Date: Wed, 28 Apr 2021 08:55:13 +1000

> like Word and are not interested in switching.

That's a failure on our part then.  That process is not a good result for anyone involved.

It is relatively easy to use word to drive a process that produces XML.  The one I used recently involves all the same command-line tools, but with a step that converts the word into markdown.

It should be possible to build something more direct than that if there was enough interest.  I'm not volunteering, but suggesting that some of the LLC tools budget might be pointed in that direction.

> it's hard to tell what changed in the final version.

I just got an XML diff from the RPC in AUTH48.

---

From: "John Levine" <[email-masked]>
Date: 28 Apr 2021 21:12:30 -0400

It appears that Martin Thomson  <[email-masked]> said:
>It is relatively easy to use word to drive a process that produces XML.  The one I used recently involves all the same
>command-line tools, but with a step that converts the word into markdown.

In principle, it's straightfoward to translate Word files into other
formats. (The file format is ISO/IEC 29500, a bunch of XML files in a
zip archive. The bad news is that the spec is over 5,000 pages.)
We could build tools to turn Word files into our dialect of
markdown or XML, give or take some negotiations about stylesheets
and markup conventions.

On the other hand, I expect that people who don't want to write
documents in markdown wouldn't be too enthusiastic about reviewing
them in markdown, so before setting off on a project like this I would
want to understand what's going to work for our Word users for the
entire document production process. Maybe it's two way Word <->
markdown, maybe something else. We have a fair number of markdown
users, so if we're building tools that deal with markdown, it would be
nice if they found them useful, too.

This is why I like Jay's approach of trying to figure out what our I-D
authors actually do from writing to editing to AUTH48 to RFC, and see
what the gaps are before trying to guess what tools to update or
create. Also, how are people are likely to use them, e.g., do we want
stuff that runs on our own PCs, or on a web site, or maybe build them
with an API that can be driven either through the web or a command-line
stub.

R's,
John

> I just got an XML diff from the RPC in AUTH48.

Are you sure it wasn't a text diff of the XML?  That works OK if the
incoming XML is in a format close to what the RPC uses, not so great
otherwise.

---

From: "Martin Thomson" <[email-masked]>
Date: Thu, 29 Apr 2021 12:57:41 +1000

> markdown or XML, give or take some negotiations about stylesheets

Sounds good.  We probably need to stop saying "build" though.  I used code that I found on a website and it worked nicely.

My sense is that these tools just cherry pick a very narrow set of things, so it should be possible to re-tread the path they have taken, but I haven't seen any need to do that.

> authors actually do [...]

I thought that the survey results were clear here on this point.  There are far too many people who use Joe Touch's Word template.  I think that it's clear that these people need a better alternative (or Joe's template needs some attention so that it can produce XML).

> incoming XML is in a format close to what the RPC uses, not so great

You are right, of course.  And that is perfectly adequate for my purposes.  Mostly.  Maybe a true XML diff wouldn't mangle things so badly when things get out of sync.

---

From: Lars Eggert <[email-masked]>
Date: Thu, 29 Apr 2021 13:17:16 +0300

Hi,

On 2021-4-29, at 4:12, John Levine <[email-masked]> wrote:
> create.

do we have any data on the number of Word authors, whether the numbers are going up or down, and whether there are clusters in certain WGs and/or areas?

I wonder if we could push more docs to XML by assigning editors to documents, esp. WG documents, that are up to speed with authoring in XML or Markdown.

Thanks,
Lars

---

From: "John Levine" <[email-masked]>
Date: 29 Apr 2021 12:04:37 -0400

>Sounds good.  We probably need to stop saying "build" though.  I used code that I found on a website and it worked nicely.

While I am all in favor of using or adapting stuff that already
exists, I don't get the impression that you're offering to train all
the other IETFers that might want to use that code, fix bugs when
people find them, make sure it works on other systems than yours, and
so forth. Maybe "build" isn't the best word but the IETF has a budget
for tools so we can support the stuff we care about.

> I thought that the survey results were clear here on this point.  There are far too many people who use Joe Touch's Word
> template.  I think that it's clear that these people need a better alternative (or Joe's template needs some attention
> so that it can produce XML).

it does produce XML, but it's ISO 29500 XML, not our XML.  As I said, it should be possbible to build
tools that use style tags in their XML to translate to something that works better in our publication 
process but I would like to understand the whole path the documents take so we don't end up with something that
sort of works in theory but acutal authors don't use.

R's,
John

-- 
Regards,
John Levine, [email-masked], Primary Perpetrator of "The Internet for Dummies",
Please consider the environment before reading this e-mail. https://jl.ly