Subject: Re: [IPP] Posted 1.1 release candidate tools

From: "Kennedy, Smith \(Wireless & IPP Standards\) via ipp" <[email-masked]>
Date: Sat, 16 May 2020 23:07:56 +0000

Hi Mike,

Just now getting time to test these tools with a printer that is already IPP Everywhere™ 1.0 certified, and I'm observing some interesting deltas / issues.

1. The printer is failing test I-10:

    I-10. Get-Printer-Attributes Operation (default)                     [FAIL]
        RECEIVED: 13004 bytes in response
        status-code = successful-ok (successful-ok)
        EXPECTED: finishing-template-supported
        EXPECTED: finishing-template-supported
        EXPECTED: finishings-col-database OF-TYPE collection (got no-value)
        EXPECTED: finishings-col-database/finishing-template OF-TYPE keyword|name (got no-value)
        EXPECTED: finishings-col-default OF-TYPE collection (got no-value)
        EXPECTED: finishings-col-default/finishing-template OF-TYPE keyword|name (got no-value)
        EXPECTED: finishings-col-ready OF-TYPE collection (got no-value)
        EXPECTED: finishings-col-ready/finishing-template OF-TYPE keyword|name (got no-value)

Here's a dump of the Printer's attributes relating to finishing:

Serenity: ipptool-server-captures [503]$ grep finishing _Kennedy\ DeskJet\ 3755_.conf
ATTR keyword job-creation-attributes-supported "copies","finishings","finishings-col","job-pages-per-set","sides","orientation-requested","media","print-quality","printer-resolution","output-bin","media-col","output-mode","print-content-optimize","pclm-source-resolution","print-color-mode","ipp-attribute-fidelity","job-name","page-ranges","multiple-document-handling","job-mandatory-attributes","print-rendering-intent","print-scaling"
ATTR no-value finishings-col-database
ATTR no-value finishings-col-ready
ATTR keyword finishings-col-supported "finishing-template"
ATTR enum finishings-default 3
ATTR no-value finishings-col-default
ATTR enum finishings-supported 3

The problematic line in ipp-tests.test seems to be this:

EXPECT finishings-supported OF-TYPE enum IN-GROUP printer-attributes-tag DEFINE-MATCH HAVE_FINISHINGS

Since the Printer's "finishings-supported" contains only 3 (none), I think the line should instead be:

EXPECT finishings-supported OF-TYPE enum IN-GROUP printer-attributes-tag WITH-VALUE >3 DEFINE-MATCH HAVE_FINISHINGS

2. The new "I-9" test for Identify-Printer never gets exercised because it has SKIP-IF-NOT-DEFINED HAVE_IDENTIFY_PRINTER at the start, but HAVE_IDENTIFY_PRINTER isn't defined until test I-10. I moved I-9 to after I-10.7 and renumbered so that I-10.7 is now I-9.7, I-9 is now I-10. Now it gets executed.

3. The current I-10.5 and I-10.6 are failing on my IPP Everywhere 1.0 certified printer. I'm wondering whether these tests ought to be enabled only if some DEFINE is defined ("PEDANTIC" / "WARNINGS" etc.). I know they should pass, but if that hasn't been exercised previously, that can cause some additional thrash. I don't want to lower the bar and remove those tests, but if there was some grace period, that might help teams who are trying to certify in the near term. I'm not sure what others think. Asking my firmware teams whether fixing that in pending devices' certifications will cause thrash.

Still testing but wanted to share this for now.

Smith

/**
    Smith Kennedy
    HP Inc.
*/

> On Apr 8, 2020, at 2:23 PM, Michael Sweet via ipp <[email-masked]> wrote:
> 
> All,
> 
> I have posted the first release candidate of the IPP Everywhere 1.1 Printer Self-Certification Tools to:
> 
> https://ftp.pwg.org/pub/pwg/ipp/wd/sw-ippeveselfcert11-20200408-macos.zip <https://ftp.pwg.org/pub/pwg/ipp/wd/sw-ippeveselfcert11-20200408-macos.zip>
> https://ftp.pwg.org/pub/pwg/ipp/wd/sw-ippeveselfcert11-20200408-rhel.tar.gz <https://ftp.pwg.org/pub/pwg/ipp/wd/sw-ippeveselfcert11-20200408-rhel.tar.gz>
> https://ftp.pwg.org/pub/pwg/ipp/wd/sw-ippeveselfcert11-20200408-ubuntu.tar.gz <https://ftp.pwg.org/pub/pwg/ipp/wd/sw-ippeveselfcert11-20200408-ubuntu.tar.gz>
> https://ftp.pwg.org/pub/pwg/ipp/wd/sw-ippeveselfcert11-20200408-windows.msi <https://ftp.pwg.org/pub/pwg/ipp/wd/sw-ippeveselfcert11-20200408-windows.msi>
> 
> These tools incorporate the new "ippevesubmit" tool for submitting your self-certification results, fix all of the known issues in the previous version of the tools, and follow the latest draft of the IPP Everywhere 1.1 and IPP Everywhere 1.1 Printer Self-Certification Manual documents:
> 
> https://ftp.pwg.org/pub/pwg/ipp/wd/wd-ippeve11-20200326.pdf <https://ftp.pwg.org/pub/pwg/ipp/wd/wd-ippeve11-20200326.pdf>
> https://ftp.pwg.org/pub/pwg/ipp/wd/wd-ippeveselfcert11-20200312.pdf <https://ftp.pwg.org/pub/pwg/ipp/wd/wd-ippeveselfcert11-20200312.pdf>
> 
> ________________________
> Michael Sweet
> 
> 
> 
> _______________________________________________
> ipp mailing list
> [email-masked]
> https://www.pwg.org/mailman/listinfo/ipp <https://www.pwg.org/mailman/listinfo/ipp>

_______________________________________________
ipp mailing list
[email-masked]
https://www.pwg.org/mailman/listinfo/ipp

---

From: Michael Sweet via ipp <[email-masked]>
Date: Mon, 18 May 2020 09:04:12 -0400

Smith,

Go ahead and file Github issues for the first two issues and I'll get them fixed.

https://github.com/istopwg/ippeveselfcert/issues/new

The last two issues probably need a little discussion (see below)

> On May 16, 2020, at 7:07 PM, Kennedy, Smith (Wireless & IPP Standards) <[email-masked]> wrote:
> 
> Hi Mike,
> 

+1

> 2. The new "I-9" test for Identify-Printer never gets exercised because it has SKIP-IF-NOT-DEFINED HAVE_IDENTIFY_PRINTER at the start, but HAVE_IDENTIFY_PRINTER isn't defined until test I-10. I mov…

Better to define HAVE_IDENTIFY_PRINTER sooner, as a renumbering will require changes to the self-cert manual...

> 3. The current I-10.5 and I-10.6 are failing on my IPP Everywhere 1.0 certified printer. I'm wondering whether these tests ought to be enabled only if some DEFINE is defined ("PEDANTIC" / "WARNINGS…

These tests (for requested-attributes='printer-description' and ='job-template') were added because the non-conformance to STD92 was causing problems with getting full IPP Everywhere support working in CUPS.

I'd like to know what the failures were - too many attributes returned, too few?  Too many isn't a serious interoperability issue - might lead to some laziness on the Client side but testing will catch that - but too few prevents Clients from working which *is* a serious interoperability issue...

________________________
Michael Sweet

---

From: "Kennedy, Smith \(Wireless & IPP Standards\) via ipp" <[email-masked]>
Date: Mon, 18 May 2020 15:10:43 +0000

> On May 18, 2020, at 7:04 AM, Michael Sweet <[email-masked]> wrote:
> 
> Smith,
> 
> https://github.com/istopwg/ippeveselfcert/issues/new <https://github.com/istopwg/ippeveselfcert/issues/new>
> 

Done - https://github.com/istopwg/ippeveselfcert/issues/47 <https://github.com/istopwg/ippeveselfcert/issues/47>

> Better to define HAVE_IDENTIFY_PRINTER sooner, as a renumbering will require changes to the self-cert manual...

OK - perhaps an I-9.1 for a Get-Printer-Attributes detecting whether Identify-Printer is supported, and then an I-9.2 that tests the Identify-Printer operation?

Added - https://github.com/istopwg/ippeveselfcert/issues/48 <https://github.com/istopwg/ippeveselfcert/issues/48>

> I'd like to know what the failures were - too many attributes returned, too few? Too many isn't a serious interoperability issue - might lead to some laziness on the Client side but testing will ca…

I'll look into this - more soon.

---

From: "Kennedy, Smith \(Wireless & IPP Standards\) via ipp" <[email-masked]>
Date: Thu, 21 May 2020 14:46:58 +0000

> On May 18, 2020, at 9:10 AM, Kennedy, Smith (Wireless & IPP Standards) via ipp <[email-masked]> wrote:
> 
>> On May 18, 2020, at 7:04 AM, Michael Sweet <[email-masked] <mailto:[email-masked]>> wrote:
>> 
> 

OK, so the "job-template" returns basically nothing other than "attributes-charset" and "attributes-natural-language", which seems too few. 😞 It seems the firmware is equating 'job-template' with 'none' and 'printer-description' with 'all'.
_______________________________________________
ipp mailing list
[email-masked]
https://www.pwg.org/mailman/listinfo/ipp

---

From: Michael Sweet via ipp <[email-masked]>
Date: Thu, 21 May 2020 11:18:22 -0400

Smith,

> On May 21, 2020, at 10:46 AM, Kennedy, Smith (Wireless & IPP Standards) <[email-masked]> wrote:
>>> ...

I'm happy with relaxing the tests to allow 'all' behavior for group names (lazy, but not an issue for interoperability and an implementation choice I made early in CUPS history and later updated to Do The Right Thing...), but clearly the printer thinks 'job-template' is an attribute name and is filtering out all attributes except 'job-template'... :/

The current expectation (based on the RFC text that dates back to IPP/1.0) is that 'job-template' will return all of the 'xxx-default', 'xxx-ready', and 'xxx-supported' Printer attributes that correspond to the supported Job Template attributes for the printer.

....

The current ad-hoc best practice (as implemented by CUPS and cups-filters) is to send "requested-attributes"='all','media-col-database' as an IPP/2.0 request and then (if that fails) send "requested-attributes"='all' as an IPP/1.1 request and deal with the lack of "media-col-database" information...

When monitoring status, CUPS (and others) typically provide a list of (status) attributes they require, and largely that seems to work with most printers.

So at the very least we need to make sure that 'all','media-col-database' works, as well as requests for specific attributes.  I would also like to see that 'printer-description' and 'job-template' work (to the extent that they return at least the corresponding attributes without an error if they return more), although perhaps we could provide automatic exceptions (i.e. not treat them as hard failures) for some period of time (perhaps for products released through the middle of 2021?) since they are not as critical to interoperability with existing Clients?

________________________
Michael Sweet

---

From: "Kennedy, Smith \(Wireless & IPP Standards\) via ipp" <[email-masked]>
Date: Thu, 21 May 2020 15:44:23 +0000

Hi Mike,

I have reconsidered my position - I think we need to be disciplined about testing what is required. So, let's do what you suggested - have the tests pass if a group returns more than expected, but fail if they return less than expected. We need to draw the line and hold the line.

Smith

/**
    Smith Kennedy
    HP Inc.
*/

> On May 21, 2020, at 9:18 AM, Michael Sweet <[email-masked]> wrote:
> 
> Smith,
>