Subject: [IPP] The Proposed PWG Raster

From: "Petrie, Glen" <[email-masked]>
Date: Thu, 3 Feb 2011 08:31:01 -0800

Hello All,

I have been thinking more on the details of the proposed raster format.

1. I would propose the name: PWG HCD Raster  (PHR)

------------------------------------------------------------------------
------------------------------ 

Assumptions:

==========

1. A PHR is a separate entity from the Job Ticket; that is, the PHR is
not contained in the Job Ticket but is referenced by the Job Ticket

=> A PHR must have sufficient header data to describe "What is this?"
But does not contain HCD Job Ticket attributes.

2. A PHR is usable-by or generated-by one or more HCD Services (Print,
Scan, Fax, Copy).

=> A PHR must have sufficient header data to describe "What is this?"

3. A PHR is usable-by or generated-by one or more external Services
(Print-Preview, Post-Processing, Transforms).

=> A PHR must have sufficient header data to describe "What is this?"

4. A PHR may contain 1-to-many rasters.

=> For more than one raster, the individual raster may be on one or
separate page.

Example: Scan multiple regions of the same page. (Smart form processing)

Example: N-up the next four raster.

=> The bounding region of the raster must be recorded.

5. A PHR is usable-in or generated-in a mobile, cloud, soho, business or
production environment.

=> There may be limitation or requirement based on system resources.

------------------------------------------------------------------------
------------------------------ 

Discussion Items

=============

1.	Full page raster => Each raster is the same size as the media
size.

	a.	To support assumption 5 for the mobile environment; this
need is impractical.  Device in the mobile environment typical have
insufficient system resources to generate a full page raster at a print
dpi.   One could argue that the mobile device/application could
"stream-out" the full page raster.  This is often referred to as banded
output of the raster content.  However, I still contend that encoding
useless top/bottom/left/right white space is wasteful. 

=> A requirement to make PHR input/output streamable.

	b.	To support assumption 2, this need is not required.
Example of scanning a region.  The region may be 1in x 1in or 3in x 10in
or 8.5in x 11.0in but to put these in a PHR of 8.5 in x 11.0 has not
value and consume resources.

=> The bounding region should be recorded and raster should be the size
of the bounding region.

Practical examples:

Print: I would like to place the next 10 raster in the PHR on a single
page.  Yes, the client could have done this but would be required to.

Scan: I would like scan 3 areas of the 8.5 in x 11.0 in document and
record in a PHR.   By recording the bounding region of each raster; then
each raster could be associated with a field on the original scanned
document such as date, name, amount, etc.  This would be extremely
usefully to external Services.

2.	Term "HW Resolution"

	a.	The reference to 'HW' within the PHR is not important to
HCD services.  The reference to "resolution" is not technically correct.
What is important to HCD/External services is the Raster DPI.  In
addition, there is concept of fast-scan / slow-scan and fast-print /
slow-print directions which imply different DPI's in each direction.
Any scaling (integer or otherwise) between the "preferred (actual) DPI"
of the raster and the Job Ticket DPI is determinable by the HCD service.

=> The PHRHDR should record the raster horizontal and vertical DPI's. 

------------------------------------------------------------------------
------------------------------ 

Requirements

===========

1. All internal PHR rasters shall have the same set of header content

=> There is no separate pre-header within a PHR

PHRHDR === PWG HCD Raster Header

2. A PHR shall be identifiable.

=> There is an explicit identifier in the PHRHDR

=> A PHRHDR contains the PHR format version

3. A PHRHDR shall record the horizontal (witdth) and vertical (height)
DPI of the raster.

a. If a PHR contains raster for the same page of different DPI, all
raster will be rendered by the HCD service at the largest DPI and shall
be scaled by the HCD.

4. A PHRHDR shall record the width (horizontal) and height (vertical)
number of pixels of the raster.

5. A PHRHDR shall record the number of (8-bit) bytes in single
horizontal raster line.

6. A PHRHDR shall record the top and left position, relative to the
media's top-left physical corner, of the raster.

7. A PHRDR shall record the Color Space, Color Order, Bits-per-pixel and
Bits-per-color for the raster.

8. A PHRHDR shall record the raster number for this raster and total
number raster in the PHR

9. A PHR shall be input and output streamable.

10. A PHRHDR shall record a new page indicator.

11. Byte order of PHR shall follow network convention of "big endian".

12. A PHRHDR shall record the duplex parameter for the raster.  This
does mean that the duplex parameters are to be performed by the HCD; it
means what duplex parameter were applied by the creator of the raster in
the PHR.

13. A PHRHDR shall record the type of compression used for the raster
data.

------------------------------------------------------------------------
------------------------------ 

Terms and Definitions

=================

Later.... 

------------------------------------------------------------------------
------------------------------ 

Structure and Format of PHRHDR

==========================

(I was not picky on order)

Bytes                            Value
Description 

---------                           ------------------
------------------------------

[01] --- [10]                    PHR xxxxxx                  PHR
identifier/Version

[11] --- [11]                    RasterNumber               Number, of
total, for this raster

[12] --- [12]                    TotalRaster                    Total
number of raster in PHR

[13] --- [16]                    RasterWidth                  Raster
width in pixels

[17] --- [20]                    RasterHeight                 Raster
height in pixels

[21] --- [24]                    RasterBPL                    Raster
bytes-per-line

[25] --- [28]                    HorizDPI                       Raster
DPI in horizontal direction

[29] --- [32]                    VertDPI                         Raster
DPI in vertical direction

[33] --- [36]                    TopOffset                      Raster
Top offset at raster vertical DPI

[37] --- [40]                    LeftOffset                      Raster
left offset at raster horizontal DPI

[41] --- [41]                    NewPage                      Raster is
on a new page flag

[42] --- [42]                    ColorSpaceID
Identification number for color space

[43] --- [43]                    BitsPerPixel                  Number of
bits per pixel

[44] --- [44]                    BitsPerColor                  Number of
bits per color 

[45] --- [45]                    ColorOrderID
Identification number for color ordering in a pixel

[46] --- [46]                    DuplexID                       Id
number for duplex operation performed

[47] --- [47]                    CompressionID              Id number
for raster compression method

[48] --- [64]                    0XFF
Reserve byte for future by PWG

[65] --- [95]                    0xEE                            Reserve
byte for Vendor/Service/App

[96] --- [99]                    CmpRasterLen               Length in
bytes of compressed raster to follow

[100] ----- >                    compressed raster data.

See CUPS Raster for possible values of ID variables.

-- 
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.

_______________________________________________
ipp mailing list
[email-masked]
https://www.pwg.org/mailman/listinfo/ipp

---

From: Michael Sweet <[email-masked]>
Date: Thu, 3 Feb 2011 11:40:29 -1000

Glen,

Let me preface my response with this: I don't want to define an entirely new and complex raster format. I believe the consensus at the F2F was the same.

That said, my comments are inline below...

> 1. I would propose the name: PWG HCD Raster  (PHR)

We only use "HCD" in the IDS group, and as several people pointed out in the F2F we don't want the name or format to be too closely tied to any one aspect of a HCD/MFD/printer/whatever.
 
> ------------------------------------------------------------------------------------------------------ 
> Assumptions:
> ==========
> 1. A PHR is a separate entity from the Job Ticket; that is, the PHR is not contained in the Job Ticket but is referenced by the Job Ticket
> => A PHR must have sufficient header data to describe "What is this?"  But does not contain HCD Job Ticket attributes.

I don't believe that the job ticket referenced the document; rather, the job associates the job ticket with the document(s) in the job.
>  
> 2. A PHR is usable-by or generated-by one or more HCD Services (Print, Scan, Fax, Copy).

Why? What problem are you trying to solve?
 
> 3. A PHR is usable-by or generated-by one or more external Services (Print-Preview, Post-Processing, Transforms).

Again, why? What problem are you trying to solve?
 
> => For more than one raster, the individual raster may be on one or separate page.

Let me reiterate a point I made in the F2F - we really don't want to design yet another raster format. IIRC PDF/is does allow for more than one image stream per page and is far better suited to this particular use case. PWG Raster/CUPS Raster is a simple page-based raster format.
 
> 5. A PHR is usable-in or generated-in a mobile, cloud, soho, business or production environment.

This is already discussed in the draft spec and it the motivation behind using PackBits/RLE as the basic compression algorithm over higher-performing (but higher-resource) algorithms.
 
> ------------------------------------------------------------------------------------------------------ 
> Discussion Items
> =============
> Full page raster => Each raster is the same size as the media size.
> To support assumption 5 for the mobile environment; this need is impractical.  Device in the mobile environment typical have insufficient system resources to generate a full page raster at a print dpi.   One could argue that the mobile device/application could “stream-out” the full page raster.  This is often referred to as banded output of the raster content.  However, I still contend that encoding useless top/bottom/left/right white space is wasteful.
>  

Yes, and if you'll notice this is in the current draft already and a specific feature of CUPS Raster.

(and sending whitespace for the margins is trivial, very space efficient, and generally simpler for the client)

> To support assumption 2, this need is not required.  Example of scanning a region.  The region may be 1in x 1in or 3in x 10in or 8.5in x 11.0in but to put these in a PHR of 8.5 in x 11.0 has not value and consume resources.
>  
> => The bounding region should be recorded and raster should be the size of the bounding region.
>  
> Practical examples:
> Print: I would like to place the next 10 raster in the PHR on a single page.  Yes, the client could have done this but would be required to.
> Scan: I would like scan 3 areas of the 8.5 in x 11.0 in document and record in a PHR.   By recording the bounding region of each raster; then each raster could be associated with a field on the original scanned document such as date, name, amount, etc.  This would be extremely usefully to external Services.

PDF/is can address the multi-image per page case.

For the simple 1-image case the PWG Raster can just provide the area scanned (and not be a full page - again this isn't a print-only format), and could even provide multiple "pages", one per area scanned.
 
> Term “HW Resolution”
> The reference to ‘HW’ within the PHR is not important to HCD services.  The reference to “resolution” is not technically correct.  What is important to HCD/External services is the Raster DPI.  In addition, there is concept of fast-scan / slow-scan and fast-print / slow-print directions which imply different DPI’s in each direction.  Any scaling (integer or otherwise) between the “preferred (actual) DPI” of the raster and the Job Ticket DPI is determinable by the HCD service.
>  
> => The PHRHDR should record the raster horizontal and vertical DPI’s.

Actually we want Pixels Per Inch (PPI). WRT the format, PPI == DPI == HWResolution, the difference is in the name and not the actual semantics.

> ...
> ... 
> [96] --- [99]                    CmpRasterLen               Length in bytes of compressed raster to follow

This kills streaming...

________________________________________________________________________
Michael Sweet, Senior Printing System Engineer, PWG Chair

---

From: "Petrie, Glen" <[email-masked]>
Date: Thu, 3 Feb 2011 14:45:34 -0800

...

	... 

	[96] --- [99]                    CmpRasterLen
Length in bytes of compressed raster to follow

This kills streaming...

[gwp] Oops, this should do after the raster content to entire the entire
raster was received.

---

From: Michael Sweet <[email-masked]>
Date: Thu, 3 Feb 2011 15:11:34 -1000

On Feb 3, 2011, at 12:45 PM, Petrie, Glen wrote:
> [gwp] Oops, this should do after the raster content to entire the entire raster was received.

How would that help? And what about raster data larger than 4GiB?

---

From: "Petrie, Glen" <[email-masked]>
Date: Thu, 3 Feb 2011 17:14:26 -0800

It was proposed only as a check value - it could be dropped  (or use
something like a check sum)

________________________________

From: Michael Sweet [mailto:[email-masked]] 
Sent: Thursday, February 03, 2011 5:12 PM
To: Petrie, Glen
Cc: [email-masked]; [email-masked]; [email-masked]
Subject: Re: [IPP] The Proposed PWG Raster

		...

		... 

	This kills streaming...

	[gwp] Oops, this should do after the raster content to entire
the entire raster was received.

---

From: "Petrie, Glen" <[email-masked]>
Date: Fri, 4 Feb 2011 05:56:46 -0800

After more consideration, I was not happy with requirement 3  and
propose the following change 

Was

 

To

a. If a PHR contains raster for the same page of different DPI, all
raster will be rendered by the HCD service at the DPI of the first
raster for the page in PHR and shall scale all other raster for the same
page to the DPI of the first raster.

Based on emails with Mike, I modified my proposal of the byte data at
the end to 

Was

To

[65] --- [99]                    0xEE                            Reserve
byte for Vendor/Service/App

 

From: [email-masked] [mailto:[email-masked]] On Behalf Of
Petrie, Glen
Sent: Thursday, February 03, 2011 8:31 AM
To: [email-masked]; [email-masked]; [email-masked]
Subject: [MFD] The Proposed PWG Raster

Hello All,

Assumptions:

==========

Discussion Items

=============

Practical examples:

2.	Term "HW Resolution"

Requirements

===========

 

Terms and Definitions

=================

Later.... 

Bytes                            Value
Description 

[48] --- [64]                    0XFF

 

-- 
This message has been scanned for viruses and 
dangerous content by MailScanner <http://www.mailscanner.info/> , and is

believed to be clean.

---

From: "Petrie, Glen" <[email-masked]>
Date: Fri, 4 Feb 2011 16:03:06 -0800

Played with writing some sample code and changed the length of a few of
the values

[01][04]  PHR identifier 

[05][10]  PHR Version 

[11][14]  Number, of total, for this raster 

[15][18]  Total number of raster in PHR 

[19][22]  Raster width in pixels 

[23][26]  Raster height in pixels 

[27][30]  Raster bytes-per-line 

[31][34]  Raster PPI in horizontal direction 

[35][38]  Raster DPI in vertical direction 

[39][42]  Raster Top offset at raster vertical PPI 

[43][46]  Raster left offset at raster horizontal PPI 

[47][47]  Raster is on a new page flag 

[48][48]  Identification number for color space 

[49][49]  Number of bits per pixel 

[50][50]  Number of bits per color  

[51][52]  Identification number for color ordering in a pixel 

[53][53]  Id number for duplex operation performed 

[54][54]  Id number for raster compression method 

[55][69]  Reserve byte for future by PWG 

[70][79]  Reserve byte for future by vendor 

[80][89]  Reserve byte for future by services

[90][99]  Reserve byte for future by apps 

From: [email-masked] [mailto:[email-masked]] On Behalf Of
Petrie, Glen
Sent: Friday, February 04, 2011 5:57 AM
To: [email-masked]; [email-masked]; [email-masked]
Subject: RE: [MFD] The Proposed PWG Raster

Was

 

To

Was

To

 

Hello All,

Assumptions:

==========

Discussion Items

=============

Practical examples:

2.	Term "HW Resolution"

Requirements

===========

 

Terms and Definitions

=================

Later.... 

Bytes                            Value
Description 

[48] --- [64]                    0XFF

 

believed to be clean. 

believed to be clean.

---

From: "Petrie, Glen" <[email-masked]>
Date: Mon, 7 Feb 2011 06:58:45 -0800

---

From: "Petrie, Glen" <[email-masked]>
Date: Mon, 7 Feb 2011 11:47:19 -0800

After discussion with my Japanese colleague, I had decided to withdraw
my proposal/suggestion for the PWG raster format.

Glen