WP Processor; Online/Offline

The role of a PWP Processor is to shield the specificities of handling a PWP from the rendering engine. This rendering engine, typically the core rendering part of a user agent, should consider that all content it renders, or it needs for rendering (e.g., CSS or font files) are retrieved from the Web via, typically, HTTP(S) as separate resources. A WP Processor makes it possible to bring (Packaged) Web Publications under the same abstraction as a Web Resource identified by a URL.

More specifically, a WP Processor has to achieve several tasks:

• Provide some sort of a local storage, or cache, of the WP content. To achieve this, the processor has to catch the HTTP(S) requests stemming from the rendering engine and has to, possibly, serve the content from its cache or the file. In other words, the WP Processor has to act as a network proxy.
• Take care of the conversion of locators to the consituent resources of a WP, possibly convert from locators provided via state locators or a state independent locator: the differences among these different locators should become transparent to the rendering engine.

The latest evolution of browser technologies around Web Workers [[web-workers]] and Service Workers [[service-workers]] make the development of a PWP Processor feasible. Service Workers provide a flexible and programmable way to efficiently implement local caching of Web Resources. Caching is implemented as a programmable network proxy, meaning that the browser’s rendering engine becomes oblivious to whether a resource originates from the local cache or directly from the network: these are indeed some of the basic functionalities a PWP Processor must provide.

Addressing and Identification

HTTP(S) URLs serve as the fundamental method for addressing a resource, or a fragment thereof, on the Web. Such URLs can also be used to uniquely identify a resource; however, conceptually, the role of addressing and identification are different. Both of these functionalities should be available for Web Publications: a publication should be uniquely identified for, e.g., library catalogues or archival, and a resource locator should be available so that the user could access the content. In other words, a Web Publication SHOULD have both one or even possibly several identifier(s) and one locator. These may be be identical but may also be different: e.g., an identifier may refer to a specific publication by a publisher (e.g., using an ISBN), whereas the locator may refer to a personal copy of that publication that the owner can freely annotate for personal use.

A typical use case for the presence of an identifier beyond the need for a locator is in academic and scholarly publishing. There are currently several methods for citing online works, but there is no equivalent standard method for citations to ebooks. Even if a reflowable ebook is cited by a scholar, the author must refer to the PDF, paper copy, or HTML version to cite it in her bibliography. Identifiers attached to (Packaged) Web Publications should enable stable citations.

A general [[URI]] (which includes the notion of [[URL]]) MAY serve as an identifier using, e.g., the [[ISBN-URN]] or [[UUID]] schemes. But an identifier does not necessarily resolve to a location on the Web, although it is a good practice to have a dereferencable identifier.

There is no ubiquitously accepted method for identifying a publication among the various document formats (whether electronic or printed). Within the scholarly publishing industry, for example, initiatives such as DOI and CROSSREF have addressed this problem, whereas traditional “trade” publishing rely more on ISBN related services. Some of these identifier schemes provide resolver services or a “standard” representation in term of [[URL]]. The definition of (Packaged) Web Publication should be oblivious to the exact identification used; this issue is left to specialized services and industry organizations. Architecturally, the only requirement, regarding identifiers, is that the complete manifest of a (Packaged) Web Publication MUST include a manifest item for one, or more, identifiers, and that these should be stable across, for example, copying the publication or changing its location on the Web. This document concentrates on addressing only, i.e., on the management of locators.

WP Manifests

The manifest of Web Publications is a Web Resource that includes information pertaining to the overall publication structure, such as the default logical reading order(s) of the set of resources that comprise the publication (the “spine”), as well as predictable user-facing meta-structures, such as one or several tables of contents, glossaries, etc. The WP manifest may also include various metadata (either directly or via further references) that are essential for the overall publishing workflow. The exact definition of all possible manifest items, the internal structure and serialization syntax of a manifest, etc., will require additional work as part of a more detailed specification of Web Publications.

A fundamental question that must be answered is how does a WP Processor get hold of the manifest. Because the publication patterns of WP-s can be different depending on publishers, authors, etc., there should be different ways of accessing the manifests: it can be referred to via a link element in an HTML file, can be in an agreed-upon-position within a package, or can be conveyed through a LINK header of an HTTP(S) request. The WP Processor gets hold of the manifest by following a hierarchy of these different possibilities; a separate section in the appendix provides more details. (Note that accessing a manifest also has some security issues; the “Obtaining a manifest” of the “Web App Manifest” [[web-manifest]] document provides some further details that could be adopted, too.)

This algorithm is typically performed by the PWP Processor when initialized with the state independent locator of a particular WP instance.

The “Web App Manifest” [[web-manifest]], currently developed at W3C, is one example of a technology that could be used, or adopted, to define the final formats for PWP manifests. Although the current [[web-manifest]] drafts is geared towards Web Applications, it also includes extension points to define further manifest items necessary for PWP. Some of these extra manifest items have been already explored within the framework of the “Browser Friendly Manifestation” work of the EPUB3.1 Working Group at the IDPF.

Metadata: discovery

Throughout the digital publishing industry, highly specialized metadata vocabularies, and serialization forms thereof, are being used. Within book trade publishing as an example, ONIX [[ONIX]] has attained a dominant status as a metadata package that typically exists (in XML form) independently of the publication, and contains not only bibliographic metadata, but also trade information such as pricing. Scholarly publishing, on the other hand, often uses various derivatives of the ubiquitous BibTeX vocabulary.

While not contradicting the obvious use cases for out-of-line metadata records as used by publishers, retailers and libraries, (Packaged) Web Publications must define a syntax for basic in-line metadata records that is agnostic to the publication’s states. This means that the syntax must seamlessly support discovery and harvesting by both generic Web search engines, as well as dedicated bibliographic/archival/retailer systems. While it is expected that (Packaged) Web Publications will define a minimal set of required metadata (cf. the section ), development and adoption of other vocabularies in (Packaged) Web Publications will most likely be deemed as out of scope. In other words, domain-specific metadata requirements are up to the domains themselves to define via a profiling mechanism, or similar yet-to-be-defined means.

The adoption of HTML as the vehicle for expressing publication-level metadata (i.e., using RDFa [[html-rdfa]] and/or Microdata [[microdata]] for metadata, like authors or title) would have the added benefits of better I18N support than XML or JSON formats.

Additional API-s

Styling and Layout, Pagination

As outlined in [[dpub-latinreq]] or [[dpub-css-priorities]], the Open Web Platform in general, and CSS in particular, is still lacking solutions for meeting all of the publishers’ expectations on satisfactory typography and layout for digital publications. While improved presentation fidelity will be of paramount importance to the overall success and adoption rate of (Packaged) Web Publications, it is clear that many of these issues are going to be addressed on a case-by-case basis by the CSS Working Group over a longer period of time. STM publishing, for example, where the faithful representation and rendering of, say, mathematical or chemical formulae is of a paramount importance, has particularly severe requirements that must be fulfilled by the Open Web Platform technologies. Similarly, dynamic pagination of reflowable content is not natively supported by browsers today, and as a result Reading System developers, for example, are forced to implement pagination using various ad-hoc approaches, all coming with a significant penalty in terms of development costs, performance and stability.

It is anticipated that native support for pagination (in CSS and/or in the DOM) is going to be put forward by stakeholders as a critical component of Web Publications; thus the finalization of Web Publications may be contingent on the availability of a native pagination model for Web content.

Note that the “Houdini” Task Force, recently started jointly by the W3C CSS WG and the W3C TAG, may open new avenues to handle pagination.

Presentation Control and Personalization

When reading long-form (and sometimes mission-critical) publications, personalization—i.e., the ability for users to adapt the presentation to suit their needs—is of a great importance. While technologies such as CSS Media Queries have come a long way in terms of adapting content to devices, this is not the same thing as adapting to a user. Presentation control features are often available in e-book readers of different kinds, for example the possibility to dynamically change font size or background/foreground color schemes, but implementations are brittle and limited due to the lack of an underlying framework that explicitly supports user adaptation.

Web Publications needs to incorporate an explicit framework for achieving advanced and predictable user-triggered presentation control. (Note that from this perspective, accessibility can be seen just a radical case of personalization.)

Archive formats

Regardless of the details of the practical architecture realizing (Packaged) Web Publications, an archive format is necessary for the storage of a publication as one file (e.g., for distribution or possibly archiving) defined as the packed state of a (Packaged) Web Publication. A variety of formats for offline/archival storage of collections of digital resources exist today (e.g., [[OCF]], [[ODF]], and [[OOXML]]), but none of them is universally recognized and supported across all ecosystems. Depending on the general architecture, (Packaged) Web Publications may use one of the deployed formats (e.g., the current EPUB packaging format based on [[OCF]]), or an archive format that is generic and native to the Open Web Platform.

W3C’s Web Platform Working Group has published a Working Draft for a Streamable Package Format for the Web [[web-packaging]] to encompass the needs of various applications (like installing Web Applications or downloading data for local processing). It is not clear at this moment whether browsers will adopt this format, though.

However, the importance of streaming is not paramount for (Packaged) Web Publications. Indeed, the same publication may be accessed by the same user from different clients; if some user-dependent management also keeps track of the latest reading position in the publication, switching from one client to the other may mean that a client would have to “jump” into the content, thereby bypassing streaming. Nevertheless, if browsers, eventually, do converge towards a browser and streaming friendly packaging format, adopting it for (Packaged) Web Publications may become a real alternative. The community will have to balance native browser availability against the the wide availibility of tooling and industry distribution based on [[OCF]].

Canonical Locators Mapping in PWP Processors

A PWP is published on the server. This PWP includes, e.g., the resource for an image, and is published in at least one of the different states:

• Simply on the Web. The “top level” of this state is available, say, through the URL https://example.org/books/1/: in this setup, the URL of the image is, say, https://example.org/books/1/img/mona_lisa.jpg. This is the absolute locator of the image in this state.
• In a package. This is available through the URL, say, https://example.org/packed-books/1/package.zip. This is the absolute locator of the PWP in this state, and the locator for the image depends on the structure of the package format.

The published PWP is also assigned a state independent locator, e.g., the URL https://example.org/published-books/1. Reflecting the unpacked state in terms of (file) structure the state independent locator of the same image is https://example.org/published-books/1/img/mona_lisa.jpg.

To ensure the smooth transtion among states the PWP Processor must ensure a smooth transition among the different locators of any resource within a PWP (the image file in this example).

From a user/author point of view, whenever possible, the state independent locator should be used when referring to the PWP in, e.g., annotations. This is also true for URL-s derived from the state independent locator, like https://example.org/published-books/1/img/mona_lisa.jpg. This means the addressing unequivocal; however, a PWP Processor must be prepared for the more general case.

In view of the above, the functionalities of the PWP processor can be divided into two steps:

1. finding, based on the state independent locator, the right values of the various state locators
2. extract, based on a specific state locator, the exact locators for additional, internal resources

This functionality is based on the requirement that the complete manifest of a PWP MUST include both the state independent locator as well as all available state locators. Consequently, in order to retrieve the state locators, the PWP must first retrieve the PWP manifest using the state independent locator. (See the separate section on the details of the complete manifest retrieval.) Once the complete manifest is available a resource within a PWP can be retrieved by first deriving the relative locator for a resource (e.g., img/mona_lisa.jpg) and then combining it with the absolute locator corresponding to the state of the PWP as referred to by the rendering engine.

There may be “smarter” PWP Processors that make use of local facilities like caching, but those do not modify these conceptual approaches.

Security Models

The security model of the Web, based primarily on the same-origin policy and the concept of “site”, does not apply to portable documents, as the notion of “origin” is based on HTTP properties that are invalidated/non-existent when a document transitions from its online state to the portable state. (Packaged) Web Publications must incorporate a state agnostic security model that defines rules for both the online and portable states.