------------draft------------

"Don't I Already Have that Image?"
Issues in Equivalency of Digital Images

Submitted to First ACM International Conference on Research and Development in Digital Libraries

Howard Besser
Visiting Associate Professor
School of Information & Library Studies
University of Michigan
Ann Arbor, MI 48109-1092
howardb@umich.edu

John Weise
Systems Research Programmer
Information Technology Division
535 W. William St.
University of Michigan
Ann Arbor, MI 48109-4943
jweise@umich.edu

Abstract

The proliferation of images available in digital form will inevitably lead to a situation where the "same" image is available from a variety of sources in a variety of formats. Though encapsulation and encryption methods will guarantee that an image does not differ from what the sender intended to deliver, these tools by themselves will not help determine whether images obtained from different sources are equivalent. Currently there is no reliable method for determining the equivalency of images having different storage and compression formats. (For example, a bit-by-bit comparison of the results of two different types lossless compressions on the same image will yield no similarity whatsoever.) For digital library search engines to avoid returning numerous duplicates, we must first agree on what elements we will need in order to assess equivalency, then we must agree upon common ways of expressing those elements within metadata.

This paper examines the issues of image equivalency. It first reviews the questions raised about image equivalency, and shows that different user groups will disagree over the equivalency of a set of images. The paper then introduces concepts such as visual data equivalency and intellectual equivalency, and uses these concepts to formalize a number of the contrasting views on whether a set of images are equivalent.

The authors analyze and adapt concepts and terminology from the analog world (particularly from how visual resources managers organize slides, and how librarians treat editions). They then develop a framework and terminology to treat a digital image, its derivation history, and its edition and versioning <see attached image>.

The paper then introduces concepts of pre-process and post-process equivalency as further tools in determining whether two images are "the same". The authors hope that by identifying the issues in image equivalency and providing a common set of vocabulary, a group of digital library designers will be able to come to agreement on the metadata elements which will be needed in order to determine image equivalence.

Digital image technology is empowering image collection managers with new ways to preserve and distribute images. In order to manage a digital image collection, librarians, archivists, curators, and information specialists need to understand the relationships that can exist among images of a collection. A key step in the direction of understanding is the definition of vocabulary and the identification of methods for determining whether or not two images are equivalent. We will first describe what we mean by equivalency, and then suggest methods for evaluating relationships between images.

Definition of Digital Images In the Context of Digital Image Reproduction

Visual data is the data that represents the location and color of every pixel of an image. Visual data is displayed on computer screens, and output to printers, video and by other means. When an image is modified, for example, by correcting the color or touching up a blemish, the visual data is directly affected. Header data is information that accompanies the visual data and describes the image. Header data for an image may include keywords, a description, and information about who holds the copyrights for the image. Header data also might indicate the modifications that have been made to an image. For instance, it could contain a note that explicitly states that a blemish was touched up-- along with the name of the person who made the modification, and the date of the change. The image file format defines the structure for how the header data and visual data of the image is stored on the computer's permanent storage device, i.e., hard disk, tape, etc. For example, a file format might state that the header data must first be written to the file, followed by the visual data.

Digital images are unique, in relation to analog images, in that they can be reproduced with absolutely no loss to the image's visual data. In such a case, mathematical, bit-by-bit, comparison of the source image's visual data with the reproduction image's visual data reveals absolutely no difference between them. In fact, it is possible to recreate the source from the reproduction. Therefore, when the visual data of the reproduced image is unequivocally identical to the visual data of the source image, it can be said that the reproduced image's visual data integrity has been upheld through the reproduction process, and that an exact reproduction has been created.

The visual data of a digital image is composed of a known number of discrete elements, i.e., pixels, each that can be directly referenced because they are in a structured image file format. The pixel elements of the digital image are described numerically. Therefore, exact reproduction of a digital image is a matter of duplicating a set of numbers-- a straightforward process with no ambiguity.

Exact reproductions can not be made of analog images. The number of discrete elements of an analog image, for instance, on film, can only be approximated, and there is no structure to the elements' format. Therefore, an image on film can not be exactly reproduced, nor can any other form of analog image.

While digital images differ from analog images in their potential to be reproduced exactly, digital images can also be reproduced using methods that compromise the image's visual data integrity. The resolution reduction of an image or the use of lossy compression schemes that throw away image data least perceivable to the human eye are examples of non-exacting reproduction techniques. Thus, both analog and digital images can be reproduced with a loss of visual information, but only digital images are exactly reproducible.

Intellectual equivalence is when a reproduced image and its source, by all visual measurements, look the same when viewed. For any reproduction process that results in an intellectually equivalent image, it can be said that the process upholds the intellectual integrity of the image. Exact reproductions are always intellectually equivalent to the source image, but an image reproduced with a process that degenerates the visual image data may also be intellectually equivalent. For example, the photographic duplication of images on slide film is a case where the resultant image is intellectually equivalent, despite compromised visual data integrity. Slides are commonly duplicated using photographic processes. Since analog images cannot be duplicated without some loss of visual data, an exact reproduction is not possible. Yet, a normal (i.e., without extraordinary magnification) visual comparison of the source slide and reproduction slide reveals no differences. Intellectually, they convey the same meaning. In the digital world, an image that is scaled down by 80% has had visual data discarded permanently. Yet the overall visual composition is equally representative of the image's intellectual meaning. Thus the intellectual integrity has been preserved and the image is considered intellectually equivalent.

To summarize, there are three types of relationships that can exist between a digital image and a reproduction of it. 1) The data integrity and the intellectual integrity are upheld; the reproduction is exact. 2) The intellectual integrity is upheld, but the data integrity has degenerated. 3) Neither the intellectual integrity nor the data integrity are upheld.

The importance of defining these relationships, and terminology to describe them, has reached a critical point. Librarians, archivists, curators, and information specialists need a common vocabulary built on a clearly defined framework in order to develop and/or adapt standards and conventions for cataloging, accessioning, retaining, migrating, organizing, and interchanging image data. As we proceed in this article to propose a framework and vocabulary for understanding, you should find that the solutions to managing digital images are embodied in descriptive information about the image (metadata).

Metadata is any information that describes an image's purpose, subject, technical specifications, or provenance. Provenance is information regarding the origin and history of an image and is essential to determining both intellectual equivalence and visual data equivalence between images. The provenance of an image is established by metadata that indicates what object or image, if any, was the source for the image at hand. Additionally, provenance is evidence of what individual or organization created the resultant image.

Defining a Framework

The illustration visually frames the relationships that can exist among digital images, and identifies vocabulary for describing the relationships. The terms edition, derivative, and version all have been known to mean different things to different people. We have found it to be valuable to extend the work that Barbara Tillett of the Library of Congress developed in her doctoral dissertation and presented at the NCSA/OCLC Metadata Workshop in March of 1995. Tillett called for agreement on the use of the terms edition and version with conformance to the traditional use of these terms in the bibliographic cataloging field. This alone does not resolve incongruities, but is a key step in the right direction.

An object is the starting point and the source of content for a digital image (see illustration). An object can be virtually anything that can be captured in an image, and could be an analog image itself, such as a slide or painting, or even an image originally created digitally. In any case, a single digital image of an object initiates a new digital image edition. An edition is a digital image and its family of derivatives that are intellectually equivalent. When a digital image portrays a non-digital object, and intellectual integrity is upheld, they could both be considered part of one edition. But since digital image editions are sufficiently complex without this association, we choose to keep analog and digital images separate in this discussion. Specifically, the complexities of digital images are due to the fact that they can be reproduced in multitudes, with or without upholding the integrity of the visual data and the intellectual meaning of the image. Digital image duplication has the added dimension of exact, reversible, reproduction.

Before we delve further into the issues of reproduction and equivalence relationships, it is necessary to define a few more terms.

A derivative is a digital image that is created through reproduction of another digital image or an object. If the intellectual integrity is upheld, then the derivative is within the same edition as the source. Otherwise, the reproduction initiates a new edition. The derivation of a reproduction is the source digital image, or an object.

A version is digital image derivative that upholds the visual data integrity and the intellectual integrity of the derivation.

A derivative level contains all versions of an edition that have equivalent visual data.

The initial image, depicting an object, inaugurates an edition. It is a level 1, version 1, derivative of an object (see illustration). An exact reproduction (i.e., visual data integrity and intellectual integrity upheld) of the initial image results in a new version of a level 1 derivative. Images on one level can be in different file formats as long as the visual data in an image can be translated to be identical to that of another image's visual data. If visual data is lost in the creation of a derivative image, it moves to a new derivative level.

The only possible data loss when creating a new version is that of metadata. Metadata stored within the header data of the source image's file format is lost if the derivative's reproduction process is not capable of translating the derivation image's header data to the new image file format. Image file formats and imaging software applications vary in their capabilities for handling header data. It is important to know the capabilities of the file formats and software one uses so that the results of image reproduction and modification processes can be accurately predicted. Again, the loss of header data does not instigate a move to a new derivative level, but the loss of visual data does.

Many reproduction processes result in a loss of visual data while upholding the intellectual integrity of an image. It is a uni-directional move that is irreversible, and may employ lossy compression, resolution reduction, color reduction or remapping to a different palette. It is impossible to exactly recreate an image's higher level derivation from the derivative image.

Metadata tends to accumulate as deeper derivative levels are generated. An image that is many derivative levels or editions deep may have a lengthy history of modifications and reproduction processes that have affected the integrity of the visual data. This information is invaluable for determining intellectual equivalency among images since it verifies an images derivation, something that cannot be accomplished by simply viewing the images.

If a derivative image is not intellectually equivalent to the derivation, a new edition is initiated. Determination of intellectual equivalency is largely a subjective matter that depends on the purpose and audience of an image. For example, a photojournalist might make several prints from a negative. Technically, each image is different due to slight variations of exposure and chemical processing. The photojournalist might consider all of the prints to be intellectually equivalent, since for the intended use, they are interchangeable. Therefore, the photographer includes them all in one edition. In another scenario, an art historian might be highly aware of differences among several prints generated from one negative. Color variations due to processing or aging are highly visible to the trained eye. The art historian may choose to consider each image as a unique edition. In the digital world, situations just as ambiguous are common.

To summarize, an edition is a set of intellectually equivalent images. A derivative image that is an exact reproduction, is a new version of the same level. An intellectually equivalent reproduction that is not an exact reproduction of the image data initiates a new derivative level. Finally, a new edition is started by any reproduction lacking intellectual equivalence to the images of the edition.

Implementation

Digital image processing methods can be separated into two categories-- those that unequivocally generate intellectually equivalent reproductions, and those that produce results requiring subjective reasoning to determine intellectual equivalency.

There are two possible approaches to making image equivalency decisions. Pre-process equivalency determination is a method where explicit choices are made prior to processing in order to establish what image reproduction processes will be applied to the image. The outcome is an image that meets pre-defined equivalency requirements. The pre-process method is highly controlled, and requires a knowledge of imaging techniques and image file format structures.

Post-process equivalency determination

is the method of evaluating images after they have been processed in order to determine equivalency. To reiterate, here are the definitions for visual data equivalency and intellectual equivalency.

Visual data equivalency

If a mathematical, bit-by-bit, comparison of an image's visual data with the visual data of another image reveals absolutely no difference between them, then they are visual data equivalents as well as intellectual equivalents and exact reproductions. Visual data equivalency can be determined with software that compares the data of the images, bit-by-bit. Otherwise, metadata that describes the steps taken to create the derivative is needed in order to determine if the integrity of the visual data was upheld.

Intellectual equivalency

If a reproduced image and its source, by all visual measurements, look the same when viewed, then they are intellectually equivalent. The first step to take when attempting to establish intellectual equivalency, is to check for visual data equivalency (see previous paragraph). The next step is to do a visual comparison of the images. This involves displaying each image in a consistent manner, ideally side by side, and visually examining them for differences. Accompanying either step should be an investigation of the image's metadata, which may reveal whether or not one image is indeed a derivative of the other. Alone this does not resolve the issue, but helps to substantiate the notion that a relationship indeed exists.

Making Appropriate Decisions

In order to make appropriate decisions regarding intellectual equivalency, it is imperative to ask the proper questions. What is the purpose of the image? Who are the users? What are their needs and expectations?

An art historian might find that a very high resolution digital image with broad dynamic range and accurate color is well suited for scholarly research on a local computer. The historian uses software for magnifying specific areas of the image to study detail. In this case, the high quality digital image is a worthy successor to the traditional film slide or a high quality print, and is deemed intellectually equivalent in the eyes of the art historian. However, the same image scaled down and JPEG compressed for distribution on a network is only useful to the art historian for quick reference to an art work. Although the same overall composition is viewable, the compromised image is not intellectually equivalent to the high quality image, nor the slide, due to a lack of detail. In this scenario, the user has very high expectations that must be accommodated. The high resolution image is part of one edition, and the low resolution is part of another. The relationships among the images can and should be documented in metadata.

To extend the above example, entertain the thought of an even higher quality image. One that is of such high quality that it puts too much of a burden on today's computers and networks to consider using it everyday, but has tremendous long term archival value. The archival digital image, is actually the initial image of a digital edition. The aforementioned image that the art historian uses for research is a level 2 derivative of the same edition, and was generated from the initial image. The two images are deemed intellectually equivalent since the differences, while technically substantial, are subtle to the human eye. The low resolution reference image still initiates a new edition.

Conclusion

Duplicates of a single image can be stored in a variety of formats then compressed in different ways. Each of these can then have its resolution reduced, color rebalanced, or be cropped. The permutations and combinations are endless. If managers of digital image collections and designers of digital library searching tools want to provide better access to their collections (and minimize the retrieval of duplicates), they must first understand the issues of image equivalency. The next step is the development and adoption of a standard set of metadata which can be used to determine equivalency.

Finally, the authors make recommendations as to standards that should be developed and incorporated as metadata accompanying images. Emerging international standards for file storage format (such as SPIFF) are briefly examined in light of the metadata needs.

Getting, and keeping, a grip on a digital image collection is dependent upon being able to establish relationships of reproduction and determine equivalency among images. All of this should be driven by the needs of current users, with anticipation of possible future uses playing a somewhat lesser but nonetheless considerable role.

(Can this model be extended to incorporate other digital media types?)

Last modified: 3/11/1997