"If itís the same museum information, why donít these look the same?" Comparing five implementations of identical data from the Museum Educational Site Licensing Project"
 
ICHIM 1997 Conference
 
Howard Besser
UC Berkeley School of Info Mgmt & Systems
http://www sims.berkeley.edu/~howard/

 

A number of Universities mounted the identical set of approximately 10,000 images and accompanying text records as part of the Museum Educational Site Licensing Project. This paper examines 5 of those implementations from the standpoint of the userís interactive experience. Eight students were given access to the implementations and a preliminary summary of their comparative observations is presented here, focusing primarily on the searching process. Because the examples in this study demonstrate the different ways the same information can be presented to the user, it should prove useful to implementers planning an interactive project.

 

 
The Museum Educational Site Licensing Project

The Museum Educational Site Licensing Project (MESL) was a major demonstration project designed to identify and resolve the problems of licensing and delivery of images and accompanying text from content providers to groups of content users.

The MESL project served as a laboratory for developing and testing the legal, administrative and technical mechanisms needed to enable the full educational use of museum collections through routine delivery of high-quality museum images and information to educational institutions (MESL 1997).

For a two-year period beginning mid-1995, 7 repositories (the Library of Congress, the National Gallery of Art, the National Museum of American Art, the Harvard University Art Museums, the Houston Museum of Fine Arts, the George Eastman House, and the Fowler Museum of Cultural History) supplied a total of 9,319 images and accompanying text records to 7 universities (the Universities of Illinois, Maryland, Virginia, and Michigan, as well as Columbia, Cornell, and American Universities). Each university received an identical set of images and text, but each chose to deliver these to their users within their own unique deployment system.

Over the course of the MESL project, the participants explored standards and mechanisms for exchanging images and data between institutions, mounting and delivering this information to university users, developing tools for incorporating images and data into the instructional process, and developing parameters for licensing of this type of content.

Much has been written about the general MESL concept and MESL aims in the arena of intellectual property licensing (Trant 1996, Trant 1995-96, Trant 1995). Technical issues related to the aggregation of images and text from diverse sources has been discussed elsewhere (Besser & Stephenson 1996). This is the first study to compare the different deployment systems for an identical set of images and text.

The Study

This paper examines 5 of the MESL university implementations from the standpoint of the userís interactive experience. Each of the implementations (American and Cornell Universities, as well as the Universities of Maryland, Michigan, Illinois, and Virginia) was accessed by the student study group through the WorldWide Web, which was the primary access mechanism for each universityís own body of users with the notable exception of Maryland. The vast majority of Marylandís users access their implementation through a local network, and their Web interface is far less rich than what most of their users see.

Eight students in a UC Berkeley graduate class were given access to the implementations for a one-month period. Students came from a variety of different backgrounds, and the expectation was that they would notice different features of the systems. None had an extensive art historical background, so their queries were expected to look more like those of naive users rather than someone familiar with the content domain. They were all given the following set of assignments:

This paper summarizes the preliminary findings of this study, focusing primarily on the searching process. Among other findings, it shows how identical queries yield different results from implementation to implementation.
Initial Presentation & Query Options

Most sites presented the user with several layers of explanatory information before allowing the user to compose a query. This information was designed to make users interested in the MESL data (roughly the equivalent of an "attract sequence" in a museum interactive exhibit), to contextualize the project and clarify its scope (more important in a searching environment than in an environment with a narrative structure), and to explain conditions of use. But most of the study group felt that "nesting the search page so deep within the web hierarchy discourages repeated user queries". Though savvy users might just bookmark the query page, designers might consider providing one set of paths for initial users and another set for repeat users.

Query screens for most implementations employed HTML forms with menu choices. Most sites provided forms for both simple and for complex searches, either as separate pages or combined on the same page. Examples of query screens for each site are shown in figures 1-5. These screendumps show how the same set of data can be presented in many different ways.

A number of students found the distinction between simple and complex searches confusing. In most cases the difference was that the "complex" searches permitted the user to search for a single value in each of two fields. Students felt that "complex" was a poor word to sue for this type of search.

Each site chose to index a different set of fields. Some sites chose to provide keyword access while others did not. Some sites provided access by categories of local interest (such as by course using the image). And in many cases "searchable fields" on the userís query form were really composed of indexes made off of a variety of different fields in the database. The fact that different sites combined their indexes in different ways was one of the factors that periodically led to radically different search results between sites on the same query.

Search Results

Students were asked to perform the same search at each site. Because the set of searchable fields presented to the user differed from site to site, students needed to use their own best judgment to replicate the search as closely as possible. These searches yielded vastly different results from site to site:

Reasons for divergent search results included: sites combining different sets of original data fields into combined indexes, different search engines, and whole-word versus character-string searches on various fields. These reasons will be discussed further in the "Observations" section (below).
Observations

Future reports from this study will examine how search results are presented to users at each of the implementing sites, and compare record display, thumbnail sizes, etc.

Reasons for divergent search results included: sites combining different sets of original data fields into unified indexes, different search engines, and whole-word versus character-string searches on various fields

Perhaps the most prominent cause of result discrepancies is the different ways in which the sites chose to map original data fields into fields that make sense to users trying to query the database. The MESL data dictionary contains 32 fields, far too many to effectively present to any but the most sophisticated user. Each implementation made its own decision on how to group sets of fields into a single user-searchable index, and what to call each of these combined indexes. Because of this, users visiting the different sites will be presented with different indexes over the same underlying content.

In many cases it is fairly easy to guess which fields were combined into which index. However it is not at all obvious which fields were indexed to form the virtual "keyword" field; there are an enormous number of possible ways of forming this index (from combining prominent fields like subject, description, and title to relying completely on the words within the label field). Careful choice of which fields to index for keywords might help prevent problems like finding an artist named "Child" when looking for portraits of children. Because most of the "simple" or "quick" searches favor the keyword approach, it is likely that most casual users would get different results from the various implementations.

This issue of how to combine fields for presentation to users can be a fundamental one for museums contemplating an interactive database for visitor access. The MESL experience of trying to make 32 fields understandable to the average user pales in comparison with the museum that must boil down scores of fields from a collection management system into something easily digestible to the user, particularly if the user is unfamiliar with curatorial language.

Another cause of divergent results had to do with the different ways in which the various search engines work. Most prominent among these is whether all matches must start exactly the same beginning from the left side of any field, whether the system looks for character-strings or whole words, whether they perform stemming, truncations, etc. Searching design decisions such as this can drastically affect search results, as we have seen in the case where a search for oil paintings picked up the word "soil".

Designers of access systems to museum information should note that a system that helps the user search for individual items differs from most current museum interactive exhibits which have some kind of narrative structure (Besser 1997). Individual item searches give the user a much wider latitude, but also allow them to easily lose their way (such as getting erroneous search results). While the design of an information retrieval system may at first appear to be trivial, decisions over how to combine indexes to present to the user and how to implement searching strategies -- these decisions are critical in determining the userís experience. By examining the different ways in which an identical data set can be searched and presented to users, implementers should be able to better design future interactive projects.

Acknowledgments

Thanks are due to the MESL participant institutions for providing permission and access to the images, records, and retrieval implementations. Students in UC Berkeleyís SIMS 296A course for Spring 1997 participated in the study.

References
Besser, Howard. (1997). The Transformation of the Museum and the way it's Perceived, in Katherine Jones-Garmil (ed.), The Wired Museum: Emerging Technology and Changing Paradigms, Washington: American Association of Museums, pages 153-169.

Besser, Howard and Christie Stephenson. (1996). The Museum Educational Site Licensing Project: Technical Issues in the Distribution of Museum Images and Textual Data to Universities, in James Hemsley (ed.) E.V.A. '96 London (Electronic Imaging and the Visual Arts), Thursday 25th July 1996 (vol 2), Hampshire, UK: Vasari Ltd, 1996, pages 5-1 - 5-15. [http://www.gii.getty.edu/mesl/about/docs/EVA.html]

Museum Educational Site Licensing Project. (1997). Worldwide Web Site. [http://www.gii.getty.edu/mesl/]

Trant, J. (1996). Enabling Educational Use of Museum Digital Materials: The Museum Educational Site Licensing (MESL) Project, A paper for the Electronic Imaging and the Visual Arts Conference, Florence, Italy, February 8-9. [http://www.io.org/~jtrant/papers/jt.eva.florence.9602.html]

Trant, J. (1995-96). "The Museum Educational Site Licensing (MESL) Project: An Update," Spectra, Winter. [http://www.io.org/~jtrant/papers/mesl.spectra9511.html]

Trant, J. (1995). The Getty AHIP Imaging Initiative: A Status Report, Archives and Museums Informatics, Cultural Heritage Information Quarterly, Vol. 9, no. 3, 1995, 262-278. [http://www.io.org/~jtrant/papers/eva.95.html]

 

 

Figure # Filename Caption

figure #1 American.jpg American University Query Screen

figure #2a Cornell-full.jpg Cornell University Full Search Query Screen

figure #2b Cornell-simple.jpg Cornell University Simple Search Query Screen

figure #3 Maryland.jpg University of Maryland Query Screen

figure #4 Michigan.jpg University of Michigan Query Screen

figure #5 Virginia.jpg University of Virginia Query Screen