Besides Working Group participation, this portion of your grade also includes participation in class discussions, interaction with other student presentations, short assignments, science fiction viewing reports, etc.

• Winter 96 final projects (class mainly humanities and social science students)
• Fall 96 final projects (class mainly MBA students)

Evaluation of Multimedia


We will explore how to evaluate multimedia programs.
• Readings on Multimedia Evaluation
  • Interface Design for Sun's Website
  • WebCHI: A guide to HCI issues of the Web (especially see Design, Design, Design
  • Criteria for Evaluation of Internet Information Resources, Victoria University of Wellington, Advanced Information Retrieval course
  • Software Evaluation Instrument, Children's Software Revue
• You will read at least 2 evaluations of multimedia programs
• Speakers will tell us how they evaluate programs in a variety of areas.
• You will find guidelines or criteria for reviewing multimedia programs
• You will find guidelines or criteria for reviewing other types of works
• As a class, we will go through various evaluation criteria and group them into basic areas (content, design, navigation, information chunking, etc.). From this we will create what amounts to a generic evaluation tool, but without weights put on the various categories. At the same time, students will gain an appreciation of the various component parts that go into making a multimedia program successful.
• We will work with the Synthesis/NEEDS Coalition to adapt our generic guidelines to their specific needs for courseware evaluation. Ideally, we will develop an evaluation instrument for them and test it on submissions to a contest they are currently running for the best online courseware.
• You will review multimedia Courseware from the Synthesis/Needs Coalition
• Review the Multimedia Categories that we developed in class, and be prepared to make suggestions on items to move around, add, delete, etc.
• Begin reviewing some of the Synthesis/Needs courseware. For April 18: Use the Multimedia Categories that we developed in class to review one Synthesis/Needs piece of courseware. By focusing on the use of our categories to do this, you will see the strengths and weaknesses of the categories as we have developed them. Be prepared to discuss the advantages and limitations of our categories as we attempt to revise them. Also, be ready to assign relative weights between a select group of categories. These are the courseware that you should choos from:
  • Virtual Disk Drive Simulation - Mac Director CD-ROM
  • Vibrating Beam Experiment - PC Toolbook CD-ROM
  • Vibrating Beam Experiment Web Site
  • Mattel Color Spin Case Study Web Site
  The two CD-ROMs should be available in 3112 Etcheverry Hall by noon on Monday, April 7. If you have any questions, call Greg Paschall 3-2400, paschall@needs.org on Monday. (He's the one burning the ROMs.) And you can review one of the NEEDS Coalition's papers on some their peer review efforts.

Analysis of Museum Educational Site Licensing Project (MESL) Sites

Seven universities have mounted an identical set of 10,000 museum images and accompanying text. Each has put these into a different system and they behave differently. This set of short assignments will analyze those differences. Note: you will only be able to access 5 of the MESL sites, and because of security controls you can only access these from a single workstation in South Hall (the Macintosh adjacent to the door). The Lab is only open regular working hours. Please also note that we are guests using the MESL dataset, and that it is critical that we not abuse that privelege. We are governed by the same Terms & Conditions as the participant MESL Universities.

Status as of 2/27/97 for the MESL sites:

• Illinois -- working and requires password
• Virginia -- working
• Cornell -- working
• Michigan -- try here (needs password), or here (no password)
• American -- working now
• Maryland -- working now
• Columbia -- will not be participating

By March 7:

• Compare user interface and display options on all the MESL sites. Look at how the user is supposed to navigate through the system (including how the information is "chunked", the order in which options are presented to the user, and the placement of buttons). Also examine search options and layout of search results. Keep notes about your results and be prepared to discuss these in class.
• Read through Joseph Squier's (Univ of Ill) students' comments on their MESL search engine, user interface, and use of images
• Compare size and quality of thumbnail and larger images on all the MESL sites. Note approximate sizes of images offered, and how these differ between implementations.

  By March 14:

• Perform 3 identical searches on all the MESL sites and note how the same query on the same data set yielded different results. (Keep careful notes, as you will later have to replicate key searches and turn in electronic copies of search results.) Be prepared to discuss your findings in class.

  By March 21:
  You will be creating a set of web pages that shows the most striking findings and differences between sites. Because we may lose our access to the MESL sites after March 21, you will need to save any raw data that you'll need before then. (A little later in the term we will discuss how to turn PICT screendumps into JPEG images and present these as part of a Web page.)

  • Use Flash-It (shift-apple-3) to save screen captures of various forms of the user interface (like to compare displays of identical results on different sites, to compare navigation tools, to compare what comes up immediately after a query, etc.)
  • Either "Save as html" or "View source" and save -- in order to capture things like the different query forms on the various sites, text display from identical queries on different sites, etc.

  By April 4:
  With your focus group, work up a set of web pages that explain the interesting contrasts that you have found between the MESL sites. Ideally, each group member should have found at least half a dozen interesting ones.

Science Fiction Assignment

As part of the Class Participation portion of your grade you will need to complete a number of short assignments. The most significant of these is listed below.

Viewings During the course of the semester, all students will watch at least two science fiction films and report back to the class (and/or discuss these within their focus groups):

• One film from the past (1950s or before) to examine whether past visions of future information technologies have come true:
  Metropolis (1927), *Things to Come (1936), Just Imagine (1930), *Wonderful World of Tomorrow (1939), Time Machine (1960), War of the Worlds (1953), Charleston (1927, Renoir), Transatlantic Tunnel (1935), *Woman in the Moon (Lang, 1929), Aelita (1924), *You Can't Get There From Here: Ephemeral Films 1946-60 (1987), To New Horizons: Ephemeral Films 1931-45 (1987), Fantastic Planet (1973)

• One modern film that contains visions of future information technologies:
  Emphasis on how collective memory/information will be handled, stored, destroyed: The Net (1995), Total Recall (1990), Rollerball (1975), *Farenheit 451 (1967), *1984 (1956, 1984), *Forbidden Planet (1956), Solaris (1972), Dreamscape (1984), Death Watch (1980), *Outer Limits (Robert Culp)

  Emphasis on how major aspects of society will change due to new information technologies: Johnny Mnemonic (1995), They Live (1988), Wild Palms (1993), Until the End of the World (1991), *Bladerunner (1982), Brazil (1985), Terminator (1984), *T-2 (1991)

  Examination of future societies and capabilities: The Jetsons (1990), *THX-1138 (1971), *Star Wars Trilogy (1977, 1980, 1983), *2001: A Space Odyssey (1968), Born in Flames (1982), Tron (1982)

Return to the Impact Spring 1997 Homepage