[ Abstract | Introduction | Methods | Results & Discussion | Conclusion | References | Bibliography ]


Whenever groups of humans and computers interact, we are bound to encounter the "prediction-defying pathologies of highly complex nonlinear dynamical systems" (Landauer, 1991, p. 61.) The fact that it is difficult to assign causality to patterns of behavior does not mean that we can not build theories that are both explanatory and to some extent predictive. Rather it means that we must be aware that our linkages and predictions are weak in the sense that they represent tendencies and not hard and fast rules. Empirical measurements such as the flow of users through information spaces (Pirolli, 1996) and the precision and recall of queries (Furnas, 1983) can provide a starting point for developing these theories. In my research, the fact that 83% of the students were able to locate useful information is a strong indication that the activity structure is successful at delivering appropriate resources. Similarly, the high usage rate of the Collaborative Search Page along with the increase in citations and saved sites suggests that the activity design for the second experiment is moving in the right direction. However, the tendency for half the class to adopt information in a consumer-oriented style indicates that further research into open-ended design tasks is needed.

Ideally students learn a process of problem definition, design planning, decision making, and justification. The iterative refinement required by these open-ended design tasks is particularly difficult to attain in traditional classrooms because of the tendency for students to select a partial solution to a problem that has many inter-related facets. However, the potential for students to develop a conceptual understanding of the generative principles that can drive the design process makes this type of activity particularly attractive.

Educational Implications & Future Research. The fact that users tend to interact repeatedly with small clusters of information (Card, 1996, p. 112) suggests that as educators we need to provide tools that help communities of learners share relevant information. Providing access to information is clearly less difficult than encouraging critical analysis. How can we help students move from blindly consuming information to experimenting with alternatives through an integrated conceptual framework? The Collaborative Search Page provides a starting point for presenting relevant alternatives. Developing a conceptual framework that allows for principled analysis of these alternatives may need to occur during the problem structuring phase rather than during the refinement or search episodes as occurred in these experiments.

The tendency not to revise initial designs suggests that the sequence of searching for evidence to support existing designs should be reordered. Searching for evidence needs to be linked more directly with the conceptualization of the problem if the located information is to have a significant impact on the project design. A number of approaches to encouraging a more dynamic and expanded problem definition can be drawn from other KIE projects such as critiquing other students' designs or developing simple prototypes highlighting the scientific principles involved. Linking the criticisms of existing designs and components of the prototypes with Internet evidence may help model the process of critical thinking that needs to occur for students to successfully complete similar design projects.

[ Abstract | Introduction | Methods | Results & Discussion | Conclusion | References | Bibliography ]