[ Abstract | Introduction | Methods | Results & Discussion | Conclusion | References | Bibliography ]

Results & Discussion

Relevance Judgment Experiment. The majority of the students (67%) were successful at searching, saved evidence to the Netbook, and incorporated that evidence into the design process. The degree of principled reasoning behind the design decisions varied considerably and did not correlate with the success of the search activity. Students generally know that heat moves from regions with higher heat energy to lower heat energy. However, they have trouble describing heat flow in a typical day-night cycle. They also know that insulators slow heat flow. The presence of varying heat sources, objects with different temperatures and heat capacities, and an unstable surrounding environment required a conceptual understanding and inductive style of reasoning that proved to be problematic for students including those that could recite the scientific principles involved.

Search. The Thesaurus/Dictionary was not used. In several cases, spelling errors resulted in failed search efforts. A blank "Why we kept it" field (used in the experimental group vs. a choice of building material, desert house, or other) produced more specific and insightful comments. The listing words/sketch and label house (before searching) in the experimental group did not produce noticeably better designs or searches. It did provide a useful point of comparison between initial conceptions and final projects and suggested that sketches are more generative than the more time-intensive computer-rendered drawings in the final projects.

The keywords generated at the beginning of the search activity by each period were:
Period 1: mud houses (not many houses made of mud), houses in Arizona, desert, insulation, heat energy, windows, dry, land, windows, energy, build, construction, air conditioning, structure, dome.

Period 2: mud house, wood house, straw house, desert climate, materials, building, insulation, heat, night, pueblos, windows, adobe, flammable, dome, *energy, *conservation.

Period 3: house (pictures, real estate, construction companies), building, materials, heat flow, adobe, insulation, Indian houses, desert house, building, colors & shapes, windows, insulators & conductors, *energy, *conservation.

Period 4: desert house, insulation, windows, glass, double pane, energy efficient, foundation, building materials, insulation materials, energy conservation.

Period 5: [no keywords generated]

The asterisks (*) denote words suggested by the teacher. The words in parentheses were the things that students thought they might locate with the keywords.

There were 149 unique queries/keyword combinations entered by the 66 groups. The ten most useful keywords are listed in the following table based on a weighted average of the ratings.

U F Keywords Very Somewhat Not At All No Rating Total 1 1 insulation 11 2 14 5 32 2 3 architecture 3 0 7 4 14 3 2 desert 2 5 7 1 15 4 7 dome 2 1 2 1 6 5 13 insulated 3 1 0 0 3 windows 6 23 Arizona 1 0 1 0 2 Houses 7 24 climate 1 0 0 1 2 8 25 solar panels 1 0 1 0 2 9 30 energy 0 1 1 0 2 conservation 10 15 heat 1 0 2 0 3 conservation

Figure 9. Keywords, Ratings, & Frequencies
(U: overall ranking for Usefulness. F: overall frequency rating for the keywords)

The creativity and inconsistency in users' approaches to processing information has driven research in HCI for years. For example, Dennis Egan in his "formative evaluation" studies with the SuperBook browser (cf. Landuaer, 1991, p. 64), noted that, "It appears to be reformatting - rearrangement and highlighting - of the page, or perhaps the search context in which it was found, [that] caused one group of users to notice and absorb the information where the others passed it over."[1] Similarly, it is difficult to determine whether the non-uniform rating of sites observed in the first experiment is due to context, the interactional disposition of the user, or some combination.

During the search phase, 44 of the 66 groups (~67%) found at least one Internet site that they located independently and thought useful enough to save to their Netbook. Percentages are rounded to the nearest whole number:

Figure 10. Sites Saved & Cited

In the final reports, sites located during the search phase were cited 50% more frequently than the sites visited during the Survey Evidence phase. The sites in the Survey Evidence phase came from the Networked Evidence Database (NED) and were selected by the activity designers as relevant to the project. These NED sites were cited by 16 of the 66 groups (~24%) as opposed to individually-located Internet sites that were cited by 25 of the 66 groups (~37%) (3 groups discarded). Of those students that cited evidence in their final reports, 28 of 34 or approximately 82% cited either NED evidence or other (individually-located evidence). Only 18% of the students that cited evidence cited BOTH types of evidence.

Only 8 of the 66 final reports (approximately 12%) did not reflect structural aspects[2] of the evidence found in the Survey Evidence phase. The influence of the sites presented in this early phase of the activity persisted in the form of dome-shaped mud dwellings similar to the Native American adobe structures that were compared with straw and wood buildings. Of the eight groups that developed designs that did not involve a dome-shaped or mud dwelling, only one group formulated their design based on a lab conducted during the semester (e.g., an energy conversion lab.) This group used water to store the heat energy during the day and release it slowly at night. The other seven groups developed innovative designs based on information that they located individually on the Internet. The average grade for these seven groups was 72.1 vs. the class average of 67.7 (rounded to one decimal place) or 6.5% above the average.

Collaborative Search Experiment. The change in activity structure from the first to second running of the intervention successfully expanded the problem definition for the majority of the students. This expanded problem definition helped students generate a wider range of keywords and locate useful information. The addition of a Revised Design Worksheet and Heat Flow Analysis Worksheet had little impact on the selection of features considered for the dwelling (e.g., windows, size of rooms, orientation of house, type of insulation, etc.) but did reveal internal conflicts within the students' explanatory framework for the temperature differences at different times of day. This cognitive dissonance occasionally resulted in a reevaluation of the design or reassessment of the explanatory mechanisms for describing heat flow. Initial designs reflected a strong bias towards the examples used to illustrate heat flow and towards the selected sites that were visited early on in the project.

Students generally retained the framework of their initial designs and used evidence as a "proof of existence" for features of a house rather than as an argument for selecting one design over another. Principled reasoning about the dwelling design came late in the process in either the Heat Flow Analysis worksheet or the Final Report. Students that did not use evidence to generate alternatives (i.e., the consumers) had better than anticipated success in searching for useful information but not in developing a well-structured argument.

From the first to second experiment, there was an increase of 23% (to 74%) in the number of groups that cited evidence in the final reports. In addition, citations of NED evidence increased 44% (to 68%) while evidence located during the search phase decreased by 9% (to 28%). There was an increase of 16% (to 83%) in the number of groups that located sites that they thought were useful. The following table shows the breakdown by period for sites saved during the search and cited in the final report. Percentages are rounded to the nearest whole number:

Figure 11. Percentage of groups saving/citing any evidence by period

Of those students that cited evidence in their final reports, 27 of 44 or approximately 61% (down from 82% in the first experiment) cited either NED evidence or other (individually-located evidence). Thus, the number of students citing BOTH types of evidence increased from 18% to 39%.

The 50% increase in citations of independently-located sites over selected NED sites in the first experiment also has multiple possible explanations. This finding may confirm the hypothesis that information located in the preliminary design phase is used to generate options while information found during later stages is used to support existing designs. An alternative explanation comes from looking at the "type" of evidence encountered during these two phases. The material presented during the Survey phase (part of the preliminary design) consisted primarily of examples of different houses (e.g., straw, wood, and mud). The material located independently by students typically dealt with specific structural aspects of the dwelling such as roofs, windows, or foundations. The Enertia site (see Figure 12) located by students during the search phase of the first experiment was an exception containing information about thermal inertia and delta T.

Figure 12. Excerpt from Enertia site[3].

This site was used in an activity that preceded the second experiment and was subsequently recalled by students during the search episode of the second experiment. However, it was not sited as frequently as the mud, straw, and wood sites though many of the designs reflected the heat flow envelope and radiant floors modeled at this site. The fact that citations by groups of both NED and independently located sites increased from 18% to 39% of the groups most probably reflects the increase in the number of sites presented on the Survey Evidence page as opposed to a change in how information processed.

Group Dispositions[4]

To analyze groups, categorical assignments were made based on the manner in which evidence was incorporated and arguments presented in the final reports. The purpose of this categorization was to provide an indication of the potential value of using categorical variables to describe the design/search process. The intent was not to report a tightly controlled experimental finding[5]. These categories were assigned based on the following criteria:

Strategizers: tried to produce what they thought the teacher wanted (e.g., two citations of Internet evidence, reference to two labs, etc.)

Conceptualizers: framed their designs using a top-down framework with scientific principles being presented up front and the details falling out of the principled framework.

Consumers+ (aggregation): incorporated whatever information they encountered into their designs in an ad hoc fashion.

Consumers= (replacement): replaced their design with some piece of information they found but did not critically analyze.

Experimenter++ (adds principles): added principles to support design alternatives.

Experimenter=> (refines ideas): refined ideas to support integrated design components.

Approximately half of the students were categorized as consumers based on their use of evidence and principled reasoning in the final reports. The other categories (e.g., strategizer, conceptualizer, and experimenter) provided roughly even divisions for the remainder of the class.

Figure 13. Class Composition Based On Interactional Dispositions

Aggregating the consumer and experimenter subtypes yields a composition of 53% (consumers) and 16% (experimenters). The high ratio of consumers to other types of IDs may be attributed to the complexity of the design task.

Innovative Designs: Only 9 of the 69 final reports (approximately 13%) did not reflect structural aspects of the evidence found in the Survey Evidence phase. This result is consistent with the first experiment. Interestingly, 6 of the 9 innovative designs were created by groups that reflected a consumer-based interactional disposition. However, the source for these innovative designs came from a Web site that was discovered in one period and shared through the exchange of keywords within and across periods.

Source Strategize Conceptual Consume+ Consume= Exp++ Exp=>
NED         0           1           1           0           
0           0           
Other       0           0           1           0           1           
Shared      0           0           2           2           1           
Totals:     0           1           4           2           2           

Figure 14. Innovative Designs: Interactional dispositions & Information Source

It seems probable that structural examples of designs have a stronger impact on the conceptualization of the design than more theoretical models of heat flow. This pattern may be explained by the observation that students tend to select alternatives and only later develop explanatory mechanisms that justify their utility. For example, the group that designed the House of Vents (see Figure 15) began by pulling the skylight component from the inital example. During the heat flow analysis stage they realized that the ground temperature varied less than the outside air.

Figure 15. The House Of Vents

In an excerpt from the final report, we can see an attempt to develop a mechanism for heat flow.

Part of our home is constructed underground. This helps us because the termpature stays more constant compared with the outside temperature. We have inserted vents that allow cooled air to lift up into the house forcing the hot air to rise to the top and escape through more vents.

These vents located at ground level force the cool air into the house. The hot air rises and is forced out through the top vents. Vents are obviously to keep the house cooler during the day. This is becuase heat energy passes through air easier than it does through glass.

This reasoning emerged in the final report and although the scientific language is non-normative (e.g., escape, cooled air [lifting] up) the line of reasoning is potentially generative.

A less successful example of developing a mechanism using principled reasoning can be seen in the House Of Water (see Figure 16). The design was copied directly from an existing Internet site.

Figure 16. The House Of Water

The design itself is complex using several adjacent rooms, a water filtration system, and a solar turbine. The principles related to heat storage and thermal mass that justify the use of plastic filled with water as an insulator are not developed as we can see from an excerpt from the final report:

The material we are using for our house is straw and plastic filled with water because both straws and plastics are very good insulators.

The plastic walls and small windows keep out the sun's heat and retain warmth at night.

Even when students begin to account for the surround and the ground temperatures, problems can emerge because of the complexity of the issues and the difficult mapping that must occur to apply labs to real-world situations. For example, in the House of Mirrors (see Figure 17) the students raise the house off the ground to avoid heat transfer between the ground and the house.

Figure 17. The House Of Mirrors

The final report indicates that they have applied knowledge from the Scattering Lab to select white as a color for the house and shiny silver for the roof.

The color we chose for our house is white because it absorbs the least amount of light and heat and will keep the house the coolest. The roof is shiny silver so as to reflect sunlight overhead. We made it shiny silver and not mirrors because in one lab we learned that shiny silver works better as a reflector than a mirror does.

However, they do not consider the problem that a metal roof may serve as a good conductor in addition to scattering the sun's light energy. Similarly, they do not consider the possibility that the temperature of the surround may have a greater effect on the dwelling than contact with the ground.

The general trend to support designs with principles late in the design process would be expected of strategizers realizing that they need to support their designs with principles as a requirement for the activity. The exception to this rule would be expected for conceptualizers but is hard to support empirically from the data because of the difficulty with assessing the presence or absence of conceptual reasoning. A finer-grained definition of categories may be needed to capture the subtleties that distinguish the different phases of the design process.

Grades: Experimenters and conceptualizers had the highest average grade score while consumers trailed by approximately 10 percentage points.

Figure 18. Mean (1) & Median (2) Grades by Interactional Disposition

The ranges for Consumers' grades (58-94 for Consumer+; 61-94 for Consumer=) were greater than for the other groups (76-97 Strategizer) with the Conceputalizers (85-99) and Experimenters (82-97 Exp++; 84-97 Exp=>) reaching the higher levels more consistently.

Citations, Queries, Saves: Experimenters and conceptualizers entered almost twice as many queries as consumers. Experimenters that added principles used evidence located during the search phase twice as frequently as experimenters that refined their ideas. Consumers had the lowest rate of NED evidence usage in their final reports. Conceptualizers saved the most number of sites that they thought useful to their Netbook.

Figure 19a
. Average citations (NED vs. Other)

Figure 19b. Average number of queries & sites saved

The increase in the number of sites that were deemed useful and saved during the search phase (up 16% to 83% in the second experiment) could be the result of an expanded problem definition, the accessibility of the Collaborative Search Page, and/or the effectiveness of generating keywords collectively. The fact that experimenters and conceptualizers entered twice as many queries as consumers reinforces the link between interactional dispositions assigned from analyzing the final reports and the independently measured search activities. Similarly, conceptualizers rarely used search evidence - an expected result if they were focusing more on the explanatory mechanisms in place of justification based on evidence. Conceptualizers along with experimenters had the highest average scores; something to be expected since the grading was based on conceptual understanding, innovation, and justification.

>1 Citation/Query/Save: Strategizers used NED evidence almost twice as frequently as consumers. Experimenters that added principles used evidence from the search phase more than twice as often as strategizers, consumers, and experimenters that refined ideas. Conceptualizers rarely used evidence from the search phase.

Figure 20. "Greater than 1" citation, query, saved site by interactional disposition

Somewhat surprisingly, experimenters that added principles had twice as many search citations as strategizers, consumers, and experimenters that refined ideas. The aggregate style with which this type of experimenter incorporated information could explain this phenomenon. The lower citation rate by consumers reflects their tendency to adopt alternatives arbitrarily without refining them or integrating them into a conceptual framework. The lower citation rate for strategizers can be clarified by examining the high rate of citation for NED-based evidence; a finding consistent with their characteristic of incorporating more reliable and acceptable pieces of information. Experimenters that refined ideas may have condensed their explanations or developed innovative designs that relied more on principles than evidence.

Collaborative Search Page. Approximately 41% (28 of 69 groups) used either the search or sorting capability of the collaborative search page. An average of 20% of the groups searched for some specific student name, set of keywords, or category. 40% of the groups that used the collaborative search page were some type of consumer.

Figure 21. Searching & Sorting by Interactional disposition
(3 discarded, % rounded)

In the two periods (one and two) that encountered a particularly useful site (e.g., Home Power: "How To Stay Cool In The Desert"), 50% of the groups were looking specifically for the Home Power site using either student's names or keywords (see Figure 22).

There was no discernible effect from the experimental condition for periods two, three, and seven. In fact, period two had a significantly lower usage rate for the CSP than the other periods: 27% vs. 40% to 54% for the other periods. However, the final design reports for those groups that did use the Collaborative Search Page in period two reflected a strong bias for, if not wholesale adoption of, the information located using the page.

Figure 22. HomePower: How To Stay Cool In The Desert site.

Not surprisingly, all of the groups looking for the site were consumers. Discounting that group of consumers, the proportion of strategizers using the search page rises to 35% with the conceptualizers, consumers, and experimenters leveling off as approximately 20% each.

A combination of the type of evidence, the stage of the design process, and the interactional disposition of the user are factors in how evidence is used. For example, consumers had the lowest rate of NED evidence usage. This finding, at first surprising, could be explained by the possibility that consumers best exemplify the tendency to justify decisions after making them. However, analysis of the usage statistics for the Collaborative Search Page reveals that consumers accounted for 40% of the activity. One possible inference is that consumers had already processed the early evidence, had located a relatively low number of useful sites, and were simply at the stage of the activity where they needed to find evidence to support their designs. From an information foraging perspective (Pirolli, 1996), the cost of accessing relevant material in the locality of the Collaborative Search Page (CSP) was much lower than seeking out information on their own. And in fact, the results from experiment #2 indicate that consumers were frequently targeting information located by other groups.

Leveraging Student Intuitions. Intuitions that students have about heat flow and energy conversion appear implicitly in the initial sketches and somewhat more explicitly in the final design reports. The following comments encountered in the final reports reflect various p-prims (diSessa, 1988) and facets (Clark, 1996) of principled reasoning. The purpose of categorizing these facets is a) to refine the categorization scheme used to classify interactional dispositions and b) to provide examples of the types of non-normative conceptions that students bring to the project. I provide examples[6] of how these non-normative conceptions can be transformed into more normative explanations. Ultimately, students intutions may be used to provide different types of activities that have been successful at producing conceptual change in students with similar intuitions. The following comments are grouped into categories for the purpose of beginning such an analysis:

Containment & Escape
2.06: "the advantage of having windows is, it will make the heat flow in the house easier so the "used air" has more openings to get out of."
3.08: "windows on our house will be one way mirrors because it will allow light to travel into the house but won't be able to escape but reflect off the mirrors and stay in the house."
3.14: "at night the desert cannot give back the energy and it is very cold."
4.02: "the heat that managed to make its way into our house will be trapped in the walls while trying to escape"
7.07: materials need holes to let heat in and out. "the heat is trying to get into the house"

Illusion Of Coolness
1.12: marble is naturally cool, white reflects heat
4.08: things that feel cool keep things cool
7.01: white gives illusion of coolness

Invented Mechanisms For Heat Flow & Exchange
2.05: small and low windows with black lines around the windows "because the sun will go toward the outside of the window. I got this idea from the baseball players (they put black around their eyes)."
3.05: "it takes longer for an object that is an insulator to heat up, but when it does heat up, it will retain the heat longer."
3.07: "heat flows the direction the sun is facing."
4.07: heat doesn't flow through thick things. "a good insulator is something that keeps you cold."
7.02: things that heat up faster have more heat energy
7.06: dense objects are good insulators
7.09: "In the daytime, heat will flow on the side of the house, run up the house, and off the roof."
7.10: "the pool water reflects sunlight and the house becomes cooler"
7.08: "we will not have too much furniture because that will not allow very much condensation of heat or cool in the desert homes."

Listing 1. Comments in Final Reports reflecting p-prims & facets

Two of the most prevalent p-prims appear to be "heat flows out of objects" and the derivative p-prim "objects must be physically connected for heat flow to take place." The clinical interview with student 4.05 provides examples of both of these p-prims:

i: so how does the heat energy from the sun affect the house temperature?

s: because it's scattering first of all the rays are scattering most of them are coming into this general area (side of house) like from here to here and it's being absorbed by the house so it's heating the house.

i: but without changing the outside temperature too much?

s: uh huh. it definitely changes the outside temperature a lot. but see i'm kind of confused because see if the sun is coming up it's got to be getting warmer so this (the outside) will be increasing and this the ground temperature will be increasing. wait unless the cold air is coming in here (from air to ground). no, yeah.

i: which do you think will increase faster the temperature of the outside air...?

s: yeah the outside air?

i: faster than the ground?

s: yeah. the air will be heating the - basically the heat is going into the air raising that temperature. that will raise the temperature of the heat and the temperature of the heat will raise the temperature of the ground. (draws arrows from sun to air to ground to house) because the house is right on the ground.

i: so if you had to draw heat flow arrows...

Figure 23. Heat Flow Arrows
s: [draws heat flow arrows] it would go from sun into air then "down" into the house then from the house into the ground. that's messed up if the ground is warmer than the air. oh i know it would be going from here to there from the ground to the house. the house might even be giving off a little energy.

i: so you think it can go both ways from the air into the house and from the house into the air?

s: um hm.

i: can it go both ways or does heat flow just one direction?

s: i'm trying to remember the lab we did. oh! heat always flows out. out of an object. you know the lab we did using the good and poor conductors. and you just have to find a way to slow it down. but it flows out if its hot if its 90 degrees or whatever if its a pot and it's 90 degrees its always going to flow out of the pot.
Another common p-prim is that heat can flow both out and in at the same time:

s: it flows out but it can also flow in. but its always always going to flow out because there is always something that's going to absorb the heat. you know what i mean? you just have to find a way to slow the process down. that's the only thing you can do.

This p-prim manifested itself in the Heat Flow Analysis in the form of arrows pointing both into and out of structures. Variants included filled vs. hollow arrows to represent "warm energy" vs. "cold energy". The Heat Flow Analysis worksheet and the functional representation of temperature changes over the course of the day helped to make students' thinking about heat flow more explicit. Making thinking visible, one of the tenets of the SKI framework, encouraged students to integrate their explanations in a manner that lead to more cohesive final design reports with normative scientific explanations.

The Design Process. How does the conceptualization of the problem affect the ability to locate and use evidence? Conceptualization in design problems occurs rapidly although the resulting framework appears to be fluid enough to encompass potentially conflicting elements. The different interactional dispositions dictate to some extent the fixity of these initial conceptions and the ways in which future ideas will be incorporated (i.e., through refinement, replacement, disregard, or aggregation.) Design can be viewed as a process of developing constraints that limit the number of alternatives and then iteratively refining elements that continue to exist within the contextual space of those constraints (Cuthbert, 1996.) But is this really what students are doing?

The clinical interview with student 7.03 reveals how the initial design was refined instead of discarded when supporting information was not found. The initial idea was to have a "natural humidifier" with "large rocks (black or other dark color) out in sun all day, then at night place them in a tub of cold water to cause steam so it is not dry." Heat storage and transfer appear to be the unarticulated principles guiding the design. The group moved from having an air space under the house to using a basement filled with water. The problem was then how to transfer the heat energy effectively from the water to the rest of the house. The group searched for and independently located a site dealing with ventilation that provided evidence to justify this refined alternative.

This particular group was coded as experimenters because of the tendency to test a series of related alternatives. A consumer would not have been as likely to refine the design while a strategizer would have retained a more prototypical or normative design. The group did not begin with a conceptual understanding of the problem. Rather, they developed explanatory mechanisms after looking at the various sites and completing the heat flow analysis. A glimpse into their explanations for how ventilation transfers heat energy from one region to another substantiates the interactional disposition categorization that was arrived at from analyzing the final report:
i: you talked a lot about ventilation i'm just wondering if that's the same kind of principle as heat energy moving through insulation.

s: maybe it would like be the same. is heat circulating the same thing as heat going through insulation? i dont know, i don't think so because i mean the actual circulation is caused by like the cool air pushing the hotter air towards the ceiling towards higher up in the altitude. that's it. when the cool air gets inside the house it pushes the hot air to higher altitudes.

A conceptualizer would have reasoned from the principle that heat energy flows from regions with higher temperatures to regions with lower temperatures.

Many of the comments reflecting p-prims that were extracted from the final reports (see Listing 1) reflect a similar explanatory gap in the understanding of heat transfer. For example, the idea that materials need holes to let heat in and out is an attempt to explain the mechanism for energy exchange. Group 2.06 states that "the advantage of having windows is it will make the heat flow in the house easier so the used air has more openings to get out of."

Figure 24. Sketch of heat flow in hybrid of How To Stay Cool House (group 2.06)

The intuition that heat energy gets "used up" or "trapped in walls" can be converted into more normative explanations by linking principles with concrete examples. During the clinical interviews many of the students were able to develop explanations by referring to labs they had conducted earlier in the semester and applying them to the novel situation of describing heat flow at the different times of day. For example, one group referred to the Coke and Potatoes lab where they had arrived at the same principle for describing how a cold Coke warms up and a hot potato cools down. They then proceeded to describe the heat flow between the house and the surround at the different times of day in terms of similarity to either the Coke or the potato.

This type of reformulation worked for helping students in the interviews reason about the functional representation of copies of their house that were either much larger or smaller in scale. To effectively solve this task students needed to refer to the Pulsing lab where they had added equal amounts of heat energy to different quantities of the same substance in this case water. Interestingly, even those students who were able to link the lab to the task at hand had difficulty determining which house would have more heat energy at the end of the day. For example, student 7.02 thought that the smaller house would have more heat energy because it had heated up faster. Frequently students can recite principles but do not understand the mechanisms that underlie the phenomenon that those principles explain.

The tendency for students to select principles that support their designs late in the design process in not necessarily undesirable. In fact, the heat flow analysis was intentionally placed before the final report so that students would begin to think about heat flow in terms of day-night cycles rather than the static view adopted by most students during the first experiment. Conceptualizing heat flow in terms of functions greatly helped the students that were interviewed develop a more coherent and integrated understanding of the relation between the various elements in their design. For example, group 7.03 was able to graph the function representing the temperature of the water relative to the temperature of the environment (sold line: environment; dashed line: water.)

Figure 25. Reconstruction of student graph

This graphical representation helped them explain how the water would work to both heat and cool the house at different times of the day.

The heat flow analysis worksheet presented students with an opportunity to critique their existing designs. This worksheet revealed the difficulty that students had in explaining how heat energy transfers from the sun to the house. The idea that heat transfer occurs through contact with some object provides the basis for many of the heat flow descriptions. However, problems arise when the "surround" is not included as an intermediary in the transfer process. This limited conceptualization of heat transfer may account for the tendency for students to disregard more principled pieces of evidence in favor of sites that describe specific structural aspects of the dwelling such as the windows or the roof. Performing a heat flow analysis or critique of existing designs as part of the problem structuring phase could help students develop a more principled approach to gathering information resources.

Qualification Of Results. The attempt to provide frames for analyzing the data reported here has a number of inherent problems. First, the fuzzy nature of interactional dispositions means that membership in a category is graded in addition to being dynamically redefined over the course of the activity. The categorization scheme does not reflect this fact. However, the purpose of analyzing the ID categories has been to discover the potential for using them to link behavior patterns in different phases of the activity in preparation for the next phase of this research project. Second, there are problems with the post hoc explanations that link behavior patterns. However, by laying out specific hypotheses about how evidence is used during the problem definition versus the detail design phase these problems are reduced to some extent. Further research is needed to determine if these explanations have strong emperical support. Third, the hierarchy of ID groupings is not clearly defined primarily because we are not sure whether it is better to approach design problems from a conceptual standpoint or to support design alternatives using principles after those alternatives have been generated. The probable outcome of this debate is that the top-down versus bottom-up approach depends on some combination of the student's ID and the task at hand.

Finally, the fact that the second experiment allocated more time (i.e., half a day) to the search activity can be disregarded as the cause of the increased rate of sites deemed relevant. We have shown that more time searching does not typically lead to the location of more relevant sites for a majority of students (i.e., there is a threshold for most students (Cuthbert, 1996)). And in reality students in both experiments ended up having similar time allocated to the search activity even though the activity plans differed slightly.

[ Abstract | Introduction | Methods | Results & Discussion | Conclusion | References | Bibliography ]