Computer Supported Collaborative Learning '97, Dec. 10-14, 1997, Toronto
University of Pittsburgh
Learning Research and Development Center
Abstract. To be successful, CSCL technology must be adopted by teachers and incorporated into the activities of the classroom. This paper describes a comprehensive approach to supporting teachers learning to implement computer-supported collaborative inquiry in their classrooms. The approach comprises (1) a networked software system, "Belvedere," that provides students with shared workspaces for coordinating and recording their collaboration in scientific inquiry; (2) activity plans worked out collaboratively with teachers; (3) "challenge problems" and Web-based materials designed to match and enrich the curriculum, and (4) self- and peer-assessment instruments given to students to guide the process of scientific inquiry. A fundamental aim of this work is to restructure the classroom and shift the initiative for learning activity to the students.
In this paper we describe a comprehensive methodology for implementing computer-supported collaborative inquiry in the classroom. The approach begins with a networked software system, "Belvedere," that provides students with shared workspaces for coordinating and recording their collaboration in scientific inquiry. The approach also includes student activity plans worked out in collaboration with teachers. Students work in teams to investigate real-world "challenge problems," designed to match and enrich the curriculum with attention to National Science Education Standards . The teams plan their investigation, perform hands-on experiments, analyze their results, and report their conclusions to others. Our classroom activity plans provide teachers with specific guidance on how to manage these activities with different levels of computer resources. Teachers and students are provided with assessment instruments designed as an integral part of the curriculum. Assessment rubrics are given to the students at the beginning of their project as criteria to guide their activities. They guide peer review, as well as helping the teacher assess nontraditional learning objectives. After describing these components of our comprehensive approach to support collaborative inquiry, we describe the current use of Belvedere in several schools and discuss evaluation efforts.
The diagramming window is shown in Figure 1, with a student-generated "inquiry diagram" and a window (in the lower right corner) displaying advice from a coach. The default "palette" (the horizontal row of icons near the top of Figure 1) makes salient the most crucial distinctions we want students to acquire in order to conduct scientific inquiry. Left to right, the icons are "data" for empirical statements, "hypothesis" for theoretical statements, "unspecified" shape statements about which students disagree or are uncertain; then links representing "against" and "for" evidential relations, and a link for conjunction . Students use the palette by clicking on an icon, typing some text (in the case of statements) and optionally setting other attributes, and then clicking in the diagram to place the statement or create the link. The remaining icons at the right end of the palette provide sources of counsel and knowledge: they are a light bulb representing "ideas" from the coach, an "in-box" that can receive information from Web pages, and (optional and not shown in the figure) icons that start other applications such as a Web browser. A "Guide" menu (upper right of Figure 1) provides students with suggestions on how to use the software through five "phases of inquiry" (explore, hypothesize, investigate, evaluate, and report).
We use a diagrammatic interface for cognitive, collaborative, and evaluative reasons. First, the cognitive: concrete representations of abstractions turn conceptual tasks into perceptual tasks. Thus the diagrams help students "see" and internalize these abstractions and keep track of them while working on complex issues. Second, the collaborative: diagrams support collaboration by providing a shared context and reference point. Third, the evaluative: student-constructed diagrams provide the teacher and the computer with a basis for assessing students' understanding of inquiry in general and of a topic area in particular. These three reasons are discussed further below.
Students learn to conduct critical inquiry by being posed with real world problems. Towards this end, we developed Web-based curriculum modules, treating controversial issues such as genetic testing, or scientific problems under active investigation such as mass extinctions. The modules take into account the National Science Education Standards (NSES) , local curricular standards, and teacher suggestions. The modules present students with authentic problems in which good solutions require consideration of multiple viewpoints and the use of evidence collected from various sources of information.
As shown in Figure 2, two menus are provided with the web based materials. A domain independent menu (left side) guides students through five phases of inquiry, providing suggestions on how to conduct scientific inquiry and how to use the Belvedere software in this process. Another menu (bottom) provides domain specific links organized in a manner relevant to the phases of inquiry. For example, students are provided with a link to a glossary of terms; access to simplified versions of articles on scientists' hypotheses, methodology, and field reports; and a link to experiments involving both hands-on manipulatives and computer simulations. The Web-pages contain "reference" icons resembling text pages (two are seen in Figure 2, one preceding each paragraph of text), which enable students to send text found on these pages into the inquiry diagram's "in-box."
The value of peer coaching in an unfamiliar practice can be limited by students' lack of knowledge of the criteria for excellent performance. Additionally, traditional assessment, considered to be the final step of instruction, does not measure inquiry skills effectively. We address both of these problems with performance-based assessment "rubrics" that we developed to guide self- and peer-assessment of critical inquiry, as well as to facilitate teacher assessment of student work. The rubrics are provided to students at the beginning of their research. They indicate expectations, show successful methods for progressing with inquiry, and give examples of excellent and poor performances, thus guiding peer assessment during collaboration. A sample is shown in Figure 3. The rubrics take into account NSES standards for content objectives and outcome skills to be measured , and use the methodology outlined in the New Standards: Performance Standards project  for evaluating student-generated artifacts and performances.
================================================================================ What you learn How you learn it How you tell how well you learned to do ================================================================================ Poor The inquiry diagram contains one appropriate hypothesis and no related data. To formulate Create Belvedere Fair The inquiry diagram shows one and revise inquiry diagrams appropriate hypothesis and one scientific that record data supporting it. explanations, different Good The inquiry diagram shows one and to use hypotheses about hypothesis with the use of evidence to a problem, evidence for it as well as against it. develop a different data Good The inquiry diagram shows several logical that can help hypotheses each connected to argument. you decide multiple pieces of data. between the hypotheses, Great The inquiry diagram shows multiple and the hypotheses with the use of relationships evidence for as well as against between the data each of these hypotheses. and hypotheses. Great The inquiry diagram indicates additional information the student would look for to support or to refute explanations. ================================================================================ Poor The inquiry diagram only contains information that is drawn from personal experience or speculation. To develop a Find out what Good The inquiry diagram contains model that specialists in references to information from integrates different only one discipline, for example concepts from disciplines Geology, Physics, Chemistry, or think of the Biology. problem. multiple Look for Good The information in the inquiry domains with information from diagrams come from one kind of different kinds different resource, for example only from of data. resources, such experiments, field observations, as online and or articles. library Great The inquiry diagram contains articles, references to information from experiments you multiple disciplines such as do, and field Geology, Physics, Chemistry, observations. Biology. Great The information in the inquiry diagrams are drawn from multiple resources, such as experiments, field observations, and articles. ================================================================================
Observations of student activity show that students were engaged and on task during the collaborative problems solving situations presented to them by the Belvedere comprehensive approach. Teachers indicated that the approach enhanced students ability to engage in collaborative tasks.
"Classroom observations of teachers and students using Belvedere show that it is being used to support cooperative problem solving, with students working in groups of 2 to 4 students. Students appeared to be engaged and on task. Teachers report that it is easy to use, and they find that it enhances students ability to engage in cooperative work, and to address scientific hypothesis testing in an organized and analytical way."
Students also found the activity structure easy to follow and helpful in integrating work with the use of various software tools and information resources such as the world wide web.
"Students report that working with Belvedere makes it easier for them to organize and review the arguments for and against a specific scientific hypothesis. They also report that they find it easy to integrate work in Belvedere with work in other applications like Word and Excel and Web Browsers. Students using Belvedere generated artifacts that demonstrated integration of the knowledge representation maps generated using Belvedere with text and graphic information taken from a variety of resources, including the Internet, and with numerical data generated as a result of classroom activities."
Teachers reported that the staff development activities provided were adequate for classroom implementation of the Belvedere approach.
"Data collected on the efficacy of staff development for teachers using Belvedere indicated that they were very satisfied with the training provided, and believed that they were well prepared to integrate use of the Belvedere software into their classrooms. The staff development provided for Belvedere compared very favorably with that provided by other application developers in the CAETI program.
The independent evaluator also reported a striking difference in classroom organization before and after the introduction of the Belvedere approach. The classroom changed from a traditional format, with students doing work at their desks in rows, to a group-centered organization, in which students were gathered around computers or hands-on activities "like campfires" and engaged in active discussion.
A number of directions for future work are under varying degrees of development in our project. Of particular interest to the CSCL community is the extension of our coaching capabilities to coach collaboration. We are piloting studies with human coaches to identify opportunities for coaching effective collaboration . Using our networked environment, students and coaches interact electronically from different rooms. The studies will identify what can be inferred about students' collaborative processes from the limited information available through the interface, and to see how students respond to particular suggestions. We also plan to further examine the role of the rubrics in enabling peer coaching. A more ambitious direction is the extension of Belvedere to support collaboration between educators who are themselves designing new challenge activities and classroom implementation plans. The larger vision is to support sustainable implementation of collaborative inquiry in the schools through a family of cognitively principled tools for all learning communities involved in the educational process.
 M. Baker & K. Bielaczyc. Missed opportunities for learning in collaborative problem-solving interactions. In J. Greer (Ed .), AI-Ed 95, the 7th World Conference on Artificial Intelligence in Education, Washington DC, August 16-19, 1995.
 Collins, A., Brown, J. S., & Newman, S. Cognitive Apprenticeship: Teaching the craft of reading, writing, and mathematics. Tech. Report No. 403, Palo Alto, CA: Institute for Research on Learning, 1989.
 J. Cummins. From the inner city to the global village: The microcomputer as a catalyst for collaborative interchange. Language, Culture and Curriculum, 1 (1), 1-13, 1988.
 D. Johnson & R. Johnson. Cooperation and Competition: Theory and Research. Interaction Book Company, 1989.
 D. Kuhn. Science as argument: Implications for teaching and learning scientific thinking skills. Science Education, 77:319-337, 1993.
 National Academy of Sciences. National Science Education Standards. National Academy Press, 1996.
 New Standards: Performance Standards. National Center of Education and the Economy, 1995.
 D. K. O'Neill & L. M. Gomez. The collaboratory notebook: A distributed knowledge-building environment for project-enhanced learning. Proceedings of Ed-Media '94, Vancouver, BC, June 1994.
 M. Paolucci, D. Suthers, & A. Weiner. Automated advice-giving strategies for scientific inquiry. Intelligent Tutoring Systems, 3rd International Conference, Montreal, June 12-14, 1996.
 M. Scardamalia & C. Bereiter. Higher levels of agency for children in knowledge building: A challenge for the design of new knowledge media. The Journal of the Learning Sciences, 1:37-68, 1991.
 R. E. Slavin. Cooperative Learning: Theory, Research, and Practice. Prentice-Hall, Englewood Cliffs, NJ, 1990.
 D. Suthers & D. Jones. An architecture for intelligent collaborative educational systems. AI-Ed 97, the 8th World Conference on Artificial Intelligence in Education, Kobe Japan, August 20-22, 1997.
 D. Suthers & A. Weiner. Groupware for developing critical discussion skills. CSCL '95, Computer Supported Cooperative Learning, Bloomington, Indiana, October 17-20, 1995.
 D. Suthers, A. Weiner, J. Connelly, & M. Paolucci, M. Belvedere: Engaging students in critical discussion of science and public policy issues. AI-Ed 95, the 7th World Conference on Artificial Intelligence in Education, Washington DC, August 16-19, 1995.
 J. Toth, D. Suthers, & A. Weiner. Providing expert advice in the domain of collaborative scientific inquiry. AI-Ed 97, the 8th World Conference on Artificial Intelligence in Education, Kobe Japan, August 20-22, 1997.
 M.L. Waugh & J. Levin. Telescience activities: Educational uses of electronic networks. The Journal of Computers in Mathematics and Science Teaching , 8(2): 29-33, 1988.
 N. Webb. Peer interaction and learning in small groups. International Journal of Education Research, 13:21-40, 1989.
 A. Weiner & E.Toth (submitted). Beyond technological literacy: Toward the construction of difficult science and math concepts in a technology-focused environment. Submitted for presentation to the Annual Meeting of the American Educational Research Association 1998.)