Computer Supported Collaborative Learning '97, Dec. 10-14, 1997, Toronto

An Integrated Approach to Implementing Collaborative Inquiry in the Classroom

Daniel D. Suthers, Eva Erdosne Toth, and Arlene Weiner

University of Pittsburgh
Learning Research and Development Center

Abstract. To be successful, CSCL technology must be adopted by teachers and incorporated into the activities of the classroom. This paper describes a comprehensive approach to supporting teachers learning to implement computer-supported collaborative inquiry in their classrooms. The approach comprises (1) a networked software system, "Belvedere," that provides students with shared workspaces for coordinating and recording their collaboration in scientific inquiry; (2) activity plans worked out collaboratively with teachers; (3) "challenge problems" and Web-based materials designed to match and enrich the curriculum, and (4) self- and peer-assessment instruments given to students to guide the process of scientific inquiry. A fundamental aim of this work is to restructure the classroom and shift the initiative for learning activity to the students.

Introduction

Technology has the potential to transform education, not just by providing students with an opportunity to learn the tools of the modern workplace, nor simply by automating aspects of the educational process. Its greater potential lies in the ability to change the organization of classes, from teacher-centered didactic instruction to student-centered collaborative inquiry [3, 8, 10]. Properly designed technology supports and facilitates collaborative approaches to learning that are recommended by numerous researchers [4, 5, 11, 17]. However, this potential is not an attribute of technology in itself. Computer supported collaborative learning (CSCL) technology will have an impact only if it is designed along with methodologies and materials that provide support for teachers who are learning to implement nontraditional activities in their classrooms, and address concerns such as integration with the curriculum and effective utilization of inadequate computer resources.

In this paper we describe a comprehensive methodology for implementing computer-supported collaborative inquiry in the classroom. The approach begins with a networked software system, "Belvedere," that provides students with shared workspaces for coordinating and recording their collaboration in scientific inquiry. The approach also includes student activity plans worked out in collaboration with teachers. Students work in teams to investigate real-world "challenge problems," designed to match and enrich the curriculum with attention to National Science Education Standards [6]. The teams plan their investigation, perform hands-on experiments, analyze their results, and report their conclusions to others. Our classroom activity plans provide teachers with specific guidance on how to manage these activities with different levels of computer resources. Teachers and students are provided with assessment instruments designed as an integral part of the curriculum. Assessment rubrics are given to the students at the beginning of their project as criteria to guide their activities. They guide peer review, as well as helping the teacher assess nontraditional learning objectives. After describing these components of our comprehensive approach to support collaborative inquiry, we describe the current use of Belvedere in several schools and discuss evaluation efforts.

Software for Collaborative Inquiry

The "Belvedere" software described in this paper is a complete redesign and reimplementation of a system of the same name, previously reported in [13, 14]. Belvedere's core functionality is a shared workspace for constructing "inquiry diagrams," which relate data and hypotheses by evidential relations (consistency and inconsistency). The software also includes artificial intelligence coaches that provide advice, a "chat" facility for unstructured discussions, and facilities for integrated use with Web browsers.


Figure 1. Belvedere Inquiry Diagram and Advice

The diagramming window is shown in Figure 1, with a student-generated "inquiry diagram" and a window (in the lower right corner) displaying advice from a coach. The default "palette" (the horizontal row of icons near the top of Figure 1) makes salient the most crucial distinctions we want students to acquire in order to conduct scientific inquiry. Left to right, the icons are "data" for empirical statements, "hypothesis" for theoretical statements, "unspecified" shape statements about which students disagree or are uncertain; then links representing "against" and "for" evidential relations, and a link for conjunction [5]. Students use the palette by clicking on an icon, typing some text (in the case of statements) and optionally setting other attributes, and then clicking in the diagram to place the statement or create the link. The remaining icons at the right end of the palette provide sources of counsel and knowledge: they are a light bulb representing "ideas" from the coach, an "in-box" that can receive information from Web pages, and (optional and not shown in the figure) icons that start other applications such as a Web browser. A "Guide" menu (upper right of Figure 1) provides students with suggestions on how to use the software through five "phases of inquiry" (explore, hypothesize, investigate, evaluate, and report).

We use a diagrammatic interface for cognitive, collaborative, and evaluative reasons. First, the cognitive: concrete representations of abstractions turn conceptual tasks into perceptual tasks. Thus the diagrams help students "see" and internalize these abstractions and keep track of them while working on complex issues. Second, the collaborative: diagrams support collaboration by providing a shared context and reference point. Third, the evaluative: student-constructed diagrams provide the teacher and the computer with a basis for assessing students' understanding of inquiry in general and of a topic area in particular. These three reasons are discussed further below.

Cognitive Support

Diagrams help students "see" and internalize abstractions and keep track of them while working on complex issues. The inquiry diagram serves both as a record of what the students have done, and an agenda of further work (especially with the help of coaching, discussed below). The representations help guide students' thinking and activity. We have found that the choice of representational primitives has a strong effect on the content of students' collaboration, since the first action one takes when expressing an idea is to choose a category from the primitives. The earlier version of Belvedere [14] provided a large set of choices. However, in formative evaluations with dyads, the students' discussions of the choices interfered with continuation of the inquiry process. We therefore reduced the palette to the essential types, to help focus their discussion on the most essential distinctions. One of the menus provides the option of adding other primitives.

Collaborative Support

Diagrams support collaboration by providing a shared context and reference point. These advantages manifest in different ways depending on whether the students are co-present or collaborating over the network. When they are co-present, diagrams support collaboration by helping students keep track of and refer to ideas under discussion, whether using a single display, or two displays near each other. In these situations students often use gestures on the display to indicate prior statements and relationships. In some group configurations we have seen students work independently, then use gesturing on the display to re-coordinate their collaboration when one student finds relevant information. This can occur when information is brought to the group from off-line sources, such as hands-on experiments. Distally, students can work in parallel on the same workspace, as long as they are not editing the same object at the same time. On networked computers, all changes are propagated to others working with the same diagram in a "what you see is what I see" manner. In addition to the diagram, a "chat" facility and a remote pointing mechanism support unstructured natural-language conversations, needed to coordinate the more structured inquiry diagramming when collaborating at a distance.

Evaluative Support

Student-constructed diagrams provide the teacher and the computer with a basis for assessing students' understanding of scientific inquiry, as well as of subject matter knowledge. This assessment can support computer coaching of the inquiry process. As described in [9, 15], we have constructed two types of coaches. One provides general advice on the structure of the inquiry diagram from the standpoint of scientific argumentation. It helps the students understand principles of inquiry such as: hypotheses are meant to explain data, and are not accepted merely by being stated; multiple lines of evidence converging on a hypothesis is better than one consistent datum; one should seek disconfirming evidence as well as confirming evidence (addressing the confirmation bias, as shown in Figure 1); discriminating evidence is needed when two hypotheses have identical support; circular arguments are problematic; etc. The other coach performs various comparisons between the students' diagrams and an inquiry diagram provided by a subject matter expert. This coach can provide students with feedback concerning correctness, or confront students with new information (found in the expert's diagram) that challenges students in some way.

Other Features

Other features of Belvedere, briefly noted, include the following. Students can set different "belief levels" for the statements and relations, and display these as line thickness with a "filter." References to external objects can be sent from other applications to an "in-box" (right hand icon of Figure 1) for optional placement in the diagram at the students' convenience. We and our students regularly use this in-box mechanism to send references to Web pages containing relevant information. Once placed in an inquiry diagram, Belvedere provides a hyperlink back to the referenced Web page. Thus Belvedere can be used as a structured "hotlist" to organize Internet resources.

Implementation

The Belvedere application is written in Java, and is available on request[1] for MacOS, Windows `95, NT, and Solaris. It is deployed as a client within a networked architecture that is designed to provide affordable widespread access to intelligent collaborative educational functionality on a variety of desktop user platforms. See [12] for a discussion of the architecture and other aspects of the design.

Implementing Collaborative Inquiry

Pilot studies with Belvedere [13] indicated that there was a need to structure the roles and activities of students working with Belvedere (see also [16]). With teacher colleagues, we have developed a classroom implementation methodology focused on collaborative problem solving by small groups of students. The methodology calls for changes in the classroom environment, teacher's role, curriculum materials, student activities, and assessment methodology. We describe and comment on this methodology as it was carried out in our most exemplary case, in a pilot technology-infusion project.

Classroom Environment

The traditional teacher centered environment was changed to one that is more suitable for group work. Five computer stations and five tables for hands-on investigations were set up around the classroom. The computer stations became the center for collaborative exploration of Web-based curriculum materials, use of computer simulations and data analysis tools, and use of the Belvedere environment for recording results and their significance. The tables became centers for experiments with hands-on manipulatives and for paper-based work, including peer review. In less technology-rich environments, students can share work across time periods by successively working on and storing diagrams.

Teacher's Role

The teacher shifted toward the role of facilitator of student inquiry, moving among workstations, guiding student work and offering individual help. Teachers' transition into this new role was supported by involving them in the development of student activity plans (described below) for their classes. Teacher involvement provides a sense of ownership, helping to motivate the change in how they facilitate learning, and customizes the plans for different classroom contexts. We provided additional support in a form of cognitive apprenticeship [2], by conducting several classes with Belvedere activities ourselves. The teacher assumed increasing responsibility over time, both within each class and across classes. Where developer modeling is not available, electronic discussions and peer mentoring may help teachers support each other in new practices.

Curriculum Materials


Figure 2. Web-based Materials for Challenge Problem

Students learn to conduct critical inquiry by being posed with real world problems. Towards this end, we developed Web-based curriculum modules,[2] treating controversial issues such as genetic testing, or scientific problems under active investigation such as mass extinctions. The modules take into account the National Science Education Standards (NSES) [6], local curricular standards, and teacher suggestions. The modules present students with authentic problems in which good solutions require consideration of multiple viewpoints and the use of evidence collected from various sources of information.

As shown in Figure 2, two menus are provided with the web based materials. A domain independent menu (left side) guides students through five phases of inquiry, providing suggestions on how to conduct scientific inquiry and how to use the Belvedere software in this process. Another menu (bottom) provides domain specific links organized in a manner relevant to the phases of inquiry. For example, students are provided with a link to a glossary of terms; access to simplified versions of articles on scientists' hypotheses, methodology, and field reports; and a link to experiments involving both hands-on manipulatives and computer simulations. The Web-pages contain "reference" icons resembling text pages (two are seen in Figure 2, one preceding each paragraph of text), which enable students to send text found on these pages into the inquiry diagram's "in-box."

Student Activities

In our exemplary case, the activities began with ourselves or the teacher modeling the use of inquiry diagrams to the whole class, using a simple everyday example such as reasoning about why a friend's coat is wet. Then groups of 4-6 students were formed, each working with a computer. After exploring background information on the science problem and choosing hypotheses to investigate, each group was divided. One pair or triad of students conducted hands-on experiments, recorded their results, and discussed findings. The other pair or triad of students continued to investigate the computer based articles and simulations. The full group then reassembled in front of their computer to share the results of their work, and record the results and interpretation of their experiences in their inquiry diagrams (e.g., Figure 1). Finally, the student team prepared a written report to be presented to other teams. In a one-computer classroom, computer access can be interleaved with hands-on activities.

Supporting Peer Evaluation with Performance-based Assessment

The value of peer coaching in an unfamiliar practice can be limited by students' lack of knowledge of the criteria for excellent performance. Additionally, traditional assessment, considered to be the final step of instruction, does not measure inquiry skills effectively. We address both of these problems with performance-based assessment "rubrics" that we developed to guide self- and peer-assessment of critical inquiry, as well as to facilitate teacher assessment of student work. The rubrics are provided to students at the beginning of their research. They indicate expectations, show successful methods for progressing with inquiry, and give examples of excellent and poor performances, thus guiding peer assessment during collaboration. A sample is shown in Figure 3. The rubrics take into account NSES standards for content objectives and outcome skills to be measured [6], and use the methodology outlined in the New Standards: Performance Standards project [7] for evaluating student-generated artifacts and performances. 

================================================================================
What you learn   How you learn it   How you tell how well you learned           
to do
================================================================================
                                    Poor    The inquiry diagram contains one     
                                            appropriate hypothesis and no        
                                            related data.                        
To formulate     Create Belvedere   Fair    The inquiry diagram shows one        
and revise       inquiry diagrams           appropriate hypothesis and one       
scientific       that record                data supporting it.                  
explanations,    different          Good    The inquiry diagram shows one        
and to use       hypotheses about           hypothesis with the use of           
evidence to      a problem,                 evidence for it as well as against  
                                            it.                                  
develop a        different data     Good    The inquiry diagram shows several    
logical          that can help              hypotheses each connected to         
argument.        you decide                 multiple pieces of data.             
                 between                                                         
                 the hypotheses,    Great   The inquiry diagram shows multiple   
                 and the                    hypotheses with the use of           
                 relationships              evidence for as well as against      
                 between the data           each of these hypotheses.            
                 and hypotheses.                                                 
                                    Great   The inquiry diagram indicates        
                                            additional information the student   
                                            would look for to support or to      
                                            refute explanations.
================================================================================
                                    Poor    The inquiry diagram only contains    
                                            information that is drawn from       
                                            personal experience or speculation.
To develop a     Find out what      Good    The inquiry diagram contains         
model that       specialists in             references to information from       
integrates       different                  only one discipline, for example     
concepts from    disciplines                Geology, Physics, Chemistry, or      
                 think of the               Biology.                             
                 problem.                                                        
multiple         Look for           Good    The information in the inquiry       
domains with     information from           diagrams come from one kind of       
different kinds  different                  resource, for example only from      
of data.         resources, such            experiments, field observations,     
                 as online and              or articles.                         
                 library            Great   The inquiry diagram contains         
                 articles,                  references to information from       
                 experiments you            multiple disciplines such as         
                 do, and field              Geology, Physics, Chemistry,         
                 observations.              Biology.                             
                                    Great   The information in the inquiry       
                                            diagrams are drawn from multiple     
                                            resources, such as experiments,      
                                            field observations, and articles.    
================================================================================
Figure 3. Sample Assessment Rubrics

Classroom Use of Belvedere

Belvedere was used in the first semester of 1997 by 5 teacher participants in 4 Department of Defense Dependents' Schools (DoDDS) in Germany and Italy. The classes include 9th grade Science, and 9-12th grade Physics, Chemistry, and Science and Technology.

Evaluation Methodology

Third-party evaluation of the Belvedere classroom implementation was conducted by a third party evaluator, Dr. Lynne Gilfillan, who was under contract with the Defense Advanced Research Project Agency's Computer Aided Education and Technology Initiative (CAETI) program, which funded the work of ourselves and many others in the DoDDS testbed. Dr. Gilfillan's used classroom observation forms focused on CAETI program objectives and the use of CAETI infrastructure. She also videotaped selected classroom sessions. We provided her with additional observation forms to record the activities of teachers and students, and their use of components of our software and methodology. The location of schools prevented extended observations on our part, but analysis of these forms, along with analysis of student generated artifacts (such as inquiry diagrams, Excel graphs, and student reports) for learning gains, is ongoing [18].

Summary of Evaluation Results

The independent evaluator's report discusses effects of the Belvedere approach on the general nature of student activity, on teacher roles and on the classroom environment.

Observations of student activity show that students were engaged and on task during the collaborative problems solving situations presented to them by the Belvedere comprehensive approach. Teachers indicated that the approach enhanced students ability to engage in collaborative tasks.

"Classroom observations of teachers and students using Belvedere show that it is being used to support cooperative problem solving, with students working in groups of 2 to 4 students. Students appeared to be engaged and on task. Teachers report that it is easy to use, and they find that it enhances students ability to engage in cooperative work, and to address scientific hypothesis testing in an organized and analytical way."[3]

Students also found the activity structure easy to follow and helpful in integrating work with the use of various software tools and information resources such as the world wide web.

"Students report that working with Belvedere makes it easier for them to organize and review the arguments for and against a specific scientific hypothesis. They also report that they find it easy to integrate work in Belvedere with work in other applications like Word and Excel and Web Browsers. Students using Belvedere generated artifacts that demonstrated integration of the knowledge representation maps generated using Belvedere with text and graphic information taken from a variety of resources, including the Internet, and with numerical data generated as a result of classroom activities."

Teachers reported that the staff development activities provided were adequate for classroom implementation of the Belvedere approach.

"Data collected on the efficacy of staff development for teachers using Belvedere indicated that they were very satisfied with the training provided, and believed that they were well prepared to integrate use of the Belvedere software into their classrooms. The staff development provided for Belvedere compared very favorably with that provided by other application developers in the CAETI program.

The independent evaluator also reported a striking difference in classroom organization before and after the introduction of the Belvedere approach. The classroom changed from a traditional format, with students doing work at their desks in rows, to a group-centered organization, in which students were gathered around computers or hands-on activities "like campfires" and engaged in active discussion.

Conclusions

The Belvedere project integrates advanced technology with a classroom implementation methodology to support collaborative inquiry in the classroom. Networked groupware for collaborative inquiry and intelligent coaching aids were co-designed and delivered along with cognitively principled curriculum materials, activities designed to encourage collaborative inquiry, classroom implementation plans developed collaboratively with teachers, and instruments for assessment of nontraditional learning objectives that also scaffold peer coaching.

A number of directions for future work are under varying degrees of development in our project. Of particular interest to the CSCL community is the extension of our coaching capabilities to coach collaboration. We[4] are piloting studies with human coaches to identify opportunities for coaching effective collaboration [1]. Using our networked environment, students and coaches interact electronically from different rooms. The studies will identify what can be inferred about students' collaborative processes from the limited information available through the interface, and to see how students respond to particular suggestions. We also plan to further examine the role of the rubrics in enabling peer coaching. A more ambitious direction is the extension of Belvedere to support collaboration between educators who are themselves designing new challenge activities and classroom implementation plans. The larger vision is to support sustainable implementation of collaborative inquiry in the schools through a family of cognitively principled tools for all learning communities involved in the educational process.

Acknowledgments

We express gratitude to Dan Jones for work on the client-server architecture and collaborative database, Kim Harrigal for work on the client, and Joe Toth for work on the coach. Funded by DARPA's "Computer Aided Education and Training Initiative," under the title "Collaboration, Apprenticeship, and Critical Discussion: Groupware for Learning," Contract N66001-95-C-8621. A prior version of the Belvedere software was funded by the National Science Foundation.

References

[1] M. Baker & K. Bielaczyc. Missed opportunities for learning in collaborative problem-solving interactions. In J. Greer (Ed .), AI-Ed 95, the 7th World Conference on Artificial Intelligence in Education, Washington DC, August 16-19, 1995.

[2] Collins, A., Brown, J. S., & Newman, S. Cognitive Apprenticeship: Teaching the craft of reading, writing, and mathematics. Tech. Report No. 403, Palo Alto, CA: Institute for Research on Learning, 1989.

[3] J. Cummins. From the inner city to the global village: The microcomputer as a catalyst for collaborative interchange. Language, Culture and Curriculum, 1 (1), 1-13, 1988.

[4] D. Johnson & R. Johnson. Cooperation and Competition: Theory and Research. Interaction Book Company, 1989.

[5] D. Kuhn. Science as argument: Implications for teaching and learning scientific thinking skills. Science Education, 77:319-337, 1993.

[6] National Academy of Sciences. National Science Education Standards. National Academy Press, 1996.

[7] New Standards: Performance Standards. National Center of Education and the Economy, 1995.

[8] D. K. O'Neill & L. M. Gomez. The collaboratory notebook: A distributed knowledge-building environment for project-enhanced learning. Proceedings of Ed-Media '94, Vancouver, BC, June 1994.

[9] M. Paolucci, D. Suthers, & A. Weiner. Automated advice-giving strategies for scientific inquiry. Intelligent Tutoring Systems, 3rd International Conference, Montreal, June 12-14, 1996.

[10] M. Scardamalia & C. Bereiter. Higher levels of agency for children in knowledge building: A challenge for the design of new knowledge media. The Journal of the Learning Sciences, 1:37-68, 1991.

[11] R. E. Slavin. Cooperative Learning: Theory, Research, and Practice. Prentice-Hall, Englewood Cliffs, NJ, 1990.

[12] D. Suthers & D. Jones. An architecture for intelligent collaborative educational systems. AI-Ed 97, the 8th World Conference on Artificial Intelligence in Education, Kobe Japan, August 20-22, 1997.

[13] D. Suthers & A. Weiner. Groupware for developing critical discussion skills. CSCL '95, Computer Supported Cooperative Learning, Bloomington, Indiana, October 17-20, 1995.

[14] D. Suthers, A. Weiner, J. Connelly, & M. Paolucci, M. Belvedere: Engaging students in critical discussion of science and public policy issues. AI-Ed 95, the 7th World Conference on Artificial Intelligence in Education, Washington DC, August 16-19, 1995.

[15] J. Toth, D. Suthers, & A. Weiner. Providing expert advice in the domain of collaborative scientific inquiry. AI-Ed 97, the 8th World Conference on Artificial Intelligence in Education, Kobe Japan, August 20-22, 1997.

[16] M.L. Waugh & J. Levin. Telescience activities: Educational uses of electronic networks. The Journal of Computers in Mathematics and Science Teaching , 8(2): 29-33, 1988.

[17] N. Webb. Peer interaction and learning in small groups. International Journal of Education Research, 13:21-40, 1989.

[18] A. Weiner & E.Toth (submitted). Beyond technological literacy: Toward the construction of difficult science and math concepts in a technology-focused environment. Submitted for presentation to the Annual Meeting of the American Educational Research Association 1998.)

Back to Paper Index