MindWorks:  Making Scientific Concepts Come Alive—
Final Report

by Barbara J. Becker

 

 

  "The Centennial Tower One Thousand Feet High", from the cover of Scientific American, January 24, 1874.

ABSTRACT

WestEd has completed its development of MindWorks, a series of eight instructional modules for use in common secondary school physical science that address three central goals of science literacy education:  1) to motivate students who have previously shown little interest in science; 2) to accomplish deep change in students' internalized conceptions of the structure and workings of the physical world; and 3) to build greater understanding, in both teachers and students, of the process and culture of scientific activity.

This paper is a final report on the project's progress.  In it, the author outlines the structure and methodology of the large-scale pilot and presents the results of the project's summative evaluation.

*  *  *

INTRODUCTION

In October 1994, the Southwest Regional Laboratory (now WestEd) received major funding from the National Science Foundation (ESI-9450235) to support a instructional materials development project, "Making scientific concepts come alive through historical dramatizations and active student learning strategies:  A history-of-science approach to high school physical science literacy", under the direction of its principal investigator (PI), Dr. Barbara J. Becker. 

The project is now complete.  The series of eight modules, entitled MindWorks (WestEd, 2000), complement, extend, and enrich existing curricular materials on the subjects of statics, kinematics, dynamics, thermodynamics, optics, electricity and magnetism, atomic structure, as well as the application of these basic physical principles in contemporary research.  This paper is the final report on the materials' development (Becker, 2000; Becker, Younger-Flores & Wandersee, 1995).

LARGE-SCALE PILOT

In April 1996, the PI and WestEd staff recruited local teachers to participate in a large-scale pilot of the MindWorks materials during the 1996/97 school year.  Because of the unique nature of these materials, both in terms of their content and proposed implementation, we designed the summative evaluation to include two groups of pilot teachers:

In addition, we recruited a third group of teachers:

The selected teachers formed a diverse group, not only in terms of their personal background and professional training, but also in terms of the range of students they made it possible for us to evaluate.  The evaluation included:

Participating teachers administered the MWSSSU and TOSRA to students in all study groups both at the beginning and at the end of the school year.

Data were gathered on approximately 925 students from 15 TT classes (N @ 400), 9 NT classes (N @ 250), and 11 CT classes (N @ 275).  (NOTE:  Attrition in TT and NT classes was due to teacher re-assignment, intramural curricular restructuring, limited student English proficiency, teacher pregnancy, and non-completion of necessary paperwork.)

  • Student attitudes toward science.  We targeted six of the TOSRA's 7 attitude subtests in our survey.  There was little difference in the pre-test scores among the three groups, indicating that the populations shared similar science attitudes before the treatment began.  The treatment group mean gain scores (pre- to post) are shown in Table 1.  The scores themselves are item means, with each item a 5-point scale.  TT and NT groups did not differ in post-test attitudes and were combined for this analysis.

Table 1:  Test of Science Related Attitudes:  Pre- to Post-Test Gains

Normality of Scientists

Attitude to Scientific Inquiry
Adoption of Scientific Attitudes
Enjoyment of Science Lessons
Leisure Interest in Science
Career Interest in Science
TT&NT
  0.09
- 0.02

  0.01

- 0.01
- 0.02
- 0.03
CT
- 0.02
- 0.17
- 0.11
- 0.29
- 0.12
- 0.22
d
  0.11
  0.15
  0.12
  0.28
  0.10
  0.19
E.S.
+.21*
+.21*
+.23*
+.36*
+.13
+.24*

d = (TT & NT) - CT
E.S. = effect size = d/s, where s = comparison group pre-test standard deviation.
E.S. is linearly related to the t-statistic, and therefore statistical significance was calculated from the value of t computed from E.S.
*p < .05, 2-tailed

Students in the TT and NT groups generally stayed constant in their attitudes.  In contrast, the comparison (CT) students' attitudes became more negative, significantly so on 4 of the 6 subtests.  Because student attitudes towards school-related issues are generally more positive in the fall, these data suggest that the MindWorks curriculum materials had a positive effect on student attitudes, on the order of one-fourth of a standard deviation (E.S.=.24).

  • Student understanding of basic science concepts.  MWSSSU post-tests were administered in June 1997.  Table 2 shows the results of our analysis of these data.  Again, treatment group mean gain scores are shown.

Table 2:  MindWorks Survey of Student Science Understanding:  Pre- to Post-Test Gains

Falling Bodies
Structures
Collisions
Heat
Light and Color
E & M
Scale of Solar System
Atoms and Matter
Quest'g Sci'fic Claims
Overall Perf.
TT&NT
  0.08
  0.10
  0.29
  0.05
  0.22
  0.18
  0.14
  0.12
  0.20
  0.15
CT
  0.07
- 0.08
  0.44
- 0.05
- 0.02
- 0.11
- 0.13
- 0.07
  0.07
  0.02
d
  0.01
  0.18
- 0.15
  0.10
  0.24
  0.29
  0.27
  0.19
  0.13
  0.13
E.S.
+.01
+.12
-.17*
+.15
+.27*
+.22*
+.27*
+.18*
+.11
+.20*

Notes:  See Table 1.

Overall, only the MindWorks students (TT & NT) improved in performance.  On specific subtests, comparison (CT) students outperformed TT and NT students on one subtest (Collisions), while MindWorks students showed substantially greater improvement than the CT students on 6 subtests.  For 4 of the subtests (Light & Color, Electricity & Magnetism, Scale of Solar System, and Atoms & Matter), and for the total test score, the difference favoring MindWorks was statistically significant at p<.05.  MindWorks students were also somewhat more apt to question scientific claims attributed to purported authorities (newspapers and scientists) than were traditionally taught students.

  • Teacher response to implementation.  For each module they piloted, TTs and NTs completed a summary report to inform the materials revision process.  We asked teachers to rate (using Likert scale format) both their own and their students' responses to the video, student activities, and supplementary materials.  Open-ended questions afforded them the opportunity to make extended remarks on their experience with MindWorks materials in comparison with other curricula they have used.

Overall, teachers found that MindWorks materials adapted easily to their existing program of instruction.  Most (84%) used the materials directly as suggested, or made only minor changes.  Those who made changes, did so largely to blend MindWorks more seamlessly into their customary instructional format.

Half reported that it took more time to prepare for MindWorks lessons, but 90% indicated that the time they spent was well worth the effort.  All teachers made use of the Teacher Guide provided in each module.  Eighty-two percent rated the Teacher Guide very good or excellent as a resource.  Open-ended comments and suggestions for improving the Teacher Guide, particularly those from NTs, have been especially helpful in the revision process.

Most teachers (73%) rated their students' response to the videos as very good or excellent, and 65% showed the video associated with a module two or more times.  Teachers reported that their students were genuinely moved by the very human qualities of the scientists portrayed in each video drama:  Newton's frustration with Hooke's criticism, Galileo's joy in the language of mathematics, Voltaire's inability to persuade Châtelet to adopt his view, Count Rumford's impatience with his daughter, Marie Curie's radium-damaged hands....  Scientists are people, tenaciously curious people of all kinds.  They don't always know what to do, or how best to do it.  They don't always communicate their ideas successfully to others.  They argue, collaborate, worry, laugh, and complain.

Students were tantalized by these brief video encounters with interesting people from other times and places.  Teachers reported that their students researched the de' Medici family, read H. G. Wells' Invisible Man, related Rumford's work as a British spy during the Revolutionary War to their studies in American history, learned to play croquet, and avidly followed the progress on the Mars Pathfinder mission.

The video dramas not only opened the door for students to glimpse the basic science concepts presented in each module, but invited them in to walk around and experience firsthand the thought and work space of others who have studied the same subject long before them.  In this way, the videos made students aware of themselves as investigators capable of developing and pursuing their own lines of inquiry into the workings of the natural world.

A majority of teachers indicated that their students asked more questions (59%), spent more time on task (74%), expressed more enthusiasm (75%), and engaged in more teamwork (66%) than students they've taught using other instructional materials.  Eighty-one percent rated the overall student response to the materials as very good or excellent.  Nearly all (95%) reported that their students understood basic concepts the same or better following MindWorks instruction, compared with other curricula they have used.

Most teachers expressed satisfaction with the student activities provided (94%), the lesson plans (87%), the videos (95%), and the historical documents (90%).  They liked the flexibility of the lessons, the interdisciplinary nature of module themes, the emphasis placed on uncertainty and argument in science, the human face that the videos and historical materials put on the scientists featured in each module, and, perhaps most important, the intellectual stimulation they personally derived from teaching with the MindWorks materials.  Finally, 94% indicated they would likely use the materials again.

We used data from the large-scale pilot to improve the instructional impact of each module.  We made the Benchmark issues, which had been tacitly understood as undergirding each lesson, more explicit.  We made activities more student-directed and collaborative in structure.  We modified activity instructions to ease their independent use by students.  We designed new activities to improve the articulation among lessons.  We added numerous diagrams and examples to student activities and the teacher guides.  We made historical documents more directly accessible to students.  We strengthened the links between the historical documents and the basic science concepts contained in each module.

FOLLOW-UP PILOT

Finally, we conducted an ad hoc follow-up pilot during the 1997/98 school year.  Our aim was to obtain additional information from both training and non-training teachers on their second year of incorporating the MindWorks materials into their instructional program.

We invited all large-scale pilot teachers (TT and NT) to participate.  Eight agreed to do so.  To eliminate one important instructional variable in the 1996/97 pilot, namely that introduced by the use of a separate group of individuals as comparison teachers, follow-up pilot teachers were asked to teach some of their classes using MindWorks materials and to instruct their other classes the way they always had, without MindWorks.  Most had teaching schedules that enabled them to comply with this request.  All agreed to administer the MWSSSU and TOSRA as they had in the large-scale pilot, and to submit "Unit and Lesson Delivery Information" reports as each module was taught.

The information gained from these reports proved invaluable in revising and expanding the teacher guide notes.  Although the number of non-MindWorks students participating in the follow-up pilot were insufficient to permit rigorous comparison of their pre- and post-test scores, qualitatively these results confirmed those obtained during the large-scale pilot in 1996/97.

CONCLUSION

The body of data collected during MindWorks' classroom implementation demonstrates that approaching scientific inquiry in the context of history is an effective means of increasing scientific literacy.  The use of elaboration in the form of episodes and story lines assist learners in retaining both complex information and content-specific knowledge.  Historical vignettes illustrate others' efforts to critically describe, codify, quantify, and explain the physical world.  They provide students with a glimpse of the economic, political, and social hurdles that investigators have overcome in achieving their goals.  Our field study results show that students are avidly interested in these nitty gritty issues.

Integrating history into science instruction does not have to involve a recitation of the accomplishments of a handful of well-known figures.  Many kinds of people have made valuable contributions to the development of our modern understanding of the natural world.  Our field tests show that their stories connect with students' imaginations at a familiar and comfortable level.  They raise students' curiosity about specific physical phenomena and provide them with illustrations of the range of means individuals have employed in the past to satisfy that curiosity.  Rather than miring the students in the past, the past offers them a provocative yet structured guideline for conceiving the future and their potential for shaping it.

BIBLIOGRAPHY

Becker, B. J.:  2000, 'MindWorks:  Making scientific concepts come alive', Science & Education  9, 269-278; reprinted 2002, J. J. Hirschbuhl and D. Bishop (eds.), Computers in Education. 2002/2003, 10th ed., McGraw-Hill/Dushkin, Guilford, CT, pp. 129-133.

Becker, B. J., Younger-Flores, K., &Wandersee, J. H.: 1995, 'MindWorks:  Making scientific concepts come alive'.  In F. Finley, D. Allchin, D. Rhees, and S. Fifield (eds.), Proceedings of the Third International History, Philosophy, and Science Teaching vol. 1, The University of Minnesota, Minneapolis, MN, pp. 115-125.

WestEd:  2000, MindWorks.  Series includes Student Reader, Teacher Guide and Video for each of the following titles:  Statics & Structures, Kinematics, Dynamics, Thermodynamics, Light & Color, Electricity & Magnetism, Atoms & Matter, and Tomorrow's Challenges, Kendall-Hunt Publishing, Dubuque, IA.