Project Report

The Impact of Networking on K-12 Education Reform

NSF Report Prepared for Principal Investigator Meeting June 3-4, 1999

© May 21, 1999

1. Overview

    

   Updated Abstract

   This report describes an evaluation effort conducted 1996-99 to determine whether the combination of constructivist teaching strategies and Internet-based resources correlates positively with increases in student performance. The constructivist teaching model evolved out of numerous statewide efforts (Statewide Systemic Initiatives or SSI) to achieve education reform, especially in science and mathematics, with support from the National Science Foundation.

   We conducted a survey of teachers of grades 4-12, ranging from constructivist to traditional, and heavy Internet users to light or no use of Internet. Using a series of standard instruments, we evaluated their levels of technology use and comfort and learning style. We surveyed the students of these same teachers to determine their learning style, and levels of technology comfort, motivation for learning and interest in subject matter.

   The data show no statistically significant association between student performance (motivation and interest in subject matter) and their teacher's constructivist teaching strategies, or Internet resource integration. Moreover, the interview data (focus groups, individuals) suggest that networking replaces existing communication media (letter, phone, school flyer), rather than increasing overall parental involvement in the school or their children's education. This finding is consistent with several other studies that find no significant differences (Ehrich and McCreary 1997, among others). We are in the process of determining whether we have a null result because the teachers were not strongly constructivist or heavy Internet integrators.

   Primary Research Question

   The primary aim of this study is to determine whether the introduction of network-based computing is supporting education reform and whether that correlates positively with SSI training and increased community involvement in education.

   Research Method

   To achieve these aims, we conducted a longitudinal (12 month) evaluation among the teachers, students, administrators, and community members at four school districts in Virginia. Two sites in addition to Montgomery County, were selected from the following: Goochland County Public Schools, Giles County Public Schools, and Alexandria City Schools. The participating sites were selected on the basis of their commitment to the Statewide Systemic Initiative (SSI) in Virginia, VQuest, and to networking applications in the classroom and with the community. VQuest provides a certain uniformity in approach and goals among Virginia school districts implementing reform. These uniformities facilitate comparison of evaluation results among districts. Taken together, the selected schools provide a rich mix of urban inner city, suburban, and rural environments with an appropriate blend of SSI-trained, and non-SSI-trained teachers, with and without network integration, and varying levels of local community networking.

   All sites used the same methodologies, procedures and instruments, selected or adapted collaboratively. Blacksburg, Montgomery County (home of the Blacksburg Electronic Village or BEV) was the lead site for the project. The authors supervised the project and coordinated activities among sites. Each site implemented evaluation tasks in their own districts, collected data locally, and transferred data (e.g., completed surveys) for compilation and statistical analysis.

   The evaluation plan is an attempt to provide a design for both a short-term and long-term, multi-dimensional, data collection effort and an organizational framework for interpreting that data. Because the scope of questions can be infinite, we limited impact descriptors and guiding questions to four types of learning environments where teachers:

   • integrate learner-centered teaching strategies and network resources into the curriculum
   • integrate learner centered teaching strategies, but not network resources, into the curriculum
   • do not integrate learner-centered teaching strategies, but do integrate network resources into the curriculum
   • do not integrate learner-centered teaching strategies or network resources into the curriculum

   The focus of the evaluation is on the process of schooling within the larger context of a networked community, including what was done to promote the infusion of CMC into the selected school districts and what happened as a result of these efforts. The goal of evaluation is to identify change, not to attribute the cause of the change. For projects with a scope as broad as the role of networking on education reform, it is not feasible to establish a one-to-one relationship between a specific intervention and the resultant effect. Evaluation becomes a matter of documenting Internet-related practices and shifts in teaching/learning process which emerge simultaneously with the implementation of constructivist teaching strategies, broadly-based Internet capabilities in the schools, and in interaction with the community.

   We evaluated the proposition that community involvement improves education by examining traditional vs. new means to involve community. New communication modes (email, newsgroups, Wide Web) allow new mentoring relationships to develop among peers, older and younger students, community organizations, and citizens. These relationships have the potential to enhance learning by fostering cooperation and a spirit of collaboration. Moreover, networking can foster better relationships among teachers, parents and students.

   To answer these questions, we used the following data collection methodologies: focus group interviews, observations, record review, and personal interviews. Stephen Parson, a Virginia Tech faculty member involved with the project, guided the evaluation of the community-school interaction in collaboration with participating school districts and with assistance from a graduate research assistant in the College of Human Resources and Education of Virginia Tech.

2. Advances

   Principal Discoveries

   The data that were analyzed are scores from surveys, given to teachers and students in three Virginia school districts (elementary and secondary levels). The analysis has three parts:

   1. From the survey designed to measure certain aspects of "learning style" (Vocational Learning Style Inventory), seventeen variables were created by averaging over groups of responses to individual items from the survey. The same survey was given to several teachers and to several of each teacher's students. Correlation was calculated between teacher and student scores for each of the seventeen variables. This was done separately for several groups of students, where the grouping is based on the students' ages.
   2. The data from the second part of the analysis is from three surveys. The Survey of Technology Use generated ten variables, with one observation of each variable for each teacher. These variables were calculated by averaging groups of responses to individual survey items. The Teacher Selection Form generated one average score for each teacher. The School Attitude Measure generated five variables for each student. Again, the scores were calculated by averaging individual item scores. This survey has four forms, which were analyzed separately. Canonical correlation analysis was used to search for relationships between the teacher scores (from the first two surveys) and the student scores (from the third survey).
   3. The third part of the analysis was based on the data from the Teacher Selection Form (as above) and the Nowicki-Strickland Personal Reaction Survey. The latter is a 40 item survey which generates one score for each teacher. Correlation was calculated between the scores form the two surveys.

   The data from part (i.) consisted of seventeen measurements on teachers and students, from the Vocational Learning Style Survey. The same seventeen measurements were made on both groups. Correlation coefficients were calculated, relating teacher and student responses for each measurement. These results are given for each student group considered, in Tables 1A and 1D at the end of this article.

   Correlations were calculated in two ways. First, the entire original data set for each group was used to calculate Pearson's Correlation Coefficient. P-values were generated for testing the null hypothesis of zero correlation against the alternative hypothesis of positive correlation for each group. These p-values were based on "Fisher's Z" a statistic which follows a normal distribution, asymptotically. The standard error of Fisher's Z is a function of the number of observations used in calculating the correlation. In the tables, a small p-value (near zero) indicates support for the alternative hypothesis of positive correlation, while a large p-value (near 1) provides evidence of negative correlation. As can be seen from the tables, there is little evidence for positive correlation between teacher and student responses. Although some p-values are near zero, some are also near 1. There is little consistency across groups as to which measurements had significant correlations. Finally, it should be noted that in conducting many hypothesis tests, it is likely that some p-values will be near zero, purely by chance.

   The procedure mentioned in the previous paragraph does assume that the observed data are mutually independent. Since that is not the case for the data used here, it seems likely that the standard errors for "Fisher's Z", used in conducting hypothesis tests, were too small. Indeed, when the bootstrap was used to estimate these standard errors, the estimates were larger. Hence, the resulting hypothesis tests provided even less evidence in support of positive correlation.

   In light of the above results, it was decided that correlations would also be calculated based on mean student scores, rather than individual student scores. That is, all of the student scores corresponding to each teacher were averaged, and correlations were calculated using the resulting average student scores and the teacher scores. This procedure was not recommended at the outset, because it tends to inflate correlations, making interpretation difficult. It was hoped that some significant results would be found. However, this was not the case. These results are also listed in Tables 1A through 1D.

   In part (ii.) of the analysis, the variables considered consisted of ten measurements made on teachers and five measurements made on students for each of the three districts. Canonical correlation analysis was used to search for relationship between the sets of variables. Among the four student groups, a significant result was found only for student group D. Before attempting to interpret the canonical coefficients, it is advisable to perform the analysis on data from other counties, so that comparisons can be made. The consistency of results across counties may help determine how reliable the results are.

   The analysis of part (iii.), consisted of calculating correlation between teachers' scores on the Teacher Selection Form and the Nowicki-Strickland Personal Reaction Survey. The correlations and two-sided p-values are shown in the table below. Since all p-values are much greater than zero, none of the correlations can be considered statistically significant.

   The data show no statistically significant association between student performance (motivation and interest in subject matter) and constructivist teaching strategies, or Internet resource integration. Moreover, the interview data (focus groups, individuals) suggest that networking replaces existing communication media (letter, phone, school flyer), rather than increasing overall parental involvement in the school or their children's education.

   This finding is consistent with several other studies that find no significant differences (Ehrich and McCreary 1997, among others). However, it is possible that teachers appear to be more constructivist on surveys than they actually are in the classroom. This may also be the case for the integration of Internet resources into the daily curriculum.

3. Potential Utility in Solving Educational Problems

   Research to practice continuum: This project would be positioned on such a continuum towards the practice end, as its findings related to investments in teacher training and in technology acquisition and upgrades.

   Transferable? Yes

   Influence on improved achievement in math and science? Not clear

   Connection to ongoing NSF efforts and other projects? Exchange of information at conference talks noted below, particularly VT Institute of Connecting Science to the Classroom.

4. Capacity Building

   Publications (in process: draft manuscripts)

   The Impact of Constructivism and Computer Networking and on Student Performance by Andrea Kavanaugh and John Burton

   The Impact of Computer Networking on Parental Involvement in Education by Sarah Laughon

   Conference Presentations:

   "Education via the Internet" invited panelist at the United Nations conference on the Exploration and Peaceful Uses of Outer Space (UNISPACE 99), Vienna, July 19-, 1999.

   "The Effect of Computer Networking on Families in Rural Virginia " invited paper for a conference on 'Family and the Internet', sponsored by the Public Policy Center of the Annenberg School for Communication, University of Pennsylvania, May 4, 1999.

   "The Role of the Internet in K-12 Education Reform," Guest Speaker Series, full day workshop, Spring Branch Independent School District. Houston, Texas, February 16, 1999.

   "Is Public Education in our Future?" (co-presenter with Steven Hodas) annual conference of the Public Education Network, Washington, D.C., November 16-17, 1998.

   "Pushing the Envelope: Connecting parents, teachers and children in educational collaboration." Guest speaker, Institute for Educational Dialogue Series. Board of Cooperative Educational Services (BOCES) of Nassau County, Long Island, NY. January 15, 1998

   Kavanaugh, A. and S. Laughon. "The Internet Style of Learning vs. Traditional Learning Styles: Findings of the National Science Foundation Study." Invited panelist for conference of the Virginia Education Association (District 1), October 7, 1997.

   "Evaluating the Use and Impact of Networking on K-12 Education Reform," NSF sponsored conference of the VT Institute for Connecting Science to the Classroom "Enhancing Instruction in Science, Mathematics and Technology," March 11, 1996.

   Role of research collaborators: local coordination of data collection, consultation on instruments and methods

   Role of teachers: subjects and consultants in design

   Role of policy makers: none

   Role of students: Graduate student research assistants assisted with data collection and analysis; non undergraduate students participated; grades 4-12 students were subjects

   Role of technology: subject of study (evaluation grant)

5. Survey of Research Activities

   Project methodology: surveys, focus groups, one-on-one interviews

   Subject Sample: K-12 education reform

   Grade Level Addressed by Project: 4-12

   Measures and Instrumentation for Subjects: survey instruments, interview protocol

   Data Collection procedures: survey questionnaires, archival records, interviews

   Working papers not yet presented: Impact of Computer Networking on Parental Involvement (graduate dissertation).

   Keywords: education reform, constructivism, Internet, K-12, Internet