Preliminary Evaluation of the Blacksburg Electronic Village

Executive Summary

Scott J. Patterson, Ph.D. Virginia Polytechnic Institute

Ann Bishop University of Illinois

Andrea Kavanaugh, Ph.D. Virginia Polytechnic Institute

The purpose of the research reported here was to perform a pilot evaluation of BEV technologies and services. Specific objectives were to:

• Establish evaluation methods and techniques which will help the developers of the electronic village make the system responsive to the needs of the people using it.
• Determine which information services have the greatest value to users.
• Evaluate the impact of new electronic information services on libraries and individuals.
• Assess the user interface and other technical aspects of the service offered by the BEV project.

The sampling frame utilized in this study consisted of the the 183 persons identified as having received the beta-version of the BEV user interface. These people were members of the university and/or local community who had expressed interest to the BEV design team in participating in the development of the BEV system. Three data collection techniques were utilized in this study: Self-report background questionnaires, a self-report survey of experiences with the Blacksburg Electronic Village, and focus group interviews with BEV users.

Analysis of the data focused on four key topic areas: Motivations for using BEV (why did people join); present and future uses of BEV; problems and sources of user support; and perceived impacts of BEV. Key findings in each of these four areas are:

Motivations

• To gain network access
• To increase ease of use of network resources
• To facilitate and improve work activities
• To enhance educational opportunities
• For entertainment or to pass time
• To improve the quality of life in Blacksburg

Present Uses

• Gather information
• Communicate with other people

Future Uses

• Develop and access local information resources
• Develop and access local information resources
• Develop and utilize commercial and business services
• Incorporate BEV into the educational structures of the community

Problems, Improvements and Support

• Poor user interface
• Difficulty understanding the Internet
• Too much jargon or specialized knowledge required
• Too much red-tape in registering for BEV
• Too few services offered
• Concern over relationship between town and university
• Lack of a clear project leader
• Inadequate user training and support
• Users most often turned to friends for help, followed by BEV staff
• The most effective support was available from BEV staff

Positive Impacts

• Increased connections to local community
• Increased access to individuals and resources across the planet
• Potential to attract new businesses to Blacksburg

Negative Impacts

• Creating a class society of information rich and poor
• Decreased human interaction within the community

Recommendations

Based on these patterns of findings, the researchers offer the following recommendations for both the Blacksburg Electronic Village as well as future evaluation efforts: For the Blacksburg Electronic Village:

• Recognize that libraries do and will increasingly play a vital role introducing users to the BEV system. Substantial time and resources should be devoted to assisting libraries in training activities and in providing public access to BEV's electronic public spaces.
• Facilitate communication between BEV project staff, potential system users, and other partners, such as libraries.
• Increase resources devoted to user support and marketing.
• Establish a clear leadership structure within the Blacksburg Electronic Village project.
• Accelerate, where possible, the number of BEV users.
• Accelerate, where possible, increases in the quantity, quality, and variety of services offered.

For future evaluation efforts:

• Future evaluation efforts should include face-to-face discussions. --These should be conducted at times and locations more amenable to participants.
• Questionnaires should be expanded to include items related to library use, social/psychological measures, and those that could be used to assess user outcomes.
• Evaluation activities should include greater coordination between BEV staff and researchers.
• Increase participation of researchers from disciplines beyond communication and library science in future evaluation activities.
• The general goal of future evaluation activities should be to help make the Blacksburg Electronic Village responsive to the people who use it.

Also inluded in this study was the identification and compilation of background information relevant to subsequent BEV evauations. A clippings file of stories on BEV that appeard in the local media was created. In addition, existing sources of data on local/regional library use were identified. Finally, the report outlines a specific course of action for assessing the impact of the Blacksburg Electronic Village on the community and university library system. This evaluation plan was developed in conjunction with representatives of the Montgomery-Floyd Regional Libraries and the Virginia Tech University Library.