Center for Advanced Educational Services Massachusetts Institute of Technology Center for Advanced Educational Services Search Home About People Groups CECI Research Services Programs Products

MIT > CAES > Research > PIVoT
The Physics Interactive Video Tutor
A Web-Based Supplement to a Core Science Course

Prof. Richard C. Larson and Laura Koller
Submitted to Syllabus 2000 Boston for inclusion in Conference Proceedings

    It's a late October night, around 1:00 a.m. You're a college freshman, and you're studying frantically for an Introductory Physics exam that's just a few hours away. You're stuck on the difference between angular velocity and angular frequency, and you know there's going to be an exam problem like the one you struggled with in last week's homework. Walter Lewin - PIVoTIf only you could ask the professor some questions, listen to a few minutes of his lecture from three weeks ago, or maybe get an on-line chat going with other students who are puzzling over the same difficult concepts. If you could try a few practice problems and get instant feedback on your answers, or explore a Java physics simulation, it might help you grasp the material.

    Lucky for you, on-line tutorial help is available 24 hours a day in the form of PIVoT, the Physics Interactive Video Tutor. PIVoT provides a unique opportunity for students to conduct "virtual office hours" with a renowned MIT physics professor using streaming digital video and the Internet. The PIVoT web site offers a comprehensive on-line multimedia learning environment to students enrolled in MIT’s introductory physics course in Classical Mechanics, one of the most challenging core courses required for graduation from MIT, and the one with the highest failure rate. This presentation will cover the development history of the PIVoT web site, results of a student evaluation, different models of using PIVoT in teaching, and lessons learned during two years of development.

Development History

    The PIVoT project began at MIT’s Center for Advanced Educational Services (CAES) in spring 1998. The project's key research hypothesis was that the web, streaming video and other multimedia content, a relational database, and various digital storage and delivery mechanisms could be combined to provide a web-based virtual mentoring environment, simulating "office hours" conversations between students and professors. If this was true, we expected students’ use of such web-based tutors to result in measurable improvements in their mastery of course content. We viewed this type of tool not as a replacement for professors and courses, but rather as a rich, individualized ancillary environment designed to augment and reinforce learning experiences.

    PIVOT's primary funding came from an anonymous donor. [1] It was planned as a two-year project which would derive content from course 8.01, Classical Mechanics. The proposal was based on archival videotaped "help sessions" given by acclaimed MIT Physics Professor Walter Lewin, which had been produced for broadcast over the campus cable network. The sessions were focused on explaining difficult concepts and homework problem solutions. To get the project started, help videos from two recent years were digitized. Prof. Lewin and a graduate student reviewed the tapes, identified short video segments, and specified keywords and topics for each segment. The keyword list and a curriculum-based topic tree were developed to provide the content framework for the web site. Prof. Lewin created several hours’ worth of new help videos, as well as updates to the archival sessions.

    Walter Lewin - PIVoTAfter PIVoT got underway, Prof. Lewin offered to teach course 8.01 in fall 1999 so that his 35 lectures could be videotaped, digitized, and added to the PIVoT content repository. Funding was allocated for the production, and Prof. Lewin spent the summer getting ready: each one-hour lecture, including up to a dozen live demonstrations, required 15 hours of preparation time. One of the largest lecture halls on campus was retrofitted to support high-quality three-camera video production. Throughout the semester, each lecture was digitized and posted to the PIVoT web site one week after live delivery — Prof. Lewin's attempt to encourage lecture attendance to remain at a reasonably high level.

    A textbook was selected by Prof. Lewin to include in PIVoT. Web rights were negotiated with the publisher, and the task of converting the 600-page book (for which no electronic text files were extant) began with OCR scanning, followed by a lengthy and arduous cleanup phase. The content team developed a list of Frequently Asked Questions (FAQs) with answers, as well as a bank of multiple choice questions, many of which provided hints if wrong answers were chosen. Custom tools were developed so that information about each piece of media could be logged into an Informix database. Video was "soft-segmented," with start and stop frames specified. Keywords and topics were entered for each segment, textbook passage, simulation, FAQ, and practice problem; many items were cross-referenced so that content queries would produce rich results.

The beta version of PIVoT was released to MIT students in Fall 1999, though a small number of features — most notably a search tool — were still under development. Testing and debugging proceeded, and bug fixes and minor improvements were deployed during the semester. A doctoral student developed a feature called the "Personal Tutor" as part of his thesis research. This feature would follow a student's trajectory through the PIVoT site, keep track of the keywords and topics being investigated, and suggest and provide links to other relevant PIVoT content. The final full-featured version of PIVoT was released to students in Fall 2000.

Near the beginning of this two-year development process, a group called the National Center for Accessible Media was seeking funding to study web accessibility issues for deaf and blind users. Because of the complex scientific content included in PIVoT, it had been identified as an excellent candidate for the object of this three-year research project. National Science Foundation funding was secured a year and a half into PIVoT, and the work, which is ongoing, will include captioning and audio description of the 50+ hours of streaming video, as well as a review of overall site design, with particular focus on visual and audio representations of equations and symbols. The final deliverable resulting from this research will be a recommendation for universal access design standards in the creation of science-based web sites.

Results of a Student Evaluation

During the fall 1999 semester, students’ PIVoT usage and experience was evaluated under the direction of Dr. Alberta Lipson, MIT’s Associate Dean for Educational Research. Methods included a student survey, a focus group, and the analysis of data collected in the PIVoT database. The purpose was to learn about web site functionality, student usage patterns, student attitudes toward PIVoT, and whether use of the web site had any impact on final physics grades.

Although PIVoT was an entirely supplementary study aid, of the 482 students who completed the course, 92% of this group logged on to PIVoT an average of 10 times each during the semester. Many students who used PIVoT reported it to benefit their understanding of the materials taught in 8.01. Some of the findings showed that students often discussed PIVoT among themselves, and when survey respondents were asked to comment on what others were saying about PIVoT, their remarks were quite revealing, indicating the multiple ways in which PIVoT can help student learning. These comments are particularly interesting since they come from students’ own experiences rather than from answers to structured survey questions.

Some of the more frequent problems reported by students included finding time in their schedules to look at PIVoT, particularly if they did not have computers in their dorms and had to go to public computers on campus; difficulty playing videos; and difficulty finding the content they wanted to access. The latter problem has since been addressed, as the final version of PIVoT includes a search tool. Video playback problems are of concern, as user logs showed that the videos were the most frequently accessed PIVoT feature. Video streaming performance is generally good on the MIT network, though it can slow down during high-traffic periods in the late afternoon and early evening. This will certainly improve as the campus network infrastructure gets upgraded over time. At present, both low- and high-bandwidth versions of the videos (80kbps and 300 kbps) are offered to help ameliorate this problem.

As for the original hypothesis about the influence a web site like PIVoT might have on grade outcomes: when all other variables are held constant, PIVoT usage, as measured by number of logons, had a positive influence on final physics grades. Though PIVoT was not the strongest predictor (performance in math class was the strongest), it was nonetheless significant in a multiple regression analysis. [2]

This fall, PIVoT has been deployed at Rensselaer Polytechnic Institute (RPI) through the Anderson Center for Innovation in Undergraduate Education. Dr. Karen Cummings, RPI's freshman physics course manager, is conducting a controlled experiment with PIVoT; half of the approximately 500 enrolled students are using PIVoT as one learning resource. At the end of the semester, PIVoT and non-PIVoT user experiences and learning outcomes will be compared in a formal evaluation. RPI will be a primary source of data for a final comprehensive evaluation of PIVoT. In addition, approximately 60 freshman physics students at Wellesley College are using PIVoT this fall, providing another data source for the evaluation.

How PIVoT Relates to the Traditional Lecture Course

Instructors offering PIVoT to their students have a variety of student usage models they can employ.

The first is as an ancillary learning environment. This mode of usage represents the project's original design goal. Students are told that they have PIVoT available 24/7 on their desktop computers as a supportive tutoring environment, but instructors make no homework or other direct assignments to any PIVoT content. The student is free to use or to not use PIVoT in this mode. If she does use PIVoT, she must learn to navigate the web site in a way that is similar to navigating a library. The content is simply too voluminous to be experienced entirely by any one student, so she must learn how to pick and choose. Three different navigation tools built into PIVoT assist this process. The results are nonlinear tours through the PIVoT content, each student in effect selecting his/her own path. This is how PIVoT is being used at MIT during the fall 2000 semester. Prof. Lewin, while teaching the course during the fall 1999 semester, modified this idea slightly by giving hints as to which parts of PIVoT might be helpful in various homework assignments.

The second usage is tight coupling to the course. In this mode, instructors include specific PIVoT viewing assignments directly into regular "problem sets." They might say, for instance, "Under ‘angular velocity,’ look at the video ‘Angular Frequency and Velocity,’ then do problems 1 and 2 in your homework." In effect, they use PIVoT to teach or reinforce new concepts or problem formulations. In this mode PIVoT becomes an integral required part of the course. This is the choice of instructors at Wellesley and RPI during the fall 2000 semester.

A third usage is as a stand-alone physics course. Since the repository of PIVoT content includes all 35 lectures of the regular course, numerous video help sessions, a textbook, FAQs, animated simulations and illustrative test questions, in theory students could learn Newtonian mechanics directly from PIVoT alone. We have no experience with this usage mode, and we believe that interacting with live instructors is an invaluable part of the learning process. In the future, if we decide to license PIVoT to other universities, there may be attempts at this mode of learning from PIVoT.

If these and perhaps other modes of incorporating video tutors into teaching and learning are in fact tried in different settings, an interesting and potentially very useful evaluation/assessment would focus on which mode in which setting is most effective for students.

Lessons Learned

At the end of the project, the PIVoT team had a number of recommendations for the next project, most pertaining to optimizing the development cycle by making efficient and well thought-out design decisions early on:

  • Plan and design evaluation activities at the beginning of the project, and integrate them throughout the development process. Data mining tools that are developed along with the product itself will produce data more easily and efficiently and will simplify the work of the evaluation team.

  • Familiarize your team with web accessibility guidelines (for deaf and blind users) from the beginning of the design cycle, attempting to incorporate as many features as possible into your site. Trying to retrofit these features after the fact may result in some serious design compromises. For excellent resource information, see the W3C Web Accessibility Initiative site (URL below).

  • Choose printed materials for your web project carefully. Conversion from print to web can be time-consuming and expensive. Most OCR scanning software results in only 98% accuracy, so allow adequate time for cleanup, formatting, and proofreading of print materials that have been converted with this technique.

  • Beware of getting too attached to a demonstration version of your web site. If it is created early in the project, it may get rushed to production before the design and architecture have been given full consideration. People may grow accustomed to the demo, resulting in discomfort and resistance when needed changes are implemented.

For more information

Footnotes

[1] Additional support was received from the Lord Foundation, the Lounsbery Foundation, IBM, the Stratford Foundation, SeaChange International, and the National Science Foundation through a collaboration on web-accessible media with the WGBH Education Foundation of Boston.

[2] Source: "An Examination of Student Usage Patterns and Attitudes Toward PIVoT: A Pilot Study," Dr. Alberta Lipson, MIT Office of Academic Services, June 24, 2000 (internal document).

(10/25/2000)


Home | About | People | Groups | CECI Research | Services | Programs | Products

-
Copyright © 2000 MITCopyrightMIT homepageTop of page