The research experiment used a 2 (ALIVE vs. Classroom) X 3 (NYC vs. Chicago vs. Bloomington) X 3 (Wind-driven, Lightweight construction, Fatigue / Sleep Management) mixed factorial design, with both within and between subject variables:
Between subject variables:
1. Training. We had two conditions of training. Subjects were randomly assigned to either a traditional Classroom training condition, with a live teacher present, or to the ALIVE training condition, in which they worked through ALIVE modules on individual computers.
The ALIVE modules and the Classroom course-plans for each of the three topics were created from the same subject outline and materials, covered the same range and depth of material, and were reviewed to assure equivalence of content. In each case, the material was designed to be completed in approximately one hour, although there was some variation according to material (the fatigue module was slightly shorter, the lightweight construction module was somewhat longer, and since the ALIVE training was self-paced, individual times varied considerably).
In order to assure that the classroom condition represented a high quality of training, each of the three fire departments was asked to select their best and most experienced teachers. Three teachers for each site were chosen, and all teachers received the same PowerPoint file to use for presentation of material to the class. The PowerPoint presentation was developed by topic experts, and these same experts provided training for the instructors. Instructors were given a packet of materials to review, and were observed during practice training sessions. In all cases, trainers appeared to be comfortable with the material and received high marks on teaching quality from experts and from their students.
2. Cities. The experiment was carried out in three sites: New York (FDNY); Chicago (CFD); and Bloomington, MN (BFD). We used on-site training facilities in each of the three designated cities. Because the BFD did not have access to enough firefighters to fill subject numbers required, firefighters from several surrounding town volunteer departments, with whom it normally worked on cooperative agreements, also participated. In each case approximately one week was spent on site collecting data during full day training sessions.
Within subject variable:
3. Training Modules. All subjects in the Classroom and ALIVE conditions were provided with training on three different modules:
a. Wind-driven high-rise fires: This module reviews the current level of understanding of wind-driven fires, particularly as they occur in high-rise buildings. This includes the history of their occurrence, their initiation and progression, physical principles underlying their development, and strategies and tactics for fighting these fires, including use of special equipment.
b. Fires in modern, lightweight residential construction: This module discusses modern lightweight residential construction, which, along with modern materials commonly found in homes, presents special risks and problems for firefighters, and requires special approaches for successful and safe firefighting.
c. Fatigue and sleep management for firefighters: As per the research conducted by Division of Sleep Medicine at Brigham and Women's Hospital - Harvard University, firefighters commonly experience issues with fatigue and sleep disorders, related to a significant degree to the nature of their shifts. This module describes the best current information about the medical understanding of these issues, approaches to recognizing sleep related pathologies, and reducing fatigue related problems.
The order in which the modules were presented during the course of the training day was counterbalanced each day to control for the possibility that order effects, such as fatigue, could influence outcomes.
Measures
The dependent variables of the experiment consisted of three different measures:
1. Knowledge tests. Two 15-item tests were created for each module by our subject experts that addressed substantive issues presented in the modules. These questions were pretested to assure clarity, equivalency, and range of scores. The availability of two equivalent tests allowed different questions to be used for pre- and post-tests.
2. Self-knowledge tests. A seven-point scale was developed that asked questions about perceived levels of self-knowledge of the subject matter. Firefighters were given this scale during pre-test and post-test.
3. Satisfaction test. Fifteen-item scales were also developed which asked firefighters to rate the quality of the training session.