Appendix C-1
Canada Data Quality Control Tables

Data Cleaning

This section documents further data cleaning on AP+ ® acquired data with recorded speeds of at least 30 mph. Data from Canada Study Phase 1 1 is used to illustrate the processes involved in data quality control.

Excluded drivers

The following lists the identification numbers of 7 Canadian drivers whose data were excluded from the cleaned analysis sample used for technology assessment because of insufficient data under at least one of the two experimental conditions (FEEDBACK and NO FEEDBACK).

Driver 3 (no AP+ ® data under the NO FEEDBACK condition-presumed lost

• due to technical problems in recording data on truck)

Driver 9 (no AP+ ®data under the FEEDBACK condition-presumed lost

• due to technical problems in recording data on truck)
• Driver 14 (no data under FEEDBACK condition-dropped out of study for a personal matter; re-enrolled later as Driver 28.)
• Driver 16 (insufficient data due to noncompliance to protocol)
• Driver 20 (no AP+ ®data under the FEEDBACK condition-presumed lost
• due to technical problems in recording data on truck)

Driver 24 (too little AP+ ® data to analyze-presumed lost due to technical

• problems in recording data on truck)

Driver 27 (too little AP+ ® data to analyze-presumed lost due to technical

• problems in recording data on truck)

Driver 28 (no AP+ ® data under the NO FEEDBACK condition-presumed

• lost due to technical problems in recording data on truck)

This resulted in analyzable AP+ ®black box data for n = 20 Canadian drivers (i.e., 71% complete data acquisition rate relative to the n = 28 drivers empanelled). These were used in technology assessment analyses and hypothesis testing. For data acquisition procedures other than AP+, data on more drivers was often available. For example, completed post-experimental debriefing surveys (i.e., Human Factors Questionnaire, see Appendix F) were obtained on all Canadian drivers who completed both segments of the study, regardless of whether electronic data from the AP+ ®black box were available. Consequently, the sample sizes for different dependent variables within and between measurement systems can vary considerably as to the number of drivers on which data are available.

Decisions made by the research team while meeting on October 11, 2002

During a research team meeting at ATRI in Alexandria, VA in the fall of 2002, the Biostatistics-Analysis Core (directed by G. Maislin) distributed documentation of additional problematic. These were discussed in depth and strategic data handling decisions were made as described below.

Records with durations greater than 30 seconds

The AP+ ®device stored data electronically in 1 second or longer bins. Bins longer than 1 second occurred when there were no changes in parameters being stored by the AP+ ®. In the "no records excluded" Canadian study phase sample, more than 80% of AP+ ®records (82% from the NO FEEDBACK condition and 81% from the FEEDBACK condition) summarized truck and driver status over 1 second. Approximately 12% summarized truck and driver status for 2 seconds. The remaining records (approximately 6%) summarized truck and driver status for 3 or more seconds. The distributions of record duration categories by driver and feedback condition from the Canada Study Phase 1 data are summarized in Data Quality Table 2 ( Appendix C-1). Data Quality Table 3 ( Appendix C-1) repeats this summary for the sample defined as "at least 30 mph" (n = 20 Canadian drivers used for data analyses on effects of FMT feedback). Further investigation identified extraordinarily long duration records that appeared to result from the AP+ ®clock registering elapsed time while the truck was not in use. Therefore, when attention was restricted to records in which speed was at least 30 mph, these records were excluded with some few exceptions. These exceptions were investigated for data validity. Data Quality Table 4 ( Appendix C-1) for the Canada Study Phase 1 compares the total duration of records for each driver under both conditions for the "no excluded records" samples and for the sample restricted to records with speeds of at least 30 mph. The main difference in total durations of records is exclusion of records in which the truck was standing still while other records were excluded because the truck was traveling at less than 30 mph.

All AP+ records in the ³ 30 mph sample with durations longer than 20 seconds were examined. A decision was made to exclude records with durations greater than 30 seconds, because it was deemed unlikely that none of the multiple AP+ measured truck and driver parameters would have failed to change for even a second over a period of 30 seconds or longer when the truck was operating at speeds of at least 30 mph. Excluded records amounted to 9 out of 4.7 million records (i.e., only 0.0000019%). Data Quality Tables 5 and 6 ( Appendix C-1) provide driver specific distributions of record durations under the NO FEEDBACK and FEEDBACK conditions, respectively. The same procedures were applied to data acquired in the USA drivers in study phase 2 (Appendix C-2).

Lane tracking confidence < 50%

Data Quality Table 7 ( Appendix C-1) provides the SafeTRAC ®Lane Tracking (Offset) Confidence Category (0% vs. > 0 to < 100% vs. 100%) in the "no records excluded" sample. As is evident in Data Quality Table 7 (Appendix C-1), a number of drivers had lane confidence values equal to 0%, unreliable values at certain times. After discussion with AP+ ®and SafeTRAC ®representatives, lane-tracking offset values were deemed insufficiently reliable if lane tracking confidence was less than 50%. Therefore, lane tracking offsets were set to missing for all records with lane tracking confidence less than 50%. A total of 3.8% and 2.1% of records in the ³ 30 mph sample had confidence values below 50% from SafeTRAC ®. Lateral distance, a primary outcome measure, is a function of lane tracking and so this was set to missing also for these records. Data Quality Table 8 ( Appendix C-1) provides the same information for the cleaned analysis sample. Data Quality Tables 9 and 10 ( Appendix C-1) provide greater detail with regard to the distributions of Lane Tracking Confidence values. The same procedures were applied to data acquired in the USA drivers in study phase 2 (Appendix C-2).

Driver 13 SafeTRAC ® variables

Further examination of AP+ ®records for Driver 13 indicated technological failure. Therefore, all SafeTRAC ® variables were set to missing for this driver (i.e., lane track confidence, lane track offset, and lateral distance including driver's alertness, ambient light, light/dark, indicator, event category). Also, this driver was eliminated CoPilot ® (for PERCLOS) with daylight = 0 since daylight information came from SafeTRAC ®, which was deemed nonfunctional on this Driver's truck.

Vehicle offset and lateral distance

Data Quality Table 11 for the Canada Study Phase 1 ( Appendix C-1) provides summary statistics after all exclusions contained in the cleaned analysis sample for the values of the SafeTRAC ® vehicle offset indicator as recorded by the AP+ ®system (Byte 6 of SafeTRAC ®). These exclusions include the setting of values with offset confidence < 50% to missing.

According to AssistWare Technology, the makers of SafeTRAC ®, the final five values in the 0 - 255 range of the lateral offset data byte are never used, so the translated range goes from -250 to +250cm in 2-cm increments, with a data byte value of 125 corresponding to a 0-cm lateral offset. Therefore, numbers lower than 125 indicate the vehicle is left of center while numbers higher than 125 indicate the vehicle is right of center. Specifically, a value of 0 for this byte in the output message indicates a SafeTRAC ® assessment that the vehicle's current lateral offset to be -250 cm, or 2.5 meters left of lane center. A value of 1 for this byte in the output message corresponds to a lateral offset of -248 cm, etc. In order to provide a more interpretable value, lateral distance was computed from the vehicle-offset values as:

Lateral Distance = (2*Vehicle Offset-250)

Data Quality Table 12 for the Canada Study Phase 1 (Appendix C-1) provides summary statistics for the computed lateral distance values analogous to the summary in Data Quality Table 11 provided for vehicle offset. Data Quality Tables 13 and 14 ( Appendix C-1) provide details regarding the distributions of lateral distances in the NO FEEDBACK and FEEDBACK conditions, respectively. In these tables, the distributions are summarized by providing the minimum and maximum values as well as the 10 th, 20 th, 30 th, 40 th, 50 th (median), 60 th, 70 th, 80 th, and 90 th percentiles.

SafeTRAC ® Driver Alertness

"Driver alertness" is an output provided by the SafeTRAC ®based on a proprietary algorithm using lane tracking input. The output ranges from 0 to 99. When the system is not measuring "alertness," the score is equal to 100. Data Quality Table 15 for the Canada Study Phase 1 ( Appendix C-1; comparable data for the USA study phase is in the Table 15 of Appendix C-2) provides summary statistics for SafeTRAC ®alertness for each driver under both conditions using the cleaned analysis. Data Quality Tables 16 and 17 ( Appendix C-1) provide details of the distributions for each driver in the NO FEEDBACK and FEEDBACK conditions, respectively. Examination of Driver 11's NO FEEDBACK distributions of SafeTRAC ® Driver's Alertness and lateral distance (computed from vehicle offset) revealed extreme values. Therefore, this Driver's alertness and vehicle offset (and lateral distance) were set to missing under both conditions for the cleaned analysis sample. Data Quality Table 34 ( Appendix C-1) provides the distributions of SafeTRAC events for each driver under each condition.

Steering Wheel and Wheel Movements

Examination of the AP+ ®variables "steering wheel movements" and "wheel movements" suggested that these values were reversed in some Canadian trucks due to sensor reversal at installation, which was a function of truck manufacturer type. ("Steering wheel movements" were supposed to be relatively lower values compared to "wheel movements," which were relatively larger values.) Initial programming was implemented to fix wheel movement/steering wheel movement data problems assuming the problem was truck specific. Taking into account information on which truck each driver drove, the values of wheel movements and steering wheel movements were switched for Drivers 7, 9, 13, 17, 22, 22, and 28.

We expected that within truck, there would be consistency if the cause was a mechanical switching of sensors. This proved to be the case, although there remained a few instances of inconsistencies. For example, Drivers 1, 2, and 4 had values in the range expected for steering wheel movements in both channels. Driver 11 had values that appeared to flip channels going from the NO FEEDBACK and FEEDBACK condition. That is, Driver 11 had wheel movement and steering wheel movement data switched only for FEEDBACK condition. This was the only driver with a reversal in one condition only. This is impossible under the assumption that Driver 11 used the same equipment during both conditions. Thus, wheel movement data for Drivers 1, 2, 4, 6, and 11 could not be reconciled with expected values and consequently "wheel movement" and "steering wheel movement" data for these drivers were set to missing.

Data Quality Table 18 for the Canada Study Phase 1 ( Appendix C-1) provides summary statistics for AP+ ®recorded steering wheel movement for each driver under both conditions using the cleaned analysis sample. Data Quality Tables 19 and 20 ( Appendix C-1) provide details of the cleaned analysis sample steering wheel distributions for each driver in the NO FEEDBACK and FEEDBACK conditions, respectively. Data Quality Table 21 ( Appendix C-1) provides summary statistics for AP+ ®recorded wheel movement for each driver under both conditions using the cleaned analysis sample. Data Quality Tables 22 and 23 ( Appendix C-1) provide details of the cleaned analysis sample wheel distributions for each driver in the NO FEEDBACK and FEEDBACK conditions, respectively. (The comparable tables numbered for the USA study phase can be found in Appendix C-2.)

CoPilot ® measures of PERCLOS

All CoPilot ® measures of PERCLOS values recorded when the AP+ ® daylight sensor indicated the presence of daylight (value of 1) were set to missing in the cleaned analysis sample, since the infrared-based CoPilot ® was considered accurate primarily at night. Data Quality Tables 28 and 29 for the Canada Study Phase 1 ( Appendix C-1) provide the number of records in the no exclusions sample and the cleaned analysis sample, respectively, in which the daylight indicator was equal to 0 (no daylight) or 1 (daylight). In addition, Data Quality Table 24 provides a summary of the ambient light values for each driver under both conditions.

Data Quality Table 25 for the Canada Study Phase 1 ( Appendix C-1) provides summary statistics for PERCLOS values recorded during no daylight records in the cleaned analysis sample. Data Quality Tables 26 and 27 ( Appendix C-1) provide details of the cleaned analysis sample PERCLOS distributions for each driver in the NO FEEDBACK and FEEDBACK conditions, respectively. (Appendix C-2 has comparable data for the USA study phase.)

Vehicle speed, engine rotation, longitudinal and lateral acceleration

Data Quality Tables 30 to 33 for the Canada Study Phase 1 ( Appendix C-1) provide summary statistics for vehicle speed, engine rotation, longitudinal ("X") and lateral ("Y") acceleration respectively. The ranges of the longitudinal and lateral acceleration are both from -1.28g to +1.28g.

DQ Table 1: No Records Excluded Speed Categories (mph)

DQ Table 2: No Records Excluded Record Duration Category (seconds)

DQ Table 3: Cleaned Analysis Sample Record Duration Category (seconds)

Data Quality Table 4: No Exclusions and >=30 mph Sum of Record Durations (hours)

DQ Table 5: Cleaned Analysis Sample NF Record Duration Distributions (seconds)

DQ Table 6: Cleaned Analysis Sample FB Record Duration Distributions (seconds)

DQ Table 7: No Records Excluded SafeTRAC Lane Tracking (Offset) Confidence Category

DQ Table 8: Cleaned Analysis Sample SafeTRAC Lane Tracking (Offset) Confidence Category

DQ Table 9: Cleaned Analysis Sample NF SafeTRAC Lane Tracking (Offset) Confidence Category

DQ Table 10: Cleaned Analysis Sample FB SafeTRAC Lane Tracking (Offset) Confidence Category

Data Quality Table 11: Cleaned Analysis Sample SafeTRAC Vehicle Offset (unweighted by record duration)

Data Quality Table 12: Cleaned Analysis Sample Lateral Distance (2*Vehicle Offset-250) (unweighted by record duration)

Data Quality Table 13: Clean Analysis Sample NF Percentiles of Lateral Distance (2*Vehicle Offset-250)

Data Quality Table 14: Clean Analysis Sample FB Percentiles of Lateral Distance (2*Vehicle Offset-250)"

Data Quality Table 15: Cleaned Analysis Sample SafeTRAC Driver Alertness Summary (unweighted by record duration)

Data Quality Table 16: Cleaned Analysis Sample NF SafeTRAC Driver Alertness Distribution (unweighted by record duration)

Data Quality Table 17: Cleaned Analysis Sample FB SafeTRAC Driver Alertness Distribution (unweighted by record duration)

Data Quality Table 18: Cleaned Analysis Sample Steering Wheel Movements Summary (unweighted by record duration)

Data Quality Table 19: Cleaned Analysis Sample NF Steering Wheel Movements Distribution

Data Quality Table 20: Cleaned Analysis Sample FB Steering Wheel Movements Distribution

Data Quality Table 21: Cleaned Analysis Sample Wheel Movements Summary (unweighted by record duration)

Data Quality Table 22: Cleaned Analysis Sample NF Wheel Movements Distribution

Data Quality Table 23: Cleaned Analysis Sample FB Wheel Movements Distribution