PhD

HUMAN-HUMAN MULTI-THREADED SPOKEN DIALOGS IN THE PRESENCE OF DRIVING
by
Alexander Shyrokov
under supervision of Dr. Andrew Kun
University of New Hampshire, May 2010

Abstract

The problem addressed in this research is that engineers looking for interface designs do not have enough data about the interaction between multi-threaded dialogs and manual-visual tasks. Our goal was to investigate this interaction. We proposed to analyze how humans handle multi-threaded dialogs while engaged in a manual-visual task. More specifically, we looked at the interaction between performance on two spoken tasks and driving. The novelty of this dissertation is in its focus on the intersection between a manual-visual task and a multi-threaded speech communication between two humans.

We proposed an experiment setup that is suitable for investigating multi-threaded spoken dialogs while subjects are involved in a manual-visual task. In our experiments one participant drove a simulated vehicle while talking with another participant located in a different room. The participants communicated using headphones and microphones. Both participants performed an ongoing task, which was interrupted by an interrupting task. Both tasks, the ongoing task and the interrupting task, were done using speech. We collected corpora of annotated data from our experiments and analyzed the data to verify the suitability of the proposed experiment setup. We found that driving and our spoken tasks influenced each other. We also found that the timing of interruption influenced the spoken tasks. The data indicate that the ongoing task was more influenced by driving than the interrupting task. On the other hand, the interrupting task influenced driving more than the ongoing task. This suggests that the multiple resource model does not capture the complexity of the interactions between the manual-visual and spoken tasks. We proposed that the perceived urgency or the perceived task difficulty plays a role in how the tasks influence each other.


Slides can be found on Slide Share. Text of the dissertation. List of publications.

Video shown during the defense:
Special thanks to Dr. Peter Heeman for his guidance throughout the research.

In the dissertation and during the defense I did not emphasize amount of engineering and software design that was done during this research. The screenshot below shows my computer screen during the experiments.

I had to control multiple different applications (Driving simulator control, Eye Tracker, Experiment Scripts, etc.) on four different computers (dispatcher computer, driving simulator, etc.), as well as to collect data from these computers. To address this situation I designed Experiment Wizard (a cross platform open source automation tool for scientific experiments). This allowed me to delegate the control over the applications to automatic scripts, while I had only to provide minimum of the input. Experiment Wizard prompted me every time my intervention was required and offered me instructions on what to do. This minimized human error on my part during the experiments. At the same time, Experiment Wizard ensured that all the experiments are done according to the defined rules. And finally, now it is possible to reproduce the experiment exactly as I have done it using the configuration script for Experiment Wizard.