Joint attention in dyads: Multimodal method for capturing the nuances of productive interaction during online collaborative problem solving

Due to the increasing complexity of everyday life, collaborative problem solving (CPS) as a discipline-free skill, is frequently mentioned as one of the key-competencies for 21st-century learners to be mastered.This paper presents a multimodal method how to better understand CPS and particularly, the establishment and the role of joint attention during CPS in dyadic interaction in remote setting. In CPS, the acquisition of joint attention is seen critical since it is seen to form the foundation of interaction predicting productive collaboration. However, in spite of increasing interest in joint attention, no unified definition exists for what is considered as an adequate account of joint attention. It is argued here that to merely focus on gaze following in dyads is not enough; it is only through communication and sharing of attention what makes joint attention joint instead of parallel. How we may, then, better understand and examine gaze as “social” and how a participant’s gaze operates in social interaction? In the current paper, CPS processes in dyads are examined in a dual-space online assessment environment (ATC21S, []). The environment comprises of a game-like visual space for overt actions (i.e. moving objects), combined with a free-form chat interface. This remote, human-to-human interaction context is challenging as it lacks a fully predefined structure for joint activity. Accordingly, in productive CPS, it is expected that a well-performing dyad would organize their efforts in serial sequences of chat and actions, comprising of initiating and responsive activities. To capture these sequences and their actual meaning, this explorative study applied a multimodal method that amalgamated synchronized gaze data evidence in dyads with log stream data (i.e. chat and actions) from the environment. Next, these data were combined with cued retrospective reporting (CRR) interviews, cued with the gaze data recorded during CPS and, were qualitatively analyzed for correspondence. In line with previous research in face-to-face setting, also the “when” question (the timing question of gaze) was a critical indicator here if compared to “where” question of gaze in social interaction. That is, the “correct” timing of gaze was essential for understanding and joint attention to occur. Also, as interactions are not continuously in lock step, how difficulties arouse in social interaction and how participants were orientated towards repairing the break-downs, were essential in understanding productive CPS. In the presentation, the analysis on CPS processes, based on the immediate measures of interaction, their orderings and relations are discussed.