Exploring the Needs of Informal Learners of Computational Skills: Probe-Based Elicitation for the Design of Self-Monitoring Interventions

Rimika Chaudhury, Taha Liaqat, Parmit Chilana

Keywords: Self-monitoring interventions, informal learning, visualization

Abstract: Informal learners of computational skills often find it difficult to self-direct their learning pursuits, which may be spread across different mediums and study sessions. Inspired by self-monitoring interventions from domains such as health and productivity, we investigate key requirements for helping informal learners better self-reflect on their learning experiences. We carried out two elicitation studies with paper-based and interactive probes to explore a range of manual, automatic, and semi-automatic design approaches for capturing and presenting a learner's data. We found that although automatically generated visual overviews of learning histories are initially promising for increasing awareness, learners prefer having controls to manipulate overviews through personally relevant filtering options to better reflect on their past, plan for future sessions, and communicate with others for feedback. Our findings have several implications for designing learner-centered self-monitoring interventions that can be both useful and engaging for informal learners.

Track: HCI/visualization

Summary Of Changes:

We thank the reviewers for the positive reviews and for providing helpful feedback. We have carefully considered and addressed the comments from all the reviewers. Below, we present the summary of changes, and offer some clarifications.

[AC, R1] We wanted to clarify that in our studies, particularly in Study 2 where the participants were shown interactive probes, they were asked to respond to a task prompt “interact with each probe and … use the visual overviews to reflect on past learning sessions, and plan ahead for future sessions” (Section 4.5). The goal was to gauge learners’ needs, perceptions and behaviors, through and with respect to each design, rather than asking them directly about their wants.
The above approach is guided by learner-centered design (Soloway [63]) which builds on the principles of user-centered design (UCD), but distinguishes itself from UCD by advocating for understanding learners’ perceptions and contexts. We have added this clarification and reference in Section 3. Additionally, in section 5.2.4, we discuss the limitation of our approach, and the possibilities of understanding the learners’ needs in a more realistic way through field studies in the future. We also included additional references from learning theories of self-explanations (Chi [12]).

[R2] To clarify the rationale behind the summary designs of the interactive probes in study 2, we have re-organized the text to separate out the motivation from the description of each design. The motivation statements tie back to the findings in study 1 and connects with the design aspect we wished to probe more into (section 4.1, 4.2 and 4.3). We wanted to offer a clarification that we continued on with 12 new participants instead of repeating, as we wanted to avoid learning effects since the designs in both studies shared commonalities. We added a clarification in section 4.5 Study Procedure.

[R3] had suggested checking for succinctness in our discussion section. We have re-organized the text (in 5.2. Key requirements: 5.2.1 and 5.2.2) to more clearly convey specific design requirements and updated the titles to better reflect the separate categories. We have also updated our paper title to better reflect our elicitation technique and the narrow technical scope of the informal learning context we were studying.

In addition to these changes, we have also fixed the typographical errors and figure references.