Bits and Pieces prototype aims to augment the processes of collecting, remembering, sensemaking and documentation of experiences at work to support informal learning when under time pressure. Time pressures currently mean that workers do not find it easy to take note of learning experiences that happen at work nor to then review and reflect on these later in a way that leads to improvements in working practice. That is, in busy working times informal learning experiences need to be quickly captured by collecting traces of bits and pieces, whereas they need to be remembered and made sense out of during more flexible working times, when time permits it again. The Bits and Pieces tool could be used for example by a general practitioner (GP) who has been given the task of reviewing a new medication and considering whether the GP Practice should move patients onto this new drug. As part of their research they will over time gather many different informal experiences traced in bits of information (e.g., links to reports on the new medication, information sheets, notes from discussions with colleagues, notes from their review of the practice’s experience with the current medication). Bits and Pieces will support them in pulling together all of relevant learning experiences captured in bits of information, reviewing and making sense out of them in a way that helps to inform their recommendation to the GP Practice. The sensemaking outcome can then be used in many ways – to guide their decision-making, to share with colleagues or to export so that it can be included as evidence of their informal learning as part of their appraisal or revalidation process.
The Bits and Pieces tool currently works with Evernote as a collection service to allow users to efficiently and easily take notes about important experiences at work (by making notes, forwarding emails, adding a photo), but will extend on other services in the future to adapt to the individual preferences. Building upon the collected traces, the tool then provides the workers with a visual interface to support their retrieval (remembering), reviewing and making sense out of the corresponding informal learning experiences (see figure 1). From a theoretical point of view this tackles the problem of remembering the informal learning experiences via traces from the episodic memory and making sense out of them via an interactive user interface to visually group them. This endeavor transforms the experiences from the episodic into the semantic memory using the process of mental categorization.
Figure 1. B&P interface with the timeline view (upper pane) and the organizing view (lower pane) including an sensemaking example about dementia enhancement services (DES) and a bit’s information pop up
As figure 1 shows, Bits and Pieces has two viewing panes: the top is the Timeline view in which the user can view all the experiences traced as bits of information they have collected. Currently this is shown as a timeline view, but other views onto the material (tag clouds, maps) are also under development. After users have remembered the informal learning experiences via time-based cues, they can pull the corresponding bits of information from this top pane down into the lower pane (the Organise view) creating a meaningful subset, which can be organized in a way that makes sense to them. Currently, the Organise pane allows users to interactively group the bits. The visual support given to organising is through using circles to visually group material, but other organisational views are also being considered (mapping and hierarchies). To access a video that explain the above described features interactively, please follow the link:
The Bits and Pieces prototype was evaluated in a formative evaluation that formed part of the ongoing co-design research approach (Dennerlein et al., 2014). It involved four professionals from two GP practices such as a diabetes specialist nurse and a doctor. The evaluation was split into two parts, a collection phase to enable the gathering of informal learning experiences about a realistic learning need (e.g. discover methods for reversing diabetes) and a supervised sensemaking phase making use of Bits and Pieces. While the usage of the prototype, the participants task was to first remember their informal learning experiences based on the collected bits with the help of the time-based cues in the timeline and afterwards structure them semantically with the help of the circles in the organizing view. In general, all four participants were positive about the tool and understood and appreciated its underlying idea and purpose. With respect to the timeline, the results indicate that it is able to prompt remembering informal learning experiences, but that there is the need of extra cues to increase the chance to find and remember an informal learning experience, such as information on the involved persons or location. Additionally, the findings point towards the existence of a re-evaluation phase of the collected bits: i.e. a person examines the bits related to informal learning experiences more deeply and assesses them again in the more flexible working time than it was possible during the more busy working time. In the same lines the healthcare professionals requested to directly manipulate e.g. tags of bits and attach comments. With respect to the organizing view, the interactive nature of sensemaking via the organizing and categorizing of bits into circles was valued highly by the participants. One participant was especially successful in her sensemaking as she achieved to meaningfully group her collected informal learning experiences on diabetes treatment according to their influence on blood pressure: i.e. blood sugar levels go down though exercise as opposed to diet. This lead to the creation of a categorization of respective informal learning experiences in the circles ‘Results from diet’ and ‘Results from exercise’ and their overlap representing results from both. It is likely that the involved representations would have been enriched and matured having had more time for sensemaking in the study. In conclusion, we found first support for the effectiveness of the Bits and Pieces interface for supporting retrieval and foraging traces for informal learning experiences via a timeline and sensemaking via an organizing view.
During the first year a decent amount of data was collected in the workshops with end-users. This data become a prompt to implement the second round of Bits and Pieces design. To do that we conducted series of activities:
- Existing transcript of all co-design and evaluation activities was analyzed by five researchers with a goal to elicit user needs. The issues identified have been categorized as
- Conceptual: issue concerns the concept or idea of the prototype itself;
- Usability: issue concerns the usability of something;
- Feature: issue concerns a feature nice to have;
- Bug: issue concerns a bug.
- Features were translated into the new user requirements. All together there were 78 user requirements identified.
- New mock-ups for user interface were designed. Application was rethought and rearranged. To find room for all new user controls new 6 toolbars were introduced.
- Considering that the application should be easy to use and easy to learn, it was decided to reduce amount of features to have the essential ones only. To achieve that three-phase voting process was designed:
- An voting workshop was conducted with end-users in Leeds. Several specific scenarios were proposed to them for implementation and limited amount of chips were given to end-users. They had to place these chips on a paper prototype that represented all new controls. In such way the most important features were observed and fixed.
- At the same time voting process among researchers was conducted. The researchers had to vote for these requirements that were important from the research perspective. This phase was implemented online by using Requirement Baazar service.
Both voting results were collected and represented in mock-ups (See Fig. 2). Then group of designers conducted judgement process, where the most voted requirements from both side were included into the final design. By reducing an amount of controls an amount of toolbars also was reduced to 4.
Fig. 2. Part of the Mock-up screen with voting results
The final version of mockups with explanations is attached below. On the base of this version developers already started the second round of development. A milestone for this development round is August 14, 2014, when we plan new evaluation workshop with end-users in Leeds.
Dennerlein, S., Rella, M, Tomberg, V. Theiler, D., Treasure-Jones, T., Kerr, M., Ley, T., Al-Smadi, M. & Trattner, C. (2014). Making Sense of Bits and Pieces: A Sensemaking Tool for Informal Workplace Learning. In: Proceedings of EC-TEL 2014. In press.