Building capacity for The Carpentries Workbench lesson development framework through documentation and community feedback

Kari Jordan, Robert Davey

Funding Area: Capacity building / Construcción de capacidad

Problem Statement: The Carpentries organisation develops materials to train those who need data skills across a range of fundamental open-source technical curricula. The Carpentries relies on a small Core Team to lead strategy and development and a community of volunteers who help maintain the lessons. Previously, lesson sites were based on the Jekyll framework, often resulting in recurring dependency issues or increased technical debt across libraries. We developed a new modularised curriculum material framework to address these issues and transitioned our lessons into the new framework, called the Workbench. The summary of the challenges is as follows: The Workbench is catering to a large and varied community. Different people within the community will interact with the Workbench in different contexts from different viewpoints. Documentation on the Workbench needs development based on those contexts and viewpoints. For Workbench sustainability, the capacity for the community to maintain and develop the infrastructure needs to be expanded. The proposed work will build capacity in the community through the following outcomes: Documenting the benefits of the Workbench with reference materials would help the community take up the new framework Lessons are central to The Carpentries activities, and documentation of the lesson infrastructure will be used by members acting in most roles within the community. Tuning the documentation based on feedback from the community.

Proposed Activities: Through previous funding for the main Workbench development process and using the Diataxis Documentation Framework, we have generated initial documentation for the Workbench that broadly follows the four themes of Tutorial, HOWTO, Reference, and Explanation: Our HOWTO content is focused on “flight note” style guides aimed at fixing bugs, previewing changes, and validating content. We have generated Explanatory text about how the modules of the Workbench interoperate and why the infrastructure exists. Reference content contains knowledge about repository organisation, Markdown syntax, command reference, and API documentation. Tutorial development has focussed on guiding community members who want to use the infrastructure to develop lessons, but some also exist for maintenance and development of the Workbench itself. We have initial tutorials in setting up particular framework elements, but these tutorials need refactoring. Our documentation must be of the highest quality to be most useful to our various community groups. There are two areas where we propose to concentrate our efforts if successful: Documentation Improvement and Community Feedback. Documentation Improvement: (2023 12 01 - 2024 08 01) Lessons comprise two main views: Learner View and Instructor View. Across these views, we need to serve four main intersecting groups who strongly contribute to capacity: Contributors, Instructors, Maintainers, and Developers. Contributors predominantly develop the Learner View by contributing to existing lessons and forming new curricula. Instructors provide updates to the Instructor View by actively teaching lessons and gathering feedback from surveys and community discussions. Maintainers do not teach but need to know the Workbench package and repository structure to run builds. Finally, Developers of the Workbench infrastructure address bugs in the code and implement new features. We will address the needs for focused information, delivering comprehensive improvements across the Workbench documentation. Through iterative testing with the community, we will ensure the accessibility and effectiveness of these user guides. Community Feedback: (2024 05 01 - 2024 12 01) The existing curricula of Carpentries lessons have recently been transitioned into the Workbench, and maintainers and contributors are starting to work with the new tools and environment. We need to gather information and canvass feedback from this community, e.g., questions asked on The Carpentries Slack server, in dedicated Maintainer meetings, mistakes made in pull requests, etc., to ensure that the documentation we provide is fit for purpose. Through monthly structured online calls, we can go back to the community to assess accessibility, utility, accuracy, and completeness and iteratively feed back further suggestions, improvements, and contributions.

Openness: The Carpentries has a 25-year history in developing, maintaining, and training from open source curricula based on strong ethical core values of openness and access for all. Since 2012, The Carpentries has supported 4,000 workshops in 65 countries and trained 4,287 Instructors to deliver our 45 collaboratively developed, open lessons to 100,000 novice learners. Improvements to documentation would result in improvements to lessons and Instructors' ability to teach them. The Workbench and related documentation is already openly available and developed using the GitHub versioning system: All Carpentries lessons are open-source and CC-BY licensed Workbench infrastructure is also open source, MIT licensed, and designed with a desire to make reuse and adaptation as easy as possible Based on our previous expertise in running community events, we will initiate monthly community coworking events over the proposed year timeframe, where we will invite community members to help us develop documentation and provide structured, regular feedback. This will: Make the documentation process more collaborative Increase sustainability for open-source technical lesson infrastructure Increase capacity for the development of new features and continuous improvement that benefits the community as a whole Provide learning opportunities for community members keen to gain experience with open source development, R package maintenance, R development, and accessibility

Challenges: Producing documentation is time-consuming, requiring an intuitive structure to be useful and effective. We will prioritise documentation development for community members to get involved with developing the Workbench infrastructure. We will then revisit all documentation section by section, ensuring we complete key parts of the materials based on those that would greatly impact capacity in the community, e.g., moving towards the documentation that prioritises lesson contributors, developers, and maintainers rather than the infrastructure itself. Producing comprehensive, effective, and accurate documentation requires feedback from both experts and community members to assess fitness for purpose. To address this, we will canvass the community early on in the work to form a cohort of reviewers that can quickly and reliably give us detailed feedback on sections of the documentation once they have been improved. The more users the workbench has, the more capacity is required to support infrastructure improvements and features. The Carpentries staff cannot completely support this, and the Workbench is an open-source project. This means we need capacity within the community for that same community to take “open ownership” of the Workbench. We have strong extensive evidence of the efficacy of this model across our existing efforts in curriculum and lesson development for our core training programmes in data science skills.

Neglectedness: The Carpentries developers have already released the Workbench infrastructure and recently transitioned the process of building lessons into websites into the new framework. The community is starting to work with the new Workbench tools and workflows, so this opportunity is very timely for us. Through this proposal, we seek funding for two critical activities: documentation improvement and iterative feedback processes from staff and community members in the effective use, maintenance, and future development of the Workbench. These efforts are crucial to ensure the wealth of open materials that the Carpentries supports are sustained into the future. Sustaining open-source infrastructure in the long term presents challenges, as many software projects lose momentum once initial funding ends. The Workbench is no different; where we have maintained funding to develop the framework but are now coming to the end of our available funds. We are looking to further our capacity-building efforts specifically to support the open-source community around the Workbench in its use and maintenance. The Chan Zuckerberg Initiative (CZI) provided Initial funding for this work from November 1, 2019, through May 31, 2021 (Grant # 2019-206334 (5022)). The Carpentries applied for a renewal grant in 2022 to lower the technical barriers for lesson authorship, maintenance, and contribution; however, that proposal was not funded.

Success: The long-term success and health of the Carpentries lesson development and maintenance community depends on this documentation existing and it being comprehensive. Success metric: An open publication of the first complete version of the Workbench documentation via the GitHub platform, publicly viewable and editable. This project will support the community to deliver their own goals of lesson development, maintenance of lessons and the Workbench infrastructure, and acting as advocates and leaders in their local communities. Success metric: A rich curriculum for developing, maintaining, and extending the Workbench. Deployment of the improved documentation will allow future work to strengthen community governance and build capacity. Our Collaborative Lesson Development Training (CLDT) program, to be launched in Q4 2023, will guide community members through the initial stages of lesson design and collaborative development, and will depend on accurate extensive documentation. Success metric: Feedback into the Workbench documentation gathered through CLDT program activities, measured through GitHub contributions. We cannot set up a governance committee and structure until the user community around the Workbench is more mature. However, towards the end of the year, we would form a plan to bring in a governance structure based on the feedback we receive throughout the proposal’s active period. Success metric: A document outlining how we will set up the governance structure in 2025.

Total Budget: 25,000 USD

Affiliations: The Carpentries

LMIE Carveout: The majority of our 16-member Core Team resides within the USA, with six of the team members working from the UK (2), Germany (1), South Africa (2) and Canada (1). Our community is truly global, with over 4000 instructors throughout the world, and our instructors and trainers represent 82 languages. The Carpentries maintains a strong presence in LMIEs, and we have regional coordinators that can provide community engagement and logistics for specific locations. For example, we employ an African Capacity Development Manager that is tasked with building and fostering our community within the African continent. Furthermore, we have focused our efforts on onboarding our most recent cohort of Trainers from Spanish-speaking regions, particularly those in Latin America.

Team Skills: Two core Carpentries teams collaborate on the Workbench in terms of forming and delivering the strategic plan: the Technology Team and the Curriculum Team. The Technology Team delivers the infrastructure and data management strategy of the Core Team, working on all aspects of development from system health scripting, to full stack web service and database applications, to GDPR compliance and data security. The Curriculum Team manages the community interactions required to continue and sustain development of lessons, and how the Workbench plays an effective part in this process. The proposed team comprises: Executive Director: Dr Kari L. Jordan leads the Carpentries organisation and will provide overall direction. Director of Technology: Dr. Robert Davey leads the Technology Team and has a track record of managing technical staff on complex multi-disciplinary projects involving different stakeholder demographics. He will carry out the documentation improvements. Director of Curriculum: Dr Toby Hodges leads the Curriculum Team and manages the organisation of the community around Workbench features and lesson development. He will lead the community interactions. We have a long-standing Code of Conduct (CoC) and CoC Committee in place for all interactions between and within the Carpentries and the community. This is a core and critical part of not just our training workshops but also our organisation our community, and we have a rigorous incident reporting and response process.

Submission Number: 32