Promoting the Use of Open Infrastructure in Africa

Edward Miguel

Funding Area: Capacity building / Construcción de capacidad

Problem Statement: The scientific community has become increasingly aware of problems with the reliability of research in the social sciences, which has led to growing consensus on the need for greater openness, integrity, and reproducibility (Christensen, Garret, Jeremy Freese, and Edward Miguel. 2019). These problems often arise from a lack of transparency—including selective presentation of results and the failure to publicly share data, materials, and code. Together with intentional scientific misconduct, these practices cast doubt on the credibility of research findings, undermining efforts to make public policy more evidence-based. In response to this challenge, the Berkeley Initiative for Transparency in the Social Sciences (BITSS) has spent the past decade building open science tools and infrastructure and promoting their widespread adoption through short-course trainings. These efforts have helped to shift research norms and practices towards greater transparency and credibility. Despite this progress, stricter norms and standards for research transparency unintentionally favor those with access to networks and resources. Without proactive efforts to make open science tools, infrastructure, and training equitable and inclusive, we risk leaving a large and important group of researchers—those based in Sub-Saharan Africa and other low-income regions—behind.

Proposed Activities: BITSS’ flagship Research Transparency and Reproducibility Training (RT2) provides a platform for training in open science infrastructure and related tools to underserved groups, including African social scientists. With support from IOI, we propose to host a 1-day RT2 ahead of the Center for Effective Global Action’s 12th Annual Africa Evidence Summit, to be held in Dar es Salaam, Tanzania in Summer 2024. By pinning our training to the Summit, we leverage the fact that over 60 members of the Network of Impact Evaluation Researchers in Africa (NIERA)—hailing from Kenya, Tanzania, Uganda, Rwanda, Burundi, and Ethiopia—will already be present. Several members of this group have already expressed a strong interest in attending such a training. In 2022, 62% of applicants to a NIERA training in Ethiopia reported interest in learning about “research transparency, reproducibility, and meta-research.” At a similar training in Ghana this year, 77% of participants were "very satisfied" with CEGA’s session on research ethics, transparency, and reproducibility. With the help of a committed funding partner, we are eager to meet growing demand for this training among underserved groups. RT2-Tanzania will accommodate roughly 30 participants, with sessions led by 4 academic instructors and 2 BITSS facilitators. We plan to recruit at least one instructor from NIERA, as a handful of NIERA members have already been trained by BITSS and have strong fluency with research transparency and related infrastructure. To meet the needs of this new format and audience, the curriculum will be adapted from our existing RT2 curriculum which covers, among other topics: threats to research credibility and reproducibility and their relation to the scientific ethos; improved research design specification; infrastructure, tools and methods for research reproducibility and collaboration; and evidence synthesis, reproducibility, and interpretation. Our focus will be to train participants in BITSS’s three open-source platforms. The training will be led by the BITSS team, including PI Edward Miguel, Assistant Project Scientist Fernando Hoces de la Guardia, and Program Manager Grace Han. Having delivered 11 successful RT2 events since 2014, BITSS is well-prepared to recruit instructors and manage the agenda and logistics of the event. Outreach and promotion will be managed by the NIERA team, led by Chairman Amos Njuguna. We expect to primarily source applicants from NIERA and its partners in the region (e.g. the African Population and Health Research Center and the Council for the Development of Social Science Research in Africa). As Chair, Njuguna is well-positioned to disseminate information about the training and identify researchers who would greatly benefit from the training. This work will require staff time to adapt the agenda, recruit instructors, select participants, prepare training materials, and plan event logistics and outreach in partnership with the Summit organizing team.

Openness: We will train scholars to use three open-source platforms managed by BITSS: the Social Science Reproduction Platform (SSRP), Social Science Prediction Platform (SSPP), and MetaArXiv. The SSRP is an openly licensed platform that facilitates the sourcing, cataloging, and review of attempts to verify the computational reproducibility—the ability to reproduce a paper’s results, tables, and figures using data, code, and materials made available by the authors—of social science research. The SSPP enables the systematic collection and assessment of expert forecasts regarding untested social programs, helping researchers and policy makers by increasing the accuracy of forecasts and improving experimental design and analysis. MetaArXiv is a pre-print service for papers focused on improving research transparency and reproducibility. By hosting RT2 with the Africa Evidence Summit, we will be engaging a large network of scholars who may not otherwise be aware of BITSS programming and resources, driving our goal for equitable and inclusive participation in the open science community. After the training, we plan to share the materials openly on a public Open Science Foundation project page and the BITSS resource library. Finally, we will encourage participants to use our curriculum and learning resources to offer the training to those in their own networks. For instance, faculty will be encouraged to trickle the learnings to their graduate students, and other researchers to their peers.

Challenges: We are confident that the training will produce the expected learning, curricular outputs, and shifts in research norms and practice for participants. However, we recognize that focusing solely on individual capacity may not address institutional constraints, and that training only a small number of African researchers when so many could potentially benefit risks perpetuating inequality in the scientific ecosystem. Additionally, we acknowledge the tradeoff between the size of our training and the level of individual support that our instructors can provide to participants; we expect there will be much greater demand for this training than we can realistically meet within budget constraints. To mitigate these issues, we will make all of our training materials open and accessible on an OSF project page and will point those whom we do not accept to BITSS’s online resource library and YouTube channel, which contain open science academic materials and tools, recordings of our Massive Open Online Course on Research Transparency and Reproducibility, and previous conferences and webinars. We will also take this opportunity to re-engage BITSS Catalysts—graduate students, faculty, and other researchers in the social sciences committed to educating the current and next generation of social scientists on research transparency—in Africa and connect them with researchers within NIERA and at the Africa Evidence Summit who are interested in research transparency.

Neglectedness: BITSS currently has funding from the National Institute of Aging to conduct RT2 for 5 years, where the emphasis is on research transparency and reproducibility practices as they pertain to aging and health disparities research. However, because it is a U.S. federal grant, we are unable to host trainings abroad with LMIE communities. We previously received funding from the William and Flora Hewlett Foundation to bring African scholars to US-based RT2s, however they stopped funding this activity when their priorities shifted. While we have identified modest sources of funding for the technical development and maintenance of our open science infrastructure, we have not come across any other funding opportunities that would allow us to strengthen the sustainability of open infrastructure through capacity building for LMIE researchers. We were delighted to learn about IOI as we believe our objectives are well aligned.

Success: We will measure success through participants’ expressed change in awareness, knowledge, and research behaviors post-training. To gain better understanding of our participants’ needs, capacities, and learning outcomes, we will collect quantitative and qualitative data using the following: 1) Pre- and post-training surveys (administered 1 month prior to and 6 months following the event): These will measure participants’ i) changes in awareness and perspectives, ii) use and mastery of specific open science tools and infrastructure, and iii) confidence in having constructive conversations about issues related to transparency and ethics with their collaborators and the wider scientific community (through mechanisms such as peer review, seminar presentations, and conducting research reproductions, replications, or extensions). 2) Interviews with a small group of randomly selected participants (6-12 months following the training): These interviews will collect qualitative data from participants on the drivers of, and barriers to, adoption of tools and practices introduced at RT2. Previous RT2 surveys have documented growing awareness of problems in research conduct and evaluation, demand for sustained support, and evidence of adoption of new tools. We have also gained valuable insight on researcher practices through interviews. We will build on these methods, adapting them for RT2-Tanzania and the goals of increasing awareness and use of our open platforms among African scholars.

Total Budget: $25,000

Affiliations: Center for Effective Global Action (CEGA), Network of Impact Evaluation Researchers in Africa (NIERA)

LMIE Carveout: While CEGA is headquartered at the University of California, Berkeley, our network includes more than 150 North American faculty, 65 African scholars, and hundreds of graduate students around the world that produce rigorous evidence about expanding education, health, and economic opportunities for people living in poverty. CEGA invests in research capacity building that empowers LMIE scholars to become knowledge-generators in their home countries and organizes events to disseminate locally-generated research and connect researchers with policy makers and other partners. With NIERA as our partner, the proposed project will be carried out in Tanzania prior to the 12th Africa Evidence Summit, a gathering to discuss the latest research on economic development and poverty alleviation in Africa.

Team Skills: Edward Miguel is Oxfam Professor of Environmental and Resource Economics and Faculty Director of CEGA at the University of California, Berkeley. His main research focuses are African economic development, for which he has conducted field work in Kenya, Sierra Leone, and Tanzania, and promoting openness and inclusion in development research. He is Faculty Director of BITSS, which has changed the landscape of research transparency norms and practices since 2012. Through the development of open science policies, protocols and platforms, and the provision of educational resources like his textbook, Transparent and Reproducible Social Science Research, Miguel has demonstrated his commitment to research ethics and role as a leader in the field. Having been an instructor at eight RT2 events, he is well-equipped to lead oversight of this training. Amos Njuguna is the Dean of the School of Graduate Studies, Research and Extension at United States International University-Africa, and a founding member and Chairman of NIERA. His research in financial economics has impacted policy outcomes in the Kenyan retirement benefits and insurance industries and public governance. As Dean, he has embedded impact evaluations into the research methodology curriculum for all graduate programs at the University and trained faculty, students, and others on impact evaluations approaches. Njuguna is a fellow of CEGA’s East Africa Social Science Translation Collaborative and an ambassador for transparency.

How Did You Hear About This Call: Word of mouth (e.g. conversations and emails from IOI staff, friends, colleagues, etc.) / Boca a boca (por ejemplo, conversaciones y correos electrónicos del personal del IOI, amigos, colegas, etc.)

Submission Number: 163