Critical cloud infrastructure for ocean research and planning in the Indian Ocean Rim: A regional JupyterHub to support the ITCOocean training center
Funding Area: Critical shared infrastructure / Infraestructura compartida critica
Problem Statement: The food security of millions relies on coastal, often artisanal, fisheries in the Arabian Sea and Bay of Bengal, including East Africa, West and East India, Bangladesh, and Myanmar. Across the global south, we know that those experiencing the brunt of climate change impacts have the least resources to deal with it. This pattern extends to scientists in the region who lack access to big data geoscience resources and platforms, unlike their global north counterparts. The International Training Centre for Operational Oceanography (ITCOocean) in Hyderabad, India, run by the Indian National Centre for Ocean Information Services (INCOIS), provides training and research support in operational oceanography that supports research and applications for marine safety, fisheries, and conservation. This proposal would support a JupyterHub provisioned for geospatial analysis and cloud data access and computing maintained by INCOIS. The JupyterHub will be used both ITCOocean trainings and as a research platform for regional scientists. The first ITCOocean training for 70 early career Indian Ocean Rim scientists is in September 2023, using a 2i2c supported JupyterHub funded via 2 months of seed money from ESIP. The funding from OI Fund 2023 would support the hub for 10 extra months to allow us to demonstrate the benefits of a regional JupyterHub for operational oceanography training and research, to support proposals to secure long-term funding.
Proposed Activities: Timeline: September 2023: Initial 2i2c JupyterHub set-up and first training course in cloud-based computing for operational oceanography using the Pangeo ecosystem of tools: September 11-22, 2023 (funded via seed money from ESIPfed.org). Instruction by Elizabeth Holmes, PhD (NOAA Fisheries). The course and student logistics are handled by Nimit Kumar, PhD (INCOIS) and Uday Bhaskar, PhD (INCOIS/director ITCOocean). https://hackweek-itcoocean.github.io/2023-Hackbook/ October-December 2023: Development of a virtual training and hackweek to expand the training options and reduce travel costs for participants (Kumar, Bhaskar, and Holmes). Monthly community calls for the September 2023 hackweek participants for 1) end user feedback for the hub and to understand the challenges of cloud-based JupyterHubs for our participants with limited internet access, esp in E Africa, Bangladesh and Myanmar, and 2) support continued project research on the hub by the Sept 2023 participants. January-April 2024: Two main activities. 1) Virtual training course developed by 2i2c to be run by ITCOocean: “Management of JupyterHubs for Research and Training Communities”. This will be taught by James Munroe (2i2c). 2) Pilot a virtual training course using the INCOIS JupyterHub. April 10-12, 2024: Presentation on the value of cloud computational platforms for ocean science for research communities in low and medium income countries at the 2024 UN Ocean Decade Conference. We will present results from the ITCOocean hackweeks supported by the JupyterHub. May-July 2024: Submit proposals to secure long-term funding for the JupyterHub to support ITCOocean and early career Indian Ocean Rim oceanographers. Training courses expertise: Dr Uday Bhaskar leads the International Training Centre for Operational Oceanography (ITCOocean) at INCOIS. ITCOocean was established in 2013 by an agreement with Intergovernmental Oceanographic Commission (IOC/UNESCO). With IOC support, ITCOocean organizes training courses related to oceanographic data, prediction of the state of ocean including tsunami and storm surges, and IOC's International Oceanographic Data and Information Exchange 'OceanTeacher' programme. IOC bears the cost of trainees/students in participating in the training courses at ITCOocean. Jupyter training: Dr. Elizabeth Holmes is lead for NOAA Fisheries Open Science initiative and affiliate faculty at the University of Washington School of Fisheries (Seattle, WA USA). She has taught statistics and reproducible science for over 20 years (R, Git, GitHub, Python, Jupyter) and heavily involved in the geoscience Open Science community. Since 2015, she has taught training courses at INCOIS (with Nimit Kumar) as part of NOAA’s international outreach program. JupyterHub admin training: James Munroe is leading 2i2c’s efforts to expand JupyterHub use by developing curriculum for teaching how to maintain and admin JupyterHubs.
Openness: 2i2c’s JupyterHub infrastructure is open source and they have a ‘free to replicate’ policy. Via the “Management of JupyterHubs for Research and Training Communities” trainings to be held at ITCOocean, we will help develop the skillbase necessary for other Indian research communities to manage their own JupyterHubs for ‘big data earth science’, helping ensure that Indian Ocean Rim scientists working in earth sciences are not ‘left out’ of training and experience in using cloud-based platforms for big data research and application development. Our training materials will all be open source and hosted on GitHub at https://github.com/Hackweek-ITCOocean
Challenges: The main challenge will be developing experience and expertise with JupyterHubs and notebooks by the ITCOocean staff. Although there is high technical computer skill and capacity at INCOIS, there is limited Jupyter notebook experience and no JupyterHub experience. Dr. Holmes has been an organizer for a number of hackweek using 2i2c JupyterHubs and will train the ITCOocean staff. This will require running a number of events (course and hackweeks) using the hub so that the current ITCOocean technical staff develop the experience. Having a 2i2c managed hub for the first year (or two) at least will be critical as 2i2c has the expertise and mission to support JupyterHub infrastructure for communities with less experience with these platforms. In addition, establishment of a 2i2c JupyterHub will allow the INCOIS staff to join the community of 2i2c JupyterHubs being established by 2i2c as part of their support for Latin American and African cloud infrastructure for big data research.
Neglectedness: This proposal would continue funding for the 2i2c JupyterHub used in the Sept 2023 course and hackweek for another 10 months so that 70 participants can continue their research projects and we can offer more in-person and virtual trainings using it to increase the INCOIS staff experience using the JupyterHub platform. The timing is unique and critical because it takes advantage of the Sept 2023 course and hackweek to kick-start an early career a user base. This funding would provide the “business” case showing concrete examples of regional research projects and trainings made possible by the ITCOocean JupyterHub. This will allow us to apply for funds for long-term support. The goal would be to identify funding opportunities via international or NGOs that support capacity building in low and medium income regions (Indian Ocean Rim) for earth science research and climate change impact response. We would begin applying during early 2024 with the goal to have secured long-term funding by the end of the 10 months covered by this proposal. Dr. Holmes is helping with the proposal writing and training, but NOAA does not support this type of “pilot” funding for cloud infrastructure that supports a non-NOAA agency. Similarly, securing pilot funding for a JupyterHub from the Ministry of Earth Sciences (MoES, Government of India) would not be possible. The training center is supported by UNESCO.
Success: Participants in the Sept 2023 hackweek continue to use the JupyterHub to develop new research projects using access to NASA and NOAA remote sensing data in the cloud and access to the community resources (Jupyter notebooks) developed for oceanography and marine applications. Development of local self-sustaining, Open Science, community of Indian Ocean Rim scientists who engage with the Open Science big data geoscience communities that are helping researchers use massive earth data sets, e.g. Pangeo (https://pangeo.io) and ESIP (https://www.esipfed.org/). Currently these communities are heavily Europe, US, and Canada dominated. Successfully securing long-term funding for the ITCOocean JupyterHub AND this being a “spark” that will help the spread of knowledge and expertise with JupyterHubs within India and Indian Ocean Rim countries.
Total Budget: 22500
Budget File: pdf
Affiliations: International Training Centre for Operational Oceanography (ITCOocean), ESSO-INCOIS, Hyderabad, India
LMIE Carveout: Yes this project fits within this category. The International Training Centre for Operational Oceanography (ITCOocean) in Hyderabad, India promotes the development and optimization of scientific base, technology and information system for operational oceanography within the Indian Ocean Rim. The training center, along with staff from Indian National Centre for Ocean Information Services (INCOIS), provide advanced training in operational oceanography for young scientists and technical persons and decision makers/ officials from India, Indian Ocean Rim countries and other developing nations on a continuous regular basis, to enable a large pool of young trained manpower with core skills in the sub-continent.
Team Skills: Dr. Elizabeth Holmes (NOAA Fisheries). Eli is lead for NOAA Fisheries Open Science initiative and affiliate faculty at the Univ of WA School of Fisheries. She teaches reproducible science (incl R & Python) and statistics. She has been supported since 2015 by NOAA Fisheries’ international program and has taught multiple training courses in fisheries statistics and R at INCOIS. Dr. TVS Udaya Bhaskar (INCOIS). Udaya heads the INCOIS Ocean Data Management division and ITCOocean. INCOIS is the central repository for marine data in India: the National Oceanographic Data Centre for the International Oceanographic Data Exchange Programme, the National Argo Data Centre, and the regional data centre for the IOGOOS Programme. Dr. Nimit Kumar (INCOIS). Nimit is part of the marine fishery advisory program at INCOIS and is involved in INCOIS’ training programmes via ITCOocean. He is one of the founders of the early career scientist network of the 2nd international Indian Ocean expedition and is the education and outreach chair of the Pan Ocean Remote Sensing Conference. Dr. James Munroe (2i2c). James is Product and Community Lead for 2i2c. He is a strong advocate for enabling scientists and students to be efficient and effective in their computational workflows and wants to bring the strength of the Jupyter ecosystem to a broad range of users. Before 2i2c, he was an associate professor in the Dept. of Physics and Physical Oceanography at Memorial Univ. of Newfoundland.
How Did You Hear About This Call: Word of mouth (e.g. conversations and emails from IOI staff, friends, colleagues, etc.) / Boca a boca (por ejemplo, conversaciones y correos electrónicos del personal del IOI, amigos, colegas, etc.)
Submission Number: 160
Loading