Skip to content

BIDS - INCF 2024 Google Season of Docs Case Study

About BIDS

The Brain Imaging Data Structure (BIDS) is an open global community driving the standardization of neuroscience data across a broad and growing range of modalities and health research disciplines. First released in June 2016, it is supported by a worldwide research network and endorsed by organizations like the International Neuroinformatics Coordinating Facility (INCF). A very large community, including ~29K yearly visitors, use the BIDS specification. More than 300 BIDS contributors currently support and maintain the BIDS community resources to structure and share their data. BIDS encompasses over 40 domain-specific and modality-specific technical specifications, open software conversion and analytics tools, and global infrastructure for collaborating on emerging standards in neuroscience (Poldrack 2024).

Case Study Authors:

Summary

# Tech Writers TW Project Hours Budget % Project Completed
1 150 $6,500 USD 100%

This project was trying to improve the BIDS Online Presence, which consists primarily of its informational websites and the BIDS specification. We were able to migrate away from a multitude of websites and condense down to mainly the one “New” BIDS Website. Overall the project was very successful with a wealth of work put in by the tech writer and mentors. One lesson learned was we could have budgeted more for mentor time as it required more mentor time than anticipated. For other open-source volunteer or community-driven projects in need of coentralizing and streamlining their own online presence, this is a great program if you can define most of the tasks early on and dedicate mentor time alongside tech writer time during the project.

Project Description

Project Proposal

https://www.incf.org/activities/gsod

The Brain Imaging Data Structure (BIDS) is a global initiative to standardize neuroscience data across various modalities. Despite its widespread use and contributions from over 300 community members, the prior online resources distributed across multiple platforms presented barriers to new user onboarding and effective contribution. This project aimed to streamline and centralize BIDS documentation and resources, making them more accessible, user-friendly, and aligned with FAIR (Findable, Accessible, Interoperable, Reusable) guiding principles.

While the BIDS main website and BIDS technical specification already see high and growing usage, this project lowered barriers in onboarding new users and facilitating new user contributions to the community by improving the BIDS main website. Improvements from this project satisfied FAIR guiding principles and improved the experience of referencing BIDS documentation for new and old users alike.

Proposal Creation Process

The BIDS maintainers, steering group, and BIDS online presence working group all saw a need for BIDS to improve and centralize its online resources. One of the BIDS maintainers, Christine Rogers, had worked with Google Season of Docs before with success and suggested we partner with INCF to make progress on the website and documentation improvement projects. Primarily Christine and Eric Earl (chair of the BIDS online presence working group and another BIDS maintainer) drove the project proposal forward with prioritization oversight from the online presence working group. The brief proposal format was effectively developed through several consultative meetings and rounds of drafting.

Budget
How much money did you ask for? 6500$ USD
How did you come up with this estimate? Writer stipend: 5000$
Mentor stipend: 500$ * 3 mentors
(following GSOC practice)
How many hours of work did you budget for the project? Writer: 4 h / wk
Mentors: 1 h / wk * 4 mentors = 4 h / wk
How many hours of work were actually needed for the project? Writer: 5 h / wk
Mentors: 1.5 h / week * 4 mentors = 6 h / wk
What other expenses did you include in your budget? None
Did you run into any budget surprises during the project (for example mis-estimates)? If so, please explain. Mentor time underestimated. Would allocate more for additional mentors or more mentor time.

Technical Writer Recruitment

We selected a technical writer candidate early on based on the advanced neuroinformatics data standards experience which was fundamental to quickly on-board for this project.

When INCF published the writer and mentor name on the project page, first the writer and then mentors were personally contacted by individuals wishing to participate.

Note: Writer name, but not his contact information was published. For that reason we took his name down and wrote that a writer had already been selected.

We were lucky to work with this writer with significant experience and technical background in the field for the full duration of the project timeline from start to end.

Other Participants

A few BIDS maintainers were recruited to mentor the project. Each stayed for the duration of the project with varying availability over the year.

Additional members of the Maintainers and Steering group were recruited to provide feedback on the work in progress. In addition, a feedback survey was distributed globally to the user community with feedback received at the 60% progress mark in the project.

Timeline

April and May were a soft start to the project, with our tech writer and mentors alike getting used to working out project tracking and bi-weekly check-in meetings with each other. A lot of technical changes happened early on to combine many documentation sites into just one website and improve navigation. By June and July we were requesting and incorporating feedback from contributor surveys. We were able to share progress and contributor views during the annual Organization for Human Brain Mapping (OHBM) Conference’s BIDS Town Hall. August and September was another round of requesting and incorporating feedback from web surveys. And finally, October and November brought the BIDS Impact page together to wrap the project with a significant effort on everyone’s part.

Deliverables

Planned Deliverables

Deliverable % Complete Relevant Links Notes
Consolidate into one website 100% new BIDS website Dev site - Planning launch in Q1 2025 to replace the old BIDS website at https://bids.neuroimaging.io/
Implement a more friendly main website structure and improve navigation 100% mkdocs TOC configuration User feedback indicates this created significant improvement ( See: Metrics)
New BIDS Impact page 100% new impact page Expanded and improved with more metrics and relevant context
Categorize and add clear summaries to each webpage 100% Pull request #498 Each page now has a lead paragraph.
Synthesized and enabled gathering of user-feedback and user-testing 100% Developed a follow-up survey and collated findings from all community feedback. Helped synthesize metrics and inputs for annual community presentation.
Add a feedback form within the main website 100% Giscus at the bottom of the front page was re-scoped and implemented by a mentor

The above table summarizes the many changes across several repositories (pull requests linked here):

Unplanned Deliverables

No unplanned deliverables arose in the course of this project.

Metrics

We performed a broad-reaching public BIDS community survey before this project started and a narrower website-specific survey at the 60% mark in this project’s timeline to assess the improvements being made to online materials. In user feedback collected on the overall utility as well as the ease of navigation of the old vs the new website (4 separate rating questions), users rated the new site as either as good or better than the old site for both utility and ease of navigation in 94% of respondents. Key improvements noted were acceleration in accessing info, more intuitive navigation in both the search bar and the navigation tabs, and the feedback reporting page.

Analysis

This was a successful project. The project resulted in positive user feedback about the changes implemented by only 60% through the timeline.

During on-boarding as we began breaking down deliverables into tasks, the mentors realized that a few deliverables could be resolved most effectively by mentors managing the most technical steps. This minor re-scoping helped move the project forward quickly. Some timelines were delayed due to outside factors during the course of the project and timelines were adjusted accordingly.

The project experience underscored the value of external support for open-source initiatives – both in terms of funding and umbrella organization administration.

Lessons Learned

What went well?

Topic What we did Lesson learned
Mentorship Had an initial pool of 5 potential mentors for support (of which 3.5 were available in reality) Planning for a larger pool of mentors was an excellent idea.
Metrics Sent a “before” and “after” community survey Do this again, and sooner/better timed to the changes
Project Deliverables Resolved some deliverables with technical features on the website (for example feedback form, search) not technical writing Re-scoping project can be okay to solve challenges in the most effective way

What could be improved?

Topic What we did Lesson learned
Budget 1 writer stipend; 3 mentor stipends split across 4. We underestimated mentor effort and would budget more mentor stipends
Communication Mattermost plus alternating weekly check-in/coworking times Allocate extra mentor time to managing and setting timelines
Onboarding Set a loose order of on-boarding tasks Encourage an order of completion for early tasks for example friction log
Project Management GitHub Project Board It took a few iterations to get this right. Deliverable timelines between meetings should be set.