A graduate student perspective on overcoming barriers to interacting with open-source software

Despite these clear benefits, graduate students face obstacles at every stage of the learning process that current standard practices in academia do not address. To the best of the authors’ knowledge, the literature currently lacks any clear synthesis of these issues from the graduate student perspective. This article is motivated by this notable gap, as well as by the experiences of the authors as Doctoral students in Computer Engineering and Biology, respectively, and as coding instructors with a combined 10 years of experience teaching open-source scientific programming to approximately 550 graduate students globally (as of late 2019). Thus, the points discussed herein are done so based largely on the shared experiences and personal observations of the authors and the diverse group of students they have interacted with and talked to about this topic.

This paper is intended for those in any discipline that involves some degree of interaction with OSS, including both scientific (e.g., computer science, ecology) and nonscientific fields (e.g., library studies, business). Written from a graduate student perspective, this discussion is intended for individuals in any research-based postgraduate degree program, those who closely work with these individuals (e.g., supervisors, researchers, administrators), and members of the OSS community; it is broadly accessible for those with both limited and well-developed computational knowledge. Best practices of coding and OSS use are not discussed herein; these topics have been discussed in detail by Wilson et al. (2014) and Jiménez et al. (2017). This paper neither discusses the existing differences in academic culture or research progression across countries or institutions, nor speaks to specific programming software or languages.

This article first provides a unique, in-depth discussion of the many barriers that graduate students face when learning, using, developing, and contributing to OSS in the context of their graduate research, including cognitive obstacles and cultural shifts. Following this, it proposes a series of practical guidelines for graduate students, supervisors, administrators, and OSS community members to help reduce and possibly eliminate these barriers. If maintained over time, these proposed actions aim to establish and normalize practices at all levels of academia and OSS that may facilitate a sustainable framework for the continued development of OSS in research while also boosting student success and confidence.

When students enter graduate school, the discovery that their research will require or improve significantly with data cleaning and other computational skills may cause significant uncertainty. Although most consider programming and the ability to use software programs “hard skills”—teachable abilities that can be defined and measured—inconsistencies and gaps in the training provided to students exist across and within disciplines. Of 55 geography departments of US institutions examined in 2017, 80% offered a programming course, but <11% of the undergraduate degrees offered by these departments required the completion of any programming course (Bowlick et al. 2017). Touchon and McCoy (2016) reported that only 25% of 154 ecological doctoral programs examined required that students take a biostatistics course, which is the primary way that most ecologists apply OSS to research. This reflects an ongoing issue; over a decade ago, in 2008, a workshop jointly organized by the International Union of Biochemistry and Molecular Biology and the Federation of Asian and Oceanian Biochemists and Molecular Biologists had already identified the absence of a standard means of measuring or classifying the computational skills of biology graduates as a major hindrance to the development of bioinformatics graduate education (Tan et al. 2009). As recently as 2017, computational training has been identified as an unmet need that continues to be underprioritized; in the case of surveyed biologists, this lack of training was seen as the most important limiting factor in their ability to use the data generated by their research (Barone et al. 2017). Today, engineering programs usually require that students complete an introductory programming course, but the links between programming and research are often unclear. While both the breadth and depth of pre- and early-graduate school computing experience and instruction varies among specific disciplines, institutions, and research labs, many new graduate students lack the necessary computing knowledge and skills required for their research project and nor are there opportunities available to gain these skills through traditional or standardized ways (Katz et al. 2018).

A significant barrier to graduate students at this early stage of their academic career is the need to assess their own programming skills and the skills needed to accomplish their research goals without having appropriate background on the topic or a means by which to undertake an assessment. The complexities surrounding the ignorance of one’s own ignorance was discussed by Dunning (2011); the well-known Dunning-Kruger Effect states that poor performers in a domain are largely unaware of their own deficient skills or knowledge. This burden weighs heavily on graduate students struggling to assess their programming skills and deciding how much time and effort they need to develop these skills. Here, a knowledgeable and objective supervisor¹ is essential for providing guidance to their students about how to plan and conduct research (Alharbi and Jacobsen 2016). However, in the context of determining current and required computational skills of students, the success of this guidance is likely to depend on a supervisor’s own computational skills and knowledge, which may be limited or outdated as a result of the relatively recent surge in the diversity and use of OSS across disciplines. Developing computational skills, whether out of necessity or in supplement, delays initial research progress. Graduate students must adhere to institutional deadlines and typically depend on limited funding that may require additional responsibilities such as teaching. These time-consuming demands ultimately add burden and stress to a valuable training process. Whether students lack awareness of the training process, or appreciate and embrace the challenges they face, committing the time to develop these skills add risk. As will be discussed in more detail herein, the pursuit of these skills introduces additional barriers to students, including cognitive obstacles during the learning process, a lack of guidance and resources, and technical difficulties linked to gaps in general knowledge and experience in using computers.

Faced with the daunting task of learning to program, students often experience obstacles that span many learning environments and are well described by cognitive and educational psychologists (Murphy and Thomas 2008; Cutts et. al. 2010). Most of the graduate students the authors have interacted with report that they seek out coding as a tool to accomplish a task related to their research objectives, rather than specifically to learn to program. This is further evidenced by researchers spending significant amounts of time applying code written by others, rather than attempting to write original code (Nowogrodzki 2019); half of UK-based academics surveyed in 2014 reported that they do not write their own software (Smith et al. 2018). Students learn code while also trying to apply these new skills to solve additional problems such as unfamiliar statistical analyses or data transformation; all in addition to managing other research tasks. According to cognitive load theory, these extraneous pieces of information can overwhelm the limited capacity of a student’s working memory, leading to problems in learning and retaining new information (Sweller et al. 2011). Students belonging to underrepresented groups in computing may experience additional mental strain in the form of stereotype threat, a fear of conforming to negative stereotypes that may reduce situational performance (Steele and Quinn 1999).

Students require clear analysis objectives to focus their learning. Overwhelmed students often miss out on necessary discussions about the relative importance of syntax specifications, programming concepts, and tool utility in the context of their research and the degree of knowledge and skills needed for each. Without clear objectives and specific knowledge of these new tools, the task of “learning to code” may seem insurmountable to many students. Often students turn to popular online resources, but these resources can be problematic if consumed out of context and the messages that often accompany these resources further increase coding anxiety. For example, some resources claim that only passionate individuals willing to put in the years of effort needed to reach a “master” level should learn programming (Flombaum 2017), providing groundless guidelines about the time commitment required—“a minimum of 3 months [on a schedule of] 11 am to 11 pm” (Kusanagi 2016)—or emphasize that “programming requires being able to absorb a huge swath of ever-changing information” (Pratt 2017).

Struggling students may suffer from a fixed mindset, believing that their abilities or talents are predetermined or innate (Dweck 2007). Comments such as “programming is not for me” or “I’m just not good at coding” reveal this attitude. Unfortunately, blogs, discussion boards, and online articles often support these misconceptions (e.g., Flombaum 2017; Pratt 2017); graduate students are likely to stumble upon such sources of information in the early stages of the learning process when alternative resources are unavailable. A reluctance to take opportunities to learn and declining effort when faced with errors often accompanies a fixed mindset (Dweck 2007). Students often interpret coding errors as a reflection of their own abilities rather than recognizing their inevitability and framing them as valuable learning opportunities (Rodrigo et al. 2009; Brown et al. 2018; Carey and Papin 2018).

A lack of available guidance and resources during the learning process often compounds psychological demotivators. In the earliest stages of learning, students often miss important conversations about tool availability and the advantages and limitations of each or miss the significance of these conversations (Bacharakas 2018). As a result, they may choose software suboptimal for addressing their research problem or supporting their long-term career goals. Initial and ongoing technical considerations such as software installation, operating system compatibility issues, version control, interface options (e.g., R Commander, RStudio, R Notebooks), software launching, directory navigation, and data formatting prompt set-up routines that introduce some degree of dependency. This is particularly problematic for students who use institutional laptops and lack the administrative permissions required to install and update software. The complexity of software installation and configuration can potentially lead to confusion or technical problems that can waste time and demotivate students, especially those with limited computer knowledge (List et al. 2017). These knowledge gaps create obstacles in many ways; for example, a student may have difficulty importing a data set without a general understanding of directory structure or if the file format does not conform to the analysis software requirements (Baker 2017).

After overcoming software installation and configuration, students may be eager to quickly solve complex research tasks, but they must first (i) learn the basics of coding, including vocabulary and grammar; (ii) be aware of how to apply code in a practical way; and (iii) understand how to approach these workflows. Students face a steep learning curve, and where they cannot access formal training through traditional course-based means, students must take responsibility in finding and using online resources, local peer-instruction groups (e.g., Dreyfuss 2017), or other learning opportunities such as workshops or tutorials (Baker 2017). In possible cases where supervisors lack familiarity with a tool, cannot keep up with the current state of research computing, or simply lack awareness of the value of these skills, students might need to defend the time dedicated to this learning process because progress toward their specific research goals may appear slow.

As students gain foundational skills and begin tackling higher-level research tasks, they will face hurdles such as interpreting error messages, applying unfamiliar functions, and uncertainty about availability of prewritten code for completing common analysis tasks. Success depends upon finding solutions to these problems, but students may hesitate to ask for assistance. At this stage, many students consult notoriously problematic OSS documentation, unaware of its limitations. A survey of GitHub users showed that 93% of respondents reported problems with incomplete, outdated, or confusing OSS documentation (Geiger et al. 2018). Even well-documented contributions include technical language outside of the scope of student’s knowledge, forcing them to seek further help online or locally.

While searching for solutions online, students often visit StackOverflow (stackoverflow.com), a valuable community question-answering website where students can browse previously submitted questions and responses or submit their own questions. Community question websites, including StackOverflow, often offer community members who answer questions at high levels of understanding or use technical terminology outside the scope of student’s knowledge, especially when the question is not placed in an appropriate context. Students provided with both a solution and explanation of the solution to a specific problem learn more effectively than those provided with only a solution, if the delivery of the explanation coincides with the level of the student’s current understanding (Webb 1989). Question quality is important because high-quality questions can attract more users and result in high-quality responses (Ravi et al. 2014). However, students unaware of what makes a high-quality question may make errors in their inquiries, such as using ambiguous terminology, not clearly stating the problem, not contextualizing the problem, or not demonstrating what background research they have done to address the problem themselves, including ensuring that the website does not already answer their question. These oversights can reduce the number and quality of responses to their question and increase the chances of receiving a rude or condescending response from a frustrated community member. Such responses may discourage students from asking future questions or lead them to simply use solution code without gaining an understanding of why or how it works.

Interaction and involvement with online communities outside offers an invaluable resource to increase understanding and use of OSS, but it requires that students gain some understanding of the structure and culture of such communities. The online presence of each OSS community is unique in scope and form, but may include activity on various social media platforms such as Twitter, chat platforms such as Slack, and community sharing through tutorials or blog posts (Carey and Papin 2018; see Amazon 2019a; Google 2019a; Microsoft 2019b). Graduate students often lack awareness of these informal but powerful means of connecting with other users and developers of OSS. Similarly, students might be unaware of local resources such as student-led study groups, formalized help centers on campus, and relevant community meet-up groups. As a result, it is possible that students unknowingly disconnect from peers at the same institution or even within the same department who face similar challenges in learning and using particular software. Lack of availability of these local resources (both on and off campus) represents a significant gap that needs to be addressed at both institutional and community levels.

As noted by Smith et al. (2018), there is a lack of systems in place for rewarding software-based contributions to research. This is reflected in graduate schools and academic culture today through the lack of associated curricula and the perceived unimportance of such contributions. While students explicitly learn about the process of publishing research results as a traditional manuscript in a peer-reviewed journal, comparatively little discussion focuses on the role or importance of publishing code or software or software-related articles. However, traditional manuscripts increasingly include research code and data as supplemental components and some scientific journals now expect it. Although this change represents an important step toward reproducible and transparent science, many students, unaware of their options, might publish “software articles” as a placeholder before receiving credit for their work in traditional formats (Smith et al. 2018). Current publication practices may not acknowledge other code or software-based contributions as primary research products; for instance, these contributions may be omitted from calculations of author metrics on systems such as Google Scholar Citations and Mendeley (Martín-Martín et al. 2018).

Reward systems driving career progression such as tenure reviews and funding applications focus on traditional research contributions and subsequently measure productivity and success in terms of publications and citations of peer-reviewed research papers (Hey and Payne 2015). These metrics do not adequately capture the full range of contributions in modern computational research (Smith et al. 2018). Similarly, degree requirements and institutional expectations of dissertation structure also bias toward traditional research contributions and rarely recognize code—even well-documented, peer-reviewed code—as a comparable product. The computational portion of a student’s work, which typically represents a major time commitment, valuable skill development, and potentially significant contributions to the discipline, may end up relegated to appendices or ignored altogether. Other means of contributing to the OSS community, such as creating tutorials or organizing existing resources are recognized as extracurricular volunteer work, at best, and play little or no role in degree progression. Thus, those students who devote time to publishing their code and contributing to OSS do so at the risk of delays in degree or career progress, and may face criticism from supervisors and peers despite the inherent value of these tasks, which may be sought by industry (Learner et al. 2006; Smith et al. 2018).

In addition to the challenges associated with academic culture, few graduate students are prepared for roles as contributors to OSS. A recent survey by Mozilla of university students who wanted to contribute to OSS but were unable to revealed that nearly 70% cited “not knowing where to start” as the primary reason they did not contribute (Bacharakas 2018). Graduate school curricula may not include the practical skills needed for contributing, including where and how to share code, where and how to ask for feedback, and general knowledge about licensing and maintenance responsibilities. These knowledge gaps may be particularly daunting and complex in instances where graduate students have adapted another user’s code for their own purposes, which is the case most often seen by the authors. Unfortunately, newcomers who join the OSS community with limited practical knowledge of contributing rarely become long-term contributors, in part because of a notable absence of mentoring in the OSS community (Mendez et al. 2018). Steinmacher et al. (2015) provided evidence that a lack of social interaction between newcomers and the OSS community may represent as significant a barrier to contributing as a lack of technical skills.

Would-be contributors face a new set of psychological challenges. Both students and well-established scientists worry that their code and documentation are not sufficiently polished, and that they will face judgement for poor variable names, misused technical terminology, or errors in their code (Barnes 2010). Although it is important that users validate their code, especially code that may have been reverse engineered or adapted, students may fear that errors in their code will poorly reflect the quality of their research. These concerns and overall lack of confidence decrease the likelihood they will share or publish their code even if the value of such a contribution is high (Barnes 2010; Bacharakas 2018). Graduate students are not taught and are unaware that unpolished code is normal and acceptable even among professional software developers, and code that accomplishes something others are also trying to accomplish represents a valuable contribution that should be shared and reviewed (Barnes 2010).

Success in graduate school critically depends on maintaining motivation. Thus, graduate students must balance recognizing their limitations, identifying the knowledge and resource gaps they need to fill, and acknowledging their progress. To encourage a productive and motivating environment for themselves and their peers, graduate students should:

Supervisors represent the primary resource for guidance and mentorship for graduate students and can potentially greatly influence their skills, knowledge, and attitude toward OSS. To help students avoid or overcome barriers to OSS, supervisors should:

Administrators can provide resources, prioritize institutional objectives, and influence academic culture. To support students and faculty interacting with OSS in their research, administrators should:

As noted by the Open Source Initiative (2007a), “OSS and other collaborative projects benefit through, and because of, community”. OSS community members can play an important role in supporting and mentoring new collaborators, including graduate students. When working with graduate students, OSS community members should: