What’s integrity got to do with it? Second-year experiences of the Path2Integrity e-learning programme

The P2ILC is a proactive RCR programme and tackles RI challenges using the dialogical approach in two ways. First, the dialogical approach determines the overall learning objective of the programme: students “learn how to conduct a dialogue on the rejection or acceptance of norms in research integrity” (Priess-Buchheit et al. 2020a, p. 23). Second, this programme applies the dialogical approach in its methods by using storytelling, role playing, and coming to an agreement. In the P2ILC, programme trainers have three different series of RI learning cards that are targeted to the different student levels. The trainers can select single learning cards from each series matching the educational level of their participants. The learning cards are available online and free of charge. Path2Integrity has also offered workshops, in person and online, where the learning cards are used. The contents and methods of the programme are explained in the “1st year” publication (Priess-Buchheit et al. 2020a, 2020b) and on the website Path2Integrity.eu.

The P2ILC programme aligns with numerous studies, which document that role play increases classroom participation, reflexive and creative thinking, application of concepts, emotional engagement, and personal accountability (Löfström 2016; Grose-Fifer 2017; McCarthy and Anderson 2000; Poorman 2002; McWilliams and Nahavandi 2006; Poling and Hupp 2009). Studies have shown that specific exercises promote perspective-taking (Löfström 2012), and simulation (Wright-Maley 2015) and role play (Rosnow 1990; Strohmetz and Skleder 1992) help raise awareness of complexities in ethics; all of these are found in the programme. The P2ILC programme matches with the feedback from many scholars interviewed by Andorno et al. (2019, p.15), who said that the most efficient tool to learn research integrity is “the recourse to case studies combined with discussion. Cases can either be taken from real-life or may be fictitious. … Specific methods that were mentioned as helpful to improve the quality and efficacy of the teaching [in this study] are role-playing, individual and group presentations by students”.

As described in Priess-Buchheit et al. (2020b), the three series in the P2ILC programme are each supported by a portfolio of tools. For each series, a handbook explains seven to eleven learning sessions that support a culture of RI. The M- and Y-Series learning cards are for graduate and early-career researchers and enable students to rationally lay out their position on good scientific practice as well as the ways they use research in a responsible manner while understanding the research procedures and landscape. At the same time, the specific goal of the S-Series is for undergraduate and high-school students to understand the importance of research integrity’s criteria for society (Häberlein 2020). Even though the goal of the Y-Series is the same as the M-Series, the Y-Series was specifically designed for early-career researchers. The P2ILC programme is an open educational resource and can be used in secondary schools, universities, or within formal learning settings in adjunct areas.

The P2ILC programme recognizes that RCR training differs across disciplines and needs to be tailored to each target group. The revised version of the programme resulted from several drafting circles using trials and feedback for improvement. A Path2Integrity train-the-trainer programme has also accompanied the P2ILC programme since 2021 (Palianopoulou and Stock 2020).

The name P2ILC refers to the programme’s origin, a European Horizon 2020 project, and explains that this programme uses so-called learning cards, which each describe a RI/RCR learning setting. The programme was designed, evaluated, and revised in 2019 and 2020.

As described in Priess-Buchheit (2020c, p. 55), the aim of this RI/RCR learning programme is to overcome “[t]raditional methods of teaching ethics and research integrity, [because they] do not appear to be efficient in raising awareness on these issues”. Innovative teachings focus on a learner-centric approach in comparison with the more traditional teacher-centred approach of delivering the curriculum (e.g., lectures). In a learner-centric approach, students play an active role in shaping the learning experience.

The P2ILC programme follows a dialogical approach enabling its learners to “rationally lay out their position on good scientific practice as well as the ways in which one would explain and justify their position to others. As opposed to debate, participants are encouraged to build sound arguments by listening actively and (if necessary) countering good arguments” (Priess-Buchheit 2020c, p. 55).

The P2ILC programme, as an open-education source, allows trainers worldwide to use it for their teaching. Trainers can adapt the PILC programme into their different disciplines. The different learning cards and the accompanying handbooks explain the learning objectives for each card in detail.

Path2Integrity specifically tailored the learning objectives to the cards’ content and tasks. In 2019 and 2020, the developers integrated many ideas and comments from research ethics and integrity experts (see Priess-Buchheit 2020c). The P2ILC programme includes innovative methods, easy and fun tasks, and builds on the educational status quo.

While the original P2I programme (2019) concentrated on (responsible) research literacy and contained sessions regarding research procedures, research environment, research publication, data management, collaborative research, safeguards, and more, the revised programme adds sessions to support scientific citizenry. It entails sessions concentrating on reliable research and reliable research results in the S-Series. The umbrella in Figure 1 represents this expansion which took place from 2019 to 2020. The right side below the RI umbrella displays the researchers’ fields of actions in the context of RI, and the left side describes society’s perspective on RI. The right side displays the field of the learning cards first version (2019), and the left side presents the new learning cards’ fields (2020); a new series concentrates on scientific citizenry and contains the above-mentioned new learning settings in which students learn about reliable research and reliable research results from a societal perspective.

In the revised version, the P2ILC programme offers 27 learning sessions and concentrates on both research literacy and scientific citizenry. From 2020 onwards, the revised P2ILC programme has been available in printed folders and as e-learning units (learning-p2i.eu).

As Mulhearn et al. 2017 showed in their predictive modelling tool (PMT), RI/RCR training depends on many variables. “The PMT is a sophisticated tool that can be used to develop, assess and improve education for RCR” (Krom & van den Hoven 2021, p. 2). Watts et al. (2017) designed the PMT by modelling the instructional effectiveness of reasonable conduct of research education in a meta-analytic path-analysis and calculating effect sizes for different didactical factors such as use of guidelines, group activities, and different content fields. Mulhearn et al. (2017) described the validation and use of PMT. The PMT “allow[s] course developers and evaluators to make standardized predictions about the relative effectiveness of an ethics training course based on the findings of the path model.” (Mulhearn et al. 2017, p. 196–197).

We explain the didactical factors of the revised P2ILC programme in relation to the PMT’s main categories We use the PMT terms to describe the P2ILC programme and mark characteristics of the programme with (+) or (−) to indicate whether they predict a positive or negative effectiveness for the P2ILC training as indicated in Watts et al. (2017, p. 638).

As described in Table 1, the programme contains guidelines, principles, and codes, which students use to argue in favour or against specific scientific practices. The overall learning objective of the programme is to conduct a dialogue that aligns with the PMT variables: ethical awareness (−), consequences (−), motives (+), emotions (+), constraints (+), and forecasting (+).

Table 1 shows the learning content of the M-Series and exemplifies what students learn through specific learning objectives.

As shown in Table 2, the P2ILC programme uses role playing (+), small group discussions (+), and web discussion in the online version (+) next to storytelling and coming to an agreement. Also, students are asked to join large group discussions (−) and readings (−). Trainers select the sessions and tasks according to their students’ expertise.

Table 3 shows that the programme was designed for trainers who teach between 4 and 35 students onsite or between 4 and 300 students online. It contains the Y- and M-Series for students who are beginning to have an identity in their field of study and the S-Series for students who have not identified their field of study.

Most of the S-Series learning sessions refer to a story called “What happened at LONA Science Center?”. This case is introduced in the first unit and continues across different cards. The PMT predicts a positive influence using this case story on RCR learning because this story is neither realistic (−), complex (−), nor emotional (−), but it is short (+) and creates engagement when you add different cards. The story called “Hannah’s Protocol” in the M- and Y-Series follows the same methods. The P2ILC programme uses many grouped-based activities (−), often by using unspecific (+) fictional cases (+). Considering these details, the PMT sums to a positive practice characteristic for the P2ILC programme. The PMT also predicts that the negative P2I training characteristics, which are marked with (−) above, diminish or counteract its effectiveness.

How effective one instructional factor is in practice is highly dependent on how this factor supports the programme’s overall aim (Krom and van den Hoven 2021). With the overall objective of enabling learners with the ability to rationally lay out their position on good scientific practice, two of the negative P2I training characteristics may change their effect. We think that in grouped-based activities (−), as shown in Fig. 2, and large group discussions (−), students can learn how to conduct a dialogue as the discussion evolves. Furthermore, the negative P2I training characteristic readings (−) are presented in the Y-Series by discovering the relevant codes of conduct and principles to open the door to the self-policing scientific community. Although the PMT indicates readings as a negative effect, the P2ILC programme uses this activity to acquaint learners with the scientific community’s norms and values.

The 48 workshops were conducted from January to November 2020. The feedback collected from the 2020 workshops was based on the trainer’s perspective. A semi-structured questionnaire was used to better understand what worked well and to pinpoint challenges. Two questions were answered by the trainers “What difficulties occurred? (Which organisational, spatial, personal, linguistic, cultural, religious, or other problems, if any?)” and “what went well? (Which exercises made you and the participants particularly enthusiastic about the topic, and why?)”. We evaluated all comments, except for the general remarks from the workshops. Contrary to the first year, the feedback during the second year was collected solely from trainers, lecturers, and teachers who used the learning cards in their settings. The first two years of the learning program were used to continuously improve the program by the evaluation team. Feedback was collected from month 6 up to month 24. In addition to the trainers’ perspectives, the participants’ perspectives and learning development have been tracked continuously starting in month 25, and this will be discussed in a future paper.

Based on feedback from the first year, a few adjustments were made to the material (learning cards and handbooks). Consequently, since 2020 trainers have been better prepared to deliver the described training on RCR. In addition, during the second year of operation, the COVID-19 pandemic forced trainers and students alike to work or study remotely. New and different challenges arose due to the rapid shift to an e-learning format. While there are certainly benefits to working and studying remotely, the new challenges include a different attention span and finding different ways to engage students to develop different skills and to thrive.

Research suggests that hybrid learning and working will most likely become the new normal (International Commission on the Futures of Education 2020); therefore, the feedback not only gave insight into in-class teachings but also informed the evaluation of the suitability of the P2ILC programme when taught online. In this regard, one should note that the material was not originally developed for an e-learning format but was adapted to an online learning programme in 2020.

The feedback collected in the second year of operation contained 100 comments. To ensure clarity throughout, this paper uses the same categorisation in this analysis as was used in the 2019 analysis (Priess-Buchheit 2020c).

Comments were categorized as follows:

The third category “justified and valuable comments for the P2ILC programme” provided valuable insights for the second year of operation. They will be discussed further. In total, 19 comments were chosen.

After categorising the 100 comments, “justified and valuable comments” and “insights for further improvement” contained 18 comments (see Tables 4 and 5). Fourteen comments were chosen for further analysis, as shown in Table 4. Additionally, four comments were added to a separate table to show possibilities of further improvement, as can be seen in Table 5. The additional category “insights for further improvement” was added to assure the longevity of the program in case someone would like to continue this program and use its material over the three years of the project.

The comments from Table 4 were used for further evaluation and grouped into the following three categories: