On the individual and organizational capacities supporting impact assessment: the case of the Yukon Environmental and Socio-Economic Assessment Board
Abstract
Impact assessment (IA) processes rely on the ability of assessment boards and their assessors to gather, synthesize, and interpret knowledge from a variety of sources, making IA a knowledge-based activity. IA boards in northern Canada operate in a context that prioritizes pluralism, where Indigenous knowledge is a key element of decision-making and the ability of practitioners to interact with knowledge—research capacity—affects process effectiveness, credibility, and legitimacy. Drawing on common principles from existing research capacity frameworks, we identify the dimensions of capacity most relevant to more fully realizing inclusive impact assessment processes. We then examine the Yukon Environmental and Socio-Economic Assessment Board (YESAB) as a specialized environmental governance organization with assessors whose research capacity directly impacts process outcomes. Results show that while common dimensions across knowledge-based disciplines, such as sufficient resources (e.g., financial support), are often addressed in the YESAB context, others unique to IA, like contextual understanding, require further examination. The interaction between individual and organizational research capacity is a complex balance between investing in individuals and investing in organizational supports. The proposed framework facilitates multi-scalar supports for individual assessors and assessment bodies alike to navigate balancing technical and value-driven knowledge in assessments.
Introduction
Impact assessment (IA) is a widely adopted environmental governance mechanism that engages a variety of diverging perspectives and knowledge to identify potential positive and negative effects of a proposed development project (Noble and Press 2011; UNEP 2018). IA processes are generally organized through a designated primary assessment organization, which can be a centralized government agency, a sector-based agency, or a dedicated assessment board (UNEP 2018). More decentralized models of IA utilize assessment boards as a boundary-spanning organization tasked with negotiating a variety of agendas into specific recommendations for development projects (Partidario and Sheate 2013; Meuleman 2015). IA boards are responsible for gathering, interpreting, and considering evidence from a variety of knowledge sources to formulate recommendations prior to regulatory approval, making IA a distinctly knowledge-based activity (Noble and Press 2011; Bond et al. 2018a; UNEP 2018). However, the ability of these governance bodies and their employees to undertake the required knowledge gathering and management tasks—research capacity—heavily influences the effectiveness and legitimacy of the overall process (Marsh and Smith 2000; Pope et al. 2013; Howlett and Ramesh 2015; Bond et al. 2018b).
Research capacity is understood to be the ability of an actor, organization, or network to engage, produce, maintain, and use knowledge through individual and collective development (Cashmore 2004; Kolhoff et al. 2009; Nykvist and Nilsson 2009; Partidario and Sheate 2013; Wright 2014; Lonsdale et al. 2017). Research capacity therefore includes the ability to interact with knowledge beyond conventional academic or scientific research activities, which focus on knowledge production, to capture all interactions with knowledge and information. The concept has been discussed in professional fields that rely heavily on the transmission of knowledge between organizations and individual practitioners, such as public health and international development (e.g., Cooke 2005; Mugabo et al. 2015; Maag et al. 2018), yet in environmental governance the concept has rarely been engaged directly (e.g., van der Molen 2018; Rahman et al. 2019; Gustafsson et al. 2020). In this paper, we consider IA to be a knowledge-intensive governance mechanism that relies on the ability of the assessment body and their assessors to not only produce new information and knowledge, but also to maintain, use, and interpret the knowledge and information provided by other participants. Previous IA research acknowledges the need to address different perspectives, but tends to focus on the coordination of technical information rather than value-driven knowledge (e.g., Bond et al. 2018a; Darling et al. 2018).
In northern Canada, equal consideration of diverse worldviews in development activities is a public policy priority, with particular focus on IA as a mechanism to ensure inclusion in decision-making (UFA 1993; Sabin 2016; Southcott et al. 2018). There are three territorial review boards in Canada, all of which have generated interest among academics and practitioners since their respective establishments through regional land claims processes: the Mackenzie Valley Environmental Impact Review Board (Northwest Territories), the Nunavut Impact Review Board (Nunavut), and the Yukon Environmental and Socio-Economic Assessment Board (YESAB; Yukon) (e.g., Fitzpatrick et al. 2008; Gondor 2016; Peletz et al. 2020). These northern IA bodies are relatively well-funded through their foundations in land claims agreements, with a high degree of the legislative legitimacy that remains a challenge elsewhere, even in Canada (Bond et al. 2018a). Particular attention has been drawn to their consideration of Indigenous rights and title and Indigenous and Traditional Knowledge (White 2009). At the forefront of criticisms are challenges associated with the related process of territorial devolution of power and limitations on the flexibility of IA in considering nontechnical and nonscientific knowledge (Christensen and Grant 2007; Gondor 2016). While all three boards share foundations in regionally specific land claims processes, each has experienced an evolution in their roles as broader devolution negotiations and implementation unfold, including their approach to pluralism in IA processes.
Pluralism—the active engagement of the multitude of worldviews, approaches, and solutions to societal problems—is a major consideration in IA, where a variety of theoretical, methodological, and representational perspectives are included in assessment processes (e.g., Cape et al. 2018). Yet, little empirical work has been done to evaluate the extent to which assessment boards have the capacity necessary to realize desired outcomes, namely the active engagement of a plurality of worldviews to the benefit of sustainable development (e.g., Peletz et al. 2020). In this paper, we present the case of the YESAB as a specialized organization with employees whose capacity to interact with knowledge determines IA process effectiveness and legitimacy, and we use this case to test a multi-level, knowledge-focused capacity framework specific to IA. In what follows, we review the concept of capacity and present the conceptual framework that guides our analysis. We then describe the case study setting and the research methods employed, followed by the results and a discussion of their implications for IA policy and practice.
Conceptual framework
Capacity building (often termed capacity development or strengthening) is a longstanding and complex endeavor that remains ambiguous in practice, despite the variety of disciplines working to develop the concept and its application. The concept of “capacity” is broadly defined as the evolving combination of competencies (skills), capabilities (resources), and relationships at multiple levels (individual, organizational, network) that enables a system to exist, adapt, and function, which interact to enhance or limit overall capacity (Marsh and Smith 2000; Brinkerhoff and Morgan 2010; Howlett and Ramesh 2015). Recognizing the breadth and ambiguity of the concept, Potter and Brough (2004, p. 337) suggested that it is “as diagnostically useful to say, ‘there is a need for capacity building’ as to say, ‘this patient is unwell’.” A solid grounding for the concept of capacity therefore remains elusive in scholarship and application, with numerous calls for further clarification and refinement for specific contexts looking to solve capacity challenges, including IA (Harrow 2001; Howitt and Suchet-Pearson 2006; Boyd et al. 2013; Darling et al. 2018). Recognizing the multi-faceted, multi-scalar, and inter-related nature of the different types of capacity is of particular importance when conceptual frameworks are applied across disciplines and institutional contexts (Harrow 2001; Gadsby 2011; Nuyens and McKee 2005).
Interrelated capacities
Across academic disciplines, scholars have identified and categorized different types of capacity to try to add clarity for capacity-building efforts and address specific societal objectives (Darling et al. 2018; van der Molen 2018). In environmental governance literature, the most widely discussed types of capacity relate to “adaptive capacity”—associated with the ability of a group to be resilient to changes in their environment (i.e., Gupta et al. 2010); “governance and community capacity”—the ability of a group to make decisions and function (e.g., Laverack and Labonte 2000; Matarrita-Cascante et al. 2017); and “policy and institutional capacity”—the ability to provide legitimacy and support for governance functions in a group (e.g., Woodhill 2010; Krishnaveni and Sujatha 2013; Ramesh et al. 2016), which were drawn together by van der Molen (2018) under headings of adaptive, integrative, and regulatory capacity in environmental governance. Despite increasing emphasis on the need and intent to accommodate and consider different knowledge systems in environmental governance, little research has developed the knowledge dimensions of capacity, known elsewhere as research capacity (Natcher et al. 2005; Howitt and Suchet-Pearson 2006; Emerson and Baldwin 2019; Gustafsson et al. 2020; Spagnuolo 2011).
Research capacity relates to the other supporting capacities that influence the function of environmental governance mechanisms and is often identified as a sub-capacity that supports national and regional development (Velho 2004; Chan et al. 2005; Al-Roubaie 2010; Andrews et al. 2011). For example, Howlett and Ramesh (2015) and Howlett et al. (2017) outlined the dimensions of policy capacity that relate directly to knowledge as “knowledge system capacity.” When governance mechanisms, such as IA, rely heavily on evidence and knowledge, the research capacity of boundary spanning organizations can have an important influence on network-level outcomes (Marsh and Smith 2000; Velho 2004; Howlett and Ramesh 2015). Within IA networks, assessment boards interact intensively with a variety of knowledge sources, which requires certain competencies and capabilities particular to the IA context (e.g., Partidario and Sheate 2013; Bond et al. 2018a; Maag et al. 2018). Assessment boards generally gather input beyond the technical specifications of a proposed development project, including public comments and local and Indigenous knowledge. This requires a high degree of both individual and organizational research capacity, with direct implications for the performance of IA processes. As a result, there is a need to better understand the dimensions of research capacity in IA bodies tasked with the coordination and interpretation of supporting evidence and knowledge to inform best practices (Kirchhoff 2006; van Loon et al. 2010; Scott 2011; Bond et al. 2018b; Loomis and Dziedzic 2018)
Evaluating capacity in IA
Evaluating the state of capacity in any field relies on conceptual frameworks often applied across disciplines to identify potential gaps in individual or organizational abilities and then develop strategies to fill those gaps through interventions (Boyd et al. 2013). Generic capacity frameworks face the challenge of conceptual ambiguity and vagueness, leading to mismatched or imprecise goals and minimal intervention success when applied across widely varying contexts (e.g., Harrow 2001; Biesta et al. 2011; Gadsby 2011; Boyd et al. 2013; Nuyens and McKee 2005). These generalized capacity evaluations have already been applied to environmental IA systems, identifying larger systemic issues in developing countries where national governments are largely in control of IA processes and the process itself is often not well established (Doberstein 2001; Kirchhoff 2006; Kolhoff et al. 2009; van Loon et al. 2010; Kolhoff et al. 2018; Khosravi and Jha-Thakur 2019). Specialized frameworks have concentrated on individual or organizational levels of capacity, but rarely connect the two (e.g., Gupta et al. 2010; Hamel and Schrecker 2011; Gustafsson et al. 2020). The spectrum between nuanced qualitative elements, such as mentorship, and highly specific quantitative components, such as financial support, makes balanced evaluations rare (Fig. 1). The result is a lopsided emphasis on the more easily measurable elements of capacity (Cooke 2005; Bates et al. 2006; Howlett and Oliphant 2010; Armstrong et al. 2013; Kislov et al. 2014; Maag et al. 2018). In Canada, there is also growing emphasis on the acknowledgement of pluralism in decision-making that will require a better understanding of the qualitative dimensions of capacity in individuals and organizations, such as sense of place or sanctity (Udofia et al. 2017; Arsenault et al. 2019). As a result, there is a clear need for a refined understandings of capacities particular to IA and how they interact.
Fig. 1.
We reviewed and compared 24 capacity-specific conceptual frameworks drawn from six disciplines, with an emphasis on those specific to environment and research capacity, to identify common elements and then customize for IA in northern Canada (Appendix A). Using pattern coding, we identified common characteristics between frequently used and cited capacity frameworks (Appendix A). When the principles of IA are considered, the suitability of these foundational aspects of capacity for application in IA is limited to the standard operational aspects, as their interpretations tend to leave dimensions of capacity particular to IA unacknowledged (Boyd et al. 2013). To refine these characteristics, we looked to the foundational principles behind IA, where context, meaningful consideration, and participation are key elements (Natcher et al. 2005; Noble and Hanna 2015; Udofia et al. 2017). We identified three broad themes to incorporate both common and IA-specific capacity constraints: infrastructure, administrative support, and knowledge demand. These higher order themes are supported by sub-capacities that are also interrelated, where more obvious components, like financial and human resources, co-exist alongside more distinctively IA components, such as disciplinary versatility and contextual understanding. In what follows we examine these themes in more detail using a case study of the IA assessment board in the Yukon Territory, Canada.
Study setting
The Yukon Territory rests at the apex of the Rocky Mountains at the junction between the Canadian Shield and the Pacific tectonic plate, which allows for a concentration and wide variety of geological formations, and therefore unique resource opportunities, in close proximity to each other. The first inhabitants of the area, now known as the Yukon First Nations, arrived ∼24,000 years before present (Bourgeon et al. 2017). They remain a driving force behind the political and economic evolution of the territory, making up ∼23% of the 41,352 people in 2019 (YG 2020). Located in the northwest corner of Canada, between Alaska, Northwest Territories, and British Columbia, the Yukon shares peoples, cultures, and language groups with other modern jurisdictions.
Land claims and self-government agreements, established under the context of an overarching Umbrella Final Agreement in 1990, form the fabric and foundation of Yukon governance mechanisms, including interactions between the territorial and federal governments. These agreements look to establish greater local control over regional social and economic development by addressing topics such as wildlife, lands, heritage, and resource management (UFA 1993). Chapter 12 of the modern-day treaty agreement establishes the impact assessment process (referred to in the agreement as development assessment) and outlines the connections between regional and territorial advisory and review boards as well as connections with complementary processes such as land use planning. The result is a federally legislated decentralized impact assessment process under the 2003 Yukon Environmental and Socio-Economic Assessment Act that supersedes other similar processes, with jurisdiction over all development projects undertaken in the territory (Government of Canada 2003). The specialized assessment body YESAB was subsequently established in 2006. Assessors are embedded in local contexts by dividing the territory into six districts with local Designated Offices (DO) assessing small-scale projects. In YESAB, both Assessment Officers (AO) and Managers of the Designated Office (MDO) are active assessors and share similar responsibilities. Larger development projects of a certain scale, complexity and political consequence are assessed by the Executive Committee (ExComm) office and are usually large multi-year projects with potential for territory-wide social and economic impacts.
Methods
Data collection
We conducted 27 key informant interviews with former and current YESAB assessors, managers, and administrators located throughout the Yukon Territory. We maximized for variation to capture the largest diversity of related experiences and perspectives possible (Baxter and Eyles 1999). Potential key informants were identified first by using the existing organizational directory, with additional key informants identified through publicly available annual reports, in combination with snowball sampling. Snowball sampling takes advantage of informal relationships amongst key informants to identify additional potential participants, thereby shifting some control over sampling towards participants (Creswell and Clark 2007; Noy 2008). Potential informants were invited to participate if they were employed by the assessment body for more than one year and were provided specifics around participant rights and anonymity within the study as a basis for informed consent prior to interviews. Community saturation was considered fulfilled once all current assessors had been approached for an interview and no additional key informants were being identified through snowball sampling.
Semi-structured interviews lasted between 60 and 90 minutes, allowing informant perspectives to develop conversationally while maintaining a semi-formal structure (Sovacool 2010; Seidman 2013). Well-developed frameworks specific to evaluating research capacity at the individual and organizational levels provided the broad guiding themes to direct the conversation (Cooke 2005; Maag et al. 2018). Prior to data collection, the McGill University Research Ethics Board reviewed and approved all data collection protocols (#127-0717). The Yukon Scientist and Explorer’s permitting process also reviewed and approved data collection activities (license # 6800-20-1099) and a research agreement was put in place between the first author and YESAB.
Analysis
Interviews were transcribed based on audio recordings, then coded iteratively using the qualitative analysis software NVivo, with individual participants assigned a random four-digit identifier to preserve anonymity. First, structural coding was used to identify potential overarching themes with regards to research capacity, knowledge management, and assessor perceptions. Then, provisional coding was applied using the common capacity framework characteristics identified in existing evaluation frameworks (Appendix A). Based on a combination of elements drawn from both broad capacity and research capacity frameworks, the data were then re-examined for element presence, level of relevance (individual or organizational), and the perception of their existence in those contexts (Saldaña 2015). Elements that appeared repeatedly, but were not included in the existing frameworks, were noted for later examination. Finally, the most relevant quotations from participants were identified to illustrate perceptions and element presence.
Assumptions and limitations
This study assumes that former and current employees and board members of YESAB are known and identifiable to each other. There are, however, limitations because of the relatively small and dispersed nature of YESAB as an organization, which limits the potential sample size of former and current employees and board members (27 current and 60 former). As such, our focus was on current assessors that had been employed for at least one year, to ensure a minimum amount of experience within the organization. Data collection protocols, in the form of semi-structured questions, were piloted with one former and one current assessor to minimize potential researcher bias and enhance reliability. Similarly, preliminary findings were discussed with a sub-group of advising IA practitioners, particularly those who were current and former YESAB assessors, to further enhance the reliability and trustworthiness of our analysis through member-checking (Yin 2003).
Results
There are many ways to organize the different dimensions of capacity for analysis, but previous frameworks limit their acknowledgement of the interconnectedness of the different elements and their qualitative aspects (Boyd et al. 2013). Often one aspect of capacity (research or otherwise) at the individual level is directly related to and impacted by organizational- or network-level limitations (Howlett and Ramesh 2015). The following analysis does not aim to be a comprehensive evaluation of the assessment body, but rather focuses on the strengths and challenges as identified by key informants, using the guiding elements of IA-specific capacity illustrated earlier. Interconnections between different dimensions and with other capacities are acknowledged where possible and explored where necessary, but for clarity and concision we focus primarily on the dimensions presented in Table 1: infrastructure, administrative support, and knowledge demand. Figure 2 illustrates these dimensions along the qualitative/quantitative spectrum. Similarly, the analysis prioritizes challenges, if only to offer potential solutions, even while there are well-established areas at YESAB that can be drawn out as examples of strong research capacity.
Table 1.
| Infrastructure | Organizational scale components, including physical infrastructure |
|---|---|
| External coordination (relationships) | Mechanisms, institutions, and policies that support interaction with other organizations and stakeholders, also includes aspects of physical infrastructure outside of YESAB purview |
| Internal coordination | Connected to organizational culture—mechanisms, institutions, and policies that support interaction between assessors, DOs, and the Board, includes aspects of organizational knowledge and continuity through turnover. Also includes digital and physical mechanisms to facilitate the introduction of new knowledge, including subscriptions to journals, internal library resources, past consultant reports, etc. |
| Finances | Sufficient financial budget to maintain infrastructure, including competitive salaries, office needs, training opportunities, etc. |
| Legitimacy | Internal (cohesive vision): Mechanisms, institutions and policies that support a cohesive understanding of the legislation and mandate of the organization, meant to address internal biases |
| External (perception): Mechanisms, institutions, and policies in support of minimizing perceptions of organizational bias | |
| Knowledge demand | Skills and knowledge required for an individual assessor to be able to complete assessments |
| Technical expertise | Knowledge and understanding of the science and technical aspects of a project |
| Project management | A generic skill set associated with time management and relationship coordination |
| Versatility (multi-disciplinary) | An ability to engage with a variety of disciplines and types of knowledge including quantitative and qualitative data, local and Indigenous knowledge, and concepts from sciences, social sciences, health, and economics. |
| Contextual understanding | An understanding of the legislative and institutional context within with assessments are completed. |
| Administrative support | Components that relate to the daily operations that incorporates both individual and organizational aspects |
| Mentorship | The availability and ability of supervisors and colleagues to mentor new and inexperienced assessors |
| Opportunity | The availability of opportunities for professional development (i.e., training, workshops) and internal advancement (i.e., promotions, more responsibility) |
| Dedicated time | Time devoted to accessing new knowledge, broadening understanding and analysis of different knowledge types |
| Culture | Organizational institutions that develop into daily common practice, where knowledge sharing and innovative thinking from all employees is normalized and encouraged. |
Fig. 2.
Infrastructure
We use the term infrastructure to describe dimensions of capacity that are primarily organizational that support research capacity in the IA context. This includes themes of external and internal coordination, as well as perceptions of legitimacy to operate. Coordination of any kind is a shifting challenge across scales and actors, as resources and connections fluctuate and as perceptions of legitimacy, including “acceptable” evidence, evolve. These components require adequate policy and procedural capacity, as well as knowledge management structures, to be fully realized.
External coordination
External coordination for YESAB is multi-faceted, approached through both physical and digital infrastructure that relies on capacity at both the individual and organizational levels. YESAB has a decentralized model for their physical infrastructure, where DOs are established in communities central to the district with the intent of embedding assessors in the local context and facilitating external coordination and outreach. The decentralized model introduces both advantages and challenges into YESAB’s overall operations. For example, situating assessors in communities does make them more accessible to the local community and the proximal First Nation, which facilitates the maintenance of external relationships through established lines of communication and outreach activities. The reliance on face-to-face and informal communication with outside organizations makes DOs the most viable option for maintaining the local area network and ensuring the local connections are made and maintained. Participant 1214 described the advantages of being embedded in a community:
“There’s so many local keepers of knowledge that to have the [Designated Office] in that area aids in information sharing…. It’s nice to just be in the community and be known and people are more comfortable sharing that information.”
However, maintaining a presence in smaller communities has limitations associated with external factors, such as housing availability or long-term employment for spouses, which leads to challenges with filling assessor positions. Participant 0901 summarized:
“We have staff that have lived out of tents and in the back of their cars. We do also get complaints sometimes about ‘you’re hiring all these new people right out of school with no experience’ and frankly, sometimes, I feel like saying, ‘Yes, we’re lucky we got anybody’.”
The external factors were seen as limiting the number of applicants for available positions, the candidates that eventually accept the positions and how long they stay, and were considered particularly relevant in the more rural communities.
Generally, participants felt that YESAB had limited interaction with the public beyond infrequent public meetings and the comments received in the official “Seeking Views and Information” phase of assessments. Participants described this passive approach to outreach as being a challenge, where interactions with the public are mainly through individuals coming into the DO, formal public open meetings, open houses, and notifications for public comment posted at the grocery store or post offices. On a few occasions, YESAB has offered short courses introducing the assessment legislation and process have been presented to First Nations governments and the public, but these one-time events are based on the stability, availability, and the motivation of individual assessors. Participant 1248 described the challenge of communicating with the public regarding their comments:
“A lot of the time, people will have these comments, and we can’t take them into account for whatever reason because of the act or a technicality, and there’s no way to really interact with those people to let them know, and so the learning opportunity is not there.”
Many participants felt that increased outreach might help with concerns around legitimacy as well by clarifying the role and mandate of YESAB and improving the quality and usability of submitted comments.
On a related note, comments received from different actors in the IA network are often based on the work of consultants. This can lead to what participant 1006 termed the “dueling consultant problem,” where First Nations and proponents both hire consultants to address the same topics and present opposing conclusions which then translates to difficulties for assessors in terms of interpretation. As participant 1006 explained:
“There’s no linking together and saying, ‘Okay, how can we get not only the better bang for our buck? Are we asking the same questions? Are we actually talking about apples instead of apples and oranges?’ ”
YESAB was seen as potentially holding a unique position to contribute to the facilitation of resource coordination between external parties while also benefitting from this developed functionality.
Another aspect of external coordination is the dissemination of information and knowledge related to project applications. The YESAB Online Registry (YOR) is an online database of all past and current applications for development projects along with the supporting correspondence, information and knowledge, and the public comments submitted for consideration as part of the assessment. One main intent behind the YOR is to make all information and knowledge associated with assessments publicly available to enhance the transparency of the process, but limitations have been identified. As of 2019, YESAB was working to resolve aspects of this acknowledged shortcoming, confirmed by Participant 0901, “We’re in a process of creating a new online registry which will allow us some better search functions, things like that.” In response to public and proponent feedback, YESAB updated the YOR to increase accessibility in 2019. Past variations of the YOR were criticized as inaccessible (i.e., incompatible file formats, awkward search capabilities), which limited cross-application between similar projects. With the heavy reliance on previous assessments and the associated information and knowledge possessed by assessors and employees, participants viewed this upgrade positively, with the potential to increase knowledge exchange, as well as better supporting cumulative effects assessment.
One exception to the view that more knowledge exchange is better is with respect to Traditional Knowledge, which often contains sensitive information such as the location of sacred sites and traditional hunting grounds. These types of submissions have specific confidentiality policies and procedures outlined in the legislation to maintain a respectful approach to interactions with First Nations cultural heritage. Participants discussed perceived tension with proponents linked to limiting access to sensitive information, who have expressed frustration in their lack of access to many of the participants.
Internal coordination
Participants at all levels of the organization identified internal coordination, mainly centered on data management, as an ongoing struggle, both between and within levels. Participant 1439 summarized these challenges:
“Often I’ll be working on policy issues, and I’ll go to do some word searches on our ‘G’ drive, and I’ll be like ‘holy crap. This is what we’re trying to fix. We already did this in 2015 […] We had a bunch of meetings, and this was the outcome, but it hasn’t been built into our current [operations].’ ”
More established respondents generally felt that time was often spent revisiting solutions that have already been considered and discarded, which they felt led to wasted time, energy, and resources.
Better internal knowledge management has the potential to facilitate internal policy development, maintain external relationships, and support knowledge mobilization. The digital and administrative infrastructure required to support policy work was generally seen as lacking, as summarized by participant 1302: “YESAB hasn’t been very good at documenting operational practice, and that’s really what people need to know to do their jobs.” The organization added a policy-specific position in 2011 and another in 2016, a move that some participants saw as a positive partial solution, though assessors still maintain policy responsibilities alongside their assessment obligations. The other major challenge concerning internal knowledge management was identified by participants as staff turnover, particularly when knowledgeable assessors exit positions or take leaves of absence. Numerous participants identified that an ideal part of training new assessors would involve some overlap with the exiting assessors so that the incumbent could provide context and introductions to the major actors in a community, but this is often not possible. Participant 1248 described the need for more formalized knowledge management: “it’s more of a hope that the knowledge has been transferred enough… I don’t think that people are being tapped as resources effectively when they’re leaving.” Mentorship around the generic components of the assessment process can come from other offices, but without the local overlap, previously established relationships are lost. This interruption in connection between organizations can lead to wasted resources, as initiatives or projects proceeding without adequate contextualization can revisit previously identified problems that went undocumented. Participant 1214_1 articulated:
“Every time someone leaves, there is a lot of information that leaves with them and the history, because that doesn’t get captured anywhere, the history of why we do things the way we do.”
Respondents also described a reliance on previous assessments as a major knowledge source, with limited new knowledge being introduced over the course of new assessments. Access to new knowledge was seen as restricted by the amount of dedicated time available to assessors for seeking new relevant resources and the availability of “outside” knowledge. As previously identified, the internal knowledge management system that does exist was seen as having limited utility in the day-to-day operations of assessors. For example, a small internal repository of useful resources relies on individual assessors for contributions. Participant 1248 remarked on the state of the repository, “Our library needs to be fed into. It’s not really being fed into that much.” Participants reported having little dedicated time to identify pertinent and up-to-date resources. Instead, there was a reliance on previously completed assessments to provide the basic resources for assessments and training. There was also a reported tendency to rely on maintained informal connections to former assessors. Participant 1248 pointed out that this is not a guaranteed resource by saying, “I think there’s this weird idea that when the people are gone, you can call them for information.” In a sense, there is an assumption that the corporate knowledge is not lost but has transferred elsewhere within the broad informal IA network and is therefore still accessible to the assessment board. Many former assessors have moved on to positions in the Yukon Government or one of the First Nations governments, so this assumption is not completely false. However, an over-reliance on accessing previous employees potentially diminishes the impetus to develop appropriate corporate knowledge systems.
Another concern relates to the public availability of academic studies that are housed behind publisher paywalls. In the mandate of YESAB, it is explicit that all information used in assessments should be publicly available, which leaves academically published research in a questionable position. Participant 1355:
“A potential issue around that that’s never really been tested is to what level are we as an organization, as assessors allowed to review and use third-party information like that, because the basis for our assessments is that the information is public and publicly available and transparent.”
With a large portion of academic literature held behind paywalls and therefore unavailable for public consumption, the argument could be made that this should limit how much those knowledge sources should be engaged. The perceived result is a reliance on previous assessments and comments for the majority of the knowledge supporting assessment recommendations.
Financial resources
Participants identified financial resources as a strength of YESAB, with assessors being well supported to pursue professional development and attend meetings and conferences. The assessment body receives full financial support from the Government of Canada, as part of the Umbrella Final Agreement, administered by the Yukon Government. Accessing training and professional development funds is a common element in nearly all capacity frameworks, and YESAB appears to have the financial resources to support their employees. However, financial resources alone can’t guarantee functionality, as participant 0901 explained:
“You can’t just throw money and more staff at it, because well there aren’t really more staff out there and there’s no housing. You have to continue to look for more longer-term sustainable solutions, and that usually comes down to how you work.”
Legitimacy
Questions of legitimacy were seen as an ongoing challenge for YESAB. Numerous participants identified the perception of neutrality (or lack thereof) as both an external and internal challenge that has ramifications for the legitimacy of the process. External perceptions of bias were discussed from the perspectives of proponents, specialized interest groups, and First Nations governments. As participant 1006 articulated:
“If they [the public, proponents, First Nations] think that staff is undertrained, inexperienced, biased, or using technical information or research to further their own goals, then that’s not helping the organization at all.”
Some of these viewpoints can be managed by establishing and reiterating a clearer understanding of roles and responsibilities of YESAB and more specifically assessors. Externally, transparency has been established by making assessment documents publicly accessible. Questions of consistency between assessors and DOs have been addressed by developing and publishing organizational methodologies and procedures for approaching the various phases of the IA process. Concerns around assessments completed in nonlocal DOs have been raised and are acknowledged by YESAB as an ongoing concern, which they are looking to address through increasing transparency and providing deeper explanations around methodology.
Ensuring assessors maintain a clear understanding of their role in the context of IA was seen as essential to maintaining organizational legitimacy in the eyes of the public. One participant identified the perception of bias as being particularly challenging in the early days of IA, when industry had the impression that assessors held a decidedly environmental protectionist approach to development.
This challenge has led to the development of public documents and internal discussions about maintaining the external perception of neutrality around YESAB and the IA process. Participant 1006 continued: “The organization tries to, certainly through ongoing conversations and training, say ‘This is supposed to be independent, unbiased, review of the facts.’ ” This position was echoed by Participant 1302 in the context of informal connections that contribute to “…trust that conclusions are being made that are unbiased and fair, based on relevant information.” However, the position of neutrality also introduces some hesitancy, as described by Participant 1439: “YESAB is supposed to be neutral and unbiased, maybe that plays into a little bit into concerns about doing too much outreach or meeting with certain YESAB participants.” This conceptualization of neutrality also does not acknowledge the conventional understandings of evidence, facts, and knowledge.
Knowledge demand
Participants reported a suite of demands being placed on the organization and the individual assessors directly related to knowledge. The most apparent was the demand for technical expertise around environmental constraints, thresholds, and regulations, followed by project management skills. Under-acknowledged in both implementation of assessments (as identified by participants) and in research capacity frameworks is versatility—necessary in a multidisciplinary context—and contextual understanding—particular to research capacity in IA. A combination of organizational and individual-level research capacity is therefore required to meet the knowledge demands placed on an assessment board.
Technical expertise
The major knowledge demand placed on the assessment board, from an organizational perspective, is to maintain a suite of assessors with the collective technical expertise to broadly understand the specifications of any application that is submitted, along with an understanding of the contextual factors. This need can create difficulties when intersected with other challenges associated with filling and maintaining assessor positions, such as housing and living in remote communities. The majority of assessors who participated in our research reported having science-related backgrounds that lend well to the environmental aspects of assessments. However, the skills and experience needed to interact with the more qualitative and Indigenous knowledge aspects of IA were seen by participants as needing further development. The result is an unintentional focus on the environmental aspects of assessments, according to participant 1302:
“I think the environmental components often overshadow the social and economic unless the projects are occurring in a really, really sensitive area or if it’s going to have really major impacts.”
The focus generally remains on more quantitative aspects of assessments with rare exceptions.
From the individual perspective, the assessors reported focusing on generalizable skills, with the ability to interpret the knowledge and information presented to them. Participant 1302 explained:
“We’re not experts and we don’t know what areas were used historically for cultural purposes, for example. That information, the cultural impacts, the social impacts on First Nation communities for projects – We just don’t generate that internally … we try to go local and get it, but there’s no experts within YESAB who are generally qualified to speak on those issues.”
The reliance on local organizations and the local public as the source for contextual knowledge regarding potential impacts can be problematic, especially when local organizations experience their own research capacity challenges. Similarly, public input is directly related to the amount of public awareness about the process, ongoing projects, and their role in the process, so the level of outreach organized by DOs can influence how much the public interacts with the IA process.
Project management
Participants identified individual assessors as being responsible for coordinating the acquisition, documentation, and eventual interpretation of the information and knowledge for the assessments assigned to them. This requires more generic skills associated with project management, including timeline management and general administrative tasks such as coordinating meetings and outreach that are the basis for relationship building. In particular, the relationship-building and maintenance aspect was seen as central to the assessor position. Participant 1302 explained:
“I think the real skill that people need to have working for YESAB is the ability to generate and maintain informal relationships because there’s really not a lot of formal opportunities for information sharing and relationship building outside of a specific project.”
This was a particular concern when internal or external staff turnover occurs without overlap, as those relationships then need to start anew.
Versatility (multidisciplinarity)
Another important skill that assessors were seen as needing to develop related to versatility, meaning the ability to fluidly interact with both quantitative and qualitative data from across disciplines, skills that have been under-acknowledged both in IA and in research capacity contexts. Participants referred to themselves as generalists, or knowledge brokers, and that this role comes with challenges that can be met by developing an individual’s versatility. Assessors are required to develop, with minimal direct training, the ability to engage and interpret different forms of knowledge, such as health and social aspects of a project, as well as perform qualitative analysis of the public and interest group comments to identify the major valued components. Participant 1038 acknowledged the challenge: “we want to call it [IA] science but it’s like an art… And it’s a real struggle because it’s a very different form of learning from the traditional [forms].” Some assessors have approached this challenge by attending workshops led by First Nations and similar offerings from other sources that introduce different worldviews and ways of identifying community values. There appears to be room to expand the spaces for assessors to explore their own biases and develop versatility both between disciplines and worldviews that would facilitate their interactions with, and interpretations of, qualitative data and Indigenous knowledge beyond understanding the context of land claims and self-governance.
Another underacknowledged skill connected to disciplinary versatility was the ability to communicate in diverse ways with diverse audiences, including clear argumentation alongside layperson interactions. The interpretation of evidence was seen as fundamental to assessments and transparency around the process, as recommendations are founded on the information and knowledge submitted for consideration. As expressed by participant 1355: “What we always have to come back to as assessors is: based on the information you have, is your conclusion supported through your argument? If it is, then that’s fine.” In the same instance, the assessor is also the public face of the assessment board in the community, fulfilling an outreach role. As expressed by participant 1733: “We are expected to communicate with external stakeholders, with organizations and we are expected to maintain these relationships, and that’s explicit in both [AO and MDO] job descriptions.” The ability to shift between the formal writing and argumentation and informal community interactions is an important knowledge brokering skill that individual assessors develop to support YESAB’s research capacity.
Contextual understanding
The physical infrastructure of YESAB is designed to embed assessors in the community in closest proximity to the projects being assessed. The intent behind this structure is to provide space for assessors to connect with more thoroughly and better understand local values and potential impacts of projects. Participant 1355 identified the (rare) assessor who has returned to the community after leaving for post-secondary education as “…good ones to have because they know the communities, they’re part of the communities, they would like to stay part of the communities.” Tension was reported between the level of personal involvement assessors want in their communities and not overwhelming community organizations with contact. Participant 0901 framed it as a potential criticism either way: “You always hear this; you should be more involved in the community but […] they also don’t want you bugging them all the time.” Assessors who strike this balance are often more successful at building and maintaining community relationships, but many assessors also acknowledged that this requires concentrated individual effort with varied success.
The challenges around contextual understanding connect back to infrastructure, workload, and turnover. Community infrastructure in the smaller communities, most notably housing, exemplifies challenges felt in other dimensions, where contextual understanding is fostered through living in a community but limited by the ability to maintain a presence in the community. Workload is a similar impediment, as participant 1_1439 explained:
“… we have the busy assessment season that can inhibit our assessors’ time to contribute to going out and doing YESAB one-on-ones or attending conferences…I think that’s a barrier for generating and sharing knowledge.”
A related challenge connects back to the previously raised issues associated staff turnover, which also limits contextual knowledge transfer.
Administrative support
Administrative support refers to the organizational and individual abilities to contribute to day-to-day organizational functions. Factors related to this aspect include reliance on mentorship as the main training mechanism and opportunities for professional development as well as for career advancement within the organization. Dedicated time to access new knowledge and the development of an organizational culture of knowledge exchange are also part of the administrative support aspect.
Mentorship
Mentorship was reported to play an important role in YESAB, acting as the primary training mechanism for new employees, including assessors. When a new assessor is hired, the main avenue for training the recruit is through mentorship under the guidance of an MDO. The type and amount of training received is highly variable and heavily dependent on the availability and management experience of the individual MDO. Participant 1302 explained:
“… it’s just an ad hoc approach to how they’re being trained. Some people are just getting thrown into the fire. Some people it’s more methodical and they’re starting at the beginning, looking at the context and how it [YESAB] was created. Others are just going right into doing assessments and just learning as they go. Not a very consistent approach.”
Relying on mentorship as the primary orientation mechanism was seen as introducing inconsistency across the organization and the development of individual assessors, which is then perpetuated as those assessors go on to mentor others, as pointed out by Participant 0301_1439: “If there were issues in how those [previous assessments] were done, they can carry into people’s future practice.” With such reliance on mentorship for maintaining operational continuity, support for mentorship work as part of a position’s responsibilities becomes especially important. Participant 1449 articulated: “We have a lot of trouble, I think, with getting people up to snuff in terms of really understanding what it is they’re doing…. There is quite little support for staff in training other staff, at present.” When training and mentorship are combined with their other duties, including completing their own assessments, reviewing all assessments administered by their office and potentially policy development work, the ability of MDOs to fulfill all these roles becomes strained. As participant 1038 pointed out: “…if someone is floundering, I’m not sure that... there’s anybody to catch them.” Without organizational support or formalized training, turnover becomes more likely. Many participants identified that the effort and time for an assessor to become fully and independently functional is extensive, so to lose a potential long-term employee due to a lack of initial and ongoing support was viewed as being costly to the organization and mission as a whole.
Professional development opportunities
Organizationally, YESAB is well-structured to provide support for training opportunities to bolster individual expertise, such as through workshops, meetings, short courses, and conferences. Participant 1248 noted: “they’re really doing a good job of pushing different training opportunities, and there’s a budget for it.” However, it is on the individual assessors to self-identify gaps in their knowledge-base and seek out the means to fill those gaps. Participant 1214_2 pointed out limitations around accessibility: “the HR department has all the training that people have gone to, but me as an employee, I can’t see what the people have done, … so I can’t ask them questions.” While there are mechanisms for sharing back to colleagues after external training experiences (i.e., training reports), these were characterized as limited in their availability and utility. The main avenues for capturing the information gained through individual training was presenting back to the main group of assessors and completing a short form that is then stored on a shared drive. Some participants expressed frustration with this, such as participant 1248: “…it’s not useful to be growing people individually if they’re not sharing it when they come back.” The connection to internal dissemination and knowledge sharing becomes important, as the opportunities afforded to employees in terms of outside training could have broader collective utility in support of overall research capacity.
Providing opportunities for employees goes beyond training and professional development, particularly the opportunity for career advancement and individual challenge. YESAB is a relatively small organization with a flat hierarchical structure, so there is limited room for an employee to advance professionally in their communities and beyond. For example, in a DO, there are only three levels of employment: administrative assistant, AO, and MDO. Participant 1214_1 explained: “If there’s already a manager that’s there for a long time, there is no opportunity to move up unless you move out or go to Whitehorse and then become an ExComm assessor ...” The prospect of staying in the assessor position with little chance for promotion left some participants feeling as though they became stagnant in terms of career development and cited this as a reason for seeking out other opportunities. The importance of having upward mobility was underlined by Participant 1038: “I looked around and I had no mentors… ultimately, I felt like this was it, it was me…. I plateaued. There’s no place else to go.” Orienting professional development as part of organizational culture supports the development of a challenging, yet supportive environment, and is also connected to the mentorship aspect.
Dedicated time
A dimension that underlies most of the other dimensions is the idea of dedicated time for the required aspects of the various positions. In other contexts, this would refer to time that is strictly reserved for research activities. In the context of IA, this would refer to time for expanding individual and organizational knowledge bases, explicitly built into job descriptions and implementation, but perhaps lacking because of operational constraints. Participants identified that mentorship, external relationship-building, and policy work were built into their job descriptions. For example, participant 1449 remarked: “…on my annual performance review, there’s a line about advancing our understanding of assessment and internal capacity…. Most senior staff are working on these projects.” However, limited dedicated time is afforded to the development, documentation, and upkeep of guidance for new and returning assessors, which extends time spent on orientation. Much of this limitation stems from the volume of assessments that a limited number of assessors are completing annually, which then limits the time available to devote to nonassessment work. Participant 1439 expressed: “I don’t think we always have a ton of time to do self-reflection and [evaluation report] audits,” while participant 1214_2 pointed out that “usually, it’s your manager who is your source of guidance.” Without dedicated time to develop broad guidance documents, there are few mechanisms other than having AOs going to their MDOs directly for guidance.
Organizational culture
Many of the reviewed capacity evaluation frameworks reference the development of an organizational culture that leads to information and knowledge exchange that then fosters creativity and problem solving, along with supporting the development of organizational knowledge. Overall, participants agreed that YESAB fosters such an environment, where the main force behind internal exchange was asking questions, but there were few formal processes and limited documentation. Participant 1439 articulated:
“YESAB has a pretty open-door policy. All the managers, if you have questions or you need to sit down and work through something with someone, they’re usually pretty available and don’t always have the answer but give you some direction and some key points to consider in making your decision.”
This idea connects to the dimension of mentorship, where the availability of managers impacts the culture of exchange, making support for individual mentors central to how the organization functions. The other aspect is fostering an environment where assessors feel supported and comfortable interacting with their colleagues and supervisors. Participant 1248 described the environment as: “you have different tangents that you can go on that people will follow you, rabbit holes that I think will be really helpful.” These explorations help develop both the individual assessor’s ability to perform assessments and organizational knowledge, as different issues are brought to the forefront for discussion and multiple perspectives on said issue can be explored collectively.
The converse of this dimension is that the need to ask questions is compounded by limited documented guidance for assessors. Participants identified that the process for documenting internal procedures were limited and underutilized, which can lead to inconsistencies, misunderstandings, and increased demand for training and mentorship. As participant 1439 described: “I’ve heard clearly from day one that there’s a lack of guidance. Even if people go away for a year […] and come back, it’s like what’s changed?” Organizational culture is central to operationalizing other dimensions, as witnessed by other connections drawn in preceding sections, but in the context of YESAB have been found to rely on individual connections, motivation, and knowledge for cohesion across assessors and employees.
Discussion
In examining the interrelated capacities affecting YESAB, the main assessment body in the Yukon Territory, a variety of challenges related to general capacity building appear to have been met, while others particular to IA and societal goals of pluralism have arisen. This observation leads us to identify some insights we consider relevant to IA more broadly. The interaction between individual capacity and organizational capacity are intertwined in a way that makes developing research capacity a complex balance between investing in individuals and investing in organizational supports. This finding supports the previous research on capacity building generally, where multi-scalar development is considered a necessity to establish long-lasting sustainable change (e.g., Velho 2004). For example, without improved internal knowledge management infrastructure, the individual dimension of “disciplinary versatility” we have identified, known elsewhere as “close to practice” (Cooke 2005) or “evidence management skills” (Kislov et al. 2014), would not easily contribute to organizational knowledge. Similarly, organizational culture oriented towards the continuous acquisition of new knowledge, including facilitating the pursuit of pluralism by engaging multiple knowledge systems, is only useful in so far as the individual assessors participate. In the case of Yukon Territory, reliance on past assessments for access to relevant information and knowledge is being perpetuated by perceived necessity, as workload constraints diminish the time available for assessors to seek out new approaches, information, and knowledge. The idea of “dedicated time” is captured in research capacity frameworks to support the acquisition of new information and knowledge (e.g., Hamel and Schrecker 2011; Maag et al. 2018). Building organizational infrastructure to support “dedicated time” can improve “disciplinary versatility” by increasing the diversity of knowledge being accessed during an assessment. Our findings also highlight that assessors are regularly being exposed to complementary knowledge systems beyond more accepted and familiar technical considerations, which can contribute to external perceptions of legitimacy and boundary spanning (Bond et al. 2018a).
“Contextual understanding” is central to the consideration of multiple knowledge systems, since IA is inherently situated in a project’s environmental, social, and economic context locally, regionally, and nationally (Scott 2011). The intent of YESAB’s decentralized organizational structure is to embed assessors in local contexts so that they will be better placed to understand local community issues. This strategy contributes to overall assumptions that a process guaranteeing the inclusion of Indigenous and local knowledge will be more equitable (Kamarck 2007; Pollitt 2003; Hughes 2010; Glucker et al. 2013; Emerson and Baldwin 2019), and (or) less colonial (Howitt and Suchet-Pearson 2006). Due to the small size of both the assessment board itself and the communities in which assessors are situated, the research capacity of individuals will be central to the effectiveness and legitimacy of the IA process, as the assessors become the local “face” of the organization. Kirchhoff (2006), in a framework specific to IA capacity, refers to relationship-building and maintenance aspects of an assessor’s work as “network and linkages” while Cooke (2005) referenced a more general research capacity framework of “linkages and partnerships.” These aspects are also variants of what Howlett and Ramesh (2015) call “organizational political capacity,” in this case referring to the status of the assessment board amongst the network of organizations working towards development decisions. Not only are assessors tasked with maintaining the assessment board’s presence in local communities, they are also performing the actual assessments, underlining their role as a knowledge broker (Maag et al. 2018) and the need to develop their individual ability to interact with multiple knowledge systems. Here, administrative supports can help assessors establish and maintain external relationships, including initial orientation and ensuring adequate time for relationship maintenance and the development of individual contextual understanding.
We found that assessors are individually responsible for maintaining their individual technical and nontechnical expertise, but the growth and diversification of individual skills are reliant on available time, their academic and professional backgrounds, their exposure to and understanding of the specific context within which they are working, and their understanding of the concept of pluralism and multiple knowledge systems. Most capacity frameworks refer directly to the individual skills of practitioners and the ongoing development of those skills through the organization (Doberstein 2001; Cooke 2005; Hamel and Schrecker 2011). Individual and organizational capacity is therefore interlocked and reciprocal, particularly in contexts with high staff turnover and where contextual understanding is considered central to the organizational mandate. While keeping administration informed by practice is an ideal, resources are necessary to support organizational evolution and knowledge management (Howlett and Ramesh 2015). Our results suggest that individual assessors can contribute to the development of YESAB policy and procedures to a limited extent, with most of their time being spent on the main mandate of completing assessments. This factor may have slowed the evolution of internal organizational policy, which our participants identified as limiting the overall research capacity of the assessment board. A partial solution has been the expansion of the employee base to include policy analysts to support and guide overall organizational development, though assessors are still involved.
As IA processes move towards full inclusion of Indigenous Peoples and complementary worldviews in decision-making processes, lessons can be drawn from the research capacity challenges facing YESAB. Overall, YESAB has built broad internal research capacity sufficient to operationalize IA legislation on a daily basis, though there are spaces requiring organizational attention. They have built a culture of information and knowledge sharing, with spaces for interaction between individuals and designated offices. However, this informal system has translated into a reliance on individuals for knowledge management in lieu of more fully developed organizational mechanisms. Notably, there appears to be a heavy reliance on mentorship for initial training and orientation of new assessors, where participants identified inconsistencies between DOs. The IA-specific capacity framework presented in this paper offers a useful starting point for assessment bodies and assessors in contexts advocating for the effective implementation of pluralism in IA.
Conclusion
IA requires capacity development at the individual (assessor) and organizational (assessment body) level to negotiate multiple perspectives and evidence from a variety of knowledge sources to inform decisions concerning development and maintain perceptions of legitimacy and effectiveness. In northern Canada, IA is founded in land claims agreements with the express mandate to consider multiple knowledge systems, particularly local and Indigenous knowledge, in IA. Navigating the balance between the technical information and value-driven knowledge required to identify and appreciate potential impacts more fully is a multi-scalar and -dimensional task (Bond et al. 2018a). Examining research capacity constraints among individual assessors and assessment bodies highlights their essential roles as knowledge brokers at different levels (i.e., communities and governance mechanisms, respectively) with implications for broader governance objectives related to plurality (Pope et al. 2013; Meuleman 2015). Developing research capacities that support pluralism at all scales of IA therefore has the potential to both improve the effectiveness of IA practice (Maag et al. 2018) and contribute to the perceived legitimacy of the overall process (Kirchhoff 2006).
Drawing on existing capacity conceptual frameworks, we identified a suite of dimensions particular to the IA context, and then applied these to analyze the case of YESAB and its assessors. Results show that YESAB has fostered an overall culture of knowledge exchange within their organization and encouraged external engagement, maintained primarily through informal relationships. However, there is heavy reliance on individual capacity of assessors to build and maintain both internal and external relationships for knowledge exchange, including the training and mentorship of new assessors and the dissemination of knowledge. Constraints on the mechanisms supporting internal knowledge exchange were identified, particularly the loss of knowledge and expertise when well-established assessors move on to external positions. YESAB has generally benefited from financial support for professional development and dedicated policy positions to enhance organizational capacity. Paying closer attention to research capacity has the potential to facilitate inclusive IA processes that acknowledge and actively use complementary knowledge systems. In the case of YESAB, this could include formalized training and mentorship programs, improved internal knowledge management, increased dedicated time for accessing new and updated knowledge sources, and finding opportunities for assessors to experience new methods and complementary worldviews. Future research exploring the inter-institutional dynamics of interrelated capacities and knowledge systems in other environmental governance contexts would be valuable and would expand on existing literature (e.g., Rahman et al. 2019). Re-orienting our understanding of IA towards a knowledge-based activity, rather than a procedural or evaluative practice, could also lead to further theory building in support of process legitimacy and transparency, as well as provide insight into bolstering process effectiveness in Canada.
Acknowledgements
The authors would like to thank the key informants, the Yukon Environmental and Socio-Economic Assessment Board and former employees for generously volunteering their voices to this study. Funding for this work was provided by the Association for Canadian Universities for Northern Studies, the Natural Science and Engineering Research Council’s CREATE-Environmental Innovation and Northern Scientific Training Programs.
References
Albert E, and Mickan S. 2003. Closing the gap and widening the scope: New directions for research capacity building in primary health care. Australian Family Physician, 32(12): 1038–1040. ISSN:0300-8495.
Al-Roubaie A. 2010. Building Indigenous knowledge capacity for development. World Journal of Science, Technology and Sustainable Development, 7(2): 113–129.
Andrews D, Crowther F, Abawi L, Conway J, Dawson M, Lewis M, et al. 2011. Capacity building for Sustainable School improvement: An Australian Research Study. VDM Verlag Dr Muller. ISBN:9783639338089.
Armstrong R, Waters E, Dobbins M, Anderson L, Moore L, Petticrew M, et al. 2013. Knowledge translation strategies to improve the use of evidence in public health decision making in local government: Intervention design and implementation plan. Implementation Science, 8(1): 121.
Arsenault R, Bourassa C, Diver S, McGregor D, and Witham A. 2019. Including Indigenous knowledge systems in environmental assessments: Restructuring the process. Global Environmental Politics, 19(3): 120–132.
Bates I, Akoto AYO, Ansong D, Karikari P, Bedu-Addo G, Critchley J, et al. 2006. Evaluating health research capacity building: An evidence-based tool. PLoS Medicine, 3(8): e299.
Baxter J, and Eyles J. 1999. The utility of in-depth interviews for studying the meaning of environmental risk. The Professional Geographer, 51(2): 307–320.
Biesta G, Allan J, and Edwards R. 2011. The theory question in research capacity building in education: Towards an agenda for research and practice. British Journal of Educational Studies, 59(3): 225–239.
Bond A, Pope J, Retief F, and Morrison-Saunders A. 2018a. On legitimacy in impact assessment: An epistemologically-based conceptualisation. Environmental Impact Assessment Review, 69: 16–23.
Bond A, Retief F, Cave B, Fundingsland M, Duinker P, Verheem R, and Brown A. 2018b. A contribution to the conceptualisation of quality in impact assessment. Environmental Impact Assessment Review, 68: 49–58.
Bourgeon L, Burke A, and Higham T. 2017. Earliest human presence in North America dated to the Last Glacial Maximum: New radiocarbon dates from Bluefish Caves, Canada. PloS One, 12(1): 1–15.
Boyd A, Cole DC, Cho D-B, Aslanyan G, and Bates I. 2013. Frameworks for evaluating health research capacity strengthening: A qualitative study. Health Research Policy and Systems, 11(1): 46.
Brinkerhoff DW, and Morgan PJ. 2010. Capacity and capacity development: Coping with complexity. Public Administration and Development, 30(1): 2–10.
Cape L, Retief F, Lochner P, Fischer T, and Bond A. 2018. Exploring pluralism–Different stakeholder views of the expected and realised value of strategic environmental assessment (SEA). Environmental Impact Assessment Review, 69: 32–41.
Cashmore M. 2004. The role of science in environmental impact assessment: Process and procedure versus purpose in the development of theory. Environmental Impact Assessment Review, 24(4): 403–426.
Chan L, Kirsop B, and Arunachalam S. 2005. Open access archiving: The fast track to building research capacity in developing countries. SciDevNet: 1–14.
Christensen J, and Grant M. 2007. How political change paved the way for Indigenous knowledge: The Mackenzie Valley resource management act. Arctic, 60(2): 115–123.
Cooke J. 2005. A framework to evaluate research capacity building in health care. BMC Family Practice, 6(1): 44.
Creswell JW, and Clark VLP. 2007. Designing and conducting mixed methods research.
Darling S, Ogden A, and Hickey GM. 2018. Reviewing northern capacity for impact assessment in Yukon Territory, Canada. Arctic Yearbook, 2018: 162–179.
de Hek S, Kusters CSL, and Schaap M. 2008. Linking policy to research, capacity building and practice: A study to identify key factors for successful linkages between policy, research and capacity building, and practice within the Policy Support Cluster International (BOCI) Programme funded by LNV. Wageningen International.
Doberstein BA. 2001. Building capacity for environmental planning in vietnam: The role of development aid environmental impact assessment programmes. Doctoral Dissertation. University of British Columbia – School of Community and Regional Planning.
Dylan DW. 2018. The curious case of NIRB’s acquisition of jurisdiction over scientific research in Nunavut. Journal of Environmental Law and Practice, 31(2): 113–135.
Emerson K, and Baldwin E. 2019. Effectiveness in NEPA decision making: In search of evidence and theory. Journal of Environmental Policy and Planning, 21(4): 427–443.
Fitzpatrick P, Sinclair AJ, and Mitchell B. 2008. Environmental impact assessment under the Mackenzie valley resource management act: Deliberative democracy in Canada’s North? Environmental Management, 42(1): 1–18.
Gadsby EW. 2011. Research capacity strengthening: Donor approaches to improving and assessing its impact in low-and middle-income countries. The International Journal of Health Planning and Management, 26(1): 89–106.
Glucker AN, Driessen PP, Kolhoff A, and Runhaar HA. 2013. Public participation in environmental impact assessment: Why, who and how? Environmental Impact Assessment Review, 43: 104–111.
Golenko X., Pager S., and Holden L. 2012. A thematic analysis of the role of the organisation in building allied health research capacity: A senior managers’ perspective. BMC Health Services Research, 12(1): 1–10.
Gondor D. 2016. Inuit knowledge and environmental assessment in Nunavut, Canada. Sustainability Science, 11(1): 153–162.
Government of Canada. 2003. Yukon environmental and socio-economic assessment act. Government of Canada, Canada.
Gupta J, Termeer C, Klostermann J, Meijerink S, van den Brink M, Jong P, et al. 2010. The adaptive capacity wheel: A method to assess the inherent characteristics of institutions to enable the adaptive capacity of society. Environmental Science and Policy, 13(6): 459–471.
Gustafsson KM, Díaz-Reviriego I, and Turnhout E. 2020. Building capacity for the science-policy interface on biodiversity and ecosystem services: Activities, fellows, outcomes, and neglected capacity building needs. Earth System Governance, 4: 100050.
Hamel N, and Schrecker T. 2011. Unpacking capacity to utilize research: A tale of the Burkina Faso public health association. Social Science and Medicine, 72(1): 31–38.
Harrow J. 2001. ‘Capacity building’as a public management Goal-Myth, magic or the main chance? Public Management Review, 3(2): 209–230.
Howitt R, and Suchet-Pearson S. 2006. Rethinking the building blocks: Ontological pluralism and the idea of ‘management’. Geografiska Annaler: Series B, Human Geography, 88(3): 323–335.
Howlett M, Mukherjee I, and Koppenjan J. 2017. Policy learning and policy networks in theory and practice: The role of policy brokers in the Indonesian biodiesel policy network. Policy and Society, 36(2): 233–250.
Howlett M, and Ramesh M. 2015. Achilles’ heels of governance: Critical capacity deficits and their role in governance failures. Regulation and Governance, 10(4): 301–313.
Howlett MP, and Oliphant S. 2010. Environmental organizations and climate change policy capacity: An assessment of the Canadian case. Canadian Political Science Review, 4(2–3): 18–35.
Hughes O. 2010. Does governance exist? In The new public governance: Emerging perspectives on the theory and practice of public governance. Edited by SP Osborne. Routledge/Taylor and Francis, London. pp. 87–104.
Janicke M. 2004. The political system’s capacity for environmental policy: The framework for comparison. In Weidner, H. (eds.) 2002. Capacity building in national environmental policy: A comparative study of 17 countries. Springer Science & Business Media. pp. 1–18. ISBN:3-540-43158-6.
Kamarck EC. 2007 The end of government—as we know it: Making public policy work. Lynne Rienner, Boulder, CO.
Khosravi F, and Jha-Thakur U. 2019. Managing uncertainties through scenario analysis in strategic environmental assessment. Journal of Environmental Planning and Management, 62(6): 979–1000.
Kirchhoff D. 2006. Capacity building for EIA in Brazil: Preliminary considerations and problems to be overcome. Journal of Environmental Assessment Policy and Management, 8(01): 1–18.
Kislov R, Waterman H, Harvey G, and Boaden R. 2014. Rethinking capacity building for knowledge mobilisation: Developing multilevel capabilities in healthcare organisations. Implementation Science, 9(1): 166.
Kolhoff AJ. 2016. Capacity development for environmental protection: towards better performing environmental impact assessment systems in low and middle income countries (Doctoral dissertation, University Utrecht).
Kolhoff AJ, Driessen PP, and Runhaar HA. 2018. Overcoming low EIA performance-A diagnostic tool for the deliberate development of EIA system capacities in low and middle income countries. Environmental Impact Assessment Review, 68: 98–108.
Kolhoff AJ, Runhaar HA, and Driessen PP. 2009. The contribution of capacities and context to EIA system performance and effectiveness in developing countries: Towards a better understanding. Impact Assessment and Project Appraisal, 27(4): 271–282.
Kothari A, Edwards N, and Judd M. 2009. Is research working for you? Validating a tool to examine the capacity of health organizations to use research. Implementation Science, 4(1): 1–9.
Krishnaveni R, and Sujatha R. 2013. Institutional capacity building: A systematic approach. SCMS Journal of Indian Management, 10(4): 17–23.
Laverack G, and Labonte R. 2000. A planning framework for community empowerment goals within health promotion. Health Policy and Planning, 15(3): 255–262.
Lim CP, Chai CS, and Churchill D. 2011. A framework for developing pre-service teachers’ competencies in using technologies to enhance teaching and learning. Educational Media International, 48(2): 69–83.
Lonsdale J, Weston K, Blake S, Edwards R, and Elliott M. 2017. The amended European environmental impact assessment directive: UK marine experience and recommendations. Ocean and Coastal Management, 148: 131–142.
Loomis JJ, and Dziedzic M. 2018. Evaluating EIA systems’ effectiveness: A state of the art. Environmental Impact Assessment Review, 68: 29–37.
Maag S, Alexander TJ, Kase R, and Hoffmann S. 2018. Indicators for measuring the contributions of individual knowledge brokers. Environmental Science and Policy, 89: 1–9.
Marsh D, and Smith M. 2000. Understanding policy networks: Towards a dialectical approach. Political Studies, 48(1): 4–21.
Matarrita-Cascante D, Trejos B, Qin H, Joo D, and Debner S. 2017. Conceptualizing community resilience: Revisiting conceptual distinctions. Community Development, 48(1): 105–123.
Meuleman L. 2015. Owl meets beehive: How impact assessment and governance relate. Impact Assessment and Project Appraisal, 33(1): 4–15.
Minja H, Nsanzabana C, Maure C, Hoffmann A, Rumisha S, Ogundahunsi O, et al. 2011. Impact of health research capacity strengthening in low-and middle-income countries: the case of WHO/TDR programmes. PLoS neglected tropical diseases, 5(10): e1351.
Mugabo L, Rouleau D, Odhiambo J, Nisingizwe MP, Amoroso C, Barebwanuwe P, et al. 2015. Approaches and impact of non-academic research capacity strengthening training models in sub-Saharan Africa: A systematic review. Health Research Policy and Systems, 13(1): 30.
Natcher DC, Davis S, and Hickey CG. 2005. Co-management: Managing relationships, not resources. Human Organization, 64(3): 240–250.
Noble B, and Hanna K. 2015. Environmental assessment in the Arctic: A gap analysis and research agenda. Arctic, 68 (3): 341–355.
Noble BF, and Press D. 2011. Introduction to environmental impact assessment. The Canadian Geographer/Le Géographe Canadien, 56(1): 142–153.
Noy C. 2008. Sampling knowledge: The hermeneutics of snowball sampling in qualitative research. International Journal of Social Research Methodology, 11(4): 327–344.
Nuyens Y, and McKee N. 2005. No development without research: A challenge for capacity strengthening. Global Forum for Health Research. 44 p. ISBN: 9782940286379.
Nykvist B, and Nilsson M. 2009. Are impact assessment procedures actually promoting sustainable development? Institutional perspectives on barriers and opportunities found in the Swedish committee system. Environmental Impact Assessment Review, 29(1): 15–24.
Pager S, Holden L, and Golenko X. 2012. Motivators, enablers, and barriers to building allied health research capacity. Journal of Multidisciplinary Healthcare, 5: 53.
Partidario MR, and Sheate WR. 2013. Knowledge brokerage-potential for increased capacities and shared power in impact assessment. Environmental Impact Assessment Review, 39: 26–36.
Peletz N, Hanna K, and Noble B. 2020. The central role of Inuit Qaujimaningit in Nunavut’s impact assessment process. Impact Assessment and Project Appraisal, 38(5): 412–426.
Pollitt C. 2003. The essential public manager. McGraw-Hill Education, UK. 196 p. ISBN: 0335212328.
Pope J, Bond A, Morrison-Saunders A, and Retief F. 2013. Advancing the theory and practice of impact assessment: Setting the research agenda. Environmental Impact Assessment Review, 41: 1–9.
Potter C, and Brough R. 2004. Systemic capacity building: A hierarchy of needs. Health Policy and Planning, 19(5): 336–345.
Rahman HT, Po JY, Saint Ville AS, Brunet ND, Clare SM, Darling S, et al. 2019. Legitimacy of different knowledge types in natural resource governance and their functions in inter-institutional gaps. Society and Natural Resources, 32(12): 1344–1363.
Ramesh M, Saguin K, Howlett MP, and Wu X. 2016. Rethinking governance capacity as organizational and systemic resources. Lee Kuan Yew School of Public Policy Working Paper, 16–12: 1–32.
Sabin JJD. 2016. Contested colonialism: The rise of settler politics in Yukon and the Northwest Territories. Doctoral Dissertation, University of Toronto.
Saldaña, J. 2015. The Coding Manual for Qualitative Researchers. Sage Publications, ISBN: 9781473902497.
Scott C. 2011. Governmentality and strategic environmental assessment: Challenging the SEA/good governance nexus. Journal of Environmental Assessment Policy and Management, 13(1): 67–100.
Seidman I. 2013. Interviewing as qualitative research: A guide for researchers in education and the social sciences, 4th ed. Teachers College Press. ISBN: 9780807746660.
Southcott C, Abele F, Natcher D, and Parlee B. 2018. Beyond the Berger inquiry: Can extractive resource development help the sustainability of Canada’s arctic communities? Arctic, 71(4): 393–406.
Sovacool BK. 2010. A critical stakeholder analysis of the Trans-ASEAN Gas Pipeline (TAGP) Network. Land Use Policy, 27(3); 788–797.
Spagnuolo F. 2011. Diversity and pluralism in earth system governance: Contemplating the role for global administrative law. Ecological Economics, 70(11): 1875–1881.
Thornhill J, Judd M, and Clements D. 2000. CHSRF knowledge transfer:(re) introducing the self-assessment tool that is helping decision-makers assess their organization’s capacity to use research. Healthcare Quarterly (Toronto, Ont.), 12(1): 22–24.
Udofia A, Noble B, and Poelzer G. 2017. Meaningful and efficient? Enduring challenges to Aboriginal participation in environmental assessment. Environmental Impact Assessment Review, 65: 164–174.
UFA. 1993. Umbrella final agreement. Government of Canada, C. O. Y. I., Government of Yukon (ed.).
Ugolini C, and Lewis S. 2000. Evidence-based decision making: do we have the right stuff? Backgrounder for discussions of the Self-Audit Tool for Decision Making Organizations.
UNEP. 2018. Assessing environmental impacts- a global review of legislation. UN Environment Programme. 150 p. ISBN: 9789280736793.
van der Molen F. 2018. How knowledge enables governance: The coproduction of environmental governance capacity. Environmental Science and Policy, 87: 18–25.
van Loon L, Driessen PP, Kolhoff A, and Runhaar HA. 2010. An analytical framework for capacity development in EIA – the case of Yemen. Environmental Impact Assessment Review, 30(2): 100–107.
Velho L. 2004. Research capacity building for development: From old to new assumptions. Science Technology and Society, 9(2): 171–207.
White G. 2006. Cultures in collision: Traditional knowledge and Euro-Canadian governance processes in northern land-claim boards. Arctic, 59(4): 401–414.
Woodhill J. 2010. Capacities for institutional innovation: A complexity perspective. IDS Bulletin, 41(3): 47–59.
Wright G. 2014. Strengthening the role of science in marine governance through environmental impact assessment: A case study of the marine renewable energy industry. Ocean and Coastal Management, 99: 23–30.
YG. 2020. Yukon Bureau of statistics – population report third quarter, 2019, Yukon Bureau of Statistics, Info Sheet no. 60. Yukon Government: Yukon Bureau of Statistics.
Yin RK. 2003. Case study research design and methods third edition. Applied Social Research Methods Series 5, SAGE Publications. 270 p.
Appendix A
Table A1.
| Author Year | Discipline | Scale | External coordination (relationship) | Internal coordination | Finances | Legitimacy | Technical cxpertise | Project management | Versatility (multi-disciplinarity) | Contextual understanding | Mentorship | Opportunity | Dedicated time | Culture | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Internal (cohesive vision) | External (perception) | ||||||||||||||
| de Hek 2008 | Agriculture policy | Organization | x | ||||||||||||
| Gupta et al. 2010 | Climate change | Organization | x | x | x | x | x | x | |||||||
| Lim et al. 2011 | Education | Individual | x | x | x | x | x | x | x | ||||||
| Doberstein 2001 | EIA | System | x | x | x | ||||||||||
| Khosravi and Jha-Thakur 2019 | EIA | System | x | x | x | x | |||||||||
| Kirchhoff 2006 | EIA | Organization | x | x | x | x | x | ||||||||
| Kolhoff 2016 | EIA | System | x | x | x | ||||||||||
| van Loon et al. 2010 | EIA | Organization | |||||||||||||
| Janicke 2004 | Environmental policy | Organization/network | x | x | x | x | |||||||||
| Maag et al. 2018 | Environmental science | Individual | x | x | x | x | x | x | x | ||||||
| Thornhill et al. 2009 / Ugolini and Lewis 2000 / Kothari et al. 2009 | Health | Organization | |||||||||||||
| Minja et al. 2011 | Health | Organization/individual | x | x | x | x | x | x | x | x | |||||
| Mugabo et al. 2015 | Health | Individual | x | x | x | x | x | x | x | x | x | ||||
| Albert and Mickan 2003 | Health care | Individual | x | x | x | x | x | x | |||||||
| Bates et al. 2006 | Health care | Organization/Individual | x | x | x | x | |||||||||
| Cooke 2005 | Health care | Organization | x | x | x | x | x | x | x | x | x | ||||
| Golenko et al. 2012 | Health care | Individual | x | x | x | x | |||||||||
| Hamel and Schrecker 2011 | Health care | Organization | x | x | x | x | x | ||||||||
| Kislov et al. 2014 | Health care | Organization | x | x | x | x | x | ||||||||
| Pager et al. 2012 | Health care | Individual | x | x | x | ||||||||||
| Gustafsson et al. 2020 | IPBES | System | x | x | x | x | x | ||||||||
| Howlett and Oliphant 2010 | Political science | Organization | x | x | x | x | |||||||||
| Armstrong et al. 2013 | Public health | Organization | x | x | x | x | |||||||||
Note: IPBES, Intergovernmental Science-Policy Platform on Biodiversity and Ecosystem Services.
Information & Authors
Information
Published In
FACETS
Volume 7 • Number 1 • January 2022
Pages: 674 - 700
Editor: Victoria Metcalf
History
Received: 12 August 2021
Accepted: 17 February 2022
Version of record online: 5 May 2022
Copyright
© 2022 Darling et al. This work is licensed under a Creative Commons Attribution 4.0 International License (CC BY 4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original author(s) and source are credited.
Data Availability Statement
All relevant data are within the paper.
Key Words
Sections
Subjects
Authors
Author Contributions
All conceived and designed the study.
SD performed the experiments/collected the data.
SD analyzed and interpreted the data.
SD and GMH contributed resources.
All drafted or revised the manuscript.
Competing Interests
The authors have declared that no competing interests exist.
Metrics & Citations
Metrics
Other Metrics
Citations
Cite As
Samantha Darling, Blane Harvey, and Gordon M. Hickey. 2022. On the individual and organizational capacities supporting impact assessment: the case of the Yukon Environmental and Socio-Economic Assessment Board. FACETS.
7: 674-700.
https://doi.org/10.1139/facets-2021-0118
Export Citations
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.
Cited by
1. Advancing Pluralism in Impact Assessment Through Research Capacity: Lessons from the Yukon Territory, Canada