Framework and proposed indicators for the comprehensive evaluation of inclusiveness: the case of climate change adaptation

Promoting inclusion has been an on-going concern in several contexts, including development and growth, decision-making and governance, cities and urbanization, innovation, and education and the workplace. The conceptualization of inclusion and what it constitutes is surprisingly diverse, and no overarching framework exists for evaluating the degree of inclusiveness in broad or diverse contexts. Without such an overarching perspective, there is a risk of omission of contexts where inclusion matters. The risk of omission also applies to the implementation and evaluation phases. References are commonly made to equity, discrimination, justice, diversity, and inclusion (UN Womenwatch 2009; Grant 2019; Norton 2019; Berry and Schnitter 2022; Environment and Climate Change Canada 2022; Wing et al. 2022), but the translation of these concepts into practical evaluation tools still requires development.

We have selected the example of climate change adaptation (CCA) because it provides a very broad context for both human and non-human situations. CCA is defined as the process of adjusting to an actual or expected climate and its effects on natural or human systems (IPCC 2022a). Scholars have long criticized the inequitable participation in CCA planning and implementation that has undermined equitable vulnerability reduction because of the failure to overcome entrenched power relations and ensure the meaningful inclusion of the most marginalized (Hügel and Davies 2020; Eriksen et al. 2021). A number of scholars have recently argued for a more inclusive approach to stakeholder engagement in CCA (Chu and Cannon 2021; IPCC 2022b; Martin et al. 2022; Singh et al. 2022). Martin and colleagues (2022) ranked inclusion as one of ten new insights in climate science 2022, whereas Singh and colleagues (2022) argue that inclusion should be a key principle by which to evaluate the success and effectiveness of CCA.

A recent systematic review of inclusive climate change adaptation (ICCA) revealed a growing interest in the idea of “inclusiveness” in the climate change context (Pham and Saner 2021). The concept of ICCA, however, has neither been fully developed nor used in formal evaluations of public policies or practices. There are also no checklists or indicators for systematically evaluating the inclusiveness of a policy or action plan in the CCA context.

An attempt was recently made to apply the principles of equity, diversity, and inclusion to the CCA context (Hoicka et al. 2022; Tangirala 2022), but the lens of inclusion was limited to a managerial tool. We must stress that the Equity, Diversity, and Inclusion (EDI) approach used in human resources is insufficient to serve as a model. Other contexts where diversity matters need to be included, such as politics and governance or even the welfare of non-human sentient beings. Moreover, the practice of EDI often tends to be too narrow in application. For example, EDI may be focused only on the moment of hiring staff. Thus, using a logic model, we will evaluate inclusion across the entire process—from planning to evaluation.

The conceptualization of ICCA should be grounded in moral reasoning. All decisions are built on values, which are entwined in the entire decision-making cycle, from the identification of priorities in the planning process through standards setting and implementation, and finally to performance evaluation. The use of an ethical lens opens the door to conceptual and analytical tools in specialized fields such as environmental ethics, business and workplace ethics, and political ethics (Light and Rolsten 2003; Zgheib 2014; London 2021). For example, environmental ethics provides convincing arguments for the broadening of the moral circle from the traditional human focus (“anthropocentrism”) to the inclusion of sentient animals, other life forms, biodiversity, and even habitats (“non-anthropocentric ethics”) (Singer 1981). Finally, ethical considerations are especially relevant in the case of CCA, which is an exemplar of climate justice theory and practice (Robinson 2018; UN 2019).

The ethical analysis in this paper provides a broad conceptual foundation for ICCA indicators. Our conceptual approach also includes all key steps in policymaking, namely initial planning, implementation, and evaluation. Within the framework constructed this way, we then identify nine priority areas and corresponding performance indicators. We believe that this type of tool is required in climate change contexts. Our tools are applicable to several contexts, either to evaluate existing concepts of inclusion or to develop inclusion policies in new contexts. Casting the conceptual scope widely enables us to establish a conceptual basis that will bring us closer to achieving the ambition of thinking inclusively about inclusiveness. The resulting heuristic tools are applicable to contexts outside of ICCA, where the idea of inclusiveness is also gaining importance.

In this paper, we develop an argument for how inclusion, inclusiveness, or inclusivity (we use these words interchangeably) should be conceptualized, implemented, and assessed in the context of CCA and propose a set of indicators that are testable in practice.

First, we use a comparative approach to examine a broad set of related contexts. Here, the focus is on inclusiveness “in practice”, meaning how inclusiveness has been conceived, implemented, and assessed in different contexts, sectors, and locations. We therefore compared international frameworks that have been globally accepted and applied to achieve inclusiveness in development and growth, decision-making and governance, cities and urbanization, innovation, and education and the workplace. We focused on (i) frameworks that were initiated by an international organization and applied in different countries and (ii) frameworks that included performance. The input from these comparisons provides insights regarding inclusiveness evaluation in diverse contexts and can serve as a foundation for developing our framework for ICCA.

Second, we carry out a broad ethical analysis that draws upon the pluralistic values of climate change ethics, emphasizing global justice, intergenerational justice, ecological justice, and business ethics. We highlight diversity, equity, and inclusion in contemporary organizations as well as political ethics, ensuring cooperation, transparency, and accountability in governance and policymaking. While climate change ethics, with its focus on the concept of climate justice, provides rich resources for the moral reasoning of inclusiveness, we believe that business ethics and political ethics offer valuable insights concerning the practical implementation of a more inclusive approach to CCA.

Lastly, we employ a standard logic model in a systematic and visual way to describe the relationships among the resources of the operation of the program, the activities planned, and the changes or results that the program is expected to achieve (W.K. Kellogg Foundation 2004; Frechtling 2015). Once a program has been described in terms of how it works and to what end, this logic model can be used as part of an evaluation study as it provides the foundation for looking at implementation and outcomes, as well as defining boundaries and identifying critical measures of performance (Pringle 2011; Marino-Saum 2020). In our study, the logic model serves as a conceptual framework by which to facilitate the selection of indicators and identify the most relevant indicators for a specific domain, problem, and location. This will, in turn, yield an indicator set that is at once transparent, efficient, and powerful in its ability to assess the state of any program or project without losing sight of the overall climate change context.

Based on this argument, we identify a set of 15 plausible indicators, which serves as our hypothesis. We will personally attempt to “falsify” this set of indicators in working with practitioners and invite the academic and practitioner communities to do the same.

To better understand the conceptualization and assessment of inclusiveness, we reviewed five sectors in which the term is already firmly established. We found that the prefix “inclusive” is common in the following domains: (1) inclusive development and growth (OECD 2015; Gupta and Vegelin 2016; Jahanger 2023), (2) inclusive (or exclusive) governance and decision-making (OECD 2020; Mustaniemi-Laakso et al. 2023), (3) inclusive cities and urbanization (Gerometta et al. 2005; UN-Habitat 2015; Ramachandran and Di Matteo 2023), (4) inclusive innovation (Foster and Heeks 2013; Government of Canada 2016; Illalba Morales et al. 2023), and (5) inclusive education and workplace (Avramidis and Norwich 2002; Madhesh 2023).

We illustrate the conceptualization of inclusiveness in these five sectors by briefly reviewing one high-profile and international framework for each sector (Table 1). The key insights from these related contexts are as follows. First, inclusiveness has become an important topic in widely varied contexts but has by no means penetrated all academic literature where one could imagine its prominent use, including that of ICCA. The interpretation of “inclusiveness” and “inclusivity” is highly dependent on context. Some sectors represent a means-to-an-end (development, governance, innovation, education), while others are ends-in-themselves (cities and the SDGs). Some are local (cities), while others are national or global (development and the SDGs). Some sectors fall under the domain of law (governance) or economics (innovation), while others fall under the domain of geography (cities) or development (education, growth). We believe that this fragmentation provides an argument for a broader theory of “inclusive inclusiveness”. Second, in terms of commonality, the frameworks can be parsed into three foci of attention: (a) stakeholders, (b) processes, or (c) outcomes, or mixtures thereof. These three terms are often explicitly used within the above frameworks, and they were also dominant in our systematic literature review on ICCA (Pham and Saner 2021). There is a logic to this choice. The three foci represent foundational ideas in public policy, where stakeholders engage in processes to achieve outcomes (Osborne 2010). They also map onto the three major schools of thought in ethics, where virtue ethics informs the attitudes and behavior of stakeholders, deontology informs the duties and rules governing decisions and processes, and utilitarianism informs strategies and the interpretation of results (Waluchow 2003). We note that some frameworks do not cover all three of these foci and, therefore, in that sense, may lack inclusiveness.

Third, we observe that case studies are commonly used to illustrate indicators and ethical perspectives. Without taking away from the value of case studies, we argue that there is value in an argumentative, theoretical approach to add insights to what is already understood from practice. Given that inclusion is heavily linked to ethical considerations, there is clearly “moral corruption” (Gardiner 2022). At the same time, this gap in theoretical approach provides an opportunity for improvement by elaborating ethical thinking during the development of inclusion-related frameworks and engaging the perception, implementation, and evaluation of inclusiveness with issues in ethics and philosophy more generally.

In this section, we investigate the relevant ethical principles and considerations to answer the question: What are “good stakeholders”, “good processes”, and “good outcomes” for ICCA? The emphasis of the word “good” is inspired by G.E. Moore's definition (1903): “Ethics is the inquiry into what is good”. The word “good” is, in this view, a moral ideal.

We first investigate climate change ethics to find the most fundamental principles and considerations that shed light on our analysis. At the heart of climate change ethics are the principles of justice and caring that raise the key moral reasoning aspects of ICCA.

The multifaceted concept of climate justice refers to global justice, inter-generational justice, and ecological justice (Gardiner 2011, 2022). Global justice deals with vulnerable and affected people in the present whose rights could be violated by the decisions of others (governments, industries, groups, and individuals), while they are powerless to block these decisions. This undermines their vital interests. Inter-generational justice concerns the interests of future people who are currently non-existent and powerless, but they will be moral agents in the future and are thus worthy of respect and the granting of universal human rights. Ecological justice emphasizes the responsibilities of human beings to refrain from transferring the burden to the environment and supports the well-being of non-human actors, animals, plants, and ecosystems due not only to their instrumental value but also their intrinsic value. Some authors also argue for the importance of recognition justice, which requires respecting and acknowledging the cultures, values, and situations of all affected parties who should be considered and represented throughout decision-making processes when it comes to distributional, procedural, and compensational justice (Schlosberg 2012; Hourdequin 2016).

Indigenous ethics and feminist care ethics provide a care-based approach to justice that has been intensified in climate change ethics. Instead of focusing on rights, duties, or responsibilities, these strands of ethics refer to guidance for ethical decision-making about action and policy that are grounded in the virtues, practices, and knowledge associated with appropriate caring and respect for self and others (Gilligan 1982; Held 2006). These theories accelerate inclusive relations of caring within interdependent human and ecological communities, justifying ethical responsibilities for all human and non-human entities (Whyte 2020; Tschakert et al. 2021). Through these intimate relationships, Indigenous peoples and female groups produce and strengthen crucial ecological and local knowledge that must be respected and included to inform conservation strategies, environmental change adaptation, and resilience. Moreover, the ethics of caring also advocate for the empowerment of less powerful groups and communities to actively include themselves in these processes, thus caring for the social and ecological communities in which their lives and interests are interwoven (Tschakert and Machado 2012).

CCA initiates, happens, and manages within a form of organization, whereas the inclusiveness perspective in CCA is based on the ethical systems that are predominantly influenced by several characteristics of human connection, such as politics and culture. With our efforts to operate inclusiveness in CCA practices, we have extended this concept by supplementing it with organizational, institutional, and political perspectives. From an organizational perspective, inclusiveness emphasizes the importance of respecting differences in culture and values, acknowledging the systemic inequality and vulnerability that prevent certain groups from raising their voices, and shaping adaptation processes and outcomes that are just. From an institutional perspective, inclusiveness focuses on the ways in which those who are vulnerable and marginalized seek to address distributional and procedural injustice. From a political perspective, inclusiveness highlights the interaction and deliberation between groups to assert their status in the political community. We supplemented our analysis by looking into organization or business ethics and political ethics to explore principles for what makes “good stakeholders”, “good processes”, and “good outcomes”.

Rooted in business ethics, EDI has recently been used as a phrase by organizations to emphasize ongoing efforts to rectify the problems that are linked to the EDI of staff, the focus of which has broadened from gender to include other underrepresented groups (Wolbring and Lillywhite 2021). Equity aims to ensure that everyone has an equitable chance to succeed. Diversity promotes the active participation of individuals from a wide range of backgrounds, experiences, and perspectives, while inclusion emphasizes the importance of creating environments where all individuals feel valued, respected, and empowered (Oswick and Noon 2014; Garg and Sangwan 2021). EDI explicitly acknowledges that different individuals may require different levels of support or accommodations to achieve fairness and contribute their unique perspectives and talents to the development of organizations. EDI serves as the organizational perspective of ICCA. By maximizing opportunities for the full pool of potential participants, EDI helps organizations promote innovation and creativity, increase effectiveness, and strengthen the relevance and impact of their adaptation actions.

Political ethics revolves around the moral principles, values, and standards that guide the behavior and actions of individuals and institutions in political decision-making and governance (Thompson 2019). Collaborating and positively contributing to the well-being and welfare of a broad society are acknowledged as social responsibilities of political actors, institutions, and citizens (Digeser 2022; Mozumder 2022). Political ethics emphasizes transparency and accountability for public officials and institutions to answer for their actions to the public while also promoting the establishment of mechanisms and institutions that ensure accountability, such as independent oversight bodies (Fox 2007; Vian 2020). Collaboration, co-contribution, transparency, and accountability serve as the institutional perspectives of ICCA. Meaningful inclusion explicitly requires collaboration between stakeholders to carry out collective action for adapting to a changing climate while accelerating the values of accountability and transparency to maintain just and equitable processes and outcomes throughout adaptation practices.

The three-pillar framework informed by climate change ethics, business ethics, and political ethics provides a valuable foundation for moral reasoning in the context of ICCA. Climate change ethics contributes to the principles of justice and caring; business ethics provides insights on equity, diversity, and inclusion; and political ethics highlights the value of collaboration, transparency, and accountability. Together, these principles can guide decision-makers and stakeholders in developing and evaluating inclusive adaptation strategies. Based on the above ethical points, (1) we answered the question: What are “good stakeholders”, “good processes”, and “good outcomes” for ICCA? and (2) we identified the following four core components of ICCA.

We conclude, therefore, that the framework is reasonably inclusive when it covers the moral principles rooted in different ethics theories as well as the ethical considerations of organizations and institutions for adaptation in practice. All four themes must be considered in concert to attain “inclusive inclusiveness” because they interact with each other. A strong emphasis on one theme (for example, the breadth of the moral circle) will impact others (for example, support and process). We will say more about the trade-off between what is theoretically desirable and what is practically achievable in the Discussion and conclusion section of this paper.

In public policy, so-called logic models are commonly used to plan activities and develop indicators of success (W.K. Kellogg Foundation 2004). The six common steps of logic models are broad enough to cover all of the key processes of general policymaking and specific adaptation initiatives. ICCA asserts the importance of inclusiveness throughout all steps: (i) planning; (ii) inputs (resource allocation and management); (iii) activities (implementation); (iv) outputs (results); (v) impacts (success); and (vi) evaluation and learning.

These six steps could be juxtaposed with the four key themes identified in the previous section in a heuristic matrix. Such a 6 by 4 matrix, however, would have an overwhelming, and overlapping, set of possible 24 units of concern. We opted instead to propose nine priorities guided by both the inclusiveness and climate change literature, the gap in adaptation practices, and the potential contribution to adaptation debates. The following section presents a summary chart (Fig. 1) aimed at giving a broad sense of relationships, followed by a description of each of the nine priorities.

We are beginning this section with a discussion of trade-offs. We know a priori that implementing inclusiveness entails direct or opportunity costs. The most obvious costs are the increased administrative burden, the loss of time, and the loss of focus. One could argue that idealistic versions of inclusiveness are in direct conflict with the formulation of priorities, something most project and people managers value highly (discriminating what matters most is more productive than including everything). The capacity for identifying priorities is of particular importance during any crisis, including that of climate change. In response, we would argue that thinking inclusively is a better starting point for priority selection and priority sequencing. A broader initial scope does not need to result in a loss of focus, and the time spent upfront identifying good (justified and sustainable) priorities will save more time later. It is also plausible that inclusive governance will be more effective because it draws from diverse knowledge and will be more sustainable because of the broader acceptance of direction.

A further consideration is the potential clash between inclusiveness and culture, tradition, or “moral order”. This is a challenging topic, given the global scope of climate change. It is a certainty that views on inclusiveness will differ between, say, Afghanistan and New Zealand. In this context, one also needs to expect that a regime will preach inclusiveness and implement something different. We do not have a good solution to this problem. Inclusiveness is a democratic and liberal concept that will not be universally accepted. The question of how elitist or non-elitist a political system ought to be goes back to at least Plato's Republic.

As a second discussion point, we would like to state the obvious point that it is impossible to achieve perfect inclusiveness. Just as extreme reduction leads to the absurd (reductio ad absurdum), extreme expansion would be far from convincing or practical. Our selection of nine focal areas was made with this problem in mind. We are attempting a breadth of inclusiveness that is neither exotic (except for the possible point about non-anthropocentric thinking) nor impractical.

We aimed to not only provide an overview of common components, such as discrimination, but to also seek a more complete approach to building an integrative lens. We start very wide when working with ethical and public policy concepts, then we select priorities and indicators, and, finally, we will improve and narrow further during interviews with CCA practitioners (on-going and planned research).

The ultimate goal is to promote inclusiveness in a broad way, across themes and across the steps of a policy process. This should improve coordination and efficiency. It should also reduce duplication and counter-purpose initiatives. We seek to achieve organizational learning in several distinct ways that target common inclusiveness limits. To illustrate these limits, we have to distinguish between omission and commission as well as involuntary and voluntary acts, as illustrated in the matrix in Table 3.

Working with the above inclusiveness indicators, we argue, will improve learning against all four limits: blind spots, callousness, constraint, and defeat.

Discussing the questions in Table 3 should take care of the first problem, blind spots. We use blind spots here as a scientific term to express the situation in which people cannot recognize the perpetuation of exclusion and discrimination. This will not be achieved, however, if the indicators are merely part of a bureaucratic exercise. They must be presented as real issues that deserve the time and attention of debate. The dialogue should involve diverse participants, which, if patiently led, should improve understanding and empathy and thus reduce the second limit, callousness. Imagine a conversation on the topic of Indicator 1 (Who has moral standing?). Such a conversation should open everybody's eyes to the fact that ignoring the pain or suffering of any sentient being (humans of all kinds and sentient non-humans) is difficult to justify. While it may be likely that participants would disagree on the moral status of animals, the disagreement would nonetheless open a people's eyes to the breadth of values held by a community. Because the suite of 15 indicators also contains many references to implementation and performance measurement, it should be possible to also push boundaries on the “constraint” and “defeat” limits. Based on the awareness created by such dialogue, it ought to become more realistic to achieve “inclusiveness-by-design”, which would improve the ability to persevere and succeed.

An important side benefit of such debates about values and ethics is that they build a common understanding and common culture that can benefit the operation in other contexts. A checklist such as that in the above table is essential to the evaluation process. However, the answer to these questions can lead to a presentation challenge. A visual picture of the results can be helpful. Inspired by the Korn Ferry Diversity and Inclusion Maturity Model (Korn Ferry 2021), we produced sample Fig. 2. In contrast to the elaborate Korn Ferry Model, we selected a very basic “Radar Chart” because these charts can be easily generated in standard software packages. Visuals that summarize all 15 indicators in one chart can help one to make comparisons over time and across comparable organizations.

Theory can only provide a framework and a starting point for practical work. Making adaptation more inclusive requires the testing of the 9 priorities and 15 indicators in real-life, diverse settings. It will be necessary to interview policymakers, researchers, and practitioners on the inclusiveness of adaptation processes, the potential of applying the inclusiveness framework in adaptation research, policymaking, and practice, as well as the necessary adjustment of inclusive indicators to be applicable. This work is currently underway.

We are grateful to Ozan Gurcan for his comments and for suggesting the addition of a radar chart. The usual disclaimers apply.