OUTREACH & IMPACT
Informing the political priorities of the next European Commission
Evidence Submitted for the ‘ERC Feedback to Policy’ Call of the European Commission for Evidence on EU Policies and Strategies Concerning Democracy, December 13 2024
Contribution ID: 8de9f48e-21da-449d-8817-c7b0d1917b97
ERC KnowledgeLab Project
Professor Mona Simion, Cogito Epistemology REsearch Centre, University of Glasgow
(1) DEMOCRACY – countering disinformation and foreign interference (FIMI): Counter foreign information manipulation and deepfakes, focusing on enhancing digital and media literacy and employing pre-bunking.
Disinformation is widespread and harmful, epistemically and practically. We are currently facing a global information crisis that the Secretary-General of the World Health Organization (WHO) has declared an 'infodemic.' Our results (https://doi.org/10.1017/epi.2023.25) show that disinformation is currently imprecisely conceptualised, and, as a result, our best detection and protection mechanisms fail to track the most problematic varieties thereof. Disinformation need not come in the form of intentionally spread false content. Disinformation consists of content with a disposition to generate ignorance in the audience in normal conditions at the context at stake: A signal r will carry disinformation for an audience A wrt p iff A’s evidential probability that p conditional on r is less than A’s unconditional evidential probability that p, and p is true. The same communicated content will act differently depending on contextual factors such as: the evidential backgrounds of the audience members, the shared presuppositions, extant social relations, and social norms. This predicts that disinformation is much more ubiquitous and harder to track than it is currently taken to be in policy and practice: mere FactCheckers, for instance, will not be able to adequately protect us against disinformation, because disinforming does not require making false claims. Indeed, our results show that this simple way of disinforming - via false claims - is both the least efficient and the least dangerous (because easiest to detect by average cognizers).
Generating ignorance - and thereby spreading disinformation - can be done in a variety of ways that does not involve false asserted content. Among the several strategies that our work identifies, two of the most problematic and widely employed are:
(1) Disinforming via misleading evidence that is implicated rather than asserted: This category of disinformation has the capacity of stripping the audience of knowledge via undermining rational belief. A classic example is communicating the content: ‘There is disagreement about climate change in the scientific community’. This claim is strictly speaking true: there is some extremely minor disagreement in the scientific community – a few outliers. However, the claim generates the false implicature that there is substantive disagreement on the topic (because the audience presupposes that if it is not a substantive level of disagreement, it wouldn’t be relevant at the context, and thereby wouldn’t be asserted). This implicature makes it rational for the audience to suspend belief on the issue of whether climate change is happening (since high levels of disagreement justify belief suspension) and justifies inaction.
(2) Disinforming via content that has the capacity of inducing epistemic anxiety: this category of disinformation has the capacity of stripping the audience of knowledge via triggering belief loss. The paradigmatic way to do this is via artificially raising the stakes at the context (e.g. ‘Are you really sure the vaccine is safe? After all, scientists do sometimes make mistakes’). The way this variety of disinforming works is via falsely implicating that these error possibilities are relevant at the context, when in fact they are not – since the scientific community has an excellent track record, the possibility of error is rationally negligeable – and, indeed, rationally neglected whenever we take an aspirin or use toothpaste.
What these ways of disinforming have in common is that they generate ignorance via exploiting contextual features such as pragmatic implicature or the audience’s background evidence/knowledge, without any false assertion. As such, some of the best Fact Checkers at our disposal will fail to capture the most efficient types of disinformation, because they are trained to track false assertions.
(10) DEMOCRACY – fostering public trust in governance: Strengthen confidence in governance through transparent, inclusive, and effectively communicated measures to secure public acceptance and support.
We have increasingly sophisticated ways of acquiring and communicating knowledge, but efforts to spread this knowledge often encounter resistance to evidence and distrust in expertise. Resistance to evidence consists in a disposition to reject evidence coming from highly reliable sources. This disposition deprives us of knowledge and understanding and comes with dire practical consequences; recent high-stakes examples include climate change denial and vaccine scepticism.
Until very recently, the predominant hypothesis in social psychology principally explained evidence resistance with reference to politically motivated reasoning: on this view, a thinker’s prior political convictions best explain why they are inclined to reject expert consensus when they do. Our results (
https://doi.org/10.1017/9781009298537) show that this hypothesis is not supported by the evidence. Political group identity is often confounded with prior rational beliefs about the issue in question; and, crucially, reasoning can be affected by such beliefs in the absence of any political group motivation. This renders much existing evidence for the hypothesis ambiguous.. Furthermore, the conceptual frameworks employed are ineffective in identifying crucial distinctions among a number of phenomena, such as between irrational resistance to evidence and rational evidence rejection. The latter happens in cases in which cognisers have misleading evidence in their environment that undermines the evidential value of the expert testimony. This evidence often come from sources that cognisers are rational to trust, due to a history of reliability.
Difficulties in answering the question as to what triggers resistance to evidence have very significant negative impact on our prospects of identifying the best ways to address resistance to evidence. If resistance to evidence has one main source - for instance, a particular type of mistake in reasoning, such as motivated reasoning - the strategy to address this problem will be unidirectional and targeted mostly at the individual-level. In contrast, if this phenomenon is, e.g., the result of a complex interaction of social, emotive, and cognitive phenomena - we would have to develop much more complex interventions, at both individual and societal levels.
Our results (https://doi.org/10.1017/9781009298537; https://doi.org/10.1017/S1062798724000218; https://doi.org/10.1111/phpr.12964) show that irrational resistance to evidence is rare; what is often encountered in the population, however, is rational evidence rejection, due to overwhelming (misleading) evidence present in the (epistemically polluted) environment of the agent. When agents rationally reject reliable scientific testimony, they do so in virtue of two types of epistemic phenomena: rebutting evidence, and undercutting evidence. Rebutting evidence undermines the evidential value of the expert testimony by lowering its probability for the agent. Undercutting evidence undermines the evidential value of the expert testimony by lowering the credibility of the expert testifier. These results, in turn, illuminate the best strategies to address the phenomenon of evidence resistance and distrust in expertise. Two major types of interventions are required:
(1) For combatting rational evidence rejection: engineering enhanced social epistemic environments. This requires: (1.1) combatting rebutting misleading evidence via reliable evidence flooding: evidence resistant communities, inhabiting polluted epistemic environments, cannot be reached via the average communication strategies designed to reach the mainstream population, inhabiting a friendly epistemic environment (with little to no misleading evidence). What is required is (1.1) quantitatively enhanced reliable evidence flow: this is a purely quantitative strategy, aimed to outweigh rebutting misleading evidence in the agent’s environment. More evidence in favour of the scientifically well supported facts will, in rational agents, work to outweigh the misleading evidence they have against the facts; (1.2) qualitatively enhanced reliable evidence flow: this is a qualitative intervention, that aims to outweigh misleading evidence via evidence from sources that the agent trusts – that are trustworthy vis-à-vis the agent’s environment; (1.3) quantitatively and qualitatively enhanced evidence aimed at combatting misleading undercutting evidence against the trustworthiness of experts: flooding evidence resistant communities with evidence from sources they trust in favour of the trustworthiness of sources they fail to trust.
(2) For combatting (relatively isolated) cases of irrational evidence resistance: increasing availability of cognitive flexibility training (e.g. in workplaces, schools, alongside anti-bias training). Cognitive flexibility training helps with enhancing open-mindedness to evidence that runs against one’s held beliefs, and to alternative decision pathways.
(12) DIGITAL TECHNOLOGIES - leveraging AI innovation: Promote safer and more trustworthy AI development through an applied AI strategy and by establishing the European AI Research Council to encourage industrial and usage of public-sector AI.
What is trustworthy AI? Policy makers and AI developers around the world have invested millions to answer this question. The motivation for this interest lies with the thought that societies will only ever be able to achieve the full potential of AI if trust can be established in its development, deployment, and use. If, for example, neither physicians nor patients trust an AI-based system’s diagnoses or treatment recommendations, it is unlikely that either of them will follow the recommendations, even if the treatments may increase the patients’ well-being. Similarly, if the general public doesn’t trust autonomous cars, they will never replace common, manually steered cars. Rational trust, however, requires trustworthiness: We should only trust an entity S when they are trustworthy. As such, if we are to expect users to trust a particular AI, we first need to understand what makes AIs trustworthy.
Several proposals in the form of ‘lists’ of features that make for trustworthy AIs can be found upon a simple Google search. These proposals list features allegedly constituting AI trustworthiness without also aiming to offer an underlying, unificatory rationale. It is claimed that trustworthy AI is, for instance, safe, just, explainable, human-centred, beneficent, autonomous, robust, fair, transparent, non-discriminatory, promoting social and environmental wellbeing, non-malificent, etc.
As with all list-based theories, unsurprisingly, these trustworthy AI frameworks suffer from one main problem: what is the trustworthy-making underlying property that delivers one particular list rather than another? Why should we think, for instance, that explainability belongs on the list, while transparency does not? Conversely, if we think that, on closer inspection, we should include transparency as well, why is that so? Short of having an answer to this question, we run the risk that our list merely covers paradigmatic cases of trustworthy AIs, at best. In turn, if this is so, we run the risk of relying on untrustworthy non-paradigmatic AIs and, conversely, of not trusting trustworthy non-paradigmatic incarnations thereof.
We have developed a complete model of trustworthy AI (https://doi.org/10.1007/s44204-023-00063-5). Its central idea is that whether AIs are trustworthy is a matter of whether they meet their function-based norms. This model serves provide a rationale for why a range of properties such as safety, justice, and explainability, are properties (often) instantiated by trustworthy AI, while other properties are not.
On the account of trustworthiness that we have developed in previous work (
https://doi.org/10.1111/nous.12448), a trustworthy entity is one that has a high enough disposition to meet the norms that govern it. What is attractive about a norm-centric account of trustworthiness is that it is not anthropocentric: it can apply to humans and AIs alike. For the purpose of generalising that account to trustworthy AI, is important to note that (i) artifacts have functions; and (ii) functions can generate norms. As such, we can derive norms governing Ais from their function, and assess their trustworthiness by how well they meet these norms.
AIs have two types of functions: design functions, sourced in the designer’s intentions, and etiological functions, sourced in what they are used for by users. In turn, functions generate norms of proper function. A heart, for instance, is properly functioning - functioning by the norm governing it - if it works in the way in which, in normal conditions, it reliably pumps blood in the circulatory system. Similarly, an AI will be properly functioning – thereby meeting the norms governing it – insofar as it works in the way that, in normal conditions, reliably leads to function fulfilment. Since Ais serve both design and user functions, they will only be trustworthy AIs insofar as they have strong enough dispositions to meet their function-based norms in both of these respects. For instance, a paradigmatic case of AI that is not well trusted by the population are AIs assessing credit scores. Since such AIs are paradigmatically designed to meet the function of reliably identifying credit-worthiness, they meet their design-based norms. However, users need to understand why their credit application was rejected, such that they know what to improve in the future. If the AI does not offer this information, it fails to meet its user etiological function-based norm, and is thereby not afforded rational user trust.