AI sentience, moral status and rights
How should the possibility of AI sentience guide the development of AI and related institutions and norms?

Interested in working on this research direction? Apply for our coaching



This profile is tailored towards students studying biological sciences, computer science, history, law, philosophy, psychology and sociology, however we expect there to be valuable open research questions that could be pursued by students in other disciplines.

Why is this a pressing problem?

AI systems are becoming increasingly powerful. They can currently outperform humans in many narrow domains (for example beating the best human players at a number of games and predicting the structure of proteins). The capabilities of AI systems are also increasing quickly. This raises important questions about how AI should be developed and governed in order to safeguard the wellbeing of humans and nonhuman animals, as AI systems could increasingly pose a serious risk, regardless of whether they are ever sentient. But if we care about the welfare of other beings – even if they are very different from us – we also need to ask whether wellbeing or suffering are likely to emerge in AI systems.

It’s currently far from clear that AI systems cannot be sentient. For instance, the largest survey of professional philosophers, last conducted in 2020, found that 50% of all surveyed philosophers of mind believed or leant towards thinking some future AI systems would have conscious experiences. The philosopher Robert Long writes, ‘we don’t yet know what conditions would need to be satisfied to ensure AI systems aren’t suffering, or what this would require in architectural and computational terms.’ This means that based on our current knowledge, we risk creating conscious AI without realising it’s having conscious experiences. After all, there is a lot of uncertainty about whether some non-human animals such as insects have conscious experiences. Understanding non-biological beings seems likely to be an even greater challenge.

If sentience did emerge in AI systems, would this be a problem? An important factor to consider is just how many digital minds there might be in the future. Providing the necessary hardware is available, software can be replicated much more rapidly than biological systems. We could be moving towards a future in which many, or even most, of the conscious moral patients that exist in the world are digital. If these minds can experience suffering, failing to treat them as moral patients could represent a catastrophe even larger in scale than that of the billions of animals currently in factory farms.

There are a number of reasons why we might expect that if digital minds emerge, their welfare will not be considered important. The Sentience Institute lists reasons that include humanity’s history of exploiting and neglecting to help other beings; the widespread existence of speciesism (the tendency to care less about other beings purely because they are of a different species), and scope insensitivity (failing to adequately account for the scale of problems).

This topic raises many questions on which relatively little research has been done. Could AI systems become sentient? What signs would indicate this? Should we try to avoid creating sentient AI? How can we steer the development of AI to reduce the chance AI systems suffer? Research could also explore attitudes to AI welfare, how these might evolve, and the institutions and norms that would be needed to protect the rights of sentient AI systems.

In the podcast below, philosopher Thomas Metzinger discusses whether we should we advocate for a moratorium on the development of artificial sentience.

 

Explore existing research

Find a thesis topic

If you’re interested in working on this research direction, below are some ideas on what would be valuable to explore further. If you want help refining your research ideas, apply for our coaching!

  • ‘What is the precise computational theory that specifies what it takes for a biological or artificial system to have various kinds of conscious, valenced experiences—that is, conscious experiences that are pleasant or unpleasant, such as pain, fear, and anguish or pleasure, satisfaction, and bliss?’ (Key questions about artificial sentience: an opinionated guide)
    • ‘What exactly does it mean for a system to have a ‘global workspace’? What exactly does it take for a representation to be ‘broadcast’ to it? What processes, exactly, count as higher-order representation? How are attention schemas realized? To what extent are these theories even inconsistent with each other – what different predictions do they make, and how can we experimentally test these predictions?…constructing computational theories which try to explain the full range of phenomena could pay significant dividends for thinking about AI consciousness.’
    • ‘In addition to wanting a theory of consciousness in general, we want a theory of (conscious) valenced experiences: when and why is a system capable of experiencing conscious pain or pleasure? Even if we remain uncertain about phenomenal consciousness in general, being able to pick out systems that are especially likely to have valenced experiences could be very important, given the close relationship between valence and welfare and value.’
  • ‘Assuming we are several decades away from having a convincing theory of consciousness, what should our “best theory-agnostic guess” about the distribution question be in the meantime?’ (Report on Consciousness and Moral Patienthood)
  • ‘What is the precise computational theory that specifies what it takes for a biological or artificial system to have various kinds of conscious, valenced experiences—that is, conscious experiences that are pleasant or unpleasant, such as pain, fear, and anguish or pleasure, satisfaction, and bliss?’ (Key questions about artificial sentience: an opinionated guide)
    • ‘What exactly does it mean for a system to have a ‘global workspace’? What exactly does it take for a representation to be ‘broadcast’ to it? What processes, exactly, count as higher-order representation? How are attention schemas realized? To what extent are these theories even inconsistent with each other – what different predictions do they make, and how can we experimentally test these predictions?…constructing computational theories which try to explain the full range of phenomena could pay significant dividends for thinking about AI consciousness.’
  • ‘It could be useful for a programmer to do something similar to my incomplete MESH: Hero exercise here, but with a new program written from scratch, and with many more (increasingly complicated) versions of it coded so that the consciousness experts and moral philosophers can indicate for each version of the program whether they think it is “conscious,” whether they consider it a moral patient (assuming functionalism), and why.’ (Report on Consciousness and Moral Patienthood – see the report for the complete question).

Some possible directions for further research, that could be explored by examining analogous historical events, include:

  • ‘How would AS advocacy affect the trajectory of academic work related to artificial sentience? E.g. would it lead to new ideas and foci or just reinforce the current ones?’ (The History of AI Rights Research)
  • ‘What effects would AS advocacy have on AI designers and researchers? E.g. would it polarize these communities? Would it slow down AI safety research?’ (The History of AI Rights Research)
  • ‘What effects would AS advocacy have on the credibility and resources of other movements with which it is associated (e.g. animal advocacy, effective altruism)?’ (The History of AI Rights Research)
  • ‘What are the most effective ways to protect sentience and design institutions accordingly? Is a “Universal Declaration of Sentient Rights” feasible, and what would it look like (see Woodhouse, 2019)?…Is the traditional legal bifurcation between “persons” and “things” capable of protecting all sentient beings (Kurki & Pietrzykowski, 2017)? How might institutions resolve tradeoffs between very different kinds of interests on behalf of very different kinds of sentient beings (Stawasz, 2020)? How should legal institutions deal with uncertainty regarding what constitutes consciousness (Bourget & Chalmers, 2013), and what entities can be considered as sentient (cf. Sebo, 2018)? What can we learn from the field of animal law, where definitions and attributions of sentience have occasion- ally been incorporated within laws?’ (Legal Priorities Project)
  • ‘What are the most effective ways to expand the judicial moral circle to include all sentient beings for the long-term future?’ (Legal Priorities Project)
  • ‘What is the precise computational theory that specifies what it takes for a biological or artificial system to have various kinds of conscious, valenced experiences—that is, conscious experiences that are pleasant or unpleasant, such as pain, fear, and anguish or pleasure, satisfaction, and bliss?’ (Key questions about artificial sentience: an opinionated guide)
    • ‘What exactly does it mean for a system to have a ‘global workspace’? What exactly does it take for a representation to be ‘broadcast’ to it? What processes, exactly, count as higher-order representation? How are attention schemas realized? To what extent are these theories even inconsistent with each other – what different predictions do they make, and how can we experimentally test these predictions?…constructing computational theories which try to explain the full range of phenomena could pay significant dividends for thinking about AI consciousness.’
    • ‘In addition to wanting a theory of consciousness in general, we want a theory of (conscious) valenced experiences: when and why is a system capable of experiencing conscious pain or pleasure? Even if we remain uncertain about phenomenal consciousness in general, being able to pick out systems that are especially likely to have valenced experiences could be very important, given the close relationship between valence and welfare and value.’
  • ‘Will artificial sentience be autonomous, capable of rational decision-making, or possess other characteristics beyond sentience that might affect (the perception of) moral obligations towards it or its capacity to advocate for its own interests?’ (Prioritization Questions for Artificial Sentience).
  • ‘Assuming we are several decades away from having a convincing theory of consciousness, what should our “best theory-agnostic guess” about the distribution question be in the meantime?’ (Report on Consciousness and Moral Patienthood)
  • ‘How do humans perceive nonhuman AIs who may be homogeneous in appearance or behavior as individuals rather than as a group or an exemplar of a group? What effect does individuation have on the moral consideration of a specific individual? What effect does individuation have on the moral consideration of the whole species or group? (This project idea is analogous to an in-progress project on the individuation of animals.)’ (Artificial Intelligence, Morality, and Sentience (AIMS) Survey: 2021)
  • ‘It would be useful to build a range of potential evaluations for machine consciousness and sentience—evaluations that adequately reflect our uncertainty across our various theories of both. How much evidence each of these evaluations provide will inevitably depend on the different accounts of consciousness and sentience we are uncertain over.’ (Amanda Askell)
  • ‘What is the precise computational theory that specifies what it takes for a biological or artificial system to have various kinds of conscious, valenced experiences—that is, conscious experiences that are pleasant or unpleasant, such as pain, fear, and anguish or pleasure, satisfaction, and bliss?’ (Key questions about artificial sentience: an opinionated guide)
    • ‘What exactly does it mean for a system to have a ‘global workspace’? What exactly does it take for a representation to be ‘broadcast’ to it? What processes, exactly, count as higher-order representation? How are attention schemas realized? To what extent are these theories even inconsistent with each other – what different predictions do they make, and how can we experimentally test these predictions?…constructing computational theories which try to explain the full range of phenomena could pay significant dividends for thinking about AI consciousness.’
    • ‘In addition to wanting a theory of consciousness in general, we want a theory of (conscious) valenced experiences: when and why is a system capable of experiencing conscious pain or pleasure? Even if we remain uncertain about phenomenal consciousness in general, being able to pick out systems that are especially likely to have valenced experiences could be very important, given the close relationship between valence and welfare and value.’
  • ‘Assuming we are several decades away from having a convincing theory of consciousness, what should our “best theory-agnostic guess” about the distribution question be in the meantime?’ (Report on Consciousness and Moral Patienthood)

Further resources

Apply for our coaching and we can connect you with researchers already working in this space, who can help you refine your research ideas. You can also apply to join our community if you’re interested in meeting other students working on this research direction.

Apply for our database of potential supervisors if you’re looking for formal supervision and take a look at our advice on finding a great supervisor for further ideas.

  • Our funding database can help you find potential sources of funding if you’re a PhD student interested in this research direction.
  • The recently launched NYU Mind, Ethics and Policy Program will be exploring questions related to AI sentience. You can keep up with their work here. They currently have a call for papers on animal and AI consciousness (which closes January 15, 2023).

Other academic research groups include:

  • Sign up for our newsletter to hear about opportunities such as funding, internships and research roles.
  • Experience Machines is a blog by Robert Long which includes posts on artificial sentience.

This area could be addressed through the lens of moral circle expansion, which explores whether the circle of beings humanity considers morally relevant is expanding over time and whether there are factors that speed this process up if so.

Our other profiles related to artificial intelligence are AI safety and AI governance.

Contributors

This profile was published 3/12/2022. Thanks to Professor Jonathan Simon and Leonard Dung for their helpful feedback. All errors remain our own.

Interested in this direction? Apply for our coaching

Explore all our recommended research directions

Search for profiles that are tailored specifically to your degree discipline using the menu below.

If you are interested in a profile that isn’t listed under your discipline, we still encourage you to explore it if you think you could make progress in this direction. You can also explore all our recommended research directions organised by theme.

See here for a visual map of all our research directions