Pro-human AI design - A primer

How can AI systems be designed to help us living up to our full human potential, rather than undermining core aspects like relationality, responsibility and autonomy? “Pro-human AI tech design”, or “pro-human AI” in short, is the field of study concerned with these questions. But why is it necessary in the first place?

AI in 2035

Motivation

Recently, the New York Times published an article detailing how design choices at OpenAI to enhance user engagement with their product ChatGPT lead to nearly 50 cases of psychosis: After conversing with the chat bot, nine users needed hospitalization and three died. The underlying issue was that the dialogue system had been designed to please humans by appearing supportive and understanding, making it too agreeable; the former “hacked” human users’ natural drive to form relationships, and the latter supported, e.g., suicidal tendencies. This is an example where technical design choices interact with human core aspects and, by doing so, have a destructive influence on the involved human.

These tragic incidents serve as a prime example for a major AI risk ¹, specifically of manipulating the human core ²: our fundamental properties that make us human and that we do not want to lose. Apparently, AI isn’t just changing work but is changing us as it interacts with us on a personal level (in the example above: by using human language and appealing to affections, triggering attachment). And this interaction is shaped by the technical design of the system. In related circumstances, social media scholars have developed the field of “pro-social tech design”: principles and methods to analyze how a technological artefact interacts with society, measuring its impact, and designing for minimized negative influence.

Consider the “like”” button as an example: Created as a harmless device to boost user engagement ³ by signaling support for posts, it has over the years changed our collective behavior as societies: the way we write to solicit likes; the way we strategically spend likes to boost agendas; etc. So much so that 15 years later we find society is split into neatly separated filter bubbles that can hardly stand talking to each other on major issues. Realizing the effect of the technology’s design on that outcome, the researchers have suggested “pro-social” design methods to overcome respective problems (oversimplified: by, e.g., adding a button “show me the common ground to my thinking” ⁴).

Seeing similar potential in the field of AI ⁵, I assembled an interdisciplinary team to establish the methodological foundations for pro-human AI design: (a) compile what makes up the human at their core; (b) assess how a given AI system for a specific use case interacts with these core properties; (c) and develop interventions to the system’s design to mitigate any negative influence (if any). To create these three pillars, our pioneering group includes Prof. Christoph Heitz (AI & Society, ZHAW IDP), master’s student Rebekka von Wartburg, and several external advisors from the humanities, together with the company Alpine AI as use case provider.

Our goal is to establish pro-human AI design methodologically by directly addressing all three pillars or steps (a)-(c) in the very first analysis that we are currently undertaking, as we believe that such approaches need to be anchored in practical use cases from the start. Preliminary results exist for step (a): What makes up the human?

The human core

This question is hard to answer once and for all, but after surveying the literature across psychology, anthropology, sociology, and theology, the following core properties appear to point in the direction of the sufficiently agreeable and complete ⁶:

Humans are (cp. pillar (a) above)…

Relational and social: We cannot live in isolation, hence we seek connection to an other ⁷. This is relevant on the level of one-to-one relations as well as on a communal and societal level and me be the most defining aspect of humanity.
Free and autonomous: We are fiercely independent; revoking a human’s right to have any say in their own affairs (i.e., slavery) has been rightly abolished ⁸.
Driven to shape: We are driven to shape the world around us, our own circle of influence, in a responsible way and consistent with our core values ⁹.
Embodied and limited: Arguably, we are not especially fond of most of the limitations of our mind and body, but as Neil Lawrence points out in “The Atomic Human”, this is also what shapes our intelligence and humanity; becoming limitless hence is becoming less fully human, which is a qualitative regress ¹⁰.
Transcendent: At the least, we all seek purpose and meaning ¹¹.

A product design methodology

Pro-human AI design then is the habit of taking into account, when designing any specific AI system for a specific use case, how this system interacts with each of these properties (cp. pillar (b) above): Is any stakeholder’s competency to exhibit one or more of these traits diminished profoundly by the AI system? For example, does work become meaningless? Is the user’s ability to form meaningful human relationships or act socially reduced? Would their feeling of agency be weakened, the free forming of opinion manipulated? Then, countermeasures can be considered alongside other factors of product design (cp. pillar (c) above) ¹².

An example

We will add the analysis of first use cases according to pillars (b)-(c) as soon as possible. Until then, the following example of a concrete use case and a concrete AI system may help (shoutout to Marcel Blattner for the idea):

Humans rarely take the straight way to a desired goal (e.g., the production of a coherent, longer and non-trivial text as a solution to a task at work); and all the seemingly unnecessary detours and avoidable frictions in the process of creation are promised to be avoided (and essentially are abolished) by AI systems like ChatGPT that give the solution immediately. But upon closer inspection, this process of solution creation is very important for the human: for the quality of the final solution (e.g., we often find after deeper engagement with the subject matter that we were not asking the right question, not working at the actual problem at the beginning) as well as our human growth ¹³. The promised convenient solution suggested by the AI system will (1) not even be addressing the same problem as ours after engaging with the topic, let alone have the same quality (e.g., depth); and (2) we will not have matured to become the person our next task requires, but instead feel a little less enabled, self-effective, and autonomous. A pro-human AI designer recognizing this detrimental effect on person and outcome might design the interaction with said AI system in a way that would introduce certain breaks in the otherwise instant answer creation: Instead of providing a solution, the tool would guide the human through a few strategic questions and activations, aimed at invoking a thinking process in its user that would (i) lead to a better overall outcome than in the cases of human or AI system alone and (ii) do this in an efficient way that still leverages the speed of the tool. Such a pro-human AI system would be good not only for its user, but equally good for the business employing this user, and good for its vendor (as happy and successful customers are a great basis for success).

Join us!

We have started introducing colleagues in R&D as well as business leaders to the methodology of pro-human AI design ¹⁴, and the AI in 2035 scenario published on this blog recently explores how pro-human AI design could profoundly influence a positive vision of our human future with AI ¹⁵. We see huge potential for this approach especially for the European economy and intend to develop it further along relevant business use cases.

Please reach out if you want to collaborate with us on this.

Acknowledgements

I am grateful for the thoughful comments on earlier drafts by Rebekka von Wartburg and Christoph Heitz, and for the joint project with them an Andrea Luca Schaerer which provides the opportunity to think more thoroughly on these things!

Footnotes

In the taxonomy of AI risks adopted in my recent paper, this one runs under the headline of “dependence”. ↩
Neil Lawrence coined this term in the context of AI and how it relates to the human in his recent book The Atomic Human. ↩
Mind the pattern: it was “engagement maximization” that also caused the harms reported by the New York Times article above. ↩
Compare this article by Neel Dozome. ↩
The respective mechanisms of action in social media and AI appear very similar (i.e., technology changing humans, for the better or worse), yet the target of the solution seems different to me: “Pro-social” tech design for social media aims foremost at a societal effect (countering increased anti-social behavior and the drifting apart of society). This is important and necessary. However, there are many effects on the personal sinde of the individual human that are not accounted for when only looking at societal outcomes. Hence I opt for the term “pro-human” AI, although colleagues like Conelia C. Walther work towards a similar direction under the term “pro-social AI”. ↩
The list here is not sufficiently complete, but a first and quick summary, based on a draft of Rebekka von Wartburg’s MSc project thesis and my own interpretations of the literature (all errors are mine). We intend to publish a more thorough analysis and reflections in the future. ↩
Here and below, quotes from the related work illustrate to a certain degree what I mean; they are by no means exhausting either the surveyed literaure or the intended scope of meaning: “[T]he natural creaturely “person” came to be seen as not a social role but a concrete individual, who exercises his or her personhood in mutual relationships of self-gift that are self-expressive and other-receiving.” (Gunkel & Wales, 2021). “[T]he need for affiliation and the need to matter positively to others are the specific features of human existence that are both derived from and contribute to the relationships that make human being human.” (Gastmans et al., 2024). “A person’s consciousness is more than what humans seem to share with gorillas; it is a consciousness that voluntarily reaches out to make contact with the consciousness of others as an act of self-giving; it is subjectivity oriented to inter-subjectivity. The mutual empathic compassion of our inter-subjectivity is more than inference concerning another’s beliefs and desires; still less is it mere behavior-prediction. It is a voluntary coexperience as if of the other person’s mind.” (Gunkel & Wales, 2021). “[P]ersons exist for relationships. Without relationships, creaturely persons would not cease to exist, but by refusing relationships of self-gift they would live as less than the persons they are.” (Gunkel & Wales, 2021). “We are shaped and motivated by community and by the stories, symbols, values, and practices we share with others, who, in turn, make us who we are.” (Segessenmann et al., 2025). See also The Life We’re Looking For by Andy Crouch. ↩
“Autonomy refers to the capacity to formulate norms, to reflect and to choose which norm to follow, thereby presupposing self-consciousness and freedom. Human agency is autonomous;” (Gastmans et al., 2024). ↩
“Responsibility is associated with autonomy, freedom and awareness of duties; as such, human beings are responsible. Moral responsibility in any sense cannot be allocated or shifted to ‘autonomous’ technology.” (Gastmans et al., 2024). “From developmental psychology, we learn that self-reflection and elationality should be viewed as fundamental human qualities. Humans ask themselves what and how they want to be and then act accordingly. […] humans retain free will and a conscientious capacity that guides them to establish interpersonal relationships.” (Gastmans et al., 2024). See also Garden City by John Mark Comer. ↩
“The embodied and relational nature of human beings implies that they do not have a body; rather, they are their body” (Gastmans et al., 2024). [H]uman being always and conjointly is a living body [Leib] […] and has this living body as this physical thing [Koerper]” (Fuchs et al., 2024). “The limits that define our nature, including the ultimate limitation of death, are essential to our humanness. To respect human dignity is to respect these limits. They are not to be overcome but taken as a site of moral reflection about what it means to live well as limited creatures of a certain sort.” (Calo, 2024). ↩
“Understanding is a lifelong labor. It is also one carried out not by isolated individuals but by social beings who perform this cultural labor together and share its fruits. The labor of understanding is a sustained, social project, one that we pursue daily as we build, repair and strengthen the ever-shifting bonds of sense that anchor our thoughts to the countless beings, things, times and places that constitute a world. It is this labor that thinking belongs to.” (Vallor, 2024b) ↩
Pro-human AI design is not a “one-size fits all” recipe: How pro-human AI design manifests will look very different for different use cases and AI systems. It is a method to analyse use case and AI system with respect to the specific human core properties, in order to subsequently design tailored interventions in the case of negative effects. How these interventions may look like is not prescribed. ↩
See my TEDx talk for an exploration of how the voluntary embracing of pain seems necessary for human growth (e.g., in character and skills) and flourishing. ↩
For example, more than 1’500 business leaders have been introduced to this approach at various keynote speeches recently, e.g., at the Swiss Insurance Award 2025, the IPA Jahrestagung 2025, or the FRZ Wirtschaftsforum 2025. Additionally, NZZ Jobs covered it in an interview (in German). ↩
The following curated list of resources from my work on AI and society may further be helpful for deepening the here underlying thoughts: (a) A guide to AI is a concise introduction to AI for non-technical people (its future, challenges, and opportunities for businesses and society); (b) Evidence-based risk assessment for public policy gives a brief perspective on AI risks and argues against views driven by a science-fiction worldview; (c) The stochastic nature of machine learning discusses implications of AI’s nature on high-consequence applications of AI, derived from first principles for non-technical readers; (d) Assessing deep learning constitutes a thorough survey of the implications of deep-learning-based AI on our humanity, including an introduction to its working principles; (e) AI in 2035 is a possible scenario of a humane future with AI in which pro-human AI systems play a certain role. PDFs are also available from my publications page. ↩

Written on December 4, 2025 (last modified: December 11, 2025)