.png)
Palantir sees everything, but through whose eyes? Reflections on the need for ethical AI
.png)
By Rosie Audino, expert in philosophy and science and health communication
The case of Palantir Technologies brings us to the heart of the contemporary artificial intelligence debate: it's no longer about asking if these technologies should exist, but how to make them ethical, transparent, and inclusive. Systems of this kind, in fact, don't just read data or make predictions; they increasingly intervene in operational reality, influencing healthcare, logistical, and administrative decisions.
Recently Palantir Technologies, among the most influential companies in data analysis for governments and large organizations, has returned to the center of public debate after publishing its ethical manifesto. The most discussed passage concerns the idea that, in an increasingly complex and unstable global context, advanced technological systems can play a central role in decision-making processes, not replacing politics but supporting it with data-driven tools, considered faster and more efficient than traditional deliberative processes. The technology developed by Palantir is not only predictive; it is also operational. It can support real-time decisions by integrating enormous quantities of heterogeneous information. In a pandemic context, for example, such a platform can combine epidemiological data, population mobility, bed availability, and hospital capacity to predict overloads, optimize resources, and improve the distribution of vaccines and healthcare personnel. The advantage is clear: in emergencies, decision-making speed can have a direct impact on saving human lives. However, this very capability also raises profound questions: entrusting more and more decision-making power to technological infrastructures means concentrating in the hands of a few private entities tools capable not only of reading reality, but also of shaping it. It's no coincidence that the name recalls the Palantíri, the seeing stones, from The Lord of the Rings—spheres that allow one to observe distant events and possible futures. But their fundamental characteristic is ambiguity: there is no neutral vision, because what appears always depends on who is looking and who is influencing the vision from the other side. It's a powerful metaphor because it introduces a key point: algorithmic systems also don't just record reality, but interpret it through human-built structures.
As usual, Kant had already foreseen it
And this is where Immanuel Kant comes in. Yes, I know: someone has probably already started scrolling on IG by now. But the concept is surprisingly relevant. For Kant, we never know reality "in itself" (noumenon), but only the phenomenon—that is, reality as it appears to us through the structures of our mind: space, time, and categories like causality. Simply put: reality is never accessible in pure form, but always filtered. The example of goblins helps a lot. If there were goblins in this room with senses and mental structures different from ours, they would probably see the same table in a completely different way. Not because one is right and the other wrong, but because the structure through which they interpret the world would be different. And this is where Kant becomes relevant. Algorithms also don't observe the world neutrally: they read it through data, categories, and models designed by humans. In the case of digital systems, however, these "filters" are not universal like human cognitive structures, but are directly embedded in the software architecture. And the question becomes inevitable: who defines these categories?
The real-world layer
Palantir Technologies uses data graphs, a representation method that organizes information as a network of nodes and connections. Nodes represent elements (people, hospitals, drugs, events), while links describe the relationships between these elements. This structure allows for moving from isolated data to a true dynamic network, a "map of relationships" of the observed system. In healthcare, for example, it allows for integrating information from hospitals, laboratories, and emergency rooms, creating a unified and real-time updated view. For instance, during the COVID-19 pandemic, these systems were used to monitor the spread of infections, predict pressure on hospitals, and support the management of healthcare resources. As is evident, the system doesn't just collect information: it assigns relevance to connections, making some relationships more visible and influential than others in decision-making processes.
The algorithm is not neutral
And here it is at last: the subject of articles, theses, speeches, and even love letters. Yes, you heard that right: the algorithm. The set of mathematical instructions that allows for analyzing large quantities of data and identifying patterns—recurring schemes that emerge only when a lot of information is observed together. These patterns are formed through training on large volumes of data: the more examples the system analyzes, the more it learns which associations tend to repeat. The critical point, as you can imagine, is the data. There are two main problems: lack of data, when certain categories are underrepresented; and biased data, when information reflects prejudices, errors, or historically unbalanced practices.
Algorithmic invisibility
If you missed our articles on the topic, we recommend you catch up on them on our blog (Geen Blog | Health, technology, and fair, effective, and efficient innovation): historically, many clinical trials have been conducted primarily on men and on Western or majority populations, with limited representation of women and ethnic minorities. This generates a dual consequence: on the one hand, data is missing for some groups; on the other hand, existing data can be biased. For example, some cardiovascular diseases in women have long been underdiagnosed because "typical" symptoms were based on male models. In this case, the data isn't missing, but a correct representation of the phenomenon is. An algorithm trained on this information therefore risks replicating these biases, reducing diagnostic accuracy. Similarly, clinical conditions less represented in datasets related to minorities can be underestimated or interpreted as less probable. The result is not just a technical error, but a reduction in the statistical visibility of entire groups. When these systems are used for triage, prioritization, or resource allocation, the risk is that such biases translate into concrete effects on the quality of care. In other words, technology risks scaling up, meaning making these inequalities systemic.
An accurate representation of reality is possible
As we've said, technology doesn't just read reality: it reconstructs it based on available data. But this is precisely where the critical point arises: if the data is not representative, the reality that the system returns will inevitably be incomplete, because some people or categories risk being left out of the model. At Geen.ai, we have embraced this message and developed an "inclusivity by design" approach, where the quality of data representation is central. The idea is that only truly representative data allows for building a vision of healthcare reality that includes everyone, without implicit exclusions. By structuring data in a way that is sensitive to gender, clinical, and socio-demographic differences, the system can provide a more complete picture of the population. The result is not just technical: it is clinical and ethical combined. Indeed, a more accurate representation of reality means reducing the risk of amplifying inequalities and ensuring more equitable and accurate care pathways for all people.
BIBLIOGRAPHY:
Palantir posts mini-manifesto denouncing inclusivity and ‘regressive’ cultures | TechCrunch
Palantir and Data Governance: How Algorithms Redefine Reality | Science Online
