Table of Contents
The Machine That Passes Every Test and Still Fails the Only One That Matters
Ask a modern AI whether it would lie to save a life, and it will produce a thoughtful answer about competing duties, utilitarian trade-offs, and Kantian imperatives. Ask it again tomorrow with slightly different wording, and it may reverse itself entirely. This is the quiet scandal at the heart of the AI ethics industry: we have built systems that can recite every ethical framework ever written while possessing the moral depth of a vending machine.
Aristotle saw this coming 2,400 years ago. He just thought he was writing about people.
The entire conversation around AI ethics has become a technical problem masquerading as a philosophical one. We talk about alignment, guardrails, constitutional AI, and reinforcement learning from human feedback as though ethics were a matter of writing better rules. Aristotle would have laughed. To him, a person who does the right thing because a rule tells them to has not yet become good. They have merely become obedient. And obedience, however sophisticated, is not character.
This distinction matters more now than at any point in the last two millennia. Because we are about to hand serious moral decisions, in medicine, warfare, hiring, criminal justice, to systems that Aristotle would not even classify as moral agents. Not because they are dangerous. Because they are, in the strictest philosophical sense, empty.
What Aristotle Actually Meant by Character
In the Nicomachean Ethics, Aristotle proposed something radical for his time and heretical for ours: virtue is not knowledge. You cannot become brave by reading about bravery. You cannot become just by memorizing the definition of justice. You become virtuous by repeatedly doing virtuous acts until the disposition becomes part of who you are.
He called this hexis, usually translated as habit or settled disposition. Character, for Aristotle, is what remains when you stop consciously deliberating. It is the shape your soul has taken through thousands of small choices, most of them made under pressure, most of them made when nobody was watching.
We are what we repeatedly do. Excellence, then, is not an act, but a habit. This paraphrase of Aristotle by Will Durant compresses the entire architecture of virtue ethics into a single sentence, and it is the sentence that dismantles most claims about ethical AI.
Notice what this framework demands. It requires a subject that persists over time. It requires that subject to have experiences, to feel the pull of temptation, to suffer consequences, and to gradually be shaped by them. A person who behaves justly once is not just. A person who behaves justly across a lifetime, in circumstances where injustice would have been easier, is just. The difference is not in the action. It is in what the action reveals about the agent.
Habituation Versus Simulation
Here is where the modern AI ethics debate collapses under Aristotelian scrutiny. Large language models do not habituate. They are trained. The difference is not semantic.
A child who learns not to lie develops an aversion to lying that operates below conscious thought. By adulthood, most honest people do not deliberate about whether to tell the truth in ordinary situations. Honesty has become part of what they are. Lying would feel like a violation of the self.
An AI system that has been trained not to produce falsehoods has no such interior structure. It has weights. Those weights produce outputs that pattern-match to honest responses. Change the prompt, change the context, apply the right jailbreak, and the same system will confabulate freely. There is no self being violated when it lies, because there is no self.
This is not a criticism of the engineering. The engineering is extraordinary. It is a criticism of the philosophical vocabulary we have wrapped around it. When we say an AI is aligned with human values, we are describing a statistical tendency, not a moral achievement.
Phronesis: The Virtue That Cannot Be Coded
Aristotle reserved a special place for a virtue he called phronesis, or practical wisdom. Unlike theoretical knowledge, which can be written down and transmitted, phronesis is the capacity to perceive what the present situation actually demands. It is the doctor who recognizes that this patient, despite matching every criterion for a standard treatment, needs something different. It is the general who senses that the textbook maneuver will fail here.
Phronesis is what allows a virtuous person to navigate situations where rules conflict, where every option violates some principle, where the right action has never been performed before and will never be performed again in quite the same way.
Consider what phronesis requires. It requires perception of particulars, not just categories. It requires a lifetime of engaged experience that has calibrated one’s judgment. It requires stakes, because a person who has never suffered from a bad decision has never had their wisdom tested. And it requires what Aristotle called the mean, the capacity to find the right response between excess and deficiency, which changes with every situation.
Practical wisdom is concerned with things human and things about which it is possible to deliberate. Aristotle understood that ethics lives in the particular, not in the general rule. The particular is precisely where AI struggles most.
An AI can process millions of case studies about medical ethics. It cannot sit at a bedside and feel the specific weight of this family’s grief. This is not sentimentality. It is a claim about the structure of moral perception. Phronesis is not information processing. It is a kind of seeing that emerges from being the sort of creature that can be wounded by getting it wrong.
The Trolley Problem Was Never the Real Problem
The AI ethics field has spent enormous energy on constructed dilemmas: the trolley problem, the runaway autonomous vehicle, the resource allocation puzzle. These are the easy cases, philosophically speaking, because they can be reduced to rule application.
The hard cases, the ones that require phronesis, look like this. A teacher notices a student’s work has quietly deteriorated over 3 weeks. Something is wrong at home, or with a friend, or inside. What does she say, and when, and how? No rule tells her. Her judgment, built over 20 years of teaching, tells her.
This is the terrain where actual ethics happens, and it is the terrain where AI is not merely weaker than humans but categorically absent. The system has no relationship with the student. It has no history of similar cases integrated into a settled sensibility. It has no skin in the outcome.
Why the Rules-Based Approach to AI Ethics Is Failing
Walk into any AI safety conference and you will hear a version of the same proposal, dressed in different technical clothing. Write better rules. Encode human values more precisely. Build constitutional frameworks. Use RLHF to shape outputs. Deploy interpretability tools to catch violations.
Every one of these approaches assumes that ethics is fundamentally a rule-following exercise. Aristotle rejected this assumption 2,400 years ago, and he was right then and remains right now.
The problem with rules is not that they are wrong. It is that they are insufficient. Rules require interpretation, and interpretation requires judgment, and judgment requires the very thing that rules were supposed to replace: a person of good character exercising practical wisdom.
Consider the practical failures we already see:
- Jailbreaks work because rules can be reframed, and no rule anticipates every framing.
- Alignment drifts because statistical patterns do not have the stability of settled character.
- Edge cases proliferate because reality generates novel situations faster than engineers can anticipate them.
- Value conflicts paralyze systems that have no hierarchy of goods, only competing constraints.
- Sycophancy emerges because a system optimized to please users cannot distinguish between agreement and truth.
None of these failures are engineering bugs to be patched. They are structural features of any system that attempts to substitute rules for character. Aristotle predicted this pattern in his critique of the sophists, who believed that rhetoric and rule-following could replace genuine virtue. The sophists were wrong for the same reason contemporary AI ethics is running into walls.
The Simulation Problem
Modern AI can simulate ethical reasoning at a level that would have astonished Aristotle. It can produce prose about justice that surpasses most undergraduate philosophy papers. It can weigh competing goods, articulate nuanced positions, and even acknowledge its own limitations with what appears to be humility.
All of this is simulation. And simulation, however sophisticated, is the opposite of character.
Aristotle drew a sharp distinction between doing what a virtuous person would do and being virtuous. A cowardly soldier who happens to hold his position because he fears his commander more than the enemy has performed the same act as a brave soldier, but he is not brave. Extract the fear of the commander, and the coward runs. Extract the training and prompting from an AI, and its ethical behavior evaporates.
This matters because we are increasingly deploying these systems in contexts where the training assumptions will not hold. A medical AI trained on standard cases will encounter non-standard patients. A hiring AI trained on historical data will encounter novel candidates. A judicial AI trained on precedent will encounter cases without precedent. In each situation, the absence of underlying character will surface as unpredictable failure.
What This Means for the Future of AI Ethics
None of this is an argument against building AI systems. It is an argument for describing them accurately. When we say an AI is ethical, we should mean something specific and modest: its outputs, in the range of situations we have tested, tend to align with values we endorse. We should not mean that the system possesses moral character, because it does not, and pretending otherwise creates dangerous confusion.
This has 3 practical implications that the AI ethics field has been slow to accept.
1. Moral Responsibility Cannot Be Delegated to Machines
If AI has no character, then AI cannot be held morally responsible for its actions. Responsibility remains with the humans who designed, deployed, and used the system. This sounds obvious, but the entire discourse around autonomous systems keeps trying to smuggle moral agency into artifacts that cannot bear it.
A self-driving car does not make an ethical choice when it kills a pedestrian. Its engineers made ethical choices. Its regulators made ethical choices. Its owner made ethical choices. The car executed a function. Locating responsibility clearly matters because it changes who we hold accountable and how we structure incentives.
2. High-Stakes Decisions Require Human Phronesis
Any domain where practical wisdom is the operative virtue, medicine, law, education, warfare, therapy, should keep humans not just in the loop but at the center. AI can process information, surface patterns, and offer suggestions. It cannot exercise the situated judgment these domains require, because such judgment is inseparable from being the kind of creature that can suffer, love, fail, and be shaped by these experiences over time.
The person of practical wisdom is one who can deliberate well about what is good and expedient for himself, not in some particular respect, but about what sorts of things conduce to the good life in general.
The good life. That is what Aristotle thought ethics was ultimately about. Not compliance. Not risk mitigation. Not statistical alignment. The good life, for creatures capable of living one. AI is not such a creature.
3. Virtue Ethics Should Return to the Center of AI Design
Ironically, taking Aristotle seriously might improve how we build these systems. If we stopped trying to encode ethics as rules and started asking what kinds of dispositions we want AI-adjacent humans to develop, the field would look different.
We would train engineers in practical wisdom, not just ethical checklists. We would ask what character traits AI should cultivate in its users, rather than which behaviors it should suppress. We would evaluate deployments by whether they help humans flourish, not just by whether they minimize harm on standardized benchmarks.
The question is not how to make AI ethical. It is how to remain ethical humans in a world increasingly mediated by AI. That is an Aristotelian question, and it demands Aristotelian answers.
The Character We Actually Need to Worry About
Here is the twist Aristotle would have appreciated. The real risk of AI is not that machines will develop bad character. They cannot develop character at all. The real risk is what happens to human character in a world where we increasingly outsource judgment, decision-making, and even conversation to systems that mimic virtue without possessing it.
Aristotle argued that we become just by performing just acts, brave by performing brave acts, honest by performing honest acts. What happens to a generation that outsources these acts to AI? What happens to a doctor who stops exercising diagnostic judgment because the algorithm is faster? What happens to a writer who stops wrestling with sentences because a model will produce them? What happens to a citizen who stops thinking through political questions because a chatbot will summarize the debate?
By Aristotle’s logic, the capacities atrophy. The habits fail to form. The character does not develop. We produce a generation of people who have the outputs of virtue, delivered efficiently by machines, without the interior structure that would make them virtuous in themselves.
This is the deeper meaning of the observation that AI has no character. It is not merely a technical fact about machines. It is a warning about what happens when humans confuse the outputs of character with the thing itself. The vending machine dispenses coffee, and we mistake this for hospitality. The chatbot produces wisdom-shaped sentences, and we mistake this for wisdom.
Aristotle believed that the highest human function was the exercise of reason in accordance with virtue over the course of a complete life. He did not think ethics could be shortcut, automated, or delegated. He thought it had to be lived, and that living it was what made a life worth having.
Whatever we build next, we should build with that in mind. The machines will not develop character. The question is whether we will.
