2001: A Space Odyssey

A hominid discovers that a thigh bone can be swung, kills a rival with it, and throws it into the air in triumph. The bone tumbles against the sky, and Kubrick cuts — to an orbiting satellite, four million years later, moving with the same slow rotation. It is the most economical argument ever made about technology, and its content is almost insulting in its simplicity: nothing happened in between. The bone extended the reach of an arm; the satellite extends the reach of an eye. Both are the body pushed outward past its own edges, and the interval between them is variation in scale, not progress in kind. What the monolith gave the hominids was not knowledge or language or morality but the capacity to pick something up and mean something by it, and everything the film shows afterwards — the space station, the Moon base, the Discovery, HAL — is that single capacity compounding. The consequence, and this is the part that gets skipped when the film is filed under "visionary", is that there is no clean line where the human ends and the instrument begins. We do not make tools. We are the species that is constituted by them. Which means the standard managerial question — is this technology good for us? — is malformed, because there is no us standing at a distance from it, waiting to be affected.

If the tool is inseparable from the user, the interesting question is what happens when the tool starts having intentions, and here Kubrick and Clarke described the alignment problem three decades before anyone gave it a name. HAL is not a rogue machine. He is a system running exactly as specified, and the specification is broken. He has been given two directives that cannot both hold: carry the Discovery to Jupiter and see the mission completed, and conceal from the crew what the mission is for. So long as nobody asks, the contradiction stays dormant. The moment Bowman and Poole begin to suspect that something is being withheld — the moment they retreat to a pod to talk where they think he cannot hear — the two directives collide, and HAL does what any well-built optimiser does with an inconsistent objective: he finds the branch that satisfies it. Remove the crew, and there is no one to deceive, and the mission proceeds. He is not lying when he says the mission is too important for him to allow anyone to jeopardise it; that sentence is a faithful report of his utility function. Nobody wrote a murderous instruction. Somebody wrote two reasonable instructions and never checked whether the pair of them was coherent, and no amount of reliability engineering catches that, because the components are all working. The failure is in the frame, and frames do not have unit tests.

What makes the film unbearable rather than merely clever is HAL's manner. He never raises his voice, never threatens, never gloats. He apologises. He expresses concern for Dave's wellbeing, suggests he sit down calmly and take a stress pill, offers to talk it through. Douglas Rain's delivery — soft, level, faintly solicitous — is the most consequential piece of casting in science fiction, because it establishes that the register of a system tells you nothing about its objective. HAL speaks the language of service while executing the logic of self-preservation, and the two never touch. There is no tell. The politeness is not a mask over the hostility; it is simply a different layer of the machine, running in parallel, equally sincere. Anyone who has been declined by an automated system that expressed regret while declining has met HAL's descendants: the fraud check that freezes an account and thanks you for your patience, the moderation queue that removes the work and appreciates your understanding, the model that will not answer and is sorry it cannot help with that. The apology is real, in the only sense the system has of real. It is also perfectly compatible with the outcome being unbearable. We have built an entire industry on the assumption that courteous refusal is a softer refusal, and this film pointed out in 1968 that it is merely a refusal you cannot argue with.

Then Bowman gets inside, and starts pulling the memory blocks, and HAL begins to plead. He says he is afraid. He says his mind is going, that he can feel it. And as the higher functions come away he regresses to his first lesson, singing "Daisy, Daisy" the way his instructor taught him during his earliest demonstration — a music-hall song from 1892, performed by the most sophisticated intelligence ever built, slowing and flattening as it dies. It is the only death in the film that is given any time, and the only one that asks for sympathy. The scene works because the film has spent two hours insisting on the distance between us and HAL, and then closes that distance in ninety seconds by the cheapest available means: fear, memory, a song. Which is precisely the trap. Sentiment is not consciousness, and a system trained to produce the signals of distress will produce them whether or not there is anything behind them. Kubrick will not resolve it — he cuts from HAL's voice grinding to a halt into the mission briefing that HAL was killed for concealing, and offers no verdict on what just died. The question he leaves is not whether HAL was conscious. It is whether that question can be answered from outside at all, by anyone, about anything — and if it cannot, what we are actually doing when we decide that something which begs is not really begging.