The Ethics of Automated Decisions

Automated systems now decide who receives a loan, a job interview and a medical referral. Their appeal is obvious: they are fast, cheap and consistent.

自动化系统如今在决定谁能获得贷款、面试机会和医疗转诊。它们的吸引力显而易见：快、便宜、一致。

Consistency, however, is not fairness. A model trained on historical data reproduces historical patterns, including the unjust ones. Removing an explicit variable such as ethnicity does not remove its influence, because correlated features carry the same information.

然而，一致并不等于公平。一个用历史数据训练出来的模型会复制历史模式，其中也包括不公正的那些。删掉「族群」这样的显性变量并不能消除它的影响，因为相关的特征携带着同样的信息。

The deeper difficulty is accountability. When a decision is distributed across data, designers and deployment, no single actor is responsible. Appeals then have nowhere to go.

更深的困难在于问责。当一项决定分散在数据、设计者和部署环节之间时，没有任何单一主体为之负责。于是申诉无处可去。

A defensible framework requires three things: documented training data, measurable error rates across groups, and a human who can reverse any decision. None of these is technically exotic. What they demand is institutional willingness.

一个站得住脚的框架需要三样东西：有据可查的训练数据、按群体可衡量的错误率，以及一个能够推翻任何决定的人。这些在技术上都不稀奇，它们要求的是机构层面的意愿。

Transparency is therefore necessary but insufficient. Publishing a model's inputs is of little use if the people affected cannot understand the output, and the algorithm decided is not an answer anyone can act on.

因此，透明是必要的，但并不充分。如果受影响的人看不懂输出，公开模型的输入就没有多大意义；而「算法决定的」也不是任何一个人可以据以行动的答复。

What makes the problem tractable is treating it as a governance question rather than a purely technical one. Technical work can measure how error rates differ between groups; only institutions can decide how much inequality is acceptable — and who must answer when the answer is none.

让这个问题变得可解的，是把它当作治理问题、而不是纯粹的技术问题。技术工作可以测量不同群体之间的错误率差异；但只有制度才能决定多大的不平等是可以接受的 —— 以及当答案是「一点也不能接受」时，由谁来负责。