While the EU once pledged to prevent a China-style social credit system, Brussels is now quietly dismantling the safeguards intended to protect citizens from automated profiling. Through its proposed Digital Omnibus, the Commission risks codifying a society where invisible algorithms decide who gets a job, a loan, or a home.
The danger lies not in one singular, massive score, but in the hundreds of smaller, opaque rankings that govern daily life. German credit agency SCHUFA, for instance, holds data on 69 million people, with scores dictating access to basic services. Recent investigations by the NGO noyb into SCHUFA’s storage of historical records highlight the impossibility of challenging results when the underlying data remains a black box.
This trend toward automated gatekeeping has already yielded concrete victims. Last month, Dutch regulators fined Uber €825 million for deactivating drivers based on algorithmic fraud detection without human intervention. Similarly, the Dutch education agency DUO used discriminatory variables—such as home location and migration background—to flag 21,500 students for fraud investigations. These systems frequently disguise systemic inequality as objective efficiency, disproportionately harming the most vulnerable.
Despite these precedents, the European Commission is moving to rewrite Article 22 of the GDPR, a cornerstone of existing digital privacy rights. The proposed change would allow companies to claim that fully automated decision-making is ‘necessary’ for contracts, even when human alternatives exist. By prioritizing corporate efficiency over transparency, Brussels is signaling that the burden of algorithmic error should shift from the institution to the individual. As the European Commission develops its own AI tools to rank job applicants, the question remains: if safeguards are treated as mere obstacles to innovation, who will be left to hold the machines accountable?
Comments (0)
No comments yet. Be the first!