Issue link: https://maltatoday.uberflip.com/i/1373323
13 maltatoday | WEDNESDAY • 19 MAY 2021 ANALYSIS THE EU looks set to ban some of the most concerning forms of artificial in- telligence (AI), such as the "social credit" surveillance system used in China, ac- cording to draft AI regulations published by the bloc. The proposed regulations, which will be reviewed by elected representatives before passing into law, will also bring some comfort to those outraged by in- stances of bias and discrimination gen- erated by AI. These include hiring algorithms found to systematically downgrade women's professional profiles, and flawed facial recognition technology that has led police to wrongfully arrest black peo- ple in the US. Such AI applications are regarded by the EU as high-risk, and will be subject to tight regulations, with hefty fines for infringement. This is the latest step in the European discussion of how to balance the risks and benefits of AI. The aim appears to be to protect citizens' fundamental rights while maintaining competitive innovation to rival the AI industries in China and the US. The regulations will cover EU citizens and companies doing business in the EU, and are likely to have far-reaching consequences, as was the case when the EU introduced data regulations in 2018. The proposals are also likely to inform and influence the UK, which is current- ly developing its own strategic approach to this area. Strong new laws Most strikingly, the draft legislation would outlaw some forms of AI that hu- man rights groups see as most invasive and unethical. That includes a broad range of AI that could manipulate our behaviour or exploit our mental vulner- abilities – as when machine-learning algorithms are used to target us with political messaging online. Likewise, AI-based indiscriminate surveillance and social scoring systems will not be permitted. Versions of this technology are currently used in Chi- na, where citizens in public spaces are tracked and evaluated to produce a trustworthiness "score" that determines whether they can access services such as public transport. The EU also looks set to take a cau- tious approach to a number of AI appli- cations identified as high-risk. Among these technologies are large-scale facial recognition systems – considered easy to deploy using existing surveillance cameras – which will require special permission from EU regulators to roll out. Many systems known to contain bias also classify as high-risk. AI that assess- es students and determines their access to education will be tightly regulated – such technology achieved notoriety after an algorithm unfairly determined UK students' grades in 2020. The same caution will apply to AI used for hiring purposes, such as algorithms that filter applications or evaluate can- didates, as well as financial systems that determine credit scores. Similarly, sys- tems that assess citizens' eligibility for welfare or judicial support will require organisations to make detailed assess- ments to ensure they meet a new set of EU requirements. To give it some teeth, and in line with the EU's existing punishment for se- rious data misuse, the AI regulations include fines for infringements of up to €20 million (£17.2 million) or 4% of global turnover, whichever is higher. Reckoning with AI Globally unique and sweeping in its application, the proposed regulation is a clear statement from Europe that it prioritises citizens' fundamental rights over technical autonomy and economic interests. But there are also concerns. Some will argue the measures go too far, stifling Europe's AI innovation. The White House in fact warned Europe not to overregulate AI in 2020, with the US aware that China's relative lack of pro- tections could see it achieve a competi- tive advantage over its rivals. On the other hand, privacy advocates and campaigners against bias in AI may be left disappointed. Some of the most problematic AI systems are excluded from the regulation, notably those used for military purposes, such as drones and other automated weapons – again speaking to fears of Chinese dominance in weaponised AI. It's also possible that other applica- tions, such as the fusion of AI with ex- isting mass surveillance capabilities, could be permitted where authorised by law. This would leave the door open for their use in law enforcement, which is exactly the area that some observers are most worried about. Such loopholes for AI-driven state surveillance systems will trouble human rights and privacy advocates Contested definitions Critics have highlighted the vague definition of AI detailed in the draft leg- islation, which focuses in particular on machine learning but may not apply to the next generation of computing tech- nologies, such as quantum computing or edge computing. As always with le- gal documents, the devil will be in the detail. Equally, there are open questions about the distinction between high-risk and low-risk AI. The regulations only apply to the former, but it's not clear whether it's always possible to deter- mine the nature of AI's risks during the development cycle. Risk is a continuum, and a dichotomy between high and low will always require an arbitrary distinc- tion which may cause problems down the line. The regulation is no doubt a bold step in the right direction. It will now be re- viewed by the European Council and the European Parliament. The process of reading, reviewing and agreeing will likely take some time, during which the questions raised here can be explored and attended to. But it stands to reason that many of the building blocks of the regulation will persist. By standing firm against forms of invasive surveillance and bias-prone AI systems, the legislation is a strong reminder that Europe takes seriously its obligation to safeguard its citizens' fundamental human rights in a period of disruptive technological change. EU is cracking down on AI, but leaves a loophole for mass surveillance