The European Parliament voted to pass the EU Artificial Intelligence Act on 13 March 2024. Together with the other pillars of EU legislation (Data Governance Act, Data Act, Digital Markets Act and Digital Services Act) the act introduces a comprehensive set of rules to regulate AI in the EU and thus will have a significant effect on the healthcare, digital health and life science sector.
Once the final formality to adopt the act is completed, the act is expected to enter into force later this year and most of its rules will be applied 24 months afterwards. However, some obligations will become binding earlier than others. For example, after the effective date of the act the ban on prohibited AI systems, eg using biometric data to categorise people or using harmful ‘subliminal techniques’, is expected to come into force within 6 months and the provisions on general purpose AI and penalties will be applied within 12 months. The obligations on high-risk AI will have a longer compliance deadline of 36 months from the effective date.
As highlighted in more detail below, AI systems that are deployed in areas such as medical aids, healthcare services, emergency healthcare patient triage systems or emotion recognition are likely to be classified, at least, as high-risk systems. Therefore, organisations in the healthcare, digital health and life science sector would need to be mindful that these systems are subject to stricter regulations and requirements under the act. That being said, a regulatory sandbox will be established to encourage innovation by allowing for the development, training and testing of AI systems (including processing personal data for such purposes) where the systems are beneficial to safeguard substantial public interest in the permitted areas (e.g. public safety and public health, disease detection, diagnosis prevention, improvement of health care systems, or protection or biodiversity etc.).
Even though the act will not apply directly to the UK, due to the global nature of AI technology, it is anticipated that AI system providers will follow the more descriptive requirements under the act which will have a domino effect on the AI supply chain. This is due to economic reality and the impracticalities of creating multiple AI products for EU and non-EU markets.
Currently the UK has adopted a principle-based approach to regulating AI. However regulators are expected to issue rules for their specific industries, including the healthcare, digital health and life science sector. AI providers and adopters should prepare for the act entering into force and any UK-specific regulations, to ensure that they comply with the regulators’ principles and requirements moving forward.
Below is a summary of some key proposals in the act, based on the current version.
The act has updated the definition of AI systems largely to align with the definition used by the OECD:
‘AI system’ is a machine-based system designed to operate with varying levels of autonomy and that may exhibit adaptiveness after deployment and that, for explicit or implicit objectives, infers, from the input it receives, how to generate outputs such as predictions, content, recommendations, or decisions that can influence physical or virtual environments.” (Article 3(1)).
This definition has gone through various rounds of negotiation and compromise. It needs to strike a balance between providing a comprehensive definition that would encompass new developments, whilst also ensuring it was not so wide as to inadvertently capture software which would not generally be classified as ‘AI’.
The regulations will apply to AI systems that are placed on the market, put into service or used in the EU, irrespective of whether the provider of the system is based within the EU or outside of the EU.
Exclusions apply to AI systems developed or used exclusively for military purposes, public authorities, international organisations, and authorities using AI systems in the framework of international agreements for law enforcement and judicial cooperation. The act will provide certain guardrails for law enforcement authorities’ use of AI for law enforcement purposes, in order to ensure protection of individuals’ fundamental rights.
The act sets out a risk-based classification approach, with varying obligations based on the risks associated with each AI system.
| Level of risk | Description | Regulation |
|
Prohibited AI practices (Article 5) |
AI systems that:
|
Prohibited to place on the market, put into services or use in EU AI systems, with little exception. Examples of exceptions: Real-time remote biometric identification systems would be allowed, subject to certain guardrails:
AI systems that infer emotions may be allowed as an exception for medical or safety reasons. |
|
High risk (Article 6) |
Two categories of high-risk AI systems(Article 6):
|
AI systems at this risk level are permitted, but in order to gain access to the EU market, they are subject to rigorous testing, proper documentation of data quality and an accountability framework that details human oversight. For example:
|
| Others |
Systems which are neither prohibited nor high risk, for example spam filters, phishing filters. |
AI systems at this level of risk are permitted but are subject to transparency obligations. |
The EU Artificial Intelligence Act also contains provisions to cover general purpose artificial intelligence (GPAI) models, which are defined as an:
‘AI model, including when trained with a large amount of data using self-supervision at scale, that displays significant generality and is capable to competently perform a wide range of distinct tasks regardless of the way the model is placed on the market and that can be integrated into a variety of downstream systems or applications are AI systems that can be used for different purposes with varying degrees of risk’ (e.g. LLM and generative AI) (Articles 52a – 52e)
All GPAI models will be subject to horizontal obligations, such as providing technical documentation to the authorities and downstream providers for the purpose of compliance with the act. Models with systemic risks are subject to additional obligations, including performing model evaluation, making risk assessments, taking risk mitigation measures, ensuring an adequate level of cybersecurity protection and reporting serious incidents to the authorities.
Non-compliance with the act and its obligations may result in significant penalties:
For small and medium-sized enterprises and start-ups, the administrative fines will be the lower of the above amount and percentages.
Violations by GPAI providers may attract a fines of up to €15 million or 3% total global turnover in the preceding financial year, whichever is higher.
The act is expected to be finally approved by the council of European Union in April 2024. It will then become effective 20 days following publication in the Official Journal later this year.
For more information, please contact our technology team.
Guidance on responsible AI in recruitment: helping to navigate data protection compliance
The benefits and risks for local authorities when adopting AI: Thanh Lanh-Connolly writes for Local Government Lawyer
AI and the environment: its application in improving environmental performance and use in environmental regulation
We produce a range of insights and publications to help keep our clients up-to-date with legal and sector developments.
Sign up
