Definition of AI system (Art. 3 AI Act)

The Digital Policy Committee (DPC) of the Organisation for Economic Co-operation and Development (OECD) promotes digital transformation in terms of human-centered and rights-oriented policy development. The DPC oversees various working groups, including the Working Group on Artificial Intelligence Governance (AIGO), which deals with policy development in the field of artificial intelligence.

On May 22, 2019, the definition of an AI system was first adopted, with a revised version of the recommendation being adopted on May 3, 2024. Further information can be found in the "OECD Explanatory Memorandum".

OECD Definition

"An AI system is a machine-based system that pursues explicit or implicit goals and derives from received inputs how it generates predictions, content, recommendations, decisions, or other outputs that can influence physical or virtual environments. AI systems differ in terms of their autonomy and adaptability after deployment."

Comparing this definition with the definition in Article 3(1) of the AI Regulation, it is noticeable that the terms are almost identical.

AI Act Definition

"AI system": a machine-based system that operates with varying degrees of autonomy, may adapt after operation has begun, and that derives from given inputs for explicit or implicit goals how it generates outputs such as predictions, content, recommendations, or decisions that can influence physical or virtual environments."

European Commission Guidelines on the Definition of an AI System

On February 6, 2025, the European Commission published its "Guidelines on the Definition of an AI System". The guidelines published by the European Commission stipulate that the definition of an AI system must be applied flexibly and on a case-by-case basis, deliberately avoiding an exhaustive list of AI systems in order not to exclude future technological developments. Due to their non-binding nature and deliberately open formulation, the guidelines thus contribute little to legal certainty in the application of the AI Regulation.

A closer look at the terms contained reveals the following characteristics and offers an indication of the legislator's regulatory idea when defining AI systems:

Machine-based system

In contrast to biological systems, an AI system according to the AI-VO must be a "machine-based" system. This initially seems self-evident, as human or animal intelligence should not be included. However, there are already remarkable advances in the research areas of bioinformatics and synthetic biology (e.g. programmable cell structures and neuromorphic chips). When making the seemingly obvious statement that "organic intelligence" should not be included, the question arises as to whether this exclusion is appropriate in the sense of comprehensive, future-proof AI regulation.

Autonomous operation (different degrees)

Recital 12 of the AI-VO states that AI systems "operate with varying degrees of autonomy, i.e. that they operate to a certain extent independently of human intervention and are able to operate without human intervention."
This characteristic does not distinguish AI systems from deterministic or stochastic systems, since a system without autonomy can also be qualified as an AI system. The wording "varying degrees of autonomy" implies that even a degree of zero is sufficient. This leads to a very broad definition of the term AI system.

Adaptability after deployment

The development and operation of AI systems (e.g. LLM) typically consists of different phases: Training phase, production phase and operating phase. These phases are often carried out repeatedly and can overlap, with feedback from the operating phase partly flowing into new training cycles. For example, in conceptual knowledge distillation, a large model is used as a teacher to compress knowledge into a smaller model. Through iterative learning and mutual feedback between the models, the system gradually improves.

However, the presence of this criterion is not mandatory for the existence of an AI system. A large proportion of current LLMs are only improved in the training phase and do not automatically learn from user input during the usage phase.

Processing of inputs for explicit/implicit goals

While AI systems process information through independent inferences and derivations, rule-based systems work according to fixed, predefined operating rules defined by humans. AI systems can pursue both explicit (clearly defined) and implicit (acquired through training) goals, while rule-based systems can only implement explicit, pre-programmed goals.

On closer inspection, however, practical difficulties also arise with this distinction. Classic software also sometimes contains adaptive rules. For example, if a "smart meter" calculates an optimal heating curve, it uses a controller that continuously compares the actual and target values and adjusts the control variables accordingly. This involves mathematical optimization, which technically corresponds to a target value search without the presence of an AI system.

Creation of outputs

The system must generate outputs from the inputs. Examples given here are predictions, content, recommendations or decisions, although the list is not exhaustive. Content here means various forms of the results of generative AI (e.g. videos, sound, images based on text input ("prompts")).

In contrast to algorithmically deterministic outputs, AI systems can also solve tasks that do not have a "correct" solution. For example, if a poem is to be written about a green frog, the content of this poem is not deterministically predetermined. A "fantasizing LLM" can solve the task of the poem in different ways and write different poems, all of which are correct.

However, the characteristic of uncertainty of the output is not necessarily given in every AI system or LLM: AI systems and LLMs can, depending on the architecture and implementation, also generate highly structured and verifiable outputs (structured outputs) such as JSON, SQL queries, program code or template-based documents. The uncertainty regarding the output result is therefore determined by the respective system architecture of the AI system and is not an inherent characteristic of AI.

Influence on environments

Finally, the output must be able to influence physical or virtual environments. The physical environment is influenced if the output data of the AI system triggers mechanical actions, for example if an AI-controlled robot arm grips and moves workpieces. A virtual environment is influenced if the AI outputs serve as input data for other systems - for example, if an AI system determines the actions of a computer-controlled character (NPC) in a video game, which then interacts with the game environment.

This criterion is also not suitable as a clear distinguishing feature from classic software. Both a conventional, algorithmically operating motion sensor in a robot and a traditionally programmed computer opponent in a video game influence their respective environment - the sensor influences the physical world through robot movements, the programmed bot influences the virtual game world. The ability to influence the environment is therefore not a unique selling point of AI systems.

Interim conclusion

The closer examination clarifies the intention of the legislator when defining AI systems. However, the legal problem remains: If concrete legal consequences such as penalties and fines are linked to the classification as an AI system, the question arises as to how a sufficient distinction can be made on the basis of such indefinite characteristics. The precise identification of an AI system within the meaning of the regulation appears to be hardly possible under these conditions - at least in cases of doubt.

In order to enable a actually practicable distinction between conventional software and AI systems, a so-called three-factor approach is proposed.

According to this approach, three fundamentally independent factors are used for evaluation:

•Factor I (Development): Does it use large amounts of data for training or is it based on explicitly specified rules?

•Factor II (Application): Does it optimize its behavior during application?

•Factor III (Outputs): Are the results indeterminate or subjectively interpretable?

Factor I: Data or expertise in development

Explanation: This factor looks at how the system was created. The aim is to determine whether the system was developed in a data-driven manner or is based on explicit, deterministic rules/knowledge.

1.Is the system developed through machine learning or on the basis of explicit rules?

Positive example: An image recognition system that learns to recognize different writing styles by training with millions of handwritings and machine writings.

Negative example: An OCR program that is based solely on predefined pattern rules for common fonts and does not use any machine learning method.

2.Is data from usage used for updates or adjustments to the system?

Positive example: A spell checker that learns from user input and automatically adapts to new word combinations or language trends.

Negative example: A permanently programmed auto-correction that only contains a static list of typos and corrections without adapting to new errors.

Factor II: Goal-oriented optimization during application

Explanation: This factor distinguishes between simple forward calculations and complex optimizations to fulfill goals in the application phase.

1.Does the system determine solutions based on heuristic or deductive methods?

Positive example: A recommendation system for online shops that analyzes user behavior and dynamically suggests suitable products based on heuristic rules or machine learning.

Negative example: A fixed set of rules for product recommendations that is based solely on manually defined criteria (e.g. "Customers who bought X also bought Y" without further adaptation to the user's behavior).

2.Does the system adapt to target specifications during application?

Positive example: An adaptive navigation system that detects traffic jams in real time and suggests alternative routes to minimize travel time.

Negative example: A static route planner that always calculates the shortest route based on a permanently stored map, regardless of current traffic data.

Factor III: Formal indeterminacy of the results

Explanation: This factor assesses whether the system results can be clearly determined formally or whether there is scope for discretion with regard to "correct" results.

1.Are the results of the system clear or is there scope for discretion?

Positive example: An automated credit rating system that weights various factors (income, payment behavior, credit history) and creates an individual risk assessment that is not clearly defined.

Negative example: A tax calculator that calculates the exact tax amount for a given income level based on fixed tax rates - without any room for interpretation.

2.Can there be different, non-deterministic outputs for the same input?

Positive example: A chatbot that generates different plausible answers to the same user question depending on the context and course of the conversation.

Negative example: A calculator that always delivers exactly the same result for the same arithmetic operation.

1. Large Language Models (e.g. ChatGPT)

A Large Language Model like ChatGPT is a clear example of an AI system. It not only uses massive amounts of data for training (Factor I), but also optimizes its behavior in the context of the conversation (Factor II) and generates non-deterministic, creative outputs (Factor III). The high expression of all three factors makes the classification as an AI system unproblematic.

2. Autonomous vehicles

The classification as an AI system is also regularly clear for autonomous vehicles. These systems learn from extensive training and sensor data (Factor I), adapt their behavior in real time to different traffic situations (Factor II) and have to make non-deterministic decisions in complex situations (Factor III).

3. Rule-based systems

The situation is different with classic rule-based systems. A booking system, for example, that is based on permanently programmed business rules, does not use any training data and delivers deterministic results, does not fulfill any of the three factors to a relevant extent and is therefore not an AI system within the meaning of the regulation.

The classification as an AI system triggers a staggered catalog of obligations, which is determined by the respective risk class. The regulations on prohibited AI practices according to Art. 5 AI Act, the extensive requirements for high-risk AI systems according to Art. 6 AI Act and the transparency obligations according to Art. 50 AI Act are central here.

Violations of the AI Act can result in significant sanctions. These range from fines of up to EUR 35 million or 7% of global annual turnover to an order to remove the system from the market. The amount of the sanction is based on the type, severity and duration of the violation as well as the size of the company.

The presented three-factor approach offers a structured methodology for the required individual case assessment. However, legally secure delimitation remains challenging in practice, especially for complex systems that contain both rule-based and dynamically learning components. To reduce compliance risks, the various components of the overall system (development, application, outputs) should be evaluated.

The documentation of the classification and the measures taken is of particular importance in order to be able to justify a comprehensible derivation and to reduce compliance risks. Further concretization through case law and administrative practice should be kept in mind.

Wendehorst/Nessler/Aufreiter/Aichinger: Der Begriff des „KI-Systems“ unter der neuen KI-VO (MMR 2024, 605).