The EU AI Act (Regulation 2024/1689) entered into force on 1 August 2024. Its prohibited practices became applicable on 2 February 2025. From August 2025, the governance provisions and obligations for General Purpose AI models will apply. From August 2026, the full weight of requirements for high-risk AI systems comes into effect. Financial institutions that have not yet begun the work of classification, documentation and architectural adaptation have a narrowing window.
This article focuses on what the AI Act requires of financial institutions in practical, architectural terms — not as an abstract regulatory summary, but as a guide to the decisions that enterprise architects, technology leaders and their governing bodies need to make now. Much of it draws on the experience of developing AI governance frameworks within a regulated German banking environment, where the AI Act intersects with obligations under DORA, GDPR and the EBA's own guidelines on machine learning in credit risk management.
Understanding the Risk Classification Framework
The AI Act is built around a risk-based framework that determines the level of obligation applicable to any given AI system. The classification of an AI system — and therefore the compliance burden that attaches to it — depends on its intended purpose and the domain in which it operates. Getting the classification right is harder than it first appears, and institutions consistently underestimate how many of their AI applications fall into the high-risk category.
For financial institutions, the critical question is how many of their AI systems are genuinely high-risk. Annex III of the Regulation lists the categories explicitly, and for banks, the most significant is the use of AI in the assessment of the creditworthiness of natural persons or the scoring of their credit. Any AI system used in lending decisions — including machine learning models for PD estimation, automated underwriting, or behavioural scoring — will typically fall within this category. Beyond credit, AI systems used in fraud detection that makes decisions affecting individuals, HR screening tools, and AI applied to the provision or denial of essential financial services are all candidates for high-risk classification.
"The classification question is not a compliance exercise to be completed once. As AI systems evolve and their scope of application broadens, their risk classification must be reassessed. A model that begins as a decision-support tool can become high-risk the moment it is used to make or recommend consequential decisions autonomously."
The Application Timeline — What Needs to Happen When
The August 2026 deadline for high-risk systems looks further away than it is. Building compliant AI architecture from scratch — establishing the model inventory, implementing logging and audit trail infrastructure, redesigning workflows for genuine human oversight, negotiating updated contracts with AI vendors, and completing conformity assessments — is not a six-month programme. Institutions beginning the work in early 2026 will be managing significant compliance risk.
What High-Risk AI Systems Must Actually Provide
For each high-risk AI system, the AI Act requires a specific set of technical and organisational capabilities. These are not documentation requirements that can be fulfilled by writing policies. They require that the underlying systems and processes are built — or rebuilt — to provide them.
-
01
Risk Management SystemA continuous, iterative process covering the full lifecycle of the AI system — from design through deployment to decommissioning. Must identify, analyse and evaluate known and reasonably foreseeable risks, and include residual risk evaluation post-mitigation. Not a one-time assessment; must be updated throughout the system's operational life.
-
02
Data Governance for Training, Validation and Testing DataTraining and test datasets must meet requirements for relevance, representativeness and freedom from bias. Complete data lineage must be maintained. Institutions using externally sourced training data — including data from third-party AI providers — must be able to demonstrate compliance with these requirements for data they may never have directly controlled.
-
03
Technical DocumentationComprehensive documentation of the system's general description, design specifications, development methodologies, training and testing processes, validation results and performance metrics. Must be maintained in current form and made available to national competent authorities on request. This is substantially more demanding than existing model risk management documentation in most institutions.
-
04
Automatic Logging and Audit TrailsHigh-risk AI systems must automatically log their operational events throughout their lifetime — sufficient to allow post-hoc reconstruction of the system's outputs and the basis on which they were produced. Logs must cover input data, outputs, and the parameters of the system at the time of each decision. Immutability and appropriate retention periods are implied by the regulatory purpose.
-
05
Transparency to DeployersProviders must supply deployers (the institutions using the AI) with documentation sufficient to allow them to use the system compliantly, understand its capabilities and limitations, and implement the human oversight measures required by the Regulation. For institutions using AI from third-party providers, this creates specific vendor contractual requirements that most current AI procurement contracts do not satisfy.
-
06
Human OversightPossibly the most architecturally demanding requirement. High-risk AI systems must be designed so that the humans overseeing them can fully understand their capabilities and limitations; monitor operation; detect and address failures, unexpected performance or risks; override, interrupt or disengage the system; and avoid over-reliance on outputs. This is not a checkbox — it requires genuine workflow redesign so that human oversight is meaningful rather than nominal.
-
07
Accuracy, Robustness and CybersecurityAppropriate levels of accuracy must be declared and maintained throughout the lifecycle. Systems must be resilient against attempts to manipulate their behaviour through adversarial inputs. Given the intersection with DORA, the cybersecurity requirements for AI systems in financial institutions effectively combine the obligations of both regulations — AI systems are ICT systems and must meet both sets of resilience requirements.
The Intersection with DORA and GDPR
Financial institutions operating in the EU are managing three overlapping regulatory frameworks simultaneously: GDPR (data protection, including rights in automated decision-making under Article 22), DORA (ICT operational resilience, directly applicable to AI systems as ICT systems), and now the AI Act. The interaction between them creates obligations that are more demanding than any single regulation in isolation.
Under GDPR Article 22, individuals have the right not to be subject to solely automated decisions that produce legal or similarly significant effects, and the right to a meaningful explanation of the logic involved. Credit decisions are the paradigm case. This means that explainability is not merely a best-practice aspiration for credit scoring models — it is a legal requirement. The AI Act reinforces this through its transparency and human oversight requirements, but GDPR has required it since 2018. Institutions with black-box credit models — particularly those relying on deep learning or ensemble methods without explainability layers — have been non-compliant with GDPR for years. The AI Act makes that non-compliance harder to ignore.
Under DORA, AI systems deployed in financial institutions are ICT systems within the meaning of the Regulation. They must be included in the ICT risk management framework, inventoried as part of the ICT asset register, covered by business continuity and incident response plans, and included in the scope of third-party risk management where the AI is provided by an external party. An AI system that fails, produces systematically erroneous outputs, or is manipulated through adversarial inputs is an ICT incident within DORA's classification framework. The incident reporting timeline — initial notification within four hours — applies.
In developing AI governance frameworks at a regulated German bank, the most consistently underestimated challenge was not the technical implementation — it was the data lineage problem. Many institutions have deployed AI models, particularly in credit risk and fraud detection, that were trained on historical datasets whose provenance, representativeness and bias characteristics are no longer fully documented. Meeting the AI Act's data governance requirements for these systems means either reconstructing the required documentation retrospectively — which is often partially impossible — or retraining models from scratch with properly governed data pipelines. Both paths are more resource-intensive than most compliance timelines assume.
What Compliant AI Architecture Looks Like
Translating the AI Act's requirements into architecture is not simply a matter of adding components to existing systems. Compliant AI architecture in a financial institution requires a foundational layer of capabilities that many institutions have not yet built.
A Model Inventory and Registry
The starting point is knowing what AI systems you have, where they are deployed, what decisions they inform or make, and how they are classified under the AI Act. Most institutions discover, during this exercise, that their AI footprint is significantly larger than their official count suggests — models embedded in vendor software, AI features enabled within SaaS platforms, and legacy statistical models whose sophistication means they now qualify as AI systems within the Regulation's definition.
Data Lineage Infrastructure
For high-risk AI systems, the provenance, composition and governance of training data must be fully traceable. This requires data pipeline tooling that captures lineage automatically rather than relying on documentation that may not be maintained. Where third-party data is used, contracts must require vendors to provide the information needed to demonstrate compliance.
Explainability by Design
Where AI informs consequential decisions, the architecture must support explainability at the individual prediction level — not just aggregate model statistics. For credit applications, this means being able to produce an explanation of why a specific application was scored as it was. This typically requires either inherently interpretable models or a purpose-built explainability layer (SHAP values, LIME or similar techniques), integrated into the operational workflow rather than retrofitted.
Immutable Audit Logging
The logging requirements of the AI Act are specific: logs must be sufficient to reconstruct how the system produced each output. Designing this from the outset — capturing input features, model version, output and confidence at each inference — is straightforward. Retrofitting it to existing systems that were not built with this requirement in mind is considerably harder.
Human Oversight Workflows
Perhaps the most significant organisational implication of the AI Act is the requirement for genuine, rather than nominal, human oversight. An override button that no one ever uses, or a review step that rubber-stamps AI outputs without meaningful scrutiny, does not satisfy the Regulation. Institutions need to assess honestly whether their current human-in-the-loop processes provide the quality of oversight that the AI Act envisions — and redesign them where they do not.
Questions for Architecture and Technology Leaders
- Have we completed a comprehensive inventory of all AI systems in use — including those embedded in vendor software and SaaS platforms — and classified each against the AI Act's risk tiers?
- For each high-risk AI system, do we have a documented risk management process, and is it iterative rather than a one-time assessment?
- Can we demonstrate full data lineage for the training, validation and test datasets of each high-risk model — including for models trained on data we sourced from third parties?
- Do our AI systems that inform individual decisions produce explainable outputs at the individual level, sufficient to satisfy both GDPR Article 22 and the AI Act's transparency requirements?
- Do our AI systems generate automatic, immutable logs sufficient to reconstruct each material decision?
- Have we reviewed all AI vendor contracts against the AI Act's provider-to-deployer documentation requirements — and do we have a programme to renegotiate those that fall short?
- Are our AI systems included in the ICT risk management framework and incident response plans required under DORA?
- Have we assessed which of our AI systems will require formal conformity assessment before the August 2026 deadline, and is that assessment programme planned and resourced?
Conclusion: Compliance as Architecture, Not Administration
The EU AI Act is frequently discussed as a compliance obligation — a set of rules to be satisfied through documentation, governance structures and process changes. That framing understates the challenge. For financial institutions deploying AI in consequential decisions, the Act requires that AI systems are built differently: with data lineage infrastructure, explainability layers, audit logging, and human oversight workflows designed in from the beginning, not grafted on at the end of a development cycle.
Institutions that treat the AI Act as an administrative exercise — producing the required documentation around AI systems that were not designed to meet its underlying requirements — will find that gap visible, sooner or later, to regulators, to auditors, or to individuals exercising their rights. The national competent authorities designated under the Act have enforcement powers. The EBA has been explicit that AI in credit risk management has been within its supervisory scope since the publication of its ML guidelines in 2021.
The more productive framing is architectural: what would it mean to build AI systems that are genuinely trustworthy, explainable and resilient? The AI Act's requirements are, in most cases, a reasonable specification for AI systems worth building. Institutions that approach compliance as an opportunity to build better AI infrastructure — rather than a constraint on the AI infrastructure they have — are likely to find that the regulatory obligation and the business interest point in the same direction.