OPINION OF THE EUROPEAN CENTRAL BANK

of 29 December 2021

on a proposal for a regulation laying down harmonised rules on artificial intelligence

(CON/2021/40)

(2022/C 115/05)

Introduction and legal basis

On 3 November 2021 the European Central Bank (ECB) received a request from the Council of the European Union for an opinion on a proposal (1) for a Regulation of the European Parliament and of the Council laying down harmonised rules on artificial intelligence (Artificial Intelligence Act) and amending certain Union legislative acts (hereinafter the ‘proposed regulation’).

The ECB’s competence to deliver an opinion is based on Articles 127(4) and 282(5) of the Treaty on the Functioning of the European Union as the proposed regulation contains provisions falling within the ECB’s fields of competence, in particular regarding the ECB’s tasks concerning the prudential supervision of credit institutions pursuant to Article 127(6) of the Treaty. In accordance with the first sentence of Article 17.5 of the Rules of Procedure of the European Central Bank, the Governing Council has adopted this opinion.

1.   General observations

1.1.

The ECB welcomes the objective of the proposed regulation to improve the functioning of the internal market by laying down a uniform legal framework for the development, marketing and use of trustworthy artificial intelligence (AI) in conformity with Union values. The ECB acknowledges the importance of setting harmonised requirements specific to AI systems to ensure a consistent and high level of protection of overriding reasons of public interest such as health, safety and fundamental rights.

1.2.

The ECB further acknowledges the increasing importance of AI-enabled innovation in the banking sector. Taking into account the inherent cross-border nature and opportunities for AI innovation in banking activities, the ECB, as the Union-level prudential supervisory authority, strongly supports the need for ensuring the harmonised implementation of the proposed regulation by credit institutions when prudential risks and requirements are concerned. In the same vein, and given the increasing importance of AI, the Union legislator is invited to consider in the future the potential of establishing an independent AI authority at Union level responsible for the harmonised application of the proposed regulation across the single market as regards matters specific to health, safety, and fundamental rights.

1.3.

Concerning high-risk AI systems provided or used by credit institutions, the ECB understands that the proposed regulation integrates certain obligations into the procedures set out in Directive 2013/36/EU of the European Parliament and of the Council (2) (hereinafter the ‘CRD’). In particular, the proposed regulation aims to enhance consistency with the CRD by integrating some of the providers’ and users’ risk management and governance obligations into the internal governance system of credit institutions (3). Because of the novelty and complexity of AI, and the high-level standards of the proposed regulation, further guidance is necessary to clarify supervisory expectations with regard to the obligations in relation to internal governance.

1.4.

The ECB welcomes the proposed regulation’s intention to avoid overlaps with the existing legislative framework by subsuming some of its provisions into the relevant provisions of the CRD (4). In this respect the ECB welcomes that the obligation of credit institution providers of high-risk AI systems to put a quality management system in place and the obligation of credit institution users of high-risk AI systems to monitor the system’s operation shall be deemed to be fulfilled by complying with the rules on internal governance arrangements, processes and mechanisms set out in the relevant provisions of the CRD (5).

1.5.

The ECB emphasises that the proposed regulation should be without prejudice to the more specific or stringent prudential obligations of credit institutions set out in sectoral regulation and supplemented by supervisory guidance. For instance, the internal governance obligations of credit institution users of AI systems under the CRD (6) extend to the effective control of outsourcing arrangements, including the identification, assessment and mitigation of all associated risks, as further informed by the EBA Guidelines on outsourcing arrangements (7). While the proposed regulation allocates different obligations to providers and users of high-risk AI systems, the EBA Guidelines on outsourcing arrangements make no such distinction in the context of outsourcing between third-party providers of technological solutions and credit institutions. In this respect, outsourcing does not lower the credit institutions’ obligation to comply with regulatory requirements, and the prudential supervisor remains competent to supervise the prudential risks posed by outsourced functions. Against this background, the ECB would welcome further clarifications regarding the applicable requirements and competent authorities with regard to outsourcing by credit institution users of high-risk AI systems.

1.6.

The ECB’s role under the proposed regulation should be clarified, in particular concerning: (1) the ECB’s prudential supervisory competences generally, and in relation to market surveillance and conformity assessment; and (2) the application of the proposed regulation to the performance of the ECB’s tasks under the Treaty.

1.7.

The ECB remains committed to a technology-neutral approach in the prudential supervision of credit institutions. Its role is to ensure the safety and soundness of credit institutions, maintaining a high standard of prudential supervision irrespective of the application of any particular technological solution. The ECB aims to maintain a level playing field for the prudential supervision of credit institutions, following the guiding principle of ‘same activity, same risks, same supervision’ (8).

2.   The ECB’s role under the proposed regulation

2.1.   Clarification of the ECB’s prudential supervisory competences in relation to market surveillance

2.1.1.

The proposed regulation provides that Regulation (EU) 2019/1020 of the European Parliament and of the Council (9) applies to AI systems covered by the proposed regulation (10), and that any reference to a product under Regulation (EU) 2019/1020 is to be understood as including all AI systems falling within the scope of the proposed regulation (11). In this respect, the ECB notes that the objective of Regulation (EU) 2019/2020 is to improve the functioning of the internal market by strengthening the market surveillance of products covered by Union harmonisation legislation to ensure that only compliant products that fulfil requirements that provide a high level of protection of public interests, such as health and safety in general, health and safety in the workplace, consumer protection, the protection of the environment and public security and any other public interests protected by Union harmonisation legislation, are placed on the Union market (12).

2.1.2.

The proposed regulation defines the market surveillance authority as the national authority carrying out the activities and taking the measures pursuant to Regulation (EU) 2019/1020 (13). Regulation (EU) 2019/1020 in turn defines the market surveillance authority as an authority designated by a Member State under Regulation (EU) 2019/1020 as responsible for carrying out market surveillance in the territory of that Member State (14). In addition, recital 9 of Regulation (EU) 2019/1020 clarifies that responsibility for enforcing Union harmonisation legislation should lie with the Member States, and their market surveillance authorities should be required to ensure that the legislation is fully complied with (15). On that basis, the ECB understands that, under the proposed regulation, the ECB is not in any way a market surveillance authority.

2.1.3.

However, the proposed regulation also provides that for AI systems placed on the market, put into service or used by financial institutions regulated by Union legislation on financial services, the market surveillance authority for the purpose of the proposed regulation shall be the relevant authority responsible for the financial supervision of those institutions under that legislation (16). In addition, recital 80 of the proposed regulation clarifies that Union legislation on financial services includes internal governance and risk management rules and requirements which are applicable to regulated financial institutions in the course of provision of those services, including when they make use of AI systems. It further clarifies that in order to ensure coherent application and enforcement of the obligations under the proposed regulation and relevant rules and requirements of the Union financial services legislation, the authorities responsible for the supervision and enforcement of the financial services legislation, including where applicable the ECB, should be designated as competent authorities for the purpose of supervising the implementation of the proposed regulation, including for market surveillance activities, as regards AI systems provided or used by regulated and supervised financial institutions. In this respect, reference is also made to the need to further enhance the consistency between the proposed regulation and the rules applicable to credit institutions regulated under the CRD.

2.1.4.

Under Article 127(6) of the Treaty, the Council may unanimously confer specific tasks upon the ECB concerning policies relating to the prudential supervision of credit institutions and other financial institutions with the exception of insurance undertakings. On that basis, Council Regulation (EU) No 1024/2013 (17) (hereinafter the ‘SSM Regulation’) confers on the ECB specific tasks concerning policies relating to the prudential supervision of credit institutions, with a view to contributing to the safety and soundness of credit institutions and the stability of the financial system within the Union and each Member State, with full regard and duty of care for the unity and integrity of the internal market based on equal treatment of credit institutions with a view to preventing regulatory arbitrage (18). In this regard, the ECB is exclusively competent, for prudential supervisory purposes, to ensure compliance with all relevant Union acts, which impose requirements on credit institutions to have in place, inter alia, robust risk management processes and internal control mechanisms (19). The ECB’s prudential supervisory role in this respect is limited to ensuring that credit institutions implement policies and processes to evaluate and manage their exposure to prudential risk, including risks related to different aspects of banks’ business models, governance and operational risk, and which arise from the use of technological solutions to ensure the safety and soundness of credit institutions and the stability of the financial system (20).

2.1.5.

Market surveillance does not aim to ensure the safety and soundness of credit institutions, but focuses instead on protecting the interests of individuals that could potentially be affected by abusive AI systems by ensuring that such systems meet the requirements needed to ensure a high level of protection of public interests such as the health and safety of persons. Consequently, the ECB understands that the Union legislator does not intend that the ECB acts as a market surveillance authority in relation to credit institutions under its supervision under the proposed regulation. This conclusion is in line with the recitals of the SSM Regulation, which clarify that the national authorities are competent to ensure a high level of consumer protection (21).

2.1.6.

On that basis, the ECB suggests that, to be consistent with the ECB’s prudential supervisory competences under Article 127(6) of the Treaty and the SSM Regulation, the text of the proposed regulation should unambiguously clarify that the ECB is not designated as a market surveillance authority or entrusted with any market surveillance tasks.

2.1.7.

While the tasks of market surveillance have not been conferred on the ECB, it may be the case that certain Member States will consider the designation of national competent authorities involved in the supervision of credit institutions as responsible for market surveillance in the context of the proposed regulation, insofar as permitted by their mandate and at least to the extent that market surveillance tasks apply to situations in which an AI system is put into service for own use. The designation of national competent authorities currently involved in the supervision of credit institutions as responsible for such market surveillance could be seen as safeguarding the coherence and cost-effectiveness of supervisory outcomes while capitalising on the expertise drawn upon by these authorities in utilising their investigatory and supervisory powers in relation to credit institutions.

2.1.8.

Finally, the ECB notes that the market surveillance provisions of the proposed regulation do not adequately address situations in which an AI system is put into service for own use. For example, the power of market surveillance authorities under the proposed regulation to recall or withdraw an AI system might not successfully bring about that system’s discontinuation in situations of own use (22). The Union legislator is therefore invited to clarify which restrictive measures and related competent authorities’ powers should apply to situations of own use.

2.2.   Clarification of the ECB’s prudential supervisory competences in the area of conformity assessment

2.2.1.

The proposed regulation provides (23) that for high-risk AI systems intended to be used to evaluate the creditworthiness of natural persons or establish their credit score (24), and that are placed on the market or put into service by credit institutions, a conformity assessment is to be carried out as part of the supervisory review and evaluation process (SREP) (25). The proposed regulation defines (26) the conformity assessment as the process of verifying whether the mandatory requirements for high-risk AI systems set out in the proposed regulation (27) have been fulfilled.

2.2.2.

As previously noted, the Council has, under Article 127(6) of the Treaty, conferred on the ECB specific tasks concerning policies relating to the prudential supervision of credit institutions, with a view to contributing, inter alia, to the safety and soundness of credit institutions and the stability of the financial system within the Union and each Member State (28). To avoid going beyond the tasks conferred on it by the Treaty, the ECB emphasises that it may be in a position to supervise the implementation of the relevant requirements in the context of the SREP while focusing on the prudential risks credit institutions may be exposed to. In this respect, the Union legislator is invited to consider the extent to which several elements of the conformity assessment might not be prudential in nature insofar as they largely concern the technical assessment of AI systems to safeguard the health and safety of persons and ensure that fundamental rights are respected by minimising the risk of erroneous or biased AI-assisted processes. In particular, the relevant provisions of the proposed regulation require high-risk AI systems to be designed or designed and developed (1) on the basis of training, validation and testing data sets that meet certain quality criteria where they make use of techniques involving the training of models with data; (2) with capabilities enabling the automatic recording of events (‘logs’) to ensure a level of traceability of the system’s functioning throughout its lifecycle that is appropriate for its intended purpose; (3) in a way that ensures that they are effectively overseen by natural persons, including human-machine interface tools, in order to prevent or minimise the risks to health, safety or fundamental rights that may emerge when a high-risk AI system is used; and (4) for the purpose of achieving an appropriate level of accuracy, robustness and cybersecurity (29). As clarified in the recitals of the proposed regulation, these requirements as to the quality of data sets used, technical documentation and record-keeping, transparency and the provision of information to users, human oversight, and robustness, accuracy and cybersecurity are necessary to effectively mitigate risks to health, safety and fundamental rights (30).

2.2.3.

Against this background, the Union legislator is invited to further reflect on the need to designate relevant competent authorities as responsible for the supervision of the conformity assessment conducted by credit institutions where matters specific to health, safety and fundamental rights are concerned, and to consider the need for ensuring the harmonised application of the proposed regulation across the single market by establishing in the future an AI authority at Union level.

2.2.4.

In addition, certain requirements for high-risk AI systems are not entirely clear or specific enough to provide a sufficient understanding to inform supervisory expectations. For example, the requirement that training, validation and testing data sets are to be relevant, representative, free of errors and complete (31) may need to be further clarified. Considering the wide scope of the mandate given to European standardisation organisations (32) and therefore the potential risk of weakening the norms set by the proposed regulation, the requirements pertaining to high-risk AI systems set out in the proposed regulation should be sufficiently specific.

2.2.5.

Finally, the ECB understands that the conformity assessment for AI systems provided by credit institutions to evaluate the creditworthiness of natural persons or establish their credit score is part of an ex ante internal control carried out by the credit institution (33). In this regard, the proposed regulation (34) should be amended to reflect the ex post nature of the specific assessment to be carried out by the prudential supervisor as part of the SREP.

2.3.   Clarification of the ECB’s prudential supervisory competences, generally

The ECB may be considered a competent authority only insofar as necessary for it to carry out the tasks conferred on it under the SSM Regulation. To avoid legal uncertainty as to whether the proposed regulation confers new tasks on the ECB, the ECB suggests that, instead of referring directly to the ECB as a competent authority, the proposed regulation should refer to competent authorities as defined in the relevant acts of Union law, such as the CRD. It would then follow from the SSM Regulation that the ECB is to be considered a competent authority only for the purpose of performing its prudential supervisory tasks (35).

2.4.   Clarification of the ECB’s independence in the performance of its tasks under the Treaty

The ECB understands that when acting as a provider placing on the market or putting into service AI systems in the Union, or as a user of AI systems located within the Union, it may itself be subject to the proposed regulation (36). The same holds true for the national central banks (NCBs). The proposed regulation provides that, when Union institutions fall within its scope, the European Data Protection Supervisor (EDPS) is to act as the competent authority for their supervision and as their market surveillance authority (37). The NCBs could be under supervision of national competent authorities (38). In this respect it is important to underline that the ECB and the NCBs should be in a position to carry out independently the tasks conferred on it by the Treaty (39), for instance when using any artificial intelligence application to define and implement monetary policy and to promote the smooth operation of payment systems (40). It must, however, be recognised that the ESCB’s independence in the performance of its tasks does not exempt it from every rule of Union law (41). The ECB understands that any potential supervision of the ECB by the EDPS and of the NCBs by the national competent authorities would be limited to proper controls over and governance of an AI system, and would not be in any way intended to impinge on the ECB’s and the NCBs’ ability to independently carry out the tasks conferred on them by the Treaty.

3.   Classification of AI systems

3.1.

The proposed regulation intends to ensure a proportionate regulatory framework in relation to its objectives by adopting a risk-based approach that imposes regulatory burdens only when an AI system is likely to pose high risks to fundamental rights and safety. Nevertheless, the proposed regulation, anticipating future developments in AI technology, defines software that qualifies as an AI system broadly. As a result, software that is developed to generate outputs such as content, predictions, recommendations, or decisions, using statistical and machine learning approaches, as well as search and optimisation methods, constitutes an AI system (42). This broad definition would cover a variety of activities undertaken by credit institutions, particularly in relation to systems intended to establish the credit score of natural persons.

3.2.

Under the proposed regulation (43), the vast majority of credit scoring activities making use of AI systems would be automatically subjected to the horizontal minimum requirements imposed on high-risk AI systems. Consequently, several activities, including acquisition targeting for marketing, collections modelling and standard credit scoring models (e.g. a scorecard using logistical regression), would need to comply with the same requirements. To enhance clarity in supervisory expectations and in line with the ECB’s technology-neutral approach, it is suggested that AI systems intended to be used to evaluate the creditworthiness of natural persons or establish their credit score and which leverage on the standalone use of linear or logistic regression or decision trees under human supervision should not be classified as high-risk AI systems, provided that the impact of such approaches to the assessment of natural persons’ creditworthiness or credit score is minor.

3.3.

As credit scoring activities are regularly carried out by credit institutions in day-to-day practice, the ECB suggests that the entry into effect of requirements that relate to the qualification of AI systems intended to be used to evaluate the creditworthiness of natural persons or establish their credit score as ‘high-risk AI systems’ should be delayed until the adoption by the Commission of common specifications (44) on the matter. In particular, these common specifications should both spell out the conditions under which high-risk AI systems in this field will be presumed to be in conformity with the applicable requirements, and define when AI systems should be considered as ‘put into service by small scale providers for their own use’, and therefore fall within the scope of the exception from the qualification as a high risk AI system (45). Against this background, the ECB should be included in the list of bodies consulted before the adoption of such common specifications, where they concern AI systems intended to be used to evaluate the creditworthiness of natural persons or establish their credit score (46).

3.4.

Finally, the ECB welcomes the possibility to update the list of high-risk AI systems included in Annex III to the proposed regulation (47), and stands ready to cooperate with, and be consulted by, the Commission on the identification of further potential risks of AI systems that may pose a risk of harm to health and safety, or a risk of adverse impact on fundamental rights. Apart from AI systems used to evaluate the credit score or creditworthiness of natural persons, the proposed regulation does not designate as high-risk other systems that might be put into service specifically by credit institutions. Nevertheless, credit institutions are developing or considering the development and use of AI data modelling linking sales, transactions, and performance data to ensure a clear overview of conduct risk in a certain area. Similarly, AI systems might be used in the real time monitoring of payments, or profiling of clients or transactions, for anti-money laundering and counter-terrorist financing purposes. Where the ECB recommends that the proposed regulation is amended, a specific drafting proposal is set out in a separate technical working document accompanied by an explanatory text to this effect. The technical working document is available in English on EUR-Lex.

---

(1)  COM(2021) 206 final.

(2)  Directive 2013/36/EU of the European Parliament and of the Council of 26 June 2013 on access to the activity of credit institutions and the prudential supervision of credit institutions and investment firms, amending Directive 2002/87/EC and repealing Directives 2006/48/EC and 2006/49/EC (OJ L 176, 27.6.2013, p. 338).

(3)  See Articles 9(9), 18(2), 20(2) and 29(5) of the proposed regulation and Article 74 of the CRD.

(4)  See Article 74 of the CRD.

(5)  See Articles 17(3) and 29(4) of the proposed regulation.

(6)  See Article 74 of the CRD.

(7)  See EBA Guidelines on Outsourcing Arrangements (https://www.eba.europa.eu/regulation-and-policy/internal-governance/guidelines-on-outsourcing-arrangements).

(8)  See Andrea Enria, Chair of the Supervisory Board of the ECB, ‘A binary future? How digitalisation might change banking’, De Nederlandsche Bank, Amsterdam, 11 March 2019, available on the ECB’s Banking Supervision website at www.bankingsupervision.europa.eu.

(9)  Regulation (EU) 2019/1020 of the European Parliament and of the Council of 20 June 2019 on market surveillance and compliance of products and amending Directive 2004/42/EC and Regulations (EC) No 765/2008 and (EU) No 305/2011 (OJ L 169, 25.6.2019, p. 1).

(10)  See Article 63(1) of the proposed regulation.

(11)  See Article 63(1)(b) of the proposed regulation.

(12)  See Article 1(1) of Regulation (EU) 2019/1020.

(13)  See Article 3(26) of the proposed regulation.

(14)  See Article 3(4) of Regulation (EU) 2019/1020.

(15)  See recital 9 of Regulation (EU) 2019/1020.

(16)  See Article 63(4) of the proposed regulation.

(17)  Council Regulation (EU) No 1024/2013 of 15 October 2013 conferring specific tasks on the European Central Bank concerning policies relating to the prudential supervision of credit institutions (OJ L 287, 29.10.2013, p. 63).

(18)  See the first paragraph of Article 1 of the SSM Regulation.

(19)  See Article 4(1)(e) of the SSM Regulation.

(20)  See recital 30 of the SSM Regulation, as well as pages 53 and 54 of the ‘ESCB/European banking supervision response to the European Commission’s public consultation on a new digital finance strategy for Europe/FinTech action plan’ (August 2020), available on the ECB’s Banking Supervision website.

(21)  See recitals 28 and 29 of the SSM Regulation.

(22)  See Article 3, points (16) and (17), Article 65(2), second subparagraph, and Articles 65(5) and 67(1) of the proposed regulation.

(23)  See Articles 19(2) and 43(2) of the proposed regulation.

(24)  See point 5(b) of Annex III to the proposed regulation.

(25)  Articles 97 to 101 of the CRD deal with the SREP.

(26)  See Article 3(20) of the proposed regulation.

(27)  See Articles 8 to 15 of Chapter 2 of Title III of the proposed regulation.

(28)  See the first paragraph of Article 1 of the SSM Regulation.

(29)  See Articles 10, 12, 14 and 15 of the proposed regulation.

(30)  See recital 43 of the proposed regulation.

(31)  See Article 10(3) of the proposed regulation.

(32)  See Article 40 and recital 61 of the proposed regulation.

(33)  See the first sentence of Article 43(2) and Annex VI to the proposed regulation.

(34)  See Article 19(2) and in particular the second sentence of Article 43(2) of the proposed regulation.

(35)  See Article 6 of the SSM Regulation.

(36)  See Article 2 of the proposed regulation.

(37)  See Articles 59(8) and 63(6) of the proposed regulation.

(38)  See Article 59(2) of the proposed regulation.

(39)  See Article 130 of the Treaty.

(40)  See the first and fourth indents of Article 127(2) of the Treaty.

(41)  See judgment of the Court of Justice of 10 July 2003, C-11/00 Commission v European Central Bank, ECLI:EU:C:2003:3, paragraphs 130 to135.

(42)  See Article 3(1) and Annex I to the proposed regulation.

(43)  See point 5(b) of Annex III to the proposed regulation.

(44)  See Article 41(1) of the proposed regulation.

(45)  Pursuant to point 5(b) of Annex III to the proposed regulation.

(46)  See Article 41(2) of the proposed regulation.

(47)  See Article 7(1) of the proposed regulation.