Swiss software firm LatticeFlow AI has rolled out Compl-AI, an evaluation framework of the European Union (EU) AI Act for Generative AI models.
The company designed the open-source framework with two research institutes- ETH Zurich and Bulgaria’s INSAIT. It includes a technical interpretation of the EU AI Act and maps regulatory requirements to technical ones.
According to LatticeFlow AI, the tool will be available online for free to enable the developers test whether their Large Language Models (LLMs) comply with the EU AI Act.
The launch of Compl-AI was also welcomed by European Commission officials.
European Commission spokesperson for digital economy, research, and innovation Thomas Regnier said: “The European Commission welcomes this study and AI model evaluation platform as a first step in translating the EU AI Act into technical requirements, helping AI model providers implement the AI Act.”
EU AI Act
The EU AI Act, which came into force in August 2024, outlines high-level regulatory requirements for the use of artificial intelligence (AI).
The European Commission has also launched a consultation on the Code of Practice for providers of general-purpose AI (GPAI) models to monitor the implementation and enforcement of the AI Act rules.
LatticeFlow AI expects that the release of COMPL-AI release will benefit the GPAI working groups.
First Evaluation of AI models
According to the company, COMPL-AI framework includes 27 benchmarks that can be used to assess LLMs.
The tool tested key public generative AI models from OpenAI, Meta, Google, Anthropic, Alibaba, and others, against an actionable interpretation of the EU AI Act.
The test identified shortcomings in several high-performing models, particularly in cybersecurity and fairness benchmarks.
However, most of the models fared well in fencing harmful content.
“With this framework, any company — whether working with public, custom, or private models — can now evaluate their AI systems against the EU AI Act technical interpretation, said LatticeFlow AI CEO and co-founder, Dr Petar Tsankov. “Our vision is to enable organisations to ensure that their AI systems are not only high-performing but also fully aligned with the regulatory requirements such as the EU AI Act.”
Read More: Survey reveals 78% of businesses unprepared for Gen AI due to poor data foundations
