A recent study by researchers from Stanford University has revealed that none of the large language models (LLMs) used by AI tools comply with the European Union (EU) Artificial Intelligence (AI) Act. The Act, which was recently adopted by the European Parliament, is the first of its kind to regulate AI on a national and regional level and also serves as a blueprint for worldwide AI regulations. However, the research shows that companies using high-scoring LLM models have a lot of work to do to attain compliance. The researchers evaluated 10 major model providers on a scale of 0 to 4, based on their degree of compliance with the 12 AI Act requirements. None of the providers scored over 75%, suggesting that there is room for significant improvement.
The study also highlighted some crucial areas of non-compliance. These included a lack of transparency in disclosing the status of copyrighted training data, energy consumption, emissions output, and methodology to mitigate potential risks. The researchers found an apparent disparity between open and closed model releases, with open releases leading to more robust disclosure of resources but involving greater challenges monitoring or controlling deployment.
The researchers proposed several recommendations for improving AI regulation, including ensuring that the AI Act holds larger foundation model providers to account for transparency and accountability. The need for technical resources and talent to enforce the Act was also highlighted, reflecting the complexity of the AI ecosystem. The main challenge, the study suggests, lies in how quickly model providers can adapt and evolve their business practices to meet regulatory requirements.
In recent months there has been a reduction in transparency in major model releases, with OpenAI making no disclosures regarding data and compute in their reports for GPT-4, citing a competitive landscape and safety implications. The researchers’ work offers insight into the future of AI regulation and argues that if enacted and enforced, the AI Act will yield a positive impact on the ecosystem, paving the way for more transparency and accountability.
