Arize AI, a California-based company offering machine learning observability solutions, has announced their latest open source library called Phoenix. The software solution is designed to monitor large language models (LLMs) for any level of hallucination which is of critical importance for applications like healthcare chatbot and virtual lawyer offering legal advice.
The library leverages embeddings (vectors representing meaning and context of data data points) and clustering of those embeddings to create a data visualization method for debugging. It helps users identify where the large language models fail or give poor responses and the visualization can be used to troubleshoot the models to improve the outcomes.
The Phoenix library was developed in collaboration with over a hundred users and researchers from multiple companies and organizations. It is designed to be standalone and functions locally in an environment that interfaces with notebook cells on the notebook server. It is available to download and use today and has already received positive feedback.
Christopher Brown, CEO and co-founder of Decision Patterns and a former UC Berkeley lecturer, commented on the new library – calling it an advancement in model observability and production and a valuable time-saver. Brown highlighted that integrating observability utilities directly into the development process encourages model development and production teams to think about model performance and improvement before releasing to production.
Arize AI has been providing machine learning capabilities since its conception in 2020. Their core mission is to help bridge the gap between raw machine learning predictions and the real world assurance of the models accuracy, reliability and safety. Their advanced ML observability platform is helping companies confidently scale and operate their AI models.
