Anyscale’s Aviary Project Enables Easy Deployment of Open-Source LLM

Anyscale, the primary commercial vendor of open-source Ray machine learning (ML) scaling technology, has launched the Aviary project to simplify open-source large language model (LLM) deployment. Despite the increasing number of open-source LLMs available to developers, deploying the models on infrastructure for real-world use has typically entailed a bespoke process. Aviary, which was built on top of the Ray project, simplifies LLM deployment with the right optimisations in place for hundreds of LLM models via pre-configured defaults and comparison tools. Aviary has been in private development at Anyscale for three months with the goal of providing a solution to the growing number of open-source models.