Amazon Bedrock, the AI service provided by Amazon Web Services Inc., is introducing a new tool alongside three new foundation models. Revealed at the AWS re:Invent 2023 event, these advancements aim to simplify the process of comparing and evaluating neural networks within the Amazon Bedrock Service.
The newly launched tool, called Model Evaluation on Amazon Bedrock, enables developers to compare different foundation models with minimal manual work. This eliminates the need for extensive custom code and streamlines the assessment process. By selecting a model and specifying the intended task, developers can upload a dataset on which the evaluation will be performed. For instance, a software team building a customer support chatbot could upload a dataset containing relevant help desk tickets.
Once the dataset is ingested, the tool automatically measures the performance of the selected Bedrock model across various predefined metrics. These metrics include accuracy, toxicity, and robustness, which determines the consistency of the model’s accuracy in different scenarios.
According to Antje Barth, AWS’ principal developer advocate for generative AI, the tool allows users to bring their own data or use curated datasets and pre-defined metrics for tasks like content summarization, question answering, text classification, and text generation.
To cater to AI projects that require assessment based on metrics not supported by the new tool, it offers the option to launch human evaluation workflows. These evaluations can be conducted by either an AWS-managed team or the company’s own employees.
In addition to the new tool, three new foundation models have been made available within Amazon Bedrock. The first is Claude 2.1, the upgraded version of Anthropic PBC’s ChatGPT rival. This general-purpose language model has been optimized for text and code generation tasks. With a 30% reduction in inaccurate answers and decreased hallucination rates, Claude 2.1 offers improved performance. It also boasts the ability to access external applications via APIs, perform simple actions within those applications, and retrieve data. Furthermore, Claude 2.1 has a significantly larger context window than its predecessors, enabling users to include more information in prompts.
The second addition to Bedrock is the 70-billion-parameter version of Meta Platforms Inc.’s Llama 2 model, which was released in July. Similar to Claude, this general-purpose large language model excels in generating text and code. Trained on two trillion tokens, Llama 2 offers a vast dataset for enhanced performance.
Lastly, Stability AI Ltd. has introduced Stable Diffusion XL 1.0, their latest image generation model. It has the capability to produce high-resolution images, with dimensions of 1024 pixels by 1024 pixels, in various styles. Compared to previous models, Stable Diffusion XL 1.0 offers improved contrast, lighting, and shadows.
As companies utilizing Amazon Bedrock gain access to these new foundation models, they can leverage the advancements in text processing tasks and open-source image generation to enhance their AI projects.
With the introduction of the Model Evaluation tool and the addition of new foundation models to Amazon Bedrock, developers now have improved resources to select the most suitable neural networks for their AI applications. These advancements contribute to the growth and development of AI technology, simplifying the evaluation process and expanding the capabilities available to developers.
