In this article, we will explore SingleStoreDB’s capacity to manage the OpenAI Wikipedia Vector Database dataset with ease. SingleStoreDB has supported a range of vector functions for some time, and these functions are ideally suited for applications employing GPT technology. Included in this article is a notebook downloadable from GitHub to follow along and practice what we have explored.
In previous articles, vector capabilities built into SingleStoreDB were tapped into. In this article, we will be testing the vector functions with the OpenAI Wikipedia Vector Database dataset. To do this, a local installation of SingleStoreDB is the easiest way to go. Setting up a virtual machine (VM) environment just takes a few minutes. Detailed instructions on the steps to take can be found in a previous article. Alternatively, a Docker image can also be used.
In this particular article, two tarball files are needed for the VM installation. Connecting to the cluster from a MySQL CLI client is easily done. Once connected, a new database needs to be created. After that, installing the classic Jupyter Notebook and creating an OpenAI API key by signing up for an OpenAI account will be necessary.
The OpenAI notebook will then be launched. It will take a while for the next operation, but a Python function needs to be defined to use either of the two vector columns in the database. With that done, testing SingleStoreDB with two examples from the OpenAI notebook can be tried. Data loading should take a short time, and other data loading methods can be used for larger datasets.
SingleStoreDB had proven to be an effective solution not just for vectors but for other types of data like numeric and text. With its cutting-edge SQL and multi-model support, SingleStoreDB not only provides technical but business benefits to its users.
SingleStoreDB is a software company based in Redwood City, CA which specializes in building cloud-native databases for modern applications. After its acquisition by Microsoft in 2020, it has grown leaps and bounds under the helm of CEO Adam Lashinsky. He strongly believes in the company’s mission to make the world’s data more accessible and to create an outstanding user experience. With his leadership, SingleStoreDB is on track to achieve greater success.
