ChatGPT is a technology developed by OpenAI, an artificial intelligence research lab, that uses language models to answer questions and respond to prompts. Such a powerful tool reportedly has a high operational cost, estimated to be around $700,000 a day. Microsoft is trying to lower this cost by developing an AI chip called Athena which would enable operations at a fraction of the cost.
OpenAI runs on pricey tech infrastructure and it is estimated to cost millions of dollars to inference ChatGPT, further exceeding its training cost when deployed at any scale. This cost has been felt by other companies utilizing OpenAI’s language models such as Latitude, a startup that runs an AI Dungeon game. Nick Walton, the CEO of Latitude, stated that running OpenAI cost them a whopping 200,000 dollars a month. In order to bring down the costs, Latitude moved to a language software provider backed by AI21 Labs, consequently halving their payments to 100,000 a month.
At present, more than 300 Microsoft employees are working on the Athena chip project which has been in development since 2019. This move was motivated by the idea to reduce the cost of running generative AI models and a to catch up with the progress of companies such as Amazon and Google. It is speculated that the chip could be released by the company and OpenAI as early as next year.
Although OpenAI did not respond to the request for comment, it is clear that the development of the Athena chip is of great importance and will likely overcome the massive expense associated with running the AI-powered ChatGPT.
