Recently, tech giant Nvidia released a new open source toolkit designed to make text-generating AI models ‘safer’. With the current state of text-generating AI models like Open-AI’s GPT-4, The Verge’s James Vincent has even gone so far as to call it an ‘emotionally manipulative liar’. Companies like Open-AI and Nvidia have been attempting to address these problematic issues by implementing filters, teams of human moderators, and other methods to minimize the errors, toxic language, and biases that have been linked with these models, however the progress have been too slow and unreliable.
Nvidia’s hopeful solution is the release of their open source toolkit called NeMo Guardrails, which is designed to make AI powered apps more secure, accurate, and appropriate. NeMo Guardrails is designed to work with most generative language models, and with a few lines of code developers can create rules that will detect toxic content, incorrect answers, and other issues. Jonathan Cohen, the VP of Applied Research at Nvidia, has stated that the company has been working on the underlying system of NeMo Guardrails for many years and only around a year ago they realized its potential as a potential fix for text-generating models.
Though the toolkit has been released without charge,some prior criticism must be noted. As mentioned before, it cannot guarantee a perfect and fool-proof system, and this too applies to NeMo Guardrails, so the effectiveness of the tech must be taken into mind. Furthermore, NeMo Guardrails does not work with certain open source options and is designed to best work for models that give clear instructions. Lastly, it must be taken into account that NeMo Guardrails is part of Nvidia’s NeMo framework, which can be accessed through the company’s computer AI software suite and cloud service, meaning it this toolkit could be used as a way to promote their own products.
Though it’s important to be cautious, NeMo Guardrails should be considered and analyzed further due to its potential in the AI industry. Companies like Zapier are already using Guardrails in order to help protect the accuracy, appropriateness and security of their models, and as more progress is made, it may well become the standard safety measure within the AI industry.
