“They may develop guardrails that are too broad or, conversely, too narrow for their use case.”Ī universal fix for language models’ shortcomings sounds too good to be true, though - and indeed, it is. “Ultimately, developers control what is out of bounds for their application with Guardrails,” Cohen said. Think keeping a customer service assistant from answering questions about the weather, for instance, or a search engine chatbot from linking to disreputable academic journals. Specifically, Guardrails can be used to prevent - or at least attempt to prevent - models from veering off topic, responding with inaccurate information or toxic language and making connections to “unsafe” external sources. Nvidia claims that the toolkit is designed to work with most generative language models, allowing developers to create rules using a few lines of code. Guardrails includes code, examples and documentation to “add safety” to AI apps that generate text as well as speech.
“AI model safety tools are critical to deploying models for enterprise use cases.” “We’ve been developing toward this release of NeMo Guardrails ever since,” Cohen told TechCrunch via email. Jonathan Cohen, the VP of applied research at Nvidia, says the company has been working on Guardrails’ underlying system for “many years” but just about a year ago realized it was a good fit for models along the lines of GPT-4 and ChatGPT. In pursuit of “safer” text-generating models, Nvidia today released NeMo Guardrails, an open source toolkit aimed at making AI-powered apps more “accurate, appropriate, on topic and secure.” Even the best models today are susceptible to biases, toxicity and malicious attacks. The companies behind these models say that they’re taking steps to fix the problems, like implementing filters and teams of human moderators to correct issues as they’re flagged.
The Verge’s James Vincent once called one such model an “emotionally manipulative liar,” which pretty much sums up the current state of things. For all the fanfare, text-generating AI models like OpenAI’s GPT-4 make a lot of mistakes - some of them harmful.
0 kommentar(er)
