From RAG to Richness: Startup Uplevels Retrieval-Augmented Generation for Enterprises
29/08/2024
The young Dutch CEO of Contextual AI had been deeply influenced by two seminal papers from Google and OpenAI, which together outlined the recipe for creating fast, efficient transformer-based generative AI models and LLMs.
Soon after those papers were published in 2017 and 2018, Kiela and his team of AI researchers at Facebook, where he worked at that time, realized LLMs would face profound data freshness issues.
They knew that when foundation models like LLMs were trained on massive datasets, the training not only imbued the model with a metaphorical brain for reasoning across data. The training data also represented the entirety of a model's knowledge that it could draw on to generate answers to users' questions.
Kiela's team realized that, unless an LLM could access relevant real-time data in an efficient, cost-effective way, even the smartest LLM wouldn't be very useful for many enterprises' needs.
So, in the spring of 2020, Kiela and his team published a seminal paper of their own, which introduced the world to retrieval-augmented generation. RAG, as it's commonly called, is a method for continuously and cost-effectively updating foundation models with new, relevant information, including from a user's own files and from the internet. With RAG, an LLM's knowledge is no longer confined to its training data, which makes models far more accurate, impactful and relevant to enterprise users.
Today, Kiela and Amanpreet Singh, a former colleague at Facebook, are the CEO and CTO of Contextual AI, a Silicon Valley-based startup, which recently closed an $80 million Series A round, which included NVIDIA's investment arm, NVentures. Contextual AI is also a member of NVIDIA Inception, a program designed to nurture startups. With roughly 50 employees, the company says it plans to double in size by the end of the year.
The platform Contextual AI offers is called RAG 2.0. In many ways, it's an advanced, productized version of the RAG architecture Kiela and Singh first described in their 2020 paper.
RAG 2.0 can achieve roughly 10x better parameter accuracy and performance over competing offerings, Kiela says.
That means, for example, that a 70-billion-parameter model that would typically require significant compute resources could instead run on far smaller infrastructure, one built to handle only 7 billion parameters without sacrificing accuracy. This type of optimization opens up edge use cases with smaller computers that can perform at significantly higher-than-expected levels.
When ChatGPT happened, we saw this enormous frustration where everybody recognized the potential of LLMs, but also realized the technology wasn't quite there yet, explained Kiela. We knew that RAG was the solution to many of the problems. And we also knew that we could do much better than what we outlined in the original RAG paper in 2020.
Integrated Retrievers and Language Models Offer Big Performance Gains The key to Contextual AI's solutions is its close integration of its retriever architecture, the R in RAG, with an LLM's architecture, which is the generator, or G, in the term. The way RAG works is that a retriever interprets a user's query, checks various sources to identify relevant documents or data and then brings that information back to an LLM, which reasons across this new information to generate a response.
Since around 2020, RAG has become the dominant approach for enterprises that deploy LLM-powered chatbots. As a result, a vibrant ecosystem of RAG-focused startups has formed.
One of the ways Contextual AI differentiates itself from competitors is by how it refines and improves its retrievers through back propagation, a process of adjusting algorithms - the weights and biases - underlying its neural network architecture.
And, instead of training and adjusting two distinct neural networks, that is, the retriever and the LLM, Contextual AI offers a unified state-of-the-art platform, which aligns the retriever and language model, and then tunes them both through back propagation.
Synchronizing and adjusting weights and biases across distinct neural networks is difficult, but the result, Kiela says, leads to tremendous gains in precision, response quality and optimization. And because the retriever and generator are so closely aligned, the responses they create are grounded in common data, which means their answers are far less likely than other RAG architectures to include made up or hallucinated data, which a model might offer when it doesn't know an answer.
Our approach is technically very challenging, but it leads to much stronger coupling between the retriever and the generator, which makes our system far more accurate and much more efficient, said Kiela.
Tackling Difficult Use Cases With State-of-the-Art Innovations RAG 2.0 is essentially LLM-agnostic, which means it works across different open-source language models, like Mistral or Llama, and can accommodate customers' model preferences. The startup's retrievers were developed using NVIDIA's Megatron LM on a mix of NVIDIA H100 and A100 Tensor Core GPUs hosted in Google Cloud.
One of the significant challenges every RAG solution faces is how to identify the most relevant information to answer a user's query when that information may be stored in a variety of formats, such as text, video or PDF.
Contextual AI overcomes this challenge through a mixture of retrievers approach, which aligns different retrievers' sub-specialties with the different formats data is stored in.
Contextual AI deploys a combination
LINK: | https://blogs.nvidia.com/blog/contextual-ai-retrieval-augmented-genera... |
See more stories from nvidia |
Most recent headlines
09/12/2024
Dalet Named an IDC Innovator in Media and Entertainment
Dalet, a leading technology and service provider for media-rich organizations, today announced that it has been named an IDC Innovator in the IDC Innovators: ...
09/11/2024
Dalet Expands Leadership Team to Fuel Next Stage of Growth
Dalet, a leading technology and service provider for media-rich organizations, today announced three new members of its executive team. Tara Bryant joins as Chi...
18/09/2024
Line up revealed for World's Most Dangerous Roads S6
Johnny Vegas & Lucy Beaumont and Babatunde Al sh & Kae Kurd arethecomedians taking part in brand new episodes of World's Most Dangerous Roads, which return...
18/09/2024
TV channel Gold's Absolutely Fabulous: Inside Out is joined by celebrities Emma Bunton, Meera Syal, Ruby Wax and more
The nation's favourite comedy channel, Gold, is set to take a reflective loo...
18/09/2024
Press Release: ToolsOnAir Wins 2024 OEM & Developer Award from Blackmagic Design at IBC
Press Release: ToolsOnAir Wins 2024 OEM & Developer Award from Blackmagic Design...
18/09/2024
A Different Man Is a Triumph That Lingers Long After Credits Role
Warning: This feature contains spoilers about the film. By Bailey Pennick Aaron Schimberg kept it brief in his introduction before A Different Man had its wor...
18/09/2024
Mi Primer Escenario' ofrece a los artistas emergentes en Mxico la oportunidad de tomar el escenario en MEXCLA Spotify
Apoyar a los artistas emergentes es parte fundamental del ADN de Spotify. Ahora ...
18/09/2024
Mi Primer Escenario' Offers Emerging Mexican Artists a Chance To Perform at MEXCLA Spotify'
Supporting emerging artists is a fundamental part of Spotify's DNA, and we...
18/09/2024
Karmic lessons and laughs with new SBS podcast Comedy Karma'
Karmic lessons and laughs with new SBS podcast Comedy Karma' 18 September, 2024 Media releases Join the ever-curious stand-up comedian Aditya Gautam a...
18/09/2024
Clear-Com Supports the Next Generation of Media Professionals at Rise Academy Summer...
eds3_5_jq(document).ready(function($) { $(#eds_sliderM519).chameleonSlider_2_1({...
18/09/2024
Nielsen launches Advanced Audiences, enhancing digital campaign precision, reach and effectiveness across Australia and New Zealand
Sydney, September 18, 2024 - Nielsen today announced the launch of Advanced Audi...
18/09/2024
Heartland Video Systems Welcomes Industry Veteran Dan Whe...
Heartland Video Systems (HVS), a premier video systems integration and consulting firm, is proud to announce the appointment of Dan Whealy as Director of Busine...
18/09/2024
Clear-Com Supports the Next Generation of Media Professio...
Clear-Com is proud to have participated in the Rise Academy Summer School 2024, a transformative experience aimed at introducing young people to the dynamic wor...
18/09/2024
Amplify Berklee Honors the Legacy and Celebrates the Future of Berklee City Music
Amplify Berklee Honors the Legacy and Celebrates the Future of Berklee City Musi...
18/09/2024
Riedel Expands Its Range of NSA Network Stream Adapters
Riedel Communications today announced the launch of two new additions to its acclaimed Network Stream Adapter (NSA) series: the NSA-003A and NSA-006A. Unveiled ...
18/09/2024
WDR Relies on Riedel Backbone for Remote Production of UE...
The German regional public broadcaster Westdeutscher Rundfunk (WDR) has implemented a Riedel backbone for communications and signal distribution for the ARD bro...
18/09/2024
Amagi Expands Footprint in Latin America with Mexico Entr...
Amagi, the global leader in cloud-based SaaS technology for broadcast and connected TV (CTV), today announced that it is entering the Mexican market. This strat...
18/09/2024
ITV to Modernize Its Media Supply Chain With Cloud-Native...
SDVI, the leading platform provider for cloud-native media supply chains, today announced that U.K. broadcaster ITV is deploying the SDVI Rally platform as part...
18/09/2024
Triveni Digital SCTE TechExpo24 Exhibitor Preview
ATSC 3.0 is gaining momentum across the U.S., and for cable operators, ensuring top-notch service quality is more important than ever. Monitoring the performanc...
18/09/2024
Amagi and BuyDRM Partner to Secure Streaming Video on Pla...
Amagi, the global leader in cloud-based SaaS technology for broadcast and connected TV (CTV), today announced a partnership with BuyDRM, a leading content secur...
18/09/2024
Charter Bumps Up Broadband Speeds, Unveils New Bundle Pricing
STAMFORD, Conn. Spectrum has made a series of announcements that include a new simplified pricing strategy, increased broadband speeds and new customer service ...
18/09/2024
Lawo Doubles the Number of HOME Apps
Lawo has announced that its HOME Apps platform now hosts nine essential processing apps, effectively doubling the previous offering with more apps to follow in ...
18/09/2024
Gray, New Orleans Pelicans Announce New Sports Network
Gray Media is partnering with the New Orleans Pelicans to create a new network that promises to bring every non-national Pelicans NBA game to its viewers....
18/09/2024
Fox Weather Expands Distribution to DirecTV
NEW YORK Fox News Media has announced that Fox Weather, a free ad-supported streaming television service (FAST), is now available to DirecTV customers....
18/09/2024
NAB Show New York Exhibitor Insight TAG Video Systems
TV TECH: What do you anticipate will be the most significant technology trends at the 2024 NAB Show New York?...
18/09/2024
Key Conversations With News, Sports Panels on Tap at NAB Show New York
Broadcast, media and entertainment leaders will gather once again at NAB Show New York in October to explore key innovations and strategies reshaping how conten...
18/09/2024
SCTE Foundation Rebrands
EXTON, Pa. The SCTE Foundation has announced a comprehensive relaunch of its efforts. The relaunch includes new branding that will be on display at the Georgia ...
18/09/2024
Viant Technology Launches New Programmatic Ad Solution, ViantAI
IRVINE, Calif. Viant Technology Inc. has launched ViantAI, an advanced AI-powered platform that it says will reshape how programmatic advertising is planned, pu...
18/09/2024
Blackmagic Design Announces Pricing for Blackmagic URSA Cine 17K 65
Blackmagic Design Announces Pricing for Blackmagic URSA Cine 17K 65 Brie Clayton September 17, 2024 0 Comments Revolutionary large format digital film...
18/09/2024
Avid | Stream IO ingest & playout solution now supports SMPTE 2110
Avid | Stream IO ingest & playout solution now supports SMPTE 2110 Brie Clayton September 17, 2024 0 Comments Avid's next-gen software-based produ...
18/09/2024
Telemundo's Celebrando Todo Lo Que Somos' Marks Hispanic Heritage Month
NBCUniversal's Telemundo Enterprises is marking Hispanic Heritage Month with year four of its multiplatform initiative with the slogan Celebrando Todo Lo Q...
18/09/2024
Hispano Soy' Marks Hispanic Heritage Month on Warner Bros. Discovery U.S. Hispanic Nets
To celebrate Hispanic Heritage Month, Warner Bros. Discovery U.S. Hispanic is la...
18/09/2024
MeTV Toons Plans Spooky Sundays in October
As Halloween approaches, MeTV Toons will feature frightful Sunday programming. On October 13, it's a scary Flintstones block 1-3 p.m., then Scooby-Doo! 3-5 ...
18/09/2024
TMZ Investigates Matthew Perry and His Death in Sept. 16 Special on Fox
TMZ takes a close look at actor Matthew Perry and the drug network that led to his death when TMZ Investigates: Matthew Perry & the Secret Celebrity Drug Ring a...
18/09/2024
Olympics Boost Broadcast, Peacock Viewing in August
The Paris Summer Olympics gave a boost to broadcast and popped on Peacock in August, according to Nielsen....
18/09/2024
Gray Forms Gulf Coast Network To Broadcast Pelicans NBA Games
Gray Media is forming a new venture called Gulf Coast Sports & Entertainment Network that will carry all of the locally televised games of the NBA's New Orl...
18/09/2024
LG Study: Asian-American Viewers Prefer Streaming Ads
LG Ad Solutions shared a new report The Inclusive Screen: Asian Americans, which found that 70% of Asian Americans feel that streaming TV ads are more relevant...
18/09/2024
VideoAmp Says It Serves as Currency in $1 Billion of Media Deals
Measurement company VideoAmp said that $1 billion in media buys have already been guaranteed using its measurement as a currency this year....
18/09/2024
Kenny Smith Will Host Season 3 of Harlem Globetrotters: Play It Forward'
Kenny Smith, former NBA star and current TNT Sports analyst, will be the host of Harlem Globetrotters: Play It Forward for Season 3, which premieres Saturday, O...
18/09/2024
ESPN's Jimmy Pitaro Reflects on ESPN Turning 45, Launch of a New App in 2025, and the Role of AI, RSNs
ESPN's Jimmy Pitaro Reflects on ESPN Turning 45, Launch of a New App in 2025...
18/09/2024
Chicago Sports Network Agrees to OTA-TV Deal With Millennial Telecommunications' WJYS-TV
Chicago Sports Network Agrees to OTA-TV Deal With Millennial Telecommunications&...
18/09/2024
New Orleans Pelicans Ink OTA-TV Distribution Deal With Gray Media; Raycom Sports To Produce All Games
New Orleans Pelicans Ink OTA-TV Distribution Deal With Gray Media; Raycom Sports...
18/09/2024
Seattle Kraken Officially Launch Kraken Hockey Network, Promise Biggest Regional Production in the NHL'
Seattle Kraken Officially Launch Kraken Hockey Network, Promise Biggest Regiona...
18/09/2024
New Victory+ Free, DTC Streaming Service From Dallas Stars and APMC Goes Live
New Victory Free, DTC Streaming Service From Dallas Stars and APMC Goes Live In addition to all Stars games, Victory will carry Anaheim Ducks games this seaso...
18/09/2024
ESPN Looking to Hire Drone Pilot
ESPN Looking to Hire Drone Pilot By Ken Kerschbaumer Wednesday, September 18, 2024 - 4:30 pm Print This Story | Subscribe Story Highlights Have the ski...
18/09/2024
Only on Netflix! With Stars like Rodrigo Santoro and Rafael Vitti, Four Brazilian Films Are Starting Production From Bahia to Bariloche
Back to All News Only on Netflix! With Stars like Rodrigo Santoro and Rafael Vi...
18/09/2024
Haivision Wins Prestigious IBC Innovation Award for its Live Video Contribution Solutions over Private 5G Networks
Haivision Wins Prestigious IBC Innovation Award for its Live Video Contribution ...
18/09/2024
Eutelsat Group Secures Additional Launches in New Agreement with Mitsubishi Heavy Industries
Photo credit: Mitsubishi Heavy Industries, Ltd Press release - 18 September 20...
18/09/2024
2024-09-18
iPadOS 18 makes the iPad experience more versatile and intelligent than ever, and is available today as a free software update. iPadOS 18 brings incredible new ...
18/09/2024
Telespazio and ABS Partner to Deliver Enhanced Connectivity Services for the Brazilian Air Force System
Telespazio and ABS Partner to Deliver Enhanced Connectivity Services for the Bra...