Puget Systems Debuts On-Premise Generative AI and Machine Learning Custom Server at SIGGRAPH 2023
11/08/2023
Brie Clayton August 10, 2023
0 Comments
Puget Systems' Specialized AI Training and Inference Server Supports Up to Four NVIDIA RTX 6000 Ada Graphics Cards to Host Web-Based Chat Interface for Large Language Models
Puget Systems (www.pugetsystems.com) today announced it will debut a custom Generative AI and Machine Learning server at SIGGRAPH 2023 in Los Angeles this week. In its booth #630 in the LA Convention Center, the team from Puget Systems will demonstrate its new specialized AI Training and Inference server, configured with four NVIDIA RTX 6000 Ada graphics cards to handle intensive generative AI and machine learning and to effectively manage real-time rendering, graphics, AR/MR/VR/XR, compute, and deep learning processing.
The Puget Systems AI Training and Inference server is a rackmount workstation capable of hosting a web-based chat server using STOA models such as the Meta-Llama-2-70b large language models (LLMs) supporting multiple simultaneous users. Puget Systems Labs conducted extensive testing of this configuration with Llama-2-70b and Falcon-40b. (Falcon-40b requires less memory space and can run with only two RTX 6000 Ada GPUs.) In addition to running a chat interface, this hardware is also suitable for base model fine-tuning within the available GPU memory limits.
Puget Labs Testing Processes and Results The Puget Systems Lab team conducted extensive testing of the new AI Training and Inference servier, utilizing a full set of four NVIDIA RTX 6000 Ada graphics cards. Labs tested the system with Meta's Llama-2-70b-chat-hf, using HuggingFace Text-Generation-Inference (TGI) server and HuggingFace ChatUI. The test model used approximately 130GB of video memory (VRAM), and the Labs confirmed that the system should work well with other LLMs that fit within available GPU memory (192GB with four cards installed).
Following are some notable performance stats from the testing:
Typical usage measured response:
Validation Time = 0.59673 ms
Queue Time = 0.17409 ms
Time per Token = 54.558 ms
Stress tested with multiple concurrent users
Data below is from a session with 114 prompts (20-30 users) over 5 minutes
Average prompt response under multi-user load:
Validation Time = 3.0312 ms
Queue Time = 4687.9 ms
Time per Token = 68.076 ms
For more information on Puget Systems AI Training and Inference server, please visit here.
August 11, 2023Indie Feature film Moon Garden Uses Cintel Scanner and DaVinci Resolve StudioBlackmagic Design today announced that the independent feature film, Moon Garden, which
August 11, 2023HP Redefining a Flexible Workstation Experience for CreatorsToday, at SIGGRAPH, HP is announcing the new HP Z4 Rack G5,
Pricing and Availability Puget Systems custom AI Training and Inference servers will be available for configuration for a wide range of generative AI applications beginning in the coming weeks. To learn more or to join the waitlist, please visit here. To learn more about Puget Systems Canadian consulting and sales operations, please visit here.
Enjoying the news? Sign up for the Creative COW Newsletter! Sign up for the Creative COW newsletter and get weekly updates on industry news, forum highlights, jobs, inspirational tutorials, tips, burning questions, and more! Receive bulletins from the largest, longest-running community dedicated to supporting professionals working in film, video, and audio.
Enter your email address, and your first and last name below!
-- Sign up: * indicates required
Email Address *
Name
--
LINK: | https://creativecow.net/puget-systems-debuts-on-premise-generative-ai-... |
See more stories from creativecow |
Most recent headlines
09/12/2024
Dalet Named an IDC Innovator in Media and Entertainment
Dalet, a leading technology and service provider for media-rich organizations, today announced that it has been named an IDC Innovator in the IDC Innovators: ...
09/11/2024
Dalet Expands Leadership Team to Fuel Next Stage of Growth
Dalet, a leading technology and service provider for media-rich organizations, today announced three new members of its executive team. Tara Bryant joins as Chi...
30/10/2024
Drone detection just got a whole lot smarter with ARDRONIS Detect from Rohde & Schwarz
Drone detection just got a whole lot smarter with ARDRONIS Detect from Rohde & S...
26/10/2024
Kelsea Ballerini and Spotify Host Intimate Listening Party To Celebrate Her New Album, PATTERNS'
I love New York City, but I can't stay at the Carlyle anymore, sings Kelsea...
26/10/2024
USA Cheer Adopts NBC Sports Next's SportsEngine Motion
MINNEAPOLIS NBC Sports Next has announced a multiyear technology agreement with USA Cheer, the governing body for sport cheering in the U.S. The organization an...
26/10/2024
Canada's MSCTV Chooses Witbe Virtual NOC for Monitoring
NEW YORK Witbe has announced that Mitchell Seaforth Cable TV (MSCTV), a Dublin, Ontario, Canada-based provider of solutions to telecommunications companies, is ...
26/10/2024
IAB Tech Lab Launches Ad Format Idol' Initiative
NEW YORK The IAB Tech Lab has launched Ad Format Idol, an initiative to standardize emerging connected TV ad formats....
26/10/2024
T-Mobile Pledges Up to $1 Million for Ongoing Hurricane Recovery Efforts
BELLEVUE, Wash. T Mobile has announced a Rally for Relief effort to help those impacted by recent hurricanes. As part of the campaign, the cellphone carrier sai...
26/10/2024
Comercio TV Taps Broadpeak for OTT Video Delivery and Monetization
CESSON-SEVIGNE, France Broadpeak said Comercio TV, the Spanish-language OTT financial-news channel, has adopted its's software-as-a-service (SaaS) platform ...
26/10/2024
Africana Studies at Berklee to Induct Kid 'n Play into Berklee's Hip-Hop Hall of Fame
Africana Studies at Berklee to Induct Kid 'n Play into Berklee's Hip-Hop...
25/10/2024
What to Watch: These 7 DFP-Supported Films From 2024 Will Inspire You to Organize for Change
It's easy to become overwhelmed when thinking about the current state of the...
25/10/2024
Fall in Love With Your Monster
PARK CITY, UTAH - JANUARY 18: (L-R) Tommy Dewey, Melissa Barrera, Kayla Foster, Meghann Fahy, and Edmund Donovan attend the 2024 Sundance Film Festival Your Mo...
25/10/2024
How Matthew Belloni Preps for a Live Recording of His Hit Ringer Podcast, The Town'
Curious about what's going on behind the scenes in Hollywood? Odds are Matth...
25/10/2024
SBS celebrates the Festivals of Lights with distinctive content to entertain all Australians
SBS celebrates the Festivals of Lights with distinctive content to entertain all...
25/10/2024
L3Harris Technologies Reports Strong Third Quarter 2024 Results, Increases 2024 Guidance
Highlights* Orders of $7.2 billion; book-to-bill of 1.4x Revenue of $5.3 billi...
25/10/2024
L3Harris, Palantir CEOs Discuss Strategic Partnership on CNBC's "Closing Bell: Overtime"
On Oct. 24, Chris Kubasik and Alex Karp sat down with CNBC's Morgan Brennan ...
25/10/2024
Meta Quest to Offer 52 NBA Games in VR for 2024-2025 Season
Meta Quest, a division of Facebook owner Meta, has announced it will continue its virtual reality broadcasts of regular-season NBA games for the fourth consecut...
25/10/2024
Disney Expected to Announce Major AI Initiative'
Disney one of the world's largest owners of intellectual property is getting ready to unveil a major AI initiative that will focus primarily on post-produ...
25/10/2024
Broadcast Solutions establishes CTO Office to drive innov...
Broadcast Solutions, one of Europe's largest media systems integration groups, has extended its technical leadership with two key appointments. Pierre Mestr...
25/10/2024
Unified Streaming fortifies customer ties with two new hi...
Unified Streaming, a leading streaming software company, has hired Valentijn Siebrands, an industry veteran, as its new streaming solutions manager. The company...
25/10/2024
Imagine Communications John Mailhot Receives 2024 David S...
Imagine Communications' John Mailhot was awarded the prestigious 2024 David Sarnoff Medal during SMPTE's annual Awards Gala, held on October 24 at the R...
25/10/2024
Independent Midwest Broadcaster selects Rohde & Schwarz TE1 liquid-cooled transmitter to expand audience reach
Independent Midwest Broadcaster selects Rohde & Schwarz TE1 liquid-cooled transm...
25/10/2024
Mexico's First Virtual Production Game Show Produced with Blackmagic Design
Mexico's First Virtual Production Game Show Produced with Blackmagic Design Brie Clayton October 25, 2024 0 Comments Club Cin polis Desaf o India...
25/10/2024
Using invisible effects to hide a stage light in Adobe After Effects
Using invisible effects to hide a stage light in Adobe After Effects Graham Quince October 25, 2024 0 Comments Lasers, dinosaurs and space VFX are coo...
25/10/2024
Improve your Color Grading using these simple steps in DaVinci Resolve 19
Improve your Color Grading using these simple steps in DaVinci Resolve 19 Kasia Jarco October 25, 2024 0 Comments Do you truly understand how to color...
25/10/2024
Brighter Eyes
Jenny Priestley talks to the BFI's Douglas Weir who oversaw the restoration of a much-loved animated classic, which returns to UK and Ireland cinemas today ...
25/10/2024
Comercio TV Chooses Broadpeak to Optimize OTT Video Deliv...
Broadpeak , a leading provider of content delivery network (CDN) and video streaming solutions for content providers and pay-TV operators worldwide, announced t...
25/10/2024
Amagi Partners With Allrites to Expand the Global Distrib...
Amagi, the global leader in cloud-based SaaS technology for broadcast and connected TV (CTV), has announced a strategic partnership with Allrites, a premium con...
25/10/2024
Studio Technologies Enhances ST 2110 Integration with Mer...
Studio Technologies, Inc., a pioneer in high-performance audio, video, and fibre-optic solutions, has long been at the forefront of the professional broadcast a...
25/10/2024
QuickLink Improves Remote Production for Saturday Night T...
QuickLink, empowering creatives to exceed modern-day standards of production, has been instrumental in easing the remote production challenges of ITV Studios...
25/10/2024
stYpe AI Glue and StypeCentral Advance the Art of TV Tale...
stYpe, a globally active innovator of virtual studio production resources, reports strong interest in the latest additions to its wide range of broadcast-qualit...
25/10/2024
Top Argentine Music Facility Casa Frida Estudio Chooses P...
Casa Frida Estudio, one of the most prestigious music facilities in Argentina, has upgraded its main control room to Dolby Atmos specifications and installed a ...
25/10/2024
Frequency expands Managed Channel Services to Europe at M...
Frequency, the engine behind many of the world's best-known streaming television channels, announces the European expansion of its Managed Channel Services ...
25/10/2024
InnovateUK and EU Approve Funding for Immersive Technolog...
Disguise, an HPA-Award-winning immersive experience company, has received £1.2 Million in funding from Innovate UK and the EU as part of a larger investment int...
25/10/2024
MASORANGE AND AGILE CONTENT RENEW THEIR COLLABORATION AGR...
Through this agreement, MASORANGE will incorporate new AGILE CONTENT technologies into its television service, enhancing customer experience with optimum servic...
25/10/2024
Swiss Indoors Basel 2024 delivered with Appear X Platform
Appear, a global leader in live production technology, is proud to announce its role as an official provider of cutting-edge broadcast transmission solutions fo...
25/10/2024
MSCTV Chooses Witbes Virtual NOC for Proactive Live TV Mo...
Witbe (Euronext Growth FR0013143872 ALWIT), a global leader in test automation and monitoring technology for video service providers, today announced that M...
25/10/2024
Unified Streaming fortifies customer ties with two hires
Unified Streaming, a leading streaming software company, has hired Valentijn Siebrands, an industry veteran, as its new streaming solutions manager. The company...
25/10/2024
Independent Midwest Broadcaster selects Rohde and Schwarz...
Competitive search leads Toledo's WLMB-TV 40 to TE1 based on energy efficiency and durability As part of its effort to reach a wider audience, the Toledo-b...
25/10/2024
IBC Accelerator Programme 2025 - Call For Industry Challe...
IBC announces the Call for Challenges for the 2025 Accelerator Media Innovation Programme, inviting proposals for transformative, fast-track innovation projects...
25/10/2024
TVN Deploys Lawo AoIP Technology in New OB Truck
HANNOVER, Germany TVN Live Production is using a variety of solutions and technologies from Lawo in its new TVN- 8UHD OB truck, which it unveiled at this year...
25/10/2024
LG Ad Solutions Joins Innovid's Harmony Initiative
NEW YORK Innovid, a software platform for the creation, delivery, measurement and optimization of connected TV (CTV), linear and digital advertising, said LG Ad...
25/10/2024
Hightouch Launches New CTV Advertising Offering
SAN FRANCISCO Consumer data platform provider Hightouch has announced a new solution it said will streamline and accelerate connected TV (CTV) advertising....
25/10/2024
Max Adds DAZN's Sports Offering in Spain
MADRID Warner Bros. Discovery's streaming service Max has reached a new agreement with DAZN that will give Max customers in Spain access to the premium spor...
25/10/2024
Beloved Genre-Benders Karate Make a Fitting Comeback
Beloved Genre-Benders Karate Make a Fitting Comeback Its one of the decades feel-good music stories: How the legendary alumni acts rediscovery led to rereleas...
25/10/2024
Berklee Presents Holiday Extravaganza: Duke Ellingtons Nutcracker Suite and Christmas Vibes Featuring Warren Wolf
Berklee Presents Holiday Extravaganza: Duke Ellingtons Nutcracker Suite and Chri...
25/10/2024
Canadian engineer Elias Ghannoum wins 2024 Lord Kelvin Award
The IEC has awarded its annual Lord Kelvin Award to the Canadian engineer, Elias Ghannoum. The award recognizes exceptional and long-term contributions to globa...
25/10/2024
Dolby and Jin from BTS Celebrate His New Single I'll Be There in Dolby Atmos
25 October 2024, 16:00 (PDT) Dolby and Jin from BTS Celebrate His New Single I'll Be There in Dolby Atmos Love Jin More In Dolby | #LoveMoreInDolby F...
25/10/2024
JazzCash and Dgpays Join Forces to Enhance Marketplace Payments in Pakistan
25 Oct 2024 JazzCash and Dgpays Join Forces to Enhance Marketplace Payments in Pakistan Amsterdam and Dubai, 25 October 2024: VEON Ltd. (Nasdaq: VEON, Euronext...
25/10/2024
Monthly Rewind: Groundbreaking Colour Innovation, Unreal Engine, The Academy Museum, and more.
October has been a wicked month across Technicolor Group! So much so, we introdu...