![](//www.4rfv.com/images/no_image.jpg)
YouTube robotics influencer Dave Niewinski has developed robots for everything from driveable La-Z-Boy chairs to an AI-guided cornhole tosser and horse-drawn chariot racing.
His recent Interactive Animatronic GLaDOS project was among nine winners in the Hackster AI Innovation Challenge. About 100 contestants vied for prizes from NVIDIA and Sparkfun by creating open-source projects to advance the use of AI in edge computing, robotics and IoT.
Niewinski won first place in the generative AI applications category for his innovative robot based on the GLaDOS guide from game series Portal, the first-person puzzle platform from video game developer Valve.
Other top winners included contestants Andrei Ciobanu and Allen Tao, who took first prize in the generative AI models for the edge and AI at the edge applications categories, respectively. Ciobanu used generative AI to help virtually try on clothes, while Tao developed a ROS-based robot to map the inside of a home to help find things.
Harnessing LLMs for Robots Niewinski builds custom applications for robotics at his Armoury Labs business in Waterloo, Ontario, Canada, where he uses the NVIDIA Jetson platform for edge AI and robotics, creating open-source tutorials and YouTube videos following his experiences.
He built his interactive GLaDOS robot to create a personal assistant for himself in the lab. It handles queries using Transformer-based speech recognition, text-to-speech, and large language models (LLMs) running onboard an NVIDIA Jetson AGX Orin, which interfaces with a robot arm and camera for interactions.
GLaDOS can track his whereabouts in the lab, move in different directions to face him and respond quickly to queries.
I like doing things with robots that people will look at and say it's not what they had immediately expected, he said.
He wanted the assistant to sound like the original GLaDOS from Portal and respond quickly. Fortunately, the gaming company Valve has put all of the voice lines from Portal and Portal 2 on its website, allowing Niewinski to download the audio to help train a model.
Using Jetson, your average question-and-answer stuff runs pretty quick for speech, he said.
Niewinski used NVIDIA's open-source NeMo toolkit to fine-tune a voice for GLaDOS, training a spectrogram generator network called FastPitch and HiFiGAN vocoder network to refine the audio quality.
Both networks are deployed on Orin with NVIDIA Riva to enable speech recognition and synthesis that's been optimized to run at many times the real-time rate of speech, so that it can run alongside the LLM while maintaining a smooth, interactive delivery.
For generating realistic responses from GLaDOS, Niewinski uses a locally hosted LLM called OpenChat that he runs in Docker from jetson-containers, saying that it was a drop-in replacement for OpenAI's API. All of this AI is running on the Jetson module, using the latest open-source ML software stack built with CUDA and JetPack.
To enable GLaDOS to move, Niewinski developed the interactions for a Unitree Z1 robotic arm. It has a stereo camera and models for seeing and tracking a human speaking and a 3D-printed GLaDOS head and body shell around the arm.
Trying on Generative AI for Fashion Fit Winner Ciobanu, based in Romania, aimed to improve the virtual clothing try-on experience with the help of generative AI, taking a top prize for his EdgeStyle: Fashion Preview at the Edge.
He used AI models such as YOLOv5, SAM and OpenPose to extract and refine data from images and videos. Then he used Stable Diffusion to generate the images, which he said was key to achieving accurate virtual try-ons.
This system taught the model how clothes fit different poses on people, which he said enhanced the realism of the try-ons.
It's quite handy as it allows users to see how clothes would look on them without actually trying them on, said Ciobanu.
The NVIDIA JetPack SDK provided all the tools needed to run AI models smoothly on the Jetson Orin, he said.
It's super-helpful to have a stable set of tools, especially when you're dealing with AI tech that keeps changing, said Ciobanu. It really cut down on the time and hassle for us developers, letting us focus more on the cool stuff we're building instead of getting stuck on tech issues.
Finding Lost Items With Robot Assistance Winner Tao, based in Ontario, Canada, created a robot to lessen the burden of searching for things lost around the house. His An Eye for an Item project took top honors at the Hackster challenge.
Finding lost objects is a chore, and recent developments in zero-shot object detection and LLMs make it feasible for a computer to detect arbitrary objects for us based on textual or pictorial descriptions, presenting an opportunity for automation, said Tao.
Tao said he needed robot computing capabilities to catalog objects in any unstructured environment - whether a living room or large warehouse. And he needed it to also perform real-time calculations for localization to help with navigation, as well as running inference on larger object detection models.
Jetson Orin was a perfect fit, supporting all functionality from text and image queries into NanoDB, to real-time odometry feedback, including leveraging Isaac ROS' hardware-accelerated AprilTag detections for drift correction, he said.
Other winners of the AI Innovation Challenge include:
George Profenza, Escalator people tracker, 2nd place, Generative AI Applications category
Dimiter Kendri, Cooking meals with a local AI assistant using Jetson AGX Orin, 3rd place, Generative AI Applications category
Vy Phan, ClearWaters Underwater Image Enhancement with Generative AI, 2nd place, Generative AI Models category
Huy Mai, Realtime Language Segment Anything on Jetson Orin, 2nd place, Generative AI Models category
Fakhrur Razi, Autonomous Int
Most recent headlines
07/09/2024
The official kickoff to the college football season provided another reminder of how football can boost audiences across all platforms, with ESPN and ABC report...
07/09/2024
CAMBRIDGE, U.K. Adder Technology, a provider of connectivity solutions and high performance IP KVM technologies, has announced a new addition to its KVM matrix ...
07/09/2024
CINCINNATI The E.W. Scripps Company (NASDAQ: SSP) has promoted Sean Franklin to vice president and general manager for WLEX, the Scripps-owned NBC affiliate in ...
07/09/2024
AMSTERDAM Cobalt Digital has announced that during the IBC Show it will be sharpening its focus on IP-based solutions by adding a scalable software-based multiv...
07/09/2024
The IBC Talent Programme will look at how the industry can collaborate to foster new talent, make training more accessible, and develop common standards for div...
07/09/2024
The non-profit association aims to reduce risks and support trust in media, in the face of increasing threat levels
By Matthew Corrigan
Published: September ...
07/09/2024
Cerberus Tech, a leader in cloud-native IP contribution and distribution, today announced a series of updates that combine to increase user control in working w...
07/09/2024
Interra Systems, a leading provider of end-to-end quality assurance solutions to the digital media industry, today announced a strategic partnership with Videli...
07/09/2024
oMoVo, an innovator in AI-driven digital asset management (DAM), is launching ioPilot, a new feature in its game-changing DAM platform. Designed to streamline c...
07/09/2024
Creativepool Annual 2024: The Most Awards Launched at Sinfonia Smith Square
Brie Clayton September 7, 2024
0 Comments
The creative industry's mo...
07/09/2024
Meredith Veloz has been named news director at WRMD Tampa, part of NBC Universal Local's Telemundo Station Group. She starts in early October and will repor...
06/09/2024
By Jessica Herndon
One of the most exciting things about the Sundance Film Fes...
06/09/2024
PARK CITY, UTAH - JANUARY 22: (L-R) Eugene Grygo, Phoebe Hoffman, Lana Wilson, Michael Kim, and Nikenya Hall attend the 2024 Sundance Film Festival Look Into M...
06/09/2024
The highly anticipated opus from director Quentin Tarantino doesn't disappoint. It's a typically outrageous conglomeration of comedy and shock laced wit...
06/09/2024
Cinema, the most voyeuristic of mediums, is uniquely placed to convey both the act of surveillance and the psychology of the surveilled. In Seberg, director Ben...
06/09/2024
In February 2019, director J.J. Abrams called Cut! for the last time during principal photography of the ninth chapter of the long-running Star Wars saga. Sta...
06/09/2024
In a cinematic universe expanding with superheroes, the creative challenge for filmmakers is to set newcomers apart from the crowd. The task is doubly important...
06/09/2024
Having photographed episodes of the Showtime horror drama Penny Dreadful's second and third seasons, John Conroy, ISC was eager to reteam with showrunner Jo...
06/09/2024
The psychological thriller Homecoming returns for its sophomore season with fresh twists, nail-biting cliff-hangers, and more of the unnerving cinematography th...
06/09/2024
Based on the series of novels by Eoin Colfer, the feature Artemis Fowl finds its namesake a precocious, hyperintelligent 12-year-old boy (played by Ferdia Sha...
06/09/2024
Vincent lives in Los Angeles, Anthony in New York City. They met on a dating app and have been talking for months, but the physical distance between them has ke...
06/09/2024
What are we going to talk about today? cinematographer Anastas Michos, ASC, GSC asks almost immediately after answering his phone. It's late July when Pana...
06/09/2024
The recent features and bear a number of similarities. Chief among them: Both were shot by cinematographer Hillary Spera. Each story also exists in the realm of...
06/09/2024
Angeliena follows the adventures and self-discovery of its namesake heroine (played by Euodia Samson), a lovable parking attendant who harbors dreams of traveli...
06/09/2024
Pair a British-born songwriter-director who's collaborated with musicians like Jay-Z with a Romanian cinematographer known for shooting intense dramas, and ...
06/09/2024
It was June 2019 when director Baz Luhrmann first approached cinematographer Mandy Walker, ACS, ASC about collaborating on the new biopic Elvis. I was extremel...
06/09/2024
Based on the best-selling mystery novel of the same name, the feature Where the Crawdads Sing tells the story of Kya (Daisy Edgar-Jones), a young woman who grew...
06/09/2024
Johannesburg, 05 September 2024 - The National Film and Video Foundation (NFVF),...
06/09/2024
Video over IP, in the sense that's used in film and television, sometimes feels like a newer idea than it really is. That might be because the high-end, unc...
06/09/2024
The ability of NFL football to attract large audiences in an era of declining viewing for linear TV was on full display in the NFL seasons opener, with an avera...
06/09/2024
MELVILLE, N.Y. Chyron has beefed up its team in EMEA and APAC with the addition of three new positions intended to enhance its growth and better serve customers...
06/09/2024
NEW YORK FutureB2B, a world-leading provider of business intelligence and networking services, is proud to announce the appointment of Joe Palombo as the publis...
06/09/2024
Adventurer A/V, a leading Taiwanese broadcast production company and equipment rental service provider based in Taipei City, has invested in latest-generation U...
06/09/2024
At IBC 2024, Hall 1 B33, leading video software provider, Synamedia, will unveil a raft of new video network solutions as it solidifies its position as a key te...
06/09/2024
LiveU today launched its Lightweight Sports Production solution making fully featured, professional live sports production accessible and affordable for every e...
06/09/2024
Osprey Video
IBC2024 Show Preview
Sept. 13-16
Various Partner Stands
At IBC2024, Osprey Video's encoding/decoding technology will be on display at sever...
06/09/2024
GB Labs, the leader in intelligent, centralized media storage, today announced the launch of NebulaNAS, a transformative new cloud storage solution developed fr...
06/09/2024
Suitest, provider of test automation for OTT applications, today announced the full availability of image- and object-based test automation capabilities in a si...
06/09/2024
France 3 Nouvelle Aquitaine (.3NoA) is a 100% regional channel that aims to promote culture and creativity at regional level.
At the beginning of 2024, the cha...
06/09/2024
At IBC 2024, Camera control specialists Videosys Broadcast will be announcing a new bidirectional radio camera system that will simplify wireless workflows, as ...
06/09/2024
Magnifi by VideoVerse, an AI-driven video technology company and powerful video-editing SaaS platform, has announced a partnership with Storyteller,a leading Sa...
06/09/2024
iWedia, a leading provider of software solutions for connected TV devices and services, is proud to announce its selection as the official vendor for the HbbTV ...
06/09/2024
LTN and Skyline Communications announce a new collaboration to drive enhanced broadcast monitoring and control for Tier 1 media, sports, entertainment, and tech...
06/09/2024
IBC2024 today announces the line-up of speakers and sessions for the first-ever dedicated IBC Talent Programme, taking place on Monday 16 September in the Showc...
06/09/2024
Kiloview, a leading provider of IP-based video transmission solutions, is set to make a significant impact at IBC 2024 with the unveiling of three groundbreakin...
06/09/2024
The newly combined entity of Fabric and Xytech Systems, currently known as Fabric x Xytech, is pleased to announce the appointment of Kira Baca as Chief Revenue...
06/09/2024
Student Spotlight: Mateo Londo o The producer talks about leaving Berklee to grieve the passing of a close friend and fellow student and then returning to fin...
06/09/2024
More rough news for grownups in traditional media. According to a new study from Precise TV, teens prefer the ads on YouTube and are turning to Netflix and YouT...
06/09/2024
A data-driven campaign created by Samsung Ads and Davis Elen Advertising for the...
06/09/2024
Glen Dacy has been named VP of news content and streaming at WFLD-WPWR Chicago, starting September 9. He most recently worked with the CBS Network News Special ...