Synthetic data has become a standard part of large language model (LLM) post-training procedures. Using a large number of synthetically generated examples from either a single or cohort of open-source, commercially permissible LLMs, a base LLM is finetuned either with supervised finetuning or RLHF to gain instruction-following and reasoning skills. This process can be seen as a knowledge��
]]>
Curating high-quality pretraining datasets is critical for enterprise developers aiming to train state-of-the-art large language models (LLMs). To enable developers to build highly accurate LLMs, NVIDIA previously released Nemotron-CC, a 6.3-trillion-token English language Common Crawl (CC) dataset. Today, the NVIDIA NeMo Curator team is excited to share that the pipeline used to build the��
]]>
As large language models (LLM) gain popularity in various question-answering systems, retrieval-augmented generation (RAG) pipelines have also become a focal point. RAG pipelines combine the generation power of LLMs with external data sources and retrieval mechanisms, enabling models to access domain-specific information that may not have existed during fine-tuning.
]]>
Evaluating large language models (LLMs) and retrieval-augmented generation (RAG) systems is a complex and nuanced process, reflecting the sophisticated and multifaceted nature of these systems. Unlike traditional machine learning (ML) models, LLMs generate a wide range of diverse and often unpredictable outputs, making standard evaluation metrics insufficient. Key challenges include the��
]]>
As robotics and autonomous vehicles advance, accelerating development of physical AI��which enables autonomous machines to perceive, understand, and perform complex actions in the physical world��has become essential. At the center of these systems are world foundation models (WFMs)��AI models that simulate physical states through physics-aware videos, enabling machines to make accurate decisions and��
]]>
Training physical AI models used to power autonomous machines, such as robots and autonomous vehicles, requires huge amounts of data. Acquiring large sets of diverse training data can be difficult, time-consuming, and expensive. Data is often limited due to privacy restrictions or concerns, or simply may not exist for novel use cases. In addition, the available data may not apply to the full range��
]]>
Action recognition models such as PoseClassificationNet have been around for some time, helping systems identify and classify human actions like walking, waving, or picking up objects. While the concept is well-established, the challenge lies in building a robust computer vision model that can accurately recognize the range of actions across different scenarios that are domain- or use case��
]]>
Llama 3.1 Nemotron 70B Reward model helps generate high-quality training data that aligns with human preferences for finance, retail, healthcare, scientific research, telecommunications, and sovereign AI.
]]>
This post is the third in a series on building multi-camera tracking vision AI applications. We introduce the overall end-to-end workflow and fine-tuning process to enhance system accuracy in the first part and second part. NVIDIA Metropolis is an application framework and set of developer tools that leverages AI for visual data analysis across industries. Its multi-camera tracking reference��
]]>
The Llama-3.1-Nemotron 70B-Reward model helps generate high-quality training data that aligns with human preferences for finance, retail, healthcare, scientific research, telecommunications, and sovereign AI. This post was updated on August 16, 2024 to reflect the most recent Reward Bench results. Since the introduction and subsequent wide adoption of large language models (LLMs)��
]]>
Synthetic data isn��t about creating new information. It��s about transforming existing information to create different variants. For over a decade, synthetic data has been used to improve model accuracy across the board��whether it is transforming images to improve object detection models, strengthening fraudulent credit card detection, or improving BERT models for QA. What��s new?
]]>
Synthetic data in medical imaging offers numerous benefits, including the ability to augment datasets with diverse and realistic images where real data is limited. This reduces the costs and labor associated with annotating real images. Synthetic data also provides an ethical alternative to using sensitive patient data, which helps with education and training without compromising patient privacy.
]]>
As vision AI complexity increases, streamlined deployment solutions are crucial to optimizing spaces and processes. NVIDIA accelerates development, turning ideas into reality in weeks rather than months with NVIDIA Metropolis AI workflows and microservices. In this post, we explore Metropolis microservices features: Managing and automating infrastructure with AI is��
]]>
AI is rapidly changing industrial visual inspection. In a factory setting, visual inspection is used for many issues, including detecting defects and missing or incorrect parts during assembly. Computer vision can help identify problems with products early on, reducing the chances of them being delivered to customers. However, developing accurate and versatile object detection models remains��
]]>
Missed GTC or want to replay your favorite training labs? Find it on demand with the NVIDIA GTC Training Labs playlist.
]]>
We are announcing our collaboration with Intrinsic.ai on learning foundation skill models for industrial robotics tasks. Many pick-and-place problems in industrial manufacturing are still completed by human operators as it is still challenging to program robots for these tasks. For instance, in a machine-tending setting, a collaborative robot could be used to pick raw material parts from a��
]]>
Learn how synthetic data is supercharging 3D simulation and computer vision workflows, from visual inspection to autonomous machines.
]]>
For robotic agents to interact with objects in their environment, they must know the position and orientation of objects around them. This information describes the six degrees of freedom (DOF) pose of a rigid body in 3D space, detailing the translational and rotational state. Accurate pose estimation is necessary to determine how to orient a robotic arm to grasp or place objects in a��
]]>
Autonomous vehicles (AV) come in all shapes and sizes, ranging from small passenger cars to multi-axle semi-trucks. However, a perception algorithm deployed on these vehicles must be trained to handle similar situations, like avoiding an obstacle or a pedestrian. The datasets used to develop and validate these algorithms are typically collected by one type of vehicle�� for example sedans��
]]>
Synthetic data can play a key role when training perception AI models that are deployed on autonomous mobile robots (AMRs). This process is becoming increasingly important in manufacturing. For an example of using synthetic data to generate a pretrained model that can detect pallets in a warehouse, see Developing a Pallet Detection Model Using OpenUSD and Synthetic Data.
]]>
Data is the lifeblood of AI systems, which rely on robust datasets to learn and make predictions or decisions. For perception AI models specifically, it is essential that data reflects real-world environments and incorporates the array of scenarios. This includes edge use cases for which data is often difficult to collect, such as street traffic and manufacturing assembly lines.
]]>
Sensor simulation is a critical tool to address the gaps in real-world data for autonomous vehicle (AV) development. However, it is only effective if sensor models accurately reflect the physical world. Sensors can be either passive, such as cameras��or active, sending out either an electromagnetic wave (lidar, radar) or an acoustic wave (ultrasonic) to generate the sensor output.
]]>
Smart cities are the future of urban living. Yet they can present various challenges for city planners, most notably in the realm of transportation. To be successful, various aspects of the city��from environment and infrastructure to business and education��must be functionally integrated. This can be difficult, as managing traffic flow alone is a complex problem full of challenges such as��
]]>
Imagine you are a robotics or machine learning (ML) engineer tasked with developing a model to detect pallets so that a forklift can manipulate them. ?You are familiar with traditional deep learning pipelines, you have curated manually annotated datasets, and you have trained successful models. You are ready for the next challenge, which comes in the form of large piles of densely stacked��
]]>
Detecting far-field objects, such as vehicles that are more than 100 m away, is fundamental for automated driving systems to maneuver safely while operating on highways. In such high-speed environments, every second counts. Thus, if the perception range of an autonomous vehicle (AV) can be increased from 100 m to 200 m while traveling at 70 mph, the vehicle has significantly more time to��
]]>
NVIDIA will present 19 research papers at SIGGRAPH, the year��s most important computer graphics conference.
]]>
Most drone inspections still require a human to manually inspect the video for defects. Computer vision can help automate and accelerate this inspection process. However, training a computer vision model to automate inspection is difficult without a large pool of labeled data for every possible defect. In a recent session at NVIDIA GTC, we shared how Exelon is using synthetic data generation��
]]>
According to the American Society of Quality (ASQ), defects cost manufacturers nearly 20% of overall sales revenue. The products that we interact with on a daily basis��like phones, cars, televisions, and computers��must be manufactured with precision so that they can deliver value in varying conditions and scenarios. AI-based computer vision applications are helping to catch defects in the��
]]>
MoveIt is a robotic manipulation platform that incorporates the latest advances in motion planning, manipulation, 3D perception, kinematics, control, and navigation. PickNik Robotics, the company leading the development of MoveIt, is exploring the use of NVIDIA Isaac Sim in an internal R&D project. The project goals are to improve perception for manipulation and augment with MoveIt Studio, PickNik����
]]>
Training AI models requires mountains of data. Acquiring large sets of training data can be difficult, time-consuming, and expensive. Also, the data collected may not be able to cover various corner cases, preventing the AI model from accurately predicting a wide variety of scenarios. Synthetic data offers an alternative to real-world data, enabling AI researchers and engineers to bootstrap��
]]>
Get to know the NVIDIA technologies and software development tools powering the latest in robotics and edge AI.
]]>
Learn how simulation and synthetic data are transforming vision AI applications at the NVIDIA Metropolis meetup on February 22 and 23.
]]>
Discover how 3D synthetic data generation is accelerating AI and simulation workflows.
]]>
Autonomous vehicle development is all about scale. Engineers must collect and label massive amounts of data to train self-driving neural networks. This data is then used to test and validate the AV system, which is also an immense undertaking to ensure robustness. Simulation is an important tool to reach this level of scale, but accuracy is key to its effectiveness. NVIDIA DRIVE Sim��
]]>
To accelerate the development of 3D worlds and the metaverse, NVIDIA has launched numerous AI research projects to help creators across industries unlock new possibilities with generative AI. Generative AI will touch every aspect of the metaverse and it is already being leveraged for use cases like bringing AI avatars to life with Omniverse ACE. Many of these projects��
]]>
Announced at GTC, technical artists, software developers, and ML engineers can now build custom, physically accurate, synthetic data generation pipelines in the cloud with NVIDIA Omniverse Replicator. Omniverse Replicator is a highly extensible framework built on the NVIDIA Omniverse platform that enables physically accurate 3D synthetic data generation to accelerate the training and accuracy��
]]>
Synthetic data is an important tool in training machine learning models for computer vision applications. Researchers from NVIDIA have introduced a structured domain randomization system within Omniverse Replicator that can help you train and refine models using synthetic data. Omniverse Replicator is an SDK built on the NVIDIA Omniverse platform that enables you to build custom synthetic��
]]>
Companies providing synthetic data generation tools and services, as well as developers, can now build custom physically accurate synthetic data generation pipelines with the Omniverse Replicator SDK. Built on the NVIDIA Omniverse platform, the Omniverse Replicator SDK is available in beta within Omniverse Code. Omniverse Replicator is a highly extensible SDK built on a scalable Omniverse��
]]>
]]>
To develop an accurate computer vision AI application, you need massive amounts of high-quality data. With a traditional dataset, you might spend months collecting images, getting annotations, and cleaning data. When it��s done, you could find edge cases and need more data, starting the cycle all over again. For years, this cycle has held back AI, especially in computer vision.
]]>
Data sits at the heart of model explainability. Explainable AI (XAI) is a rapidly advancing field looking to provide insights into the complex decision-making processes of AI algorithms. Where AI has a significant impact on individuals�� lives, like credit risk scoring, managers and consumers alike rightfully demand insight into these decisions. leading financial institutions are already��
]]>
The new NVIDIA NGP Instant NeRF is a great introduction to getting started with neural radiance fields. In as little as an hour, you can compile the codebase, prepare your images, and train your first NeRF. Unlike other NeRF implementations, Instant NeRF only takes a few minutes to train a great-looking visual. In my hands-on video (embedded), I walk you through the ins and outs of making��
]]>
Big data, new algorithms, and fast computation are three main factors that make the modern AI revolution possible. However, data poses many challenges for enterprises: difficulty in data labeling, ineffective data governance, limited data availability, data privacy, and so on. Synthetically generated data is a potential solution to address these challenges because it generates data points by��
]]>
From building cars to helping surgeons and delivering pizzas, robots not only automate but also speed up human tasks manyfold. With the advent of AI, you can build even smarter robots that can better perceive their surroundings and make decisions with minimal human intervention. Take, for instance, an autonomous robot used in warehouses to move payloads from one place to another.
]]>
]]>
Building AI models from scratch requires enormous amounts of data, time, money, and expertise. This is at odds with what it takes to succeed in the AI space: fast time-to-market and the ability to quickly evolve and customize solutions. NVIDIA TAO, an AI-Model-Adaptation framework, enables you to leverage production-quality, pretrained AI models and fine-tune them in a fraction of the time��
]]>
AI pioneer Andrew Ng is calling for a broad shift to a more data-centric approach to machine learning (ML). He recently held the first data-centric AI competition on data quality, which many claim represents 80% of the work in AI. ��I��m optimistic that the AI community before long will take as much interest in systematically improving data as architecting models,�� Ng wrote in his newsletter��
]]>
Building training and testing playgrounds to help advance sport analytics AI solutions out of the lab and into the real world is exceedingly challenging. In team-based sports, building a correct playing strategy before the championship season is a key to success for any professional coach and club owner. While coaches strive at providing best tips and point out mistakes during the game��
]]>
Perceiving and understanding the surrounding world is a critical challenge for autonomous robots. In conjunction with ROS World 2021, NVIDIA announced its latest efforts to deliver performant perception technologies to the ROS developer community. These initiatives will accelerate product development, improve product performance, and ultimately simplify the task of incorporating cutting-edge��
]]>
Deploying an autonomous robot to a new environment can be a tough proposition. How can you gain confidence that the robot��s perception capabilities are robust enough, so it performs safely and as planned? Trimble faced this challenge when it started building plans to deploy Boston Dynamics�� Spot in a variety of indoor settings and construction environments. Trimble needed to tune the machine��
]]>
Simulated or synthetic data generation is an important emerging trend in the development of AI tools. Classically, these datasets can be used to address low-data problems or edge-case scenarios that might now be present in available real-world datasets. Emerging applications for synthetic data include establishing model performance levels, quantifying the domain of applicability��
]]>
The long, cumbersome slog of data procurement has been slowing down innovation in AI, especially in computer vision, which relies on labeled images and video for training. But now you can jumpstart your machine learning process by quickly generating synthetic data using AI.Reverie. With the AI.Reverie synthetic data platform, you can create the exact training data that you need in a fraction��
]]>
Data plays a crucial role in creating intelligent applications. To create an efficient AI/ ML app, you must train machine learning models with high-quality, labeled datasets. Generating and labeling such data from scratch has been a critical bottleneck for enterprises. Many companies prefer a one-stop solution to support their AI/ML workflow from data generation, data labeling, model training/
]]>
Scene graphs (SGs) in both computer vision and computer graphics are an interpretable and structural representation of scenes. A scene graph summarizes entities in the scene and plausible relationships among them. SGs have applications in the fields of computer vision, robotics, autonomous vehicles, and so on. Current SG-generation techniques rely on the limited availability of expensive��
]]>