Metadata and the Future of USD Intelligence

The Challenge: LLMs Without Context

When you ask a generative Python model to “add a rigid body to a prim,” it may produce syntactically valid USD code — but that code often fails to align with your project’s USD hierarchy, material definitions, or joint structures.

LLMs lack the real-world context engineers design within. Without structured scene awareness, code generation becomes an unreliable guessing game. For AI to act as a dependable USD engineer, it needs structured context — a way to retrieve, reason, and verify knowledge with precision.

Introducing Metadata in ChatUSD

In ChatUSD, we use a technique called Retrieval-Augmented Generation (RAG) — a framework that helps large language models reason with factual, domain-specific data instead of relying only on their internal memory.

In practice, every piece of project information — from text fragments and code snippets to documentation paragraphs — is converted into a vector embedding, a mathematical representation of meaning, and enriched with metadata fields such as domain, product, or version.

Text fragments include small portions of manuals, simulation notes, or configuration guides that capture contextual rules. For example:

“All rigid bodies in Zone A must use PhysX material ID 3 with a friction coefficient of 0.6.”

“Each USD prim under /World/Assets/ must include a product_version metadata field.”

Code snippets are short examples of USD Python or Omniverse API usage that reveal how engineers actually structure assets and physics logic:

# Define a rigid body with material assignment
prim = stage.DefinePrim("/World/Robot/Base", "Xform")
prim.ApplyAPI(UsdPhysics.RigidBodyAPI)
prim.CreateAttribute("physics:material:friction", Sdf.ValueTypeNames.Float).Set(0.6)

‍

These embeddings are stored in a FAISS vector database, allowing ChatUSD to retrieve only the most relevant information before generating a response. This gives the model structured awareness of your USD environment — turning passive documents into active, queryable knowledge.

‍

domain       = plc
product      = XPlanar
version      = v1.2
source_type  = pdf

‍

These metadata constraints act as contextual filters, ensuring that ChatUSD retrieves knowledge tied to the relevant product line or simulation system, rather than pulling from unrelated PLC or motion-control data.

‍

The result is a domain-aware retrieval layer that dramatically reduces noise and ambiguity:

• It filters out irrelevant physics models or CAD examples.
• It resolves overlapping terminology (e.g., “mover,” “conveyor,” “carrier”) through domain context.
• It guarantees that the content feeding the LLM comes only from validated, versioned sources.

Once the top-ranked chunks are retrieved, they are dynamically injected into the LLM’s context window via ChatUSD’s RAG Modifier system, giving the model both semantic proximity and structural accuracy.
This is what enables ChatUSD to produce traceable, simulation-ready USD code — not generic answers.

‍

That’s how Champion’s RAG system converts unstructured engineering documentation into machine-interpretable design intelligence.

Metadata Implementation Example

When an engineer types a query such as “XPlanar mover safety”, ChatUSD doesn’t perform a naive keyword search.
Instead, the query is converted into the same type of vector representation as the stored documents.
The retriever then compares how similar these vectors are — using a distance measure called cosine similarity — and returns the most relevant pieces of information whose metadata matches filters such as:

for d in docs:
    d.metadata.update({
        "domain": "plc",
        "product": "XPlanar",
        "version": "v1.2-2025-09",
        "source_type": "pdf"
    })

‍

Using NVIDIA’s nv-embedqa-e5-v5 embedding model, ChatUSD encodes both technical text and code uniformly, allowing hybrid reasoning between human-readable instructions and executable logic.

Scalable Metadata Inference

Large industrial datasets are too dynamic to tag manually. Champion’s indexing framework includes a lightweight inference layer that auto-generates metadata from directory and file structures:

from pathlib import Path

for d in docs:
    path = Path(d.metadata.get("source", ""))
    d.metadata["domain"]  = path.parent.name
    d.metadata["product"] = "XPlanar" if "xplanar" in path.name.lower() else "Generic"

‍

A consistent hierarchy like this ensures that metadata mirrors the way engineering teams already organize CAD, PLC, and simulation assets:

client_docs/
 ├── plc/
 │   ├── plc_quickstart.pdf
 │   └── motion_reference.md
 ├── safety/
 │   ├── safety_spec.pdf
 │   └── maintenance.txt

‍

Metadata is no longer a manual afterthought — it’s an automated contract between file organization and AI understanding.

Metadata transforms Omniverse from a static design environment into a living data graph.
Each asset, material, and script becomes semantically indexable — and AI becomes a structured participant in the design process.

By embedding structured metadata at ingestion, ChatUSD gives Omniverse a memory system — one capable of reasoning, retrieving, and refining based on context.

This is not just AI generating code — it’s AI understanding why the code exists.

The best way to understand metadata-driven intelligence is to experience it live.ChatUSD enables engineers to query their own Omniverse environment using natural language, and receive traceable, context-grounded USD code drawn directly from their company’s knowledge base.