Beckhoff Integration with NVIDIA's USD Agent SDK

🧩 User-First Design: One-Click RAG Updates

Before diving into the technical implementation, we solved a key usability challenge — enabling non-technical users to expand the AI’s knowledge base without code.

The solution was the Drag-and-Drop RAG Import Widget, built directly into the Chat USD interface. With it, engineers can update Beckhoff documentation inside Omniverse in under a minute.

How it works:

• 🗂 Open the RAG Import section in the chat window
• 📄 Select a category — Specifications, Manuals, Product Lines, or CAD Models
• 📥 Drag and drop markdown files (or paste a file path)
• ⚡ Click Import Document — the file is processed and instantly added to the AI knowledge base

This workflow simplifies data management and keeps the assistant aligned with the latest Beckhoff product information.

‍

🔍 RAG System Implementation

To make this possible, we developed a Retrieval-Augmented Generation (RAG) pipeline optimized for Beckhoff’s technical documentation. The system indexes and retrieves domain-specific data for precise, low-latency responses.

Indexing Architecture

A custom ingestion script (beckhoff_markdown_ingestion.py) extracts metadata such as part numbers, model codes, and specifications. The content is segmented using a character-based splitter tuned for technical writing to preserve hierarchy and context.

Configuration highlights:

• 📏 Chunk size: 1,200 characters
• 🔁 Overlap: 300 characters
• 🧩 Custom separators to retain markdown structure
• 📚 Processed: 12 documents → 178 structured text chunks (≈147K characters total)

Vector Store & Retrieval

• Embeddings generated using NVIDIA’s nv-embedqa-e5-v5 model via AI Endpoints
• Indexed with FAISS (Facebook AI Similarity Search) for retrieval under 500 ms
• Registered as a retriever named beckhoff_docs in the USD Agent core module
• Default retrieval: top-k = 15, ensuring optimal context precision

This architecture allows the system to retrieve relevant technical data in milliseconds — directly from Beckhoff documentation, not the broader USD knowledge base.

‍

💬 Custom Chat USD Model — Beckhoff-Only Mode

We developed a dedicated Chat USD model called “Beckhoff Chat USD”, which operates entirely on Beckhoff data within Omniverse.

Unlike the standard Chat USD agent, this version removes connections to general USD search modules and relies solely on the Beckhoff RAG retriever. This ensures every response is based exclusively on verified Beckhoff documentation — delivering brand-accurate and context-specific results for automation engineers.

Key improvements:

• 🚫 Routing: Disabled generic USD Search nodes to prevent unrelated data retrieval
• 🎯 Retrieval: Uses only the Beckhoff RAG retriever (top-k = 15, max tokens = 5000)
• 🧠 Metadata: Updated descriptions and examples to align with Beckhoff systems
• 🔒 Isolation: Ensures 100 % Beckhoff-specific output — no generic USD context

Why we did this:
Industrial automation data is highly specialized. By isolating the model, we ensured engineers receive responses grounded in Beckhoff’s ecosystem — accurate, reproducible, and consistent across sessions.

‍

🧠 Beckhoff-Optimized Extension Configuration

To reinforce model isolation, we created a Beckhoff-specific configuration in extension.toml. This ensures the system retrieves data solely from the Beckhoff retriever while disabling other knowledge sources.

Configuration highlights:

• beckhoff_only_mode = true
• primary_retriever = "beckhoff_docs"
• beckhoff_priority = 1.0
• other_retrievers_priority = 0.0
• Similarity threshold: 0.7
• Context window: 128k tokens (Llama 4 Maverick 17B)

This approach improves consistency, reduces noise, and optimizes retrieval latency — ensuring every query remains focused on Beckhoff’s data structure and terminology.

‍

Metafunction for Automated XPlanar Layouts

To extend the system’s practical value, we added a custom metafunction that automates XPlanar tile placement using natural language input.

Example:

“Create a 10×10 grid of XPlanar tiles with 2 cm gaps.”

The function place_xplanar_tiles() supports multiple layout patterns — grid, checkerboard, U-shape, L-shape, corridor, and others — and three instancing modes (duplicate, reference, and point instancer). It automatically generates parameterized layouts with validation, optimizing scene setup for simulation and visualization.

This enables rapid scene prototyping and interactive planning of XPlanar systems without manual modeling.

‍

Performance Characteristics

| ⚙️ **Metric** | 📊 **Value** |
|----------------|---------------|
| ⚡ **Retrieval Latency** | <500 ms per query |
| 💾 **Index Size** | ~2–3 MB (178 chunks) |
| 🧮 **Embedding Dimensionality** | 768 (NV-Embed-QA) |
| 🧠 **LLM Context Window** | 128 k tokens (Llama 4 Maverick) |
| 🔍 **Top-K Retrieval** | 15 chunks (≈7,500 characters avg.) |
| 📈 **Similarity Threshold** | 0.7 (cosine similarity) |
| 🧩 **Metafunction Retrieval** | 3,114 indexed functions |

🚀 Results

The integration delivers:

• 100 % Beckhoff-specific responses with no USD knowledge leakage
• Sub-second response time across all document queries
• Instant knowledge updates via drag-and-drop import
• Natural-language scene generation for XPlanar layouts
• Simple model switching between Beckhoff-only and general USD modes

‍

🔭 Future Enhancements

• Multi-modal retrieval (PDF documents with image extraction)

• Advanced query routing with intent classification for EtherCAT, TwinCAT, and XPlanar domains

• Retrieval quality monitoring with user feedback loops

• Additional metafunctions for TwinCAT PLC code generation

• Multi-language support for international Beckhoff documentation