Traditional Retrieval-Augmented Generation (RAG) made Large Language Models (LLMs) useful for enterprise knowledge by giving them access to external documents at answer time. The usual loop is simple: encode the user question in an embedding vector, perform semantic search by retrieving the top matching chunks from a vector database, place those chunks in the prompt, and ask the model to answer. This works well when the answer lives in one obvious paragraph whose wording is close to the user's wording. However, it breaks down when the answer requires navigation across multiple layers of the knowledge base.
That limitation shows up often in technical sales and applications engineering in the semiconductor industry. A customer rarely asks in the exact language used by a datasheet, and the answer is often not on the first page that mentions the part number. It may live in a linked PDF, a product family specification, a technical appendix, an environmental compliance page, or a table that applies only after the part is classified correctly. Traditional RAG treats these as independent chunks. It does not naturally ask, "What document types did I land on?", "What document does this page link to?", or "Should I search inside that exact document next?"
Agentic RAG changes the retrieval workflow from a one-shot lookup into a controlled investigation. The recent survey on Agentic Retrieval-Augmented Generation describes this shift as adding planning, tool use, reflection, and adaptive control to RAG systems. Instead of retrieving once, the model can decide what to search, inspect intermediate evidence, narrow scope, and search again. Let's look at an example question based on TDK's public website to make the difference concrete. The question is: "Does RSEN-2030 contain PCBs?" A conventional vector search can find the RSEN-2030 product page, but the answer is not simply on that page. The useful evidence is on TDK-Lambda's environmental page about asbestos, PCBs, and mercury. The new agentic search flow first finds the RSEN-2030 page, then follows the environmental reference, then searches that exact URL. The final answer correctly states that RSEN-2030 is a production product and that production products in the latest catalog are listed as "Non-contained" for PCBs, with references to both the product page and the environmental page.
Our Approach to Agentic RAG
Knowledge Structure at ept AI
At ept AI, we manage knowledge as configured, governed sources that can be assembled for different channels, teams, and customers. A Knowledge Source can be a website, PDF library, community forum, GitHub repository, documentation, Zendesk or HubSpot knowledge base, or anonymized support tickets. A Knowledge Source Configuration combines those sources into the exact body of knowledge a channel is allowed to use.
Our Agentic RAG Tool
Fig. 1: AI workflow using Traditional RAG vs Agentic RAG. Unlike Traditional RAG, in Agentic RAG the LLM has the ability to do multiple loops of knowledge retrieval where it decides what the query is, and how the search space is constrained.
Our agentic knowledge retrieval tool for LLMs is built around that knowledge base model. The tool description tells the LLM which Knowledge Sources exist in the active Knowledge Source Configuration, including each source name, ID, type, description, and source URL. The LLM does not search an anonymous pile of chunks. It chooses which Knowledge Source to search and what to search (see Fig. 1). This capability allows users to guide LLMs how to look for information for a given task like customer support, e.g.: "first look in historical tickets, then in private documentation and finally the website before getting back to the user." Another benefit of this approach is that the LLM is in control of what queries get used for the search and can do repeated searches to optimize the search query. This turns many "I don't have enough information" answers to actual answers grounded in data. Yet, even within a Knowledge Source, the information can be too scattered and result in inaccuracies with vector search.
Fig. 2: Query search constrained to two subsets of data of two Knowledge Sources. The colored overlayed boxes represent tags applied to a set of documents. One document can have many tags (overlapping colored boxes). The dotted boxes are the constrained search space chosen by the LLM.
This shift to agentic knowledge retrieval is not a new phenomenon. With the introduction of GPT-5, tool use has become much more reliable with LLMs (see GPT-5 performance on Toolathon). What we found at ept AI is that for our customers, often large companies with a large portfolio of technical products, e.g. semiconductors, require a more granular search into their data. In addition to Knowledge Source selection, we added the capability for our system to tag data with industry-and-company-specific taxonomy and allow the LLM to constrain its search space to any subset of the tagged data (see Fig. 2). This matters most in industries where the source type changes the authority of the answer. In semiconductors and electronics, a casual webpage, a marketing catalog, a selector guide, a compliance page, and a datasheet do not carry the same weight. For engineering decisions, the answer often needs to be grounded in official datasheets, family specifications, or technical information PDFs. Those documents are the sources teams treat as binding for part selection, operating limits, storage conditions, regulatory claims, and more. Agentic RAG lets us narrow toward those industry-specific, source-specific records instead of relying on whichever chunk happens to be semantically closest.
Consider a troubleshooting question about Synaptics Astra documentation: "Playback audio doesn't match the behavior defined in the reference voice-assistant example." In this case, we have instructions in the system prompt to always search the "latest version of documentation" for that specific Knowledge Source. The LLM, having the full visibility of Knowledge Source tags, does not spread across mixed documentation branches or generic product pages. Instead, it stays entirely inside one Astra documentation version, 1.6.1 (the latest version at the time). The retrieved evidence includes the Astra Linux docs at synaptics-astra.github.io/doc/v/1.6.1/linux/index.html and all retrieved documentation URLs for this response were from the same 1.6.1 branch, which is exactly the behavior we want for version-sensitive technical questions: once the version context is present, the search stays pinned to one documentation lineage instead of blending evidence across releases.
Putting the user in control is equally important. Users are now able to express retrieval intent in normal language: "search the public website," "look inside this datasheet," "use the product page first," or "only use official docs". Under the hood, that maps to Knowledge Source Configurations, Knowledge Source IDs, queries, and tags. This gives the user a practical navigation layer over complex knowledge sources. We have already seen customers use it internally to perform documentation discovery and summary to improve their knowledge base.
Performance Comparison
We test the new Agentic RAG method in two ways: quantitatively on a set of 100 single-fact questions, and qualitatively, by inspecting traces on selected questions where the answer requires navigating across product pages, documentation versions, and linked PDFs. The point of the evaluation is not simply, "did the model answer?" It is, "did the model find the right evidence path, stay inside the right knowledge boundary, and give the user a source they can trust?"
Fig. 3: Quantitative Results. Panel A: bar chart comparing faithfulness for Traditional vs. Agentic RAG, 80% to 85%. Panel B: bar chart comparing average high-relevance retrievals per response, 4.37 to 6.
Our first test (Fig. 3) compares the new method to the old one based on 100 single-fact short technical questions (based on real production traffic). For the previous method, these questions are not particularly challenging and are mostly answered correctly. The goal of the test is to verify that the new method matches the correctness of the old method since now we have an LLM driving decisions for the knowledge retrieval. Indeed, we see that the new method produces answers that are similar to the old answer in 96% of the time. In addition, we also see that new LLM response's faithfulness improves from 80% to 85%, meaning the LLM's answer is more grounded in data than the previous method. The previous method returns, on average, 4.37 retrievals with a high relevance score to the question whereas the new method returns 6 high relevant retrievals per response. Relevance here is measured using an LLM-as-a-judge. Across the same 100 questions, traditional RAG returns 1,534 total retrievals, or almost 16 per response, while the new workflow returns 3,160. That is more retrieval work, but it is not just noise: the number of highly relevant retrievals increases. For technical support and applications engineering, that matters because the final answer is only useful if it is backed by enough precise evidence to be reviewed, cited, or escalated. In practice, we care less about the raw number of retrieved chunks and more about the shape of the evidence. A single loosely related chunk can produce a fluent but fragile answer. A larger set of high-relevance retrievals gives the model a better chance to compare sources, catch missing constraints, and cite the right document. The observed result is exactly the behavior we want: more focused evidence, higher faithfulness, and preserved answer correctness.
For the qualitative evaluation, we inspect 3 different questions on public websites where we have previously seen that the previous method fail. Consider a question about whether B84143A0008R105 (a TDK Power Line EMC Filters) can be used at 3000m altitude. A plain semantic match can find the part number page and the datasheet, but the governing detail lives in technical information about altitude derating. The agent searches broadly, finds the relevant datasheet family and technical-information PDF, then narrows retrieval to those documents. The final answer does not stop at "usable" or "not usable." It explains that standard conditions apply up to 2000m and that 3000m requires derating, giving the current derating factor of 0.925 and voltage derating factor of 0.880. That changes the response from a lookup into an engineering answer. The previous method found the part page, but it does not navigate to the technical-information PDF where altitude derating is defined, so it misses the 2000m baseline and the required 3000m derating factors.
Another example involves storage conditions for NTCG103JF103FT1, a Chip NTC Thermistor (Sensor). The product page itself is not enough. The useful information is in the NTCG series catalog linked from that product page. With the new method, the LLM first retrieves the product page, then constrains the search to the catalog PDF, and answers with the storage temperature, humidity, and handling notes from that document. This is a common pattern in component documentation: the SKU page identifies the part, but the binding storage, handling, or reliability rule is often in the family-level catalog. The previous method only reaches the product page.
Version-sensitive documentation is another place where a constrained search space matters. In a Synaptics Astra question explicitly framed around ASTRA v1.5.0 and third-party software license information, the retrieved Astra documentation stays on the 1.5.0 branch, including the EULA, v1.5.0 release notes, and a software update user guide. The retrieval set also includes a small amount of non-versioned support material, but it does not mix Astra docs from latest, main, or other versions. The search space constraint is enabled by tagging the data and giving the LLM the visibility and ability to use these tags: the model recognizes that version is part of the retrieval intent and avoids blending documentation across releases, which otherwise would cause the model to hallucinate and answer based on outdated documentation.
Challenges and Limitations
While Agentic RAG as described above results in higher accuracy, less hallucination, and better knowledge navigation, it comes with a few challenges. One of the main challenges is prompting: guiding the LLM on how to use the new tool(s). If the prompt is too permissive, the agent may over-search, spend too many tool calls, and return slowly. If the prompt is too restrictive, it falls back into one-shot Traditional RAG behavior and misses linked evidence. The best balance is to set clear goals for the LLM and make every tool orthogonal and explicit: one tool should have a clear job, clear inputs, and clear guidance for when to use it. While our agentic knowledge retrieval tool offers a sophisticated way for LLMs to navigate technical knowledge, the orthogonality question becomes especially challenging when multiple search tools are available, such as internet search or product catalog lookup. Accurate and clear tool descriptions help the LLM avoid confusion about what each tool does and when to use it.
Traceability is the second challenge. Agentic systems are harder to evaluate because the final answer is only part of the behavior. We also need to inspect the path: which tools were called, which URLs were discovered, which tag filters were applied, and whether the answer comes from the right authority. Our in-house traces already record tool calls, queries, retrieved URLs, outputs, and more. That makes it possible to let us tune prompts without guessing and also allows us to inspect latency and improve upon the speed of the agent.
Conclusion
Agentic retrieval represents the third phase of applying LLMs to question answering. The first phase was Traditional RAG, in which a single LLM call is applied to retrieved knowledge chunks. The second phase was strong orchestration, where multiple LLM calls are deployed, but in a highly structured workflow. Today, two factors enable the Agentic RAG breakthrough. First, LLMs and their associated prompting methods have become reliable enough to trust the LLM to make its own decisions about what tools to call, what knowledge to pursue, and when it is ready to compose a final answer. Second, the associated infrastructure of knowledge organization, retrieval methods, and performance management have evolved to provide LLMs with effective tools and methodologies. These become especially important in specialized knowledge domains like engineering, sales, and support of technically sophisticated products. This post illuminates some of the methods we have found to be effective at ept AI.
