Building Your First Agent on Enterprise Data with Agent Bricks

Most first agents are demos: a notebook, a folder of PDFs and a model that answers well on the five questions the builder tried. Getting from there to something a team relies on is mostly not about the model. It is about the data, the evaluation and the operations around it.

This is the path we follow on Databricks. Agent Bricks automates parts of it; the shape stays the same whether you use it or build with the agent framework directly.

The shape of the system

1. Govern the sources first

Put source documents in a Unity Catalog volume, not in a personal workspace folder. That gives you access control, lineage and an audit trail from day one. If documents come from SharePoint or a file share, ingest them with a scheduled job, and record where each document came from and when it changed.

Ask the uncomfortable question early: who is allowed to see which documents? If the answer is "it depends", the agent must enforce the same rules. Retrieval that ignores permissions is a data leak with good manners.

2. Parse and chunk with intent

Turn documents into clean text with structure preserved: headings, tables and page numbers. Then split into chunks:

• Chunk by structure, such as sections or headings, rather than a fixed character count where you can.
• Keep chunks self-contained: a few hundred tokens, with the section title prepended, so a chunk makes sense on its own.
• Store metadata with every chunk: source path, title, section, last modified date and an access label.

Write the result to a Delta table. It is now just data: you can query it, test it and fix it.

3. Index with Vector Search

Create a Vector Search index with Delta sync on the chunk table. When documents change, the pipeline updates the table and the index follows. Use hybrid search (vector plus keyword) where exact terms matter, such as product codes, policy numbers and acronyms, because pure semantic search tends to blur them.

Filter on metadata at query time, for example by department or access label, so retrieval respects the permission model from step 1.

4. Build the agent

Whether you start from an Agent Bricks template or the agent framework, keep the first version narrow:

• One job, described in plain instructions: "answer questions about our HR policies, citing the policy and section".
• One retrieval tool over the index, returning chunks with their sources.
• A small number of action tools, exposed through MCP or Unity Catalog functions with scoped permissions. Read-only to start.
• Citations required in every answer, with a fallback: "I could not find this in the policies" is a correct answer.

5. Evaluate before anyone else uses it

This is the step that separates a demo from a product.

• Build an evaluation set of 50 to 200 real questions with expected answers or expected sources, written with the people who will use the agent.
• Score every version with MLflow evaluation and LLM judges for correctness, groundedness, relevance and safety, plus simple checks such as "did it cite a source".
• Agree a quality bar with the business owner before launch, and keep the set growing as real questions arrive.

Change one thing at a time (chunk size, retrieval settings, prompt, model) and compare scores. Without this, tuning is guesswork.

6. Ship it behind a gateway

• Model Serving hosts the agent as an endpoint.
• An AI gateway applies rate limits, guardrails, PII detection and usage tracking in one place for every agent you build.
• MLflow tracing records each request: retrieved chunks, tool calls, model output and latency, so you can explain any answer after the fact.
• A feedback button in the user interface feeds real failures back into the evaluation set.

7. Run it like software

Deploy the pipeline, index and agent with Asset Bundles through CI/CD. Monitor quality scores, latency and cost per request on a dashboard the owning team actually looks at. Review a sample of traces every week for the first month.

What to expect from a first project

A focused first agent, on one document collection with a clear owner, typically takes a few weeks to reach a measured, monitored pilot. Most of that time goes into data preparation and evaluation, and that is time well spent: every agent after the first reuses the ingestion, the index patterns, the gateway and the evaluation harness.