From Dashboards to Agents, is your data ready for the change?

In my previous post, Agents Are Here. What Will Change for Analysts?, I explored how AI agents will change the analyst’s role. One part of that transition deserves a closer look: the data we have spent years preparing for BI may need a different approach when agents become its consumers.

We have been optimizing for a different consumer

For years, we have designed data models around dashboards and reports. Developers decide which metrics matter, which dimensions users can explore, and which filters they need. The model supports a curated experience in which users interact with information that has already been organized for them.

AI agents change that relationship. Instead of selecting from predefined visuals, users can ask questions in natural language, and the agent must interpret the request, identify the right metrics and attributes, construct a query, and explain the result. The agent is no longer simply presenting information that someone else selected. It is making decisions about how to retrieve and interpret the data.

That difference matters because a model that works well for a dashboard may not be equally effective for an agent. A developer can understand the purpose of a field from experience, documentation, or conversations with stakeholders. An agent must rely on the context and structure available to it, and ambiguity can lead to inconsistent answers.

More data can mean more opportunities to get it wrong

When building BI models, we often try to make them flexible enough to support many business questions. We add dimensions, measures, date fields, and relationships so developers can build different reports without repeatedly requesting new datasets. That flexibility is valuable, but it can become a liability when an agent has to decide which elements to use.

Imagine a sales model containing several date fields: order date, shipment date, invoice date, and payment date. A manager asks, “How did sales perform last quarter?” Which date should the agent use? The answer depends on the business definition of sales performance, not simply on which field is easiest to query.

The same problem appears when similar metrics exist with slightly different definitions or when business rules are embedded in SQL queries and notebooks rather than clearly expressed in the model. Even when documentation exists, it may be outdated or incomplete.

An agent might generate valid SQL and return a plausible number while answering a different question from the one the manager intended. That is more concerning than variations in the SQL itself. Different queries can be perfectly acceptable if they produce equivalent results under the same business definition. What we need to prevent is inconsistency in the meaning of the answer.

Perhaps we need smaller datasets, not bigger models

One response is to keep adding descriptions and instructions to an increasingly complex semantic model. Better metadata and clear definitions certainly help, but they cannot always compensate for a model that exposes too many competing options for a specific task.

Sometimes, the better approach is to narrow the problem. An agent responsible for sales performance may not need access to every customer service, marketing, and finance attribute in the organization. A purpose-built dataset containing the relevant measures, dimensions, and business rules can reduce ambiguity and make it easier to evaluate the agent’s answers.

For some use cases, this may mean creating a denormalized table that brings related information together. Instead of asking the agent to navigate several tables and interpret complex relationships, we provide a carefully designed dataset that directly supports the questions it is expected to answer.

This is not a recommendation to replace every dimensional model with a single table. Denormalization can introduce duplication, incorrect aggregations, and grain problems if it is not designed carefully. The goal is not to make the data model simpler at any cost; it is to make the intended business interpretation clearer.

Another option is to divide responsibilities across specialized agents. One might focus on sales performance, another on customer retention, and another on marketing effectiveness. Each would have a narrower scope and access to the data needed for its responsibilities.

Of course, specialization introduces its own challenges. Someone must maintain the datasets, keep definitions consistent, and determine which agent should handle each question. But for some business applications, this trade-off may be worthwhile if it produces more reliable answers.

There is no universal architecture. Some existing BI models will be suitable with better descriptions and a few targeted changes. Others may need dedicated datasets, revised relationships, or a more substantial redesign. The right decision depends on the questions the agent must answer and the level of reliability the business expects.

A new question for analysts

This creates an opportunity for analysts to contribute beyond defining requirements and validating dashboards. We can help determine whether the data behind our existing BI solutions is ready for agents.

That starts with understanding how managers ask questions, what decisions they need to make, and which business definitions must remain consistent. It also means identifying where rules are hidden in SQL or notebooks, where documentation no longer reflects reality, and where different teams interpret the same metric differently.

Analysts do not need to solve every engineering challenge themselves. Data engineers can help design and build the underlying datasets, while business stakeholders clarify definitions and expectations. Analysts can connect those perspectives, translate business context into explicit requirements, and test whether the agent’s answers reflect the intended meaning.

Validation becomes especially important. A successful demonstration is not enough. We need representative questions, expected results, and tests that reveal when an agent selects the wrong date, applies an incorrect definition, or produces a misleading explanation. The objective is not to prove that an agent can generate a query. It is to establish that people can trust the answer enough to use it.

This is also an extension of a principle I have written about before: understanding your users. The same dashboard can serve different people in different ways, and an agent is no exception. Knowing what a manager means by a question, what context matters, and what decision follows from the answer is essential to designing a useful solution.

As agents become another way for people to access business information, we should stop assuming that a dataset is ready simply because it already supports our dashboards. We need to evaluate it against a different consumer, different interactions, and a different standard of reliability.

The question is no longer just whether our data model supports reporting. It is whether it helps an agent reach the right business interpretation consistently.

From Dashboards to Agents, is your data ready for the change?

Start with Data, End with Value.