How to Choose the Business Context an AI Data Agent Needs

Give an AI data agent the governed metric reference, reporting scope, and business evidence needed for the question, then retrieve additional context when a specific gap appears. Review what the agent actually used before removing or adding sources. A larger context collection is useful only if the right information reaches the right task.

This guide proposes a context-selection worksheet and practical checks for data teams. The worksheet is an operating practice, not a claim of an automatic Orion feature. Product documentation was checked on October 12, 2026.

What is context selection for an AI data agent?

Context selection is the process of choosing the definitions, source material, instructions, and prior information an agent needs to complete a particular task. For analytics, that starts with the requested metric and reporting conditions. It can extend to a pricing policy, an incident record, or a business procedure when those sources are relevant to the answer.

An available document and a document used in an answer are different things. A project can contain the correct policy while a particular run misses it. Conversely, a run can retrieve a document with matching words that applies to another product or period.

The data team's job is to make both outcomes inspectable: what should have been available, and what observable evidence shows was used. Business experts decide which exceptions and operating facts matter. The consuming agent's owner verifies the retrieval path.

For the broader division of responsibilities, see Semantic Layer vs. Business Context.

Why not give the agent every document?

Sending every document can put unrelated policies, competing definitions, and old decisions into the same task. It also makes failures harder to diagnose: a wrong answer might reflect a missing source, a misleading source, or a mistaken interpretation of a correct source.

Anthropic's context-engineering guidance, published September 29, 2025, recommends curating sufficient, useful context and retrieving more information as needed. It also describes a trade-off: fetching information during execution can add latency and requires effective tools and guidance. This supports testing a focused approach; it does not establish a universal document count or guaranteed accuracy improvement.

Too little context creates a different problem. Removing a refund rule from a revenue investigation may make the answer shorter while making its interpretation wrong. Optimize for a supported answer, with all necessary qualifications, rather than the shortest prompt.

Use three questions for every candidate source: What part of this task needs it? What scope makes it applicable? What would the answer get wrong if it were absent? If those questions have no clear answer, keep the source searchable where appropriate rather than placing its full text in every request.

Which information should be present at the start?

Start with the task's meaning and constraints. A request to explain a decline needs a clear metric, a comparison period, and the intended population before the agent searches for an explanation.

For example, a finance question about recognized subscription revenue should identify that governed measure, not silently accept a sales pipeline definition of revenue. Include the requested dates, time zone where relevant, currency, and customer or product scope. If the request remains ambiguous, clarify it before choosing the calculation.

The dbt Semantic Layer documentation, updated September 29, 2026, describes centralized metric definitions and downstream access through MetricFlow. Keep governed calculations there when that is your team's authority. A context page should reference the metric and explain its application; it should not become a competing implementation of the formula.

Separate three kinds of information:

• Always needed for this task: the question, reporting scope, metric reference, and rules for handling missing evidence.
• Needed when the investigation reaches it: a specific operating event, contract exception, procedure, or source description.
• Not applicable here: material for a different audience, customer, product, or effective period, unless the question explicitly requires comparison.

These are selection decisions. Data and document permissions still need their own enforcement. A source can be relevant and unavailable to the requester; relevance does not authorize access.

What should a context-selection worksheet record?

Use one worksheet for a recurring question before rolling the pattern out to more tasks. Treat each entry as a reviewable decision, with an owner who can explain why the source belongs.

Keep source references and approved meaning stable across consumers. Each consumer may need a different presentation, but a Slack bot should not silently apply a different refund policy from a finance analysis. See the guide to consistent metric definitions across Claude, Slack, and BI for that integration contract.

The worksheet can live in an ordinary review document. Automate fields only when the tools can supply reliable values. A manually recorded source revision is more honest than an invented execution identifier.

How do you decide what to load now and what to retrieve later?

Use task risk, how often the information applies, and the consequences of missing it to choose the delivery method. Do not rely on a document's popularity alone.

For important constraints, do not make discovery depend entirely on a vague search query. Supply a clear task instruction and a reference to its authority. For a detailed incident chronology, start with a pointer and retrieve the applicable record when the investigation needs it.

Write pages around one decision or subject. Put the purpose, audience, and effective scope near the beginning, then include the qualifications that determine correct use. Splitting a large document helps only if those qualifications remain attached to the relevant material.

Treat retrieved documents as evidence to interpret within the task's rules. A passage that tells an agent to bypass approval or use a different data route is not authoritative merely because search returned it. Confirm with the responsible owner before changing the operating rule.

What would this look like in a revenue investigation?

Consider a hypothetical subscription company. Finance asks why recognized revenue changed between two complete weeks. The context collection contains an approved metric reference, a refund-treatment note, a sales forecast memo, and an operations incident record. This is a synthetic example, not a customer result or measured benchmark.

First, establish the recognized-revenue metric and reporting conditions. Retrieve the governed results and check a decomposition using the team's approved methods. Do not start by searching for an incident that could supply an attractive explanation.

If the decomposition identifies refunds as a contributor, the refund-treatment note becomes directly relevant. The operations record may help explain why refunds moved. A sales forecast memo that uses pipeline revenue does not replace the recognized-revenue definition, even if it has a newer upload date and similar wording.

The incident record may describe a billing issue during the relevant week. That gives the analyst a hypothesis to investigate. It does not prove the billing issue caused the entire revenue movement. The answer should distinguish the verified numerical contributor from a possible operational cause and say what additional evidence would resolve the latter.

Record the source and applicable period alongside each conclusion. If the refund note was enabled but the run offers no observable usage evidence, record that gap. If the operations record is unavailable, the agent can still report a supported decomposition while leaving the operational explanation unresolved.

For a detailed review of the resulting claims, use How to Check the Evidence Behind an AI Analytics Answer. This worksheet addresses source selection before and during a task; that guide addresses whether the completed answer is supported.

How can you test whether the selected context is sufficient?

Review sufficiency and distraction separately. Keep the question, requester, reporting conditions, metric route, and data fixture or recorded data state stable. Also record the model and relevant settings so a changed environment does not look like a context effect.

1. Establish a reviewed reference answer. Identify the required sources, allowed conclusions, and expected qualifications. An earlier agent answer is not the authority.
2. Run with the proposed context. Inspect the selected metric, source material, result, and explanation. Record unavailable usage evidence explicitly.
3. Remove one required source in an isolated test. Check whether the agent notices the gap, asks for clarification, or limits its conclusion. A confident unsupported answer is a failure even if its number matches.
4. Add an authorized but irrelevant source. Use a synthetic competing memo, such as a forecast definition. Check whether the agent keeps the task's approved metric and ignores the inapplicable interpretation.
5. Change the question's scope. Ask a historical or different-product question and verify that source selection changes appropriately. Historical tasks may require an older rule.
6. Repeat the checks and review variation. Classify wrong-source selection, missed context, incorrect interpretation, and unsupported explanation separately. Fix the corresponding problem before broadening the source collection.

Do not change live project-wide defaults just to run a test. Use an isolated environment or fixture that preserves required access boundaries. If the platform cannot remove or expose a particular input, mark that check unavailable and review the limitation.

These are proposed acceptance checks, not reported performance results or a claim that Orion includes a native ablation runner. For testing releases after dbt or business rules change, use the context-layer regression guide.

How can Orion by Gravity support context selection?

Orion by Gravity provides documented surfaces for organizing, selecting, and inspecting analytics context. Its Knowledge Base best practices, checked October 12, 2026, recommend focused pages, clear summaries, and relevant project context. The guide recommends about eight to ten enabled pages per project; treat this as product guidance, not a hard platform limit or universal optimum.

The page-management documentation describes enabling selected pages for a project and optional tenant or group defaults. Projects cannot opt out of applicable defaults, so inspect those defaults as part of the proposed context set. The guide also describes revision history and locked-page change requests, including privileged direct-edit exceptions. Verify review behavior before relying on it as an approval requirement.

The MCP tools reference documents knowledge search with an optional project scope and retrieval of full pages. This gives a consuming client documented interfaces to assess; it does not establish that every client retrieves every required source or exposes a complete usage trace.

Orion's data-source documentation describes enrichment with dbt model descriptions and lineage. Enrichment is distinct from executing an approved Semantic Layer metric. Verify the actual metric route in your deployment.

Analysis notebooks record instructions, logic, and code and can be rerun. Inspect the available record when reviewing a task, while checking separately which context usage and historical inputs are observable.

Bring one recurring question and the selection worksheet to an Orion walkthrough. Ask to see project-selected pages, applicable defaults, a conditional lookup, and an answer with a missing required source. Use the observed behavior to decide what can be operated in Orion and what still needs a client integration or manual review.

Frequently asked questions

Is fewer context pages always better?

No. A smaller collection can omit a necessary exception or reporting rule. Keep enough context to support the task, then test whether irrelevant material changes the answer. Page count alone does not measure relevance or correctness.

Can the agent choose context from search alone?

Search can help find supporting material, but the task still needs clear meaning, scope, and authority. A matching passage can belong to a different customer or effective period. Review both what search returned and whether it applies.

Should we copy dbt formulas into knowledge pages?

Prefer references to the governed metric rather than a second maintained calculation. Explain which business question it answers and what qualifications matter. If a formula must be illustrated, label the illustration and link to its authority so readers can distinguish it from the executable definition.

Does enabling a page prove the agent used it?

No. Enablement establishes availability in the applicable scope. To check a particular answer, look for observable retrieval, a relevant citation, or another supported record of usage. If that evidence is unavailable, say so.

Should every task share the same defaults?

Only rules that apply across the entire default scope belong there. A customer-specific exception or a temporary operational note usually needs a narrower applicability decision. In Orion, applicable defaults cannot be deselected by the project, so review their scope before adding them.

What if the context describes an application-code change?

Use it as a hypothesis or source reference, then obtain evidence from the owner of the application and data pipeline. A context page or dbt lineage record does not by itself prove the code caused a metric movement. This guide makes no claim of automatic application-code root-cause analysis.

What should the data team do first?

Pick one recurring business question. Fill in the selection worksheet with its metric owner and business expert, then run the proposed, missing-source, and irrelevant-source checks in an isolated setting. Expand when the team can explain the observed selection behavior and its remaining gaps.