Can You Trust a ChatGPT Work Data Agent Dashboard?

An analytics assistant can respect a user’s source permissions and still produce a result that is wrong, ambiguous, misleading, or unsafe to share. Those are separate questions. A permission check asks whether an account can read a record. A correctness check asks whether the metric, filters, calculation, and explanation are right. A publication check asks what data the resulting dashboard contains and who will receive it. Treating all three as “the tool is secure” hides the work that still needs an owner.

OpenAI’s current Help Center says Data queries use the connected account’s existing permissions, including applicable table, row, and column restrictions. The same guidance tells users to check the source, time period, filters, metric definition, and evidence, and warns that publishing a Site can copy the data used in an analysis into the published site. That documentation gives a useful set of boundaries. It does not report that a particular workspace’s results, permissions, or sharing setup have been tested. This is a review framework, not a claim that I ran a Data agent trial. (OpenAI’s current Data plugin guide; September 10 launch announcement)

View image detail

A reviewer checks an analysis and its source, then passes through a separate access and sharing gate.

Actual size

Choose Actual size to read the graphic closely.

A permission boundary is not a correctness stamp

The September 10 launch announcement describes a Data agent in ChatGPT Work that connects to approved company data, investigates changes, and builds interactive dashboards. The current Help page describes the Data plugin in ChatGPT Work and Codex, with setup requirements and operational guidance. OpenAI says queries use the connected account’s existing permissions. That is a meaningful design boundary: the product is not supposed to invent privileges the account lacks. But the boundary answers only one part of a larger decision.

Think of a spreadsheet analyst with permission to read the revenue table. The person can still pick the wrong table, apply the wrong quarter, include refunds when the report excludes them, divide by the wrong number of accounts, or describe a seasonal change as a campaign effect. Source access and analytical validity are related, but one does not establish the other. The same distinction applies to an AI-produced analysis. A successful connection shows that the workflow can reach some source; it does not show that the output answers the intended question.

This is not a special indictment of Data. Any analyst, dashboard, SQL query, or automated report needs a definition of correct for its intended decision. OpenAI’s documentation itself lists source, period, filters, and metric definition as things to check. Rise’s interpretation is simple: “connected” is a system state; “correct enough to rely on” is a review decision. Record them separately.

Four different checks happen before reliance

Before relying on a result, separate four checks. Authorization asks whether the connected identity could read the intended source. Analytical correctness asks whether the source, definition, period, filters, and calculation match the question. Presentation fidelity asks whether the labels, scales, comparisons, and narrative represent the result accurately. Destination suitability asks whether this dashboard, audience, or proposed action is appropriate for the information.

These checks involve different owners and evidence. An administrator may configure plugin availability and group access. A source owner can explain data permissions and definitions. An analyst can reconcile totals and investigate assumptions. A dashboard owner can review labels and audience. The person approving an action should verify its destination and payload. One person may carry several roles, but the approval does not become automatic merely because the product completed the prior step.

For a parallel example from a different connected-agent setup, read how Notion Agent MCP connections shape access and approval. It covers a different product, but the distinction between an available connection and an approved action is relevant here too.

The order also matters. It makes little sense to polish a chart if its source is wrong. It makes little sense to publish a dashboard before deciding who should see it. And a correct dashboard does not approve an email or a change to another system. The launch describes sharing findings and carrying out approved actions through connected tools; current Help says actual capabilities depend on the connected tool, its support, permissions, workspace rules, and any required approval. Treat those as separate gates instead of assuming the same behavior everywhere.

View image detail

Authorization, analytical correctness, chart fidelity, and destination suitability are separate review stages.

Actual size

Choose Actual size to read the graphic closely.

Trace the evidence behind one finding

Take one material finding and work backward. Identify the source dataset or report, the connected identity, the time range, filters, calculation, joins, and metric definition. Confirm that each detail matches the question. If the answer says a rate fell, ask for the numerator and denominator. If it describes a segment as “high risk,” ask how the segment was defined and which observations qualify. If it attributes a change to a cause, determine whether the evidence actually supports causation or only shows two things moving together.

The current Help Center encourages users to ask follow-up questions, inspect the source, period, filters, and definition, and compare against another report if the result differs. Use that guidance to turn a plausible story into a traceable claim. “The analysis is based on the support tickets table” is not enough if the result depends on whether reopened tickets count once or twice. “This comes from the canonical churn metric” is not enough if the answer cannot identify which version or period of the metric it used.

For every important sentence in a dashboard, the reviewer should be able to say what evidence would make it true. A number may be independently reproducible. A causal interpretation may require a controlled experiment, a subject-matter review, or additional context. A recommendation may be an opinion based on an assumption. Preserve those distinctions in the dashboard note or accompanying explanation. Do not allow confident prose to collapse observations, explanations, and recommendations into the same status.

View image detail

A finding can be traced from a sentence through its calculation, filters, metric, and source.

Actual size

Choose Actual size to read the graphic closely.

Reconcile like with like

When Data’s answer differs from a trusted report, compare the conditions before deciding which is wrong. Verify both source versions and refresh times. Align start and end dates, timezone, filters, cohort definitions, metric owner, currency or unit, treatment of nulls, and inclusion rules. Confirm whether one system has later-arriving data, corrected transactions, changed joins, or a restated period. If the dashboards are answering slightly different questions, matching them may be the wrong goal.

The temptation is to look at two totals and ask the assistant to “fix” one. Instead, write down the exact comparison. If one view reports net revenue and another reports booked revenue, a difference might be expected. If one uses month-to-date local time and another closes on UTC, a date boundary may account for a discrepancy. If one includes internal accounts, the denominator may differ. These explanations should be demonstrated from the definitions or data, not selected because they make a chart look tidy.

OpenAI says Data can use business terms, metric definitions, calculations, relationships, semantic layers, and trusted dashboards. The presence of those materials is useful context; it does not remove the need to confirm which definition was used. A semantic model can be incomplete, outdated, duplicated, or owned by another group. Ask the data owner to point to the approved object and review its scope. If two teams intentionally maintain different metrics, do not invent a single universal answer. State which version supports this decision.

View image detail

A comparison is fair only when each report uses the same period, definition, filters, and unit.

Actual size

Choose Actual size to read the graphic closely.

Test the access boundary with identities, not assumptions

OpenAI states that Data respects the connected account’s existing access, including applicable table-, row-, and column-level restrictions. The documentation is useful for understanding the intended behavior, but it is not a test report for your workspace. A reviewer who needs evidence should use the identities and rules that actually exist in that environment, following the organization’s approved test process.

One proposed check is a positive case and a negative case. The positive case asks an authorized user for a record or aggregate that the role should see. The negative case uses a restricted role and asks for information it should not access. The owner should check both direct records and derived results where the relevant policy requires it. Do not place sensitive data into a test environment that is not approved for that data. Do not loosen access just to see whether the agent is capable of returning something.

The result of this test is not proven by a plugin being installed or by one user successfully connecting a source. Installation controls whether a tool is available. The source provider controls what the connected identity can read. An app may require its own administrator setup or account connection. Current Help explicitly warns that these are separate states. Make a short matrix with user role, source, allowed scope, denied scope, and the expected response. If any row has no owner or expected outcome, the access test is not ready.

An aggregate deserves care too. A role may be unable to read an individual record but could still infer it from a tiny group or a sequence of filters. This possibility depends on the source and policy, so do not assert that Data exposes such information. Instead, ask the security or data owner what aggregation and inference rules apply, then include a representative case in the approved review. The purpose is to test the actual boundary, not to accuse the tool of bypassing it.

View image detail

A permitted test identity and a restricted test identity should have documented expected outcomes.

Actual size

Choose Actual size to read the graphic closely.

Read a chart as carefully as the query

Correct numbers can still produce a misleading chart. Check the axis baseline, aggregation, units, time grain, bucket labels, denominator, missing categories, and whether the comparison period is consistent. A truncated axis can make a modest change appear dramatic. A percentage without the population size can hide how few observations it represents. A weekly view may conceal a short-lived spike or a change in data freshness.

Inspect the data behind at least one chart rather than judging its styling. Confirm that each line or bar maps to the label, the chart’s title describes the measure, and the narrative does not overstate what the visualization shows. Check if the chart omits missing values or combines groups with different definitions. If the visualization is interactive, verify that filters visibly change the result in a way the reader can understand. A dashboard can be editable and attractive while still needing an analyst to validate the story.

This is particularly important when the reader was not in the original conversation. The conversation may contain assumptions or follow-up clarifications that did not make it onto the dashboard. A downstream reader sees the chart and its labels, not the complete chain of prompts. Put the metric definition, period, source freshness, and known caveat close to the visual when they change its meaning. If the caveat is essential to avoid a false conclusion, it belongs in the published artifact rather than in a private reviewer note.

View image detail

A reviewer checks scale, units, denominator, missing groups, and freshness before accepting a chart.

Actual size

Choose Actual size to read the graphic closely.

Treat the dashboard as a new artifact

Turning a conversational analysis into a dashboard changes the audience and the persistence of the work. A one-person draft answer may be easy to qualify in context. A dashboard may be revisited, refreshed, embedded in a meeting, or detached from the conversation that explains its assumptions. Decide who owns the artifact, what recurring question it answers, how often it should refresh, and what should happen when a source is late or a definition changes.

The launch announcement says Data can create dashboards that users edit, share, and refresh, and can work with a list of BI tools. Current Help makes the actual actions contingent on connected tools, source setup, supported capabilities, and permissions. Sigma’s launch post says its plugin is available to OpenAI enterprise customers with Sigma Act, Analyze, or Build licenses. It distinguishes asking and querying from dashboard creation: asking and querying are available to those licensed Sigma users, while building dashboards requires a Build license. That is the partner’s description of its own integration, not a general rule for every connector. (Sigma’s integration and license notes)

Before making a dashboard recurring, name its owner and review rhythm. Decide who checks stale data, reports errors, and changes definitions. Determine whether an old version remains accessible after a refresh or whether the product updates it in place. Those operational details were not established by the sources reviewed here, so they should be verified in the actual connected tool. If no one owns maintenance, a static report may be safer than a refreshable dashboard that silently looks current.

Sharing can copy data beyond its source context

One easily missed boundary in OpenAI’s current Help Center concerns Sites. It says publishing a Site can copy the data used in the analysis into the published Site, so the publisher should consider the data permissions when choosing the audience. This means source access and artifact access are not the same thing. A connected account may legitimately query a dataset, while a published page could make derived content available to a broader set of people than the original source role intended.

Before publication, inspect the actual artifact and audience setting. Identify whether it includes source rows, labels, annotations, chart values, filters, hidden fields, or explanatory text that should be restricted. Test what a recipient can see using an approved recipient identity. Check whether links or embeds expose the artifact outside the intended workspace. This is a recommended review, not a claim that a public exposure occurs by default. The product behavior and workspace controls need to be verified in the real publishing flow.

For a related view of shared workspace permissions and source access, see our guide to organizing shared work in ChatGPT Space. It covers Space sharing rather than Data plugin behavior, but the distinction between a reader's access to a shared artifact and access to its linked source is useful in both reviews.

Sharing is a separate decision from calculation. The metric can be correct and the access path to the data can work as designed, yet the audience may still be inappropriate. Conversely, a valid audience does not rescue a weak analysis. Record who is allowed to view the dashboard, what content they need, and who approves changes to that audience. If you cannot determine whether the published page contains copied source data, pause and ask the administrator before sharing it.

View image detail

The source permission boundary and the published dashboard audience are independent decisions.

Actual size

Choose Actual size to read the graphic closely.

Review actions as a separate approval

An analysis may suggest a next step: send a summary, assign an owner, update a CRM field, or schedule a follow-up. The fact that an action is suggested does not mean it is safe or available. OpenAI’s announcement says the Data agent can share findings through Slack or email and carry out actions users approve through connected tools. The current Help Center says sharing or actions depend on the connected tool’s supported operations, the user's permissions, workspace settings, and any approval requirement. Those conditions vary. Do not infer that every action has the same confirmation step.

Review the destination and payload before approval. Does the message include a customer name or confidential number? Is the recipient correct? Does a change in another system affect a production record? Should a person with context approve the wording? Can the effect be reversed? A dashboard's analytical review does not answer these questions. The person authorizing an action should see enough information to understand what will happen and where.

For a first evaluation, keep actions out of scope or use a non-production destination where the organization permits it. Ask for a draft or recommendation rather than a write. If a real action is later authorized, define its owner, allowed targets, review step, and rollback path. The goal is not to prevent useful follow-through. It is to prevent a plausible analysis from becoming an unreviewed change merely because the connected tool exposes a convenient button.

View image detail

A reviewed finding still needs its own destination, payload, authority, and approval before action.

Actual size

Choose Actual size to read the graphic closely.

A review matrix for the team

• Workspace access: Evidence to inspect: Data availability, source plugin, account connection, role; Responsible reviewer: Workspace administrator; Pause when: Intended role or source is not configured
• Source authority: Evidence to inspect: Dataset/report identity, owner, refresh time, account identity; Responsible reviewer: Source or data owner; Pause when: Source is unapproved, stale, or unowned
• Metric logic: Evidence to inspect: Definition, calculation, joins, filters, units, period; Responsible reviewer: Metric owner or analyst; Pause when: Definition conflicts or cannot be reproduced
• Interpretation: Evidence to inspect: Whether the explanation is observation, hypothesis, or causal claim; Responsible reviewer: Decision owner and analyst; Pause when: Wording claims more than evidence shows
• Visual fidelity: Evidence to inspect: Labels, axes, totals, missing groups, filters, freshness; Responsible reviewer: Dashboard owner or analyst; Pause when: Chart misstates or obscures the underlying answer
• Audience: Evidence to inspect: Artifact contents, recipient roles, Site sharing scope; Responsible reviewer: Publisher and administrator; Pause when: Data copy or audience is unclear
• Action: Evidence to inspect: Destination, payload, account authority, approval, reversibility; Responsible reviewer: Action owner; Pause when: Tool support or authorization is uncertain

This matrix is not a certification standard. It is a way to assign questions to someone who can answer them. If a role cannot see the actual artifact, its review may be indirect. If a source owner did not review the metric, the dashboard’s chart styling cannot fill that gap. Capture the evidence and disposition for each row that matters to the decision.

A small acceptance protocol

For a proposed pilot, prepare an expected answer from a trusted source, a restricted identity case where appropriate, and a single dashboard question. Have the analyst compare the agent's output to the known reference under the same date, filters, units, and definitions. Have the source owner verify the role and source boundary. If a chart is produced, compare its values with the evidence behind the answer. If the output will be shared, inspect the actual audience and what data the published artifact contains. If a downstream action is considered, review that action separately with an authorized owner.

Measure what your team actually cares about: whether the answer is reproducible, how many material corrections the reviewer makes, whether the right source was used, whether the result can be explained to another analyst, and whether the artifact remains current. Do not claim the tool is accurate because it generated a chart or because one answer matched. Do not use an arbitrary pass percentage unless the decision owner has established what threshold is meaningful for this analysis.

The protocol should use a deliberately small, representative task. A big dashboard that mixes five metrics and three data sources makes it harder to isolate why a mismatch happened. One source and one metric produce a cleaner diagnostic. Expand only when the team can explain the current result and maintain its context. This is Rise's proposed method, not a benchmark result or a test already completed.

What the available reporting does and does not establish

OpenAI’s launch page reports internal adoption and examples from named customers. Those examples may help explain the product’s intended use, but they remain company-provided claims. VentureBeat’s September 10 reporting describes an OpenAI interview and notes the absence of a published accuracy benchmark; it does not provide an independent reliability evaluation. Sigma’s separate partner announcement describes Sigma’s own connector and license conditions, with its own commercial perspective. The sources reviewed for this article do not establish the Data agent’s accuracy rate, failure rate, or safety performance across organizations.

That evidence boundary matters. “OpenAI says queries enforce existing permissions” is a report of a product assertion. It does not establish that the permission map in your workspace is correct, that all derived information is safe for every audience, or that the analysis contains no errors. Those questions require configuration evidence and tests in your environment. Likewise, not finding an independent study in this source review is not proof that no such evaluation exists. It is a limit of what supports this article today.

OpenAI’s current guidance to inspect source, time range, filters, definitions, and evidence is operationally useful. Use it as a review checklist, not as a guarantee. The evaluation must remain attached to the specific source, account, metric, date, artifact, and audience you inspected. A later connector change or permission update can change the relevant conditions.

Keep the status labels separate

In your notes, avoid a single “approved” label for the entire workflow. Record whether the connection is configured, the result is reconciled, the chart is checked, the dashboard audience is approved, and an action is separately authorized. A connection can be ready while its result is under review. A result can be valid for an analyst but not cleared for a broader audience. A shared dashboard can be approved while its follow-up action remains pending.

These statuses help when the analysis changes. If the metric definition changes, you know which result checks need repeating. If the dashboard’s audience expands, you know to revisit publication and access. If a new source app is added, source authority and permissions deserve a new look. The amount of process should fit the consequence of the decision. A quick internal exploration may need a light check; a high-impact financial or customer action needs a more deliberate sign-off.

Rely on the result only after its boundaries are visible

ChatGPT Work’s Data agent may help a team investigate business questions and build dashboards, but the useful claim is narrower than “the answer is secure” or “the chart is correct.” OpenAI says the connected account’s existing source permissions apply. Its Help Center also tells users to inspect sources, periods, filters, and definitions, and it warns that Site publication can copy analysis data into the published artifact. Each statement points to a different review task.

Check the source and calculation. Check the interpretation. Check the dashboard. Check the audience. Check any action. Keep an owner beside each decision, and pause when the evidence needed for that decision is missing. This is how a team can benefit from a fast first-pass analysis without confusing a working connection for business approval.