We break governance for AI in an analytics product into six decisions per feature: which data it may reach, who can switch it on, what leaves for the model provider, what gets logged, how its answers are checked and what it may cost. Enforce each decision in application or database code outside the model.
On Hacker News, someone adding ask-your-data features to a SaaS product wrote: "Feels risky to have an LLM write queries against even a read only copy of a database."
Why Governance Is Now Part of Shipping an AI Feature
In the IAPP's AI governance profession report, which surveyed more than 670 people in 45 countries and regions, "77% are currently working on AI governance". In Stanford's 2026 AI Index, the AI Incident Database recorded 362 documented AI incidents in 2025, up from 233 in 2024. The same report found that the share of businesses with no responsible AI policies fell from 24% to 11%.
In KPMG's study of more than 48,000 people in 47 countries, 66% of people using AI at work rely on AI output without evaluating its accuracy, and 56% report making mistakes in their work because of AI.
ISO/IEC 42001:2023 specifies requirements for an AI management system, and NIST's Generative AI Profile lists risks specific to generative AI. Both work at the level of an organization's AI program; the six decisions below apply that work to a single AI feature in an analytics product.
1. Which Data Each AI Feature May Reach
Snowflake Cortex Analyst, Databricks Genie and Microsoft Fabric Copilot all apply the asking user's data permissions to what the AI can reach.
| Platform | What its documentation says |
|---|---|
| Snowflake Cortex Analyst | "Cortex Analyst fully integrates with Snowflake's role-based access control (RBAC) policies" |
| Databricks Genie | "Data access is always evaluated using each end user's own Unity Catalog permissions." |
| Microsoft Fabric Copilot | "Copilot can only access data that the current user has permission to access" |
OWASP's guidance on excessive agency gives the general rule: "Limit the permissions that LLM extensions are granted to other systems to the minimum necessary."
In a product serving many customers, those permissions also have to stop at each customer's tenant. AWS described a multi-tenant text-to-SQL agent in which the server wraps every generated query in tenant-filtered common table expressions before it runs, so the model never chooses the tenant. The team reports: "In production, this architecture has processed over 50,000 queries with zero cross-tenant data exposure incidents."
Keep this control out of the prompt. In tests of seven large language models, "unsanitized user prompts can lead to SQL injection attacks" in LLM-integrated applications. A user can talk the model out of an instruction to show only one customer's data. Enforce the restriction with a row-level security policy or a server-side filter that the model's output cannot change.
2. Who Can Switch an AI Feature On
In Fabric and Amplitude, administrators control AI enablement for the whole organization. In Fabric: "Your administrator needs to enable the tenant switch before you start using Copilot." In Amplitude, AI agent settings are set for the whole organization: "These settings affect all users in your organization. Individual users can't override the organization-level state."
For an embedded or customer-facing analytics product, we recommend two layers: you decide which AI features the product offers, and each customer's administrator decides whether their users get them.
3. What Leaves for the Model Provider, and for How Long
What an AI analytics feature sends to the model depends on its design:
- Fabric Copilot sends the user's prompt and grounding data, where "grounding data might include a combination of dataset schema, specific data points, and other information relevant to the user's current task".
- Snowflake Cortex Analyst uses the semantic model's metadata "only for SQL-query generation"; the generated SQL then runs in the customer's own warehouse.
Document which of these designs each AI feature uses in your data processing documentation.
Retention at the provider differs as well:
| Provider | What its data page says |
|---|---|
| OpenAI API | "By default, abuse monitoring logs are generated for all API feature usage and retained for up to 30 days." |
| Anthropic API | "Retained data is never used for model training without your express permission." Retention differs by feature. |
| Azure-hosted OpenAI models | Prompts and completions "are NOT available to OpenAI or other providers of Models sold by Azure." |
Retention is set per provider and per feature. For each AI capability, record its model provider and the provider features it uses, and recheck the provider's retention policy whenever either changes.
4. What Gets Logged, and Who Can Read It
Log the query behind every AI answer. The AWS deployment above logs "queries, sessions, and permission-mapping changes" to an audit service. NIST's Generative AI Profile recommends "logging, recording, and analyzing GAI incidents" as a documentation practice.
Two details decide whether a log is useful:
- What is in each entry. At minimum: user, tenant, the question, the generated query, the data sources it touched and the size of the result.
- Who can read it. In Amplitude, "Only organization admins can access the Audit tab." In a customer-facing product, each customer's administrator should be able to see their own tenant's AI activity, and only theirs.
Under the EU AI Act, "high-risk AI systems shall technically allow for the automatic recording of events (logs) over the lifetime of the system", with obligations for Annex III systems applying from 2 December 2027. That article covers high-risk systems only. Check whether your product, or your customers' use of it, falls into that category.
5. How Answers Are Checked
Test for wrong numbers that look right. NIST calls this risk confabulation: "the production of confidently stated but erroneous or false content".
On Spider 2.0's 632 enterprise text-to-SQL tasks, an agent built on o1-preview "successfully solves only 21.3% of the tasks, compared with 91.2% on Spider 1.0 and 73.0% on BIRD". Using a semantic layer changes how a query fails. In dbt Labs' 2026 rerun of its comparison, 11 questions were each run 20 times. Querying through the semantic layer scored 98.2% and 100%, compared with 90.0% and 84.1% for text-to-SQL. dbt summarized the difference: "With text-to-SQL, failure looks like a plausible but incorrect answer. With the Semantic Layer, failure looks like an error message."
The practical check is a set of reference questions with SQL you trust. Databricks Genie benchmarks work this way: "accuracy is assessed by comparing the result set from your SQL query to the one generated by Genie", with up to 500 benchmark questions per agent. Execution accuracy, "the most prevalent evaluation metric, still shows many false positives and negatives" in a 2025 evaluation; two different queries can return the same rows by accident.
Run the reference set on a schedule. See also natural language query.
6. What an AI Feature May Cost
In an embedded product, track the cost of AI queries made by each customer's users. Snowflake's usage view for Cortex Analyst reports "the number of credits billed for a set of messages sent to Cortex Analyst" by user name, in hourly windows, for the last 365 days. That is the level of attribution to aim for: cost per user and per tenant, with a limit per tenant that the product enforces.
A Governance Checklist for AI Features in Analytics
| Decision | Question to answer | Evidence that it holds |
|---|---|---|
| Data access | Does the AI query with the asking user's identity and tenant scope? | A test showing a query cannot reach another tenant's data |
| Enablement | Who switches each AI feature on, for which customers? | Admin setting per customer, off until enabled |
| Provider data | What is sent to the model, and how long does the provider keep it? | Data flow per feature, with the provider page and date |
| Logging | Can every answer be traced to user, tenant, question and query? | A log entry for a real question, readable by that tenant's admin |
| Answer checks | How often are answers compared with reference queries? | Reference set, results and schedule |
| Cost | What does each tenant spend, and where does it stop? | Cost per tenant and an enforced limit |
When agents query the product through MCP, see MCP for analytics.
Where Sumboard Stands
Sumboard released an AI assistant in its SQL editor in January 2026, for the people who write queries. Sumboard keeps no copy of your data beyond cached results. Natural language querying for your customers is in progress. The feature is being designed to apply the same permission checks to the assistant's queries as to all other queries, and to produce a query you could have written yourself for each answer.
If you are choosing an embedded analytics platform that will put AI in front of your customers, take the checklist above into the evaluation. Agree on these controls before launching customer-facing AI features.
