ChatGPT for Business Intelligence: What We're Learning

We've been hearing a version of the same question in sales calls lately: "Can we add ChatGPT to our dashboards?"

It's not just one customer asking. The pattern is clear: ChatGPT changed how people expect to interact with information. And now, those expectations are spilling over into business intelligence and analytics.

The question isn't whether natural language will become part of BI, it already is. The real question is: what does that actually look like for customer-facing analytics?

The Question We're Hearing From Customers

Here's what customers are really asking for when they say they want "ChatGPT for their dashboards":

They want their users to type questions like:

• "Show me revenue by region for Q4"
• "Which customers churned last month?"
• "Compare this quarter to last year"

Instead of working through through filters, dropdowns, and date pickers.

This makes sense. ChatGPT proved that natural language is often faster than clicking through interfaces. Users got used to typing what they want and getting an answer.

But when we dig deeper into these requests, we're finding that what customers actually need isn't ChatGPT itself. It's the interaction model ChatGPT popularized: Low friction, high relevance.

What ChatGPT Changed About Business Intelligence Expectations

Before ChatGPT became mainstream, AI-powered analytics and natural language query (NLQ) in BI tools felt like nice-to-have features.

ChatGPT changed that baseline. Now users expect:

1. Zero learning curve: No training, no manual reading
2. Conversational follow-ups: "Show me last month" → "What about the month before?"
3. Plain language inputs: Not SQL, not filter syntax, just normal questions

For embedded analytics, this shift is fundamental. The old model was "learn our interface, then access your data." The new expectation is "ask your question, get your answer."

From customer feedback, we're seeing this play out in two very different contexts:

Internal BI teams are experimenting with ChatGPT plugins and API integrations to query their data warehouses. This makes sense, these are technical teams working with their own secure data.

Product teams building customer-facing analytics are asking how to bring ChatGPT-style interactions into their embedded dashboards. This is where things get complicated.

Where ChatGPT-Style Interfaces Make Sense in BI

The appeal of conversational BI is obvious: users get answers faster.

Where we're seeing it work well:

Internal Analytics Teams

• Data analysts querying their own warehouse
• Technical users who understand the data model
• Single-tenant environments (no customer data isolation needed)
• Exploratory analysis where some hallucination risk is acceptable

Customer-Facing Analytics (with caveats)

• Guided query interfaces
• Pre-defined question templates
• Structured data with clear boundaries
• Multi-tenant security built in from the start

The key difference: internal vs. external use cases have completely different requirements.

When your own team uses ChatGPT to explore data, hallucinations are annoying but manageable. When your customers use it to query their data, any incorrect response is a trust issue.

The Challenge: ChatGPT Isn't Built for Customer-Facing Analytics

This is where we see product teams hit walls.

ChatGPT (and most LLM-based solutions) were designed for open-ended, internal use cases. When you try to embed them directly into customer-facing products, you run into significant architectural problems:

1. Data Security and Multi-Tenancy

General-purpose LLMs don't natively understand multi-tenant data isolation. You can't just connect an LLM to your production database and trust it to separate Customer A from Customer B.

In embedded analytics, Row-Level Security (RLS) is non-negotiable. You need token-based authentication that ensures a user never sees data they aren't authorized to access. This is hard to enforce when an LLM is generating arbitrary SQL.

2. Hallucination Risk With Customer Data

Internal analysts can spot when an AI makes something up. Your customers can't, and shouldn't have to.

When a customer asks "What was our revenue last quarter?" and gets a confident but wrong answer, that's not a minor UX issue. It's a credibility problem for your entire product.

3. Performance and Cost at Scale

Running LLM queries for every customer interaction gets expensive fast. We're hearing from teams who experimented with ChatGPT integrations and saw:

• Unpredictable API costs (especially during traffic spikes)
• Latency issues (3-5 second response times)
• Rate limiting during peak usage

For embedded analytics that need to feel native to your product, users expect the same sub-100ms response times they get from the rest of your app.

4. The "Black Box" Problem

When an LLM generates a query, you can't easily validate it before running. With customer data, you need to know:

• Exactly what data will be queried
• Which filters are being applied
• That row-level security is enforced

The Better Approach: Structured Interactions for Embedded Analytics

Here's what we're learning from these conversations: the goal isn't to replicate ChatGPT's open-ended chaos. It's to bring the value (instant, specific answers) into embedded analytics in a way that's safe, predictable, and performant.

At Sumboard, we believe the solution for customer-facing analytics isn't "ask anything," but "guided exploration."

Guided Exploration (Not Open-Ended)

Instead of accepting any question and hoping for a valid SQL query, the most successful implementations use structured filtering and self-service builders.

• Dynamic Filters: Allow users to "speak" to the data by selecting parameters (Time, Region, Category) that immediately update the view.
• Self-Service Reports: Give users the building blocks to answer their own questions without needing to know SQL.

Users get the specific answer they need without the unpredictability of a chatbot.

Built for Multi-Tenant Isolation

This is where our platform architecture shines. Whether a user interacts via a dashboard filter, a drill-down, or an API call, every query respects row-level security by default.

Because we use token-based authentication, Customer A's interaction only touches Customer A's data. There is no "prompt engineering" hack that can bypass these hard security boundaries. This is table stakes for our embedded analytics platform.

Predictable, Verifiable Accuracy

When a user filters a dashboard or builds a report in Sumboard, the system uses deterministic query logic.

1. The user intent is captured (e.g., "Filter to last 30 days")
2. The platform applies strict security scopes
3. The optimized query runs against the verified data model

No hallucinations. No black box queries. No surprises.

Fast and Cost-Effective

By using optimized, structured queries instead of full LLM inference for every interaction, you ensure:

• Lightning-fast performance: Dashboards load in under 100ms.
• Predictable costs: No per-query API tokens or fluctuating bills.
• Scalability: Performance that holds up even as your user base grows.

The Pattern We're Seeing

ChatGPT changed user expectations around how they interact with data. That's not going away.

But for customer-facing analytics, the solution isn't to embed ChatGPT directly. It's to build interfaces that feel natural and intuitive while maintaining the security, performance, and predictability that embedded products require.

From what we're hearing in customer conversations, the teams getting this right are building AI-powered analytics that prioritize:

• Structured, guided interactions over open-ended chat
• Multi-tenancy and security built in from day one
• Speed (under 100ms load times) and cost-effectiveness
• Accuracy as non-negotiable

We're working on bringing these principles into Sumboard's platform. If you're building analytics for your customers, we'd love to help you do it securely.