Asking a database questions in plain English — "Which industries have the most companies?" or "Compare average revenue of public vs private firms" — and getting a real, data-backed answer is no longer science fiction. With Microsoft Semantic Kernel, OpenAI function calling, and a few hundred lines of C#, you can turn a normal SQL Server table into a conversational analytics assistant.

In this tutorial we walk through a working Text-to-SQL proof of concept (SemanticKernelPoc.SQL): a .NET 8 Razor Pages app where the Large Language Model (LLM) writes and executes SQL on demand, then explains the results in natural language. You'll get the full architecture, the key source code, the design decisions, the real-world performance lessons, and an honest list of pros and cons.

What We Are Building (And Why)

The goal is simple to state and powerful in practice: let non-technical users query a database without knowing SQL.

A business analyst shouldn't need to remember JOIN syntax to ask "Top 5 companies by patent count in technology, and what's their profit margin?" The app should:

1. Understand the question.
2. Know the database schema.
3. Write correct, safe SQL.
4. Execute it.
5. Interpret the rows and answer conversationally — and show the SQL it ran for transparency.

This pattern — often called Text-to-SQL or a natural-language data agent — is one of the highest-value, lowest-risk uses of LLMs in the enterprise, because the model is grounded in YOUR live data instead of hallucinating from training memory.

Why You Need This

• Self-service analytics. Business teams get answers without waiting on a data analyst.
• Faster decisions. Questions that took a ticket and two days now take ten seconds.
• Lower BI cost. Fewer one-off dashboards and ad-hoc query requests.
• Grounded AI. Answers come from real query results, dramatically reducing hallucination.
• Auditability. Every answer ships with the exact SQL that produced it.

The Tech Stack

• Runtime: .NET 8
• Web: ASP.NET Core Razor Pages + Minimal APIs
• AI orchestration: Microsoft Semantic Kernel 1.70
• LLM: OpenAI gpt-4o-mini (chat + function calling)
• Database: SQL Server
• Data access: Microsoft.Data.SqlClient
• Test data: Bogus (fake company data generator)

The project deliberately includes ONLY what a SQL chat needs — no vector database, no embeddings, no local model dependencies. It is a focused, easy-to-read reference implementation.

How It Works: The Architecture

The flow for a single question is:

User question → ChatService → Semantic Kernel → OpenAI (gpt-4o-mini)
     ↓ "I need data" (tool call) ↑
DatabasePlugin.query_database(sql) → SQL Server
     ↓ rows ↑
OpenAI interprets rows → Natural-language answer + SQL shown in UI

The magic is OpenAI function calling (wired through Semantic Kernel's auto tool invocation). The model isn't asked to RETURN SQL as text — it is given a TOOL it can call, and the kernel executes that tool automatically, feeds the results back, and lets the model continue until it has a final answer.

The Source Code, Explained

1. Registering the Kernel and Chat Service

Program.cs is tiny. We register Semantic Kernel, point it at OpenAI, register our chat service, and run a one-time database setup.

var builder = WebApplication.CreateBuilder(args);
var apiKey = builder.Configuration["OpenAI:ApiKey"]!;
var model  = builder.Configuration["OpenAI:Model"] ?? "gpt-4o";

builder.Services.AddKernel();
builder.Services.AddOpenAIChatCompletion(model, apiKey);
builder.Services.AddSingleton<ChatService>();
builder.Services.AddRazorPages();

var app = builder.Build();

// Create DB + table + seed sample data on startup
var connectionString = app.Configuration.GetConnectionString("SqlServer")!;
await DatabaseSetup.EnsureDatabaseAsync(connectionString);
await DatabaseSetup.EnsureTableAsync(connectionString);
await DatabaseSetup.SeedDataAsync(connectionString);

A single POST /api/chat minimal-API endpoint receives the question and returns the answer plus the executed queries.

2. The Database Plugin (The LLM's "Hands")

This is the tool the model is allowed to call. Notice the safety guard: only SELECT/WITH statements are permitted — the model can never modify data.

[KernelFunction("query_database")]
[Description("Executes a read-only T-SQL SELECT query and returns the results.")]
public async Task<string> QueryDatabaseAsync(
    [Description("A valid T-SQL SELECT query")] string sqlQuery)
{
    sqlQuery = StripCodeFences(sqlQuery.Trim());
    if (!sqlQuery.StartsWith("SELECT", StringComparison.OrdinalIgnoreCase) &&
        !sqlQuery.StartsWith("WITH",   StringComparison.OrdinalIgnoreCase))
        return "ERROR: Only SELECT queries are allowed.";

    await using var conn = new SqlConnection(_connectionString);
    await conn.OpenAsync();
    await using var cmd = new SqlCommand(sqlQuery, conn) { CommandTimeout = 120 };
    await using var reader = await cmd.ExecuteReaderAsync();
    // ... read rows, cap output sent to the LLM, capture for UI ...
}

Two production-minded details:

• It returns only the first 50 rows as text to the model (to save tokens) while still capturing the full set for the UI.
• It records every executed query so the front end can show the SQL and a result table — full transparency.

3. How the Model Knows Your Schema

This is the most important design choice for speed and accuracy. We inline the schema directly into the system prompt instead of making the model call a separate "get schema" tool first.

private static readonly string SystemPrompt = $$"""
    You are an intelligent data analyst assistant with access to a Company database.
    The database schema is below — you already know it, so do NOT call any schema tool:
    {{DatabaseSetup.GetTableSchema()}}

    You have ONE tool available:
    - query_database: Execute SQL SELECT queries against the database.

    GUIDELINES:
    - Only answer questions related to the Company database.
    - Always use TOP N. Never return unbounded result sets.
    - Only generate SELECT statements. Never modify data.
    """;

Because the schema is baked into the prompt, the model writes correct SQL on its FIRST attempt and skips an entire network round-trip to OpenAI.

4. Calling the Model with Auto Function Invocation

var settings = new OpenAIPromptExecutionSettings
{
    ToolCallBehavior = ToolCallBehavior.AutoInvokeKernelFunctions,
    Temperature = 0.1
};
var response = await _chatService.GetChatMessageContentAsync(history, settings, kernelWithPlugin);

AutoInvokeKernelFunctions is the line that makes it an AGENT. The kernel automatically loops: model → tool call → execute → feed results back → model → final answer. A low Temperature (0.1) keeps the generated SQL deterministic and conservative.

Performance: The Lessons That Don't Make It Into Most Tutorials

A POC that WORKS and a POC that FEELS FAST are different things. Here are the real bottlenecks we hit and fixed.

Lesson 1: Cut LLM Round-Trips

Each call to OpenAI costs latency. The naive design used three sequential calls per question (fetch schema → write SQL → interpret results). Inlining the schema into the system prompt removed an entire round-trip, dropping the model to two calls. Fewer hops = faster answers.

Lesson 2: The Database Can Be Starved, Not Slow

At one point, a trivial GROUP BY on just 10,000 rows took 24 seconds. The instinct is to blame the query or missing indexes — both were fine. The real cause was memory: the machine was low on RAM, so SQL Server's buffer pool had collapsed to 5 MB, far too small to cache a 27 MB table. Every query re-read the whole table from disk and got throttled. The fix had nothing to do with code:

EXEC sp_configure 'min server memory (MB)', 384; RECONFIGURE;
EXEC sp_configure 'max server memory (MB)', 2048; RECONFIGURE;

Result: scans went from 24s to 0.2s — a 100x+ improvement. The takeaway: before optimizing queries, check whether your database engine actually has the memory to cache its working set.

Lesson 3: Index the Columns You Filter On

A SELECT * FROM Company WHERE CompanyName = '...' did a full table scan because CompanyName wasn't indexed. Adding a single non-clustered index turned it into a one-row seek — instant, even on a cold cache.

Benefits

• Natural language access to relational data — no SQL skills required.
• Grounded answers — the model reads real rows, not its training data.
• Transparent and auditable — the exact SQL is shown with every answer.
• Safe by design — read-only guard blocks any data modification.
• Cheap to run — gpt-4o-mini is fast and inexpensive; the schema lives in the prompt, not in a vector store.
• Minimal footprint — pure SQL + Semantic Kernel, no extra infrastructure.

Pros and Cons

Pros

• Fast to build: A few hundred lines of C#; no ML pipeline.
• Accurate on structured data: SQL is exact — counts, sums, averages are correct.
• Self-documenting: Schema in the prompt doubles as living documentation.
• Low operating cost: One LLM model, no embeddings, no vector DB.
• Secure: SELECT-only enforcement; queries scoped to one table.

Cons

• Latency bound by the LLM: 5-8s per answer; use streaming and fewer round-trips.
• Schema must stay in sync: Inlined schema is manual; automate from INFORMATION_SCHEMA.
• Prompt size grows with schema: Wide schemas eat tokens; trim to relevant tables/columns.
• LLM can write a slow query: Add TOP, timeouts, and indexes; consider a query cost guard.
• Weak at fuzzy/semantic: "Companies like a solar startup" needs vector embeddings instead.
• SQL injection surface: Model writes raw SQL; the read-only guard and a least-privilege DB user are essential in production.

Production Hardening Checklist

1. Use a least-privilege, read-only SQL login — defense in depth beyond the SELECT guard.
2. Secure the API key — move it out of appsettings.json into a secret manager / env variable.
3. Add response streaming — stream the final answer token-by-token so it FEELS instant.
4. Cap rows and enforce query timeouts — protect the database from runaway scans.
5. Generate the schema dynamically — read it from the database so it never drifts.
6. Log every generated query — for auditing, debugging, and abuse detection.
7. Rate-limit and authenticate — an open Text-to-SQL endpoint is a powerful thing.

Conclusion

Text-to-SQL is one of the most practical, immediately useful applications of LLMs today. By combining Semantic Kernel's function calling with OpenAI and a plain SQL Server table, we built a conversational analytics assistant in a single, readable .NET 8 project — no vector database, no fine-tuning, no heavy infrastructure.

The hard-won lessons are as valuable as the code: inline your schema to cut latency, give your database engine enough memory to cache its working set, and index the columns you filter on. Get those right and you have an AI data assistant that is fast, accurate, transparent, and genuinely useful to non-technical users.

The LLM writes the SQL. Your database stays the source of truth. That's the sweet spot.

Frequently Asked Questions

Q: Is Text-to-SQL safe for production?

With guardrails, yes. Enforce SELECT-only queries, connect with a read-only least-privilege user, set query timeouts, and log everything. The biggest risk is an over-privileged database account, not the model itself.

Q: Why Semantic Kernel instead of calling OpenAI directly?

Semantic Kernel handles the function-calling loop (model → tool → result → model) for you, manages chat history, and makes the database a first-class "skill." You can call OpenAI directly, but you'd rebuild this orchestration yourself.

Q: Do I need a vector database?

Not for this. Vector databases shine for semantic similarity search over unstructured text. For precise, aggregate questions over structured tables, SQL is the right tool — and far cheaper.

Q: Which OpenAI model should I use?

gpt-4o-mini is a great default: fast, cheap, and strong at SQL generation. Step up to a larger model only if your schema is very complex or queries require deeper reasoning.

Q: How do I make it faster?

Reduce LLM round-trips (inline the schema), stream the final response, keep the database working set in memory, and index filtered columns. The database itself is rarely the bottleneck once it's properly resourced.