NL2SQL

This post maps the path from a natural-language query to the final response, showing which components run and in what order. It is the starting point of the series; the rationale behind each component choice and the experiments along the way are covered in the posts that follow. Why routing architecture instead of a single NL2SQL pipeline When NL2SQL is first applied to a real enterprise environment, the same problem usually appears. DDL alone does not let the LLM infer what a table named T_CUST_MST actually means, or what calculation logic “net revenue” refers to in this organization. Benchmark scores keep climbing, yet reports from real enterprise deployments still show accuracy falling below 50%. The same category of errors does not disappear just by swapping datasets. ...

In the previous ten posts, I analyzed the most common SQL generation errors. I used BIRD and Spider benchmarks, tested several models, and grouped wrong answers into types such as COLUMN_BINDING, VALUE_BINDING, and DERIVED_METRIC. As I kept looking at those SQL errors, I realized I wanted to check something different. “So how should generated SQL and supporting context appear in the UI?” I had been focused on accuracy and had missed what users actually need from the interface. In a workplace data analysis tool, the UI has to make it clear where numbers come from. Without traceable context, users end up re-running the SQL even when the result looks correct. ...

Three multi-candidate experiments from post 9 produced no meaningful improvement. Before moving on to the Schema Binding Plan, I wanted to test one more hypothesis: model tier as the bottleneck. I expected Gemini Pro to improve the current 56% accuracy baseline over flash-lite. I also ran the error-type classification and Spider experiment as part of that test. Switching to Pro Made Things Worse Prompts, context, and pipeline stayed the same. Only the model changed, and the same 50 BIRD questions ran against it. The metric is EX (Execution Accuracy): whether the generated SQL produces the correct result when executed against the database. ...

I hit 80% on my own 30-question benchmark, but only 56% on BIRD Mini-Dev’s 50 public questions. Nine experiments later, I had ruled out the multi-candidate hypothesis from three different angles. What’s left is schema understanding and methodology.

After wiring up the router, I ran a 30-question benchmark and pushed NL2SQL EX (Execution Accuracy) from 66.67% to 80%. Here’s what I fixed across four cycles and where things broke.

The term definitions are done. But when a user asks a question, who decides whether to search the graph, write SQL, or run a vector search? Things I ran into while designing the router.

Shortly after the Conway leak, Anthropic officially launched Claude Managed Agents. As agent infrastructure gets absorbed into platforms, here’s why DataNexus’s ontology layer remains safe.

How we decided on DataHub + Vanna + ApeRAG + DozerDB for DataNexus. What got eliminated from the candidate list, and why.

“What’s Your VIP Criteria?” This happened during a BI Agent project for a retail company. A business user was testing the Agent and asked, “Show me last month’s VIP customer revenue.” The system spit out a number, but the user did not look happy. “Something’s off. I think the VIP criteria are different from what our team uses.” Marketing’s VIP and CRM’s VIP were different. Same with revenue. Depending on whether you meant net revenue (순매출) or gross revenue (총매출), the difference could be hundreds of millions of won. ...