Build and update tables
SQL on the files to build or update stored (materialized) tables and views.
/ Technology · Compute Architecture · Querying & AI
Query Parquet files without a database, by SQL, chat or API.
Technical users, analysts, business users and other systems all query the same governed files. Every request is compiled with the asker’s row- and column-level access and run in place. In conversations the model writes the query without seeing the data, and large exports run at constant memory.
SQL on Parquet · Any table, any version · Conversational analytics · Metadata-only AI · Published APIs · Exports at constant memory
Ways to query
Five ways to use the same data.
Whoever asks, the request takes the shared path: compiled from governed definitions with the asker's row- and column-level access, run on the files in place, and logged.
Explore data
Query any table, at any version or business date, without loading it anywhere.
Ask in conversation
Questions in plain language, answered as tables or charts. The model never sees the data.
Published APIs
Every table is available through a published API, on an API key basis, under the same row- and column-level access.
Download and export
Large exports to CSV, TXT, JSON, Parquet or Excel, at constant memory.
SQL on files
SQL directly on Parquet files.
The SQL that runs the platform's own pipelines runs directly on the Parquet in storage. Nothing is imported into a database or warehouse first, and every query still goes through the same compiler, row- and column-level access and audit log.
- One SQL, every use: analysis, scheduled jobs, APIs and AI agents.
- Any table, any date: as it stands now or on an earlier business date.
- Nothing to sync: no import job, no second copy, no serving database.
Getting from files to an answer
Conversational analytics
How AI questions become DuckDB queries.
Conversational analytics and visualization use the same query path as everything else. The model reasons over governed metadata to write the query; it is never given the data. The query then runs through the same compiler and row- and column-level access as a dashboard or an API call.
Question in, with the asker's role
A router sends the question to the right agent, carrying who is asking.
The model sees metadata, not data
Metric and entity definitions, relationships and the glossary. Never rows.
Compiled under the user's access
The compiler adds the row and column filters, so a prompt cannot widen what a role can see.
Run on the files, returned as a chart
DuckDB runs the query in place; the answer comes back as a table or chart.
Exports
Large exports at constant memory.
Exports matter when a dataset leaves the platform: TXT or XML files for regulatory submissions, JSON for API consumers, or CSV, Parquet and Excel for analysis elsewhere. DuckDB writes the output directly to object storage; the application coordinates the job and never holds the full result set. Peak memory is set by configuration, not by export size.
Request deduplication
Requests are hashed, including the user's row and column access. In-flight jobs are joined and finished files served from cache, with configurable retention.
Asynchronous execution
The API returns a job ID at once and the export runs as a background job, so gateway timeouts don't apply.
Projection and predicate pushdown
Columns, filters, business date and row limit are compiled into the query, so unneeded row groups are never read.
Streaming writes
DuckDB writes text, JSON and Parquet straight to object storage as a multipart upload, with no staging file.
Bounded Excel writes
XLSX uses XlsxWriter's constant-memory mode, fed in batches sized from a memory budget, with Excel's row limit applied in the query.
Single job record
One record per export tracks status, row count and output location. Failures are recorded and temporary files removed.
Common questions
Frequently asked questions.
Can we query data without loading it into a database?
Yes. The same SQL the platform's pipelines use runs directly on the Parquet files in storage, on any table type and any version. Nothing is imported into a database or warehouse, and every query still applies your row- and column-level access and is logged.
Can analysts query earlier versions of a table?
Yes. Any table can be queried as it stands now, or as it was at an earlier version or business date, directly on the files and under the same row- and column-level access.
Does the AI model see our data when we ask questions in chat?
No. For conversational analytics and visualization, the model only sees governed metadata: metric and entity definitions, relationships and the business glossary. It uses that context to write the query, which is compiled with the user's row- and column-level access and run by DuckDB on the files, the same way a dashboard query runs. The model has no access path of its own, so a prompt cannot widen what a user's role can see.
Can we export millions of rows to CSV or Excel?
Yes. Source files and flattened tables export to CSV, TXT, JSON, Parquet or Excel. DuckDB writes text, JSON and Parquet directly to object storage as a multipart upload. XLSX uses XlsxWriter in constant-memory mode, up to Excel's row limit. Exports run as background jobs, and identical requests with the same access are served from cache.
Why doesn't a large export run out of memory?
No step holds the full result set. Projection and predicate pushdown limit what is read, DuckDB streams text, JSON and Parquet output to object storage, and XLSX is written one batch at a time, with batch size derived from a memory budget. Peak memory is set by configuration, not by export size.
Talk to us about your compute workloads.
Send us one report, reconciliation or pipeline that takes too long or keeps failing today. We'll run it on the platform and show you the execution path, the memory it needed and the time it took.
- Benchmarked on your data shape, not a demo set
- A walkthrough of the engine and scheduler for your architects
- A comparison with your current infrastructure spend