Technology on precomputing.com

Introduction

Fri, 25 Sep 2026 00:00:00 +0000

Answers ready before you ask: one small language keeps counts, percentiles, candles, invoices and log dashboards current in a SQLite file as the data arrives.

Precomputing is a project to make answers cheap. A short policy names the streams of events you have and the answers you want ready, and says how long each level of detail should live. From then on every event updates those answers the moment it arrives, so a question becomes a lookup, and old detail fades on the schedule you set.

How It Works

Fri, 25 Sep 2026 00:00:00 +0000

The idea. Decide the questions first. Keep their answers current as each event arrives, in one SQLite file, and let raw detail fade on a schedule while unusual events stay whole.

Before and After

Today a system that collects events usually keeps all of them and computes each answer when someone asks. A dashboard scans the last hour on every refresh. An invoice adds up a month of requests at the month’s end. Precomputing moves the work to the moment each event arrives:

The Policy Language

Fri, 25 Sep 2026 00:00:00 +0000

A policy is a short text file. It names the streams of events you have and the answers to keep ready, and says how long each level of detail survives.

The compiler turns it into plain SQLite, and the Engine runs it as it is. This post walks through a policy line by line, and shows what each line becomes in the file.

Precomputing Explained: Keep a Running Total Instead of Counting Again

Tue, 29 Sep 2026 00:00:00 +0000

A lot of software that collects data works like a shop owner with a shoebox. Every receipt goes in the box. When someone asks how much the shop sold today, she tips the box out and adds everything up, and if they ask again an hour later she does it all over again, with a bigger pile.

Precomputing works like a notepad next to the till. Each sale is added to a running total the moment it happens, so a question gets answered by reading one number. Now and then the box can be emptied. She keeps only the odd receipts (a huge refund, a sale at 3 a.m.) in case someone asks about them later.

Inside the Engine: Same Answers, 12 to 17 Times Faster

Fri, 25 Sep 2026 00:00:00 +0000

The SQL runtime does all its work inside SQLite: every insert runs a trigger that updates every summary, sketch and answer the policy asks for. That is simple and it runs anywhere, but a trigger pays SQLite’s price for each small update. The Engine runs the same policy in Go, keeps the recent work in memory and writes what changed to the same SQLite file in one transaction at every checkpoint. Its answers are the same, to the last bit, and on Demo 2’s trades it is 12 to 17 times faster.

One File: The SQLite Layout Both Runtimes Write

Fri, 25 Sep 2026 00:00:00 +0000

A Precomputing file is an ordinary SQLite file. Any SQLite tool opens it, the same SQL works on it whichever runtime wrote it, and it carries its own policy, so a reader never has to guess what is inside. This post walks through the layout for one stream.

What Is in the File

For Demo 1’s stream, latency, with the key endpoint and the value ms: