dcdb v0.1.0 · MIT · go 1.18, stdlib only · README · LICENSE

dcdb

tl;dr — filter, sort, join, hash, save. encrypted files. no server, no database, no dependencies.

dcdb is a tiny scripting language for small datasets. you keep a few hundred or few thousand rows in an encrypted .dcdb file and poke at them with short scripts or the console you get for free when you run it with no arguments.

it's not a database. it's closer to awk with a better memory and one party trick: the files are encrypted at rest.

why does this exist? because the same five jobs kept coming up — filter a table, sort it, join two files on a key, print a digest, save it back — and i got tired of pulling in a whole dataframe library every time. so i nerded myself into a weekend project that's now about 2,900 lines of standard-library go, six files, zero dependencies. it also runs in a console, because typing load users beats remembering a flag any day.

jump to

how you run it

any go toolchain works. i test with gccgo 1.18, but plain go is fine.

$ go build -o dcdb ./src
$ ./dcdb -p evening.dc          # run a script, dump the table after
$ ./dcdb -k vault-key secret.dc   # pass the encryption key
$ ./dcdb                          # or just get the console

flags come in short and long flavors — -p/--print, -t/--time, -k/--key, -dp/--data-path, -op/--output-path, -v/--version, -h/--help. nothing clever.

the console

run it bare and you get a prompt. every line is a command, data commands print the table back at you, and a typo doesn't kill your session. multi-line blocks give you a ... continuation prompt, node-style.

$ ./dcdb -k daily
dcdb 0.1.0 - built-in query console

> load orders
> only total >= 50
> show id total
> save big-orders
> .exit

get out with .help, .exit, or plain ctrl-d.

the commands

six operators, that's the whole set: > >= < <= == !=. values are auto-typed — ints, floats, bools, strings, and null for "this field isn't here". line comments start with //. the ordering operators (> and friends) want numbers or null; ordering anything else is an error, not a silent false. that one's a feature, trust me.

rows

• load <name> — read data/<name>.dcdb, or whatever path you hand it
• use <a.b> — walk into a nested object; arrays become rows, a map or leaf becomes one value row
• only <f> <op> <v> / delete <f> <op> <v> — keep / drop matching rows
• sort <f> [asc|desc], take <n>, reverse, clear
• add <f>=<v> ... — append a row; update <f> <v> <wf> <wv> — set a field where another one matches
• save <name> — write it back, encrypted. defaults to data/, same as load.
• show [f...], drop <f> ..., rename <old> <new>, unique <f>
• join <name> [left] on <lf> <rf> — hash-join the current table with another file; add left to keep unmatched rows. one merged row per match, and on name collisions the left side wins.
• index <f> / index drop [f] — an in-memory lookup on a field; only == and update == use it. anything that reshapes the table drops it, and add keeps it fresh.

flow & variables

• print, printvar <v>, set <v> <val>, echo <text>
• hash <text> — print the sha256 hex digest (see the password recipe below)
• while <v> <op> <v2> ... end, for <v> <from> <to> [step] ... end, if ... [else] ... end
• proc <name> ... end + call <name> — little subroutines; redefining a name replaces it
• include <file> — run another .dc file relative to the script, depth-capped so it can't loop forever
• ++ <v> -- <v> + <v> <n> - <v> <n> * <v> <n> / <v> <n> — mutate a variable in place
• key <pass>, sleep <ms>, exit (run is reserved for later, i have plans)

transactions

• begin — snapshot the table, the variables, and the current document
• commit — keep it. rollback — undo it.
• and if a runtime error happens inside begin ... end, it rolls itself back to the begin. that's the whole feature, and it's exactly what batch imports want.

if you remember one thing: load users then save users is a round trip. both default to data/.

stuff you'd actually do

these are the ones i use for real.

the evening report

join today's orders to the customer list and rank them. the whole script is six lines:

// evening report - join today's orders to customers
load customers
join data/orders.dcdb on id user_id
sort price desc
show name item price

$ ./dcdb -p evening.dc
[
    {
        "item": "lamp",
        "name": "Cara",
        "price": 60
    },
    {
        "item": "shirt",
        "name": "Alice",
        "price": 25
    },
    {
        "item": "hat",
        "name": "Alice",
        "price": 15
    },
    {
        "item": "mug",
        "name": "Bob",
        "price": 4
    }
]

import or nothing

batch imports either fully land or fully roll back. if the second add had blown up, the first one would've vanished too.

begin
load invoices
add id=5512 total=120
add id=5513 total=45
save invoices
commit
echo imported 2 invoices

not storing raw passwords

hash prints a sha256 digest. store that instead of the plaintext, obviously.

// myhash.dc
hash abc

$ ./dcdb myhash.dc
ba7816bf8f01cfea414140de5dae2223b00361a396177a9cb410ff61f20015ad

one account, fast

build an index once, and the equality filter and the update use it instead of scanning the whole table.

load accounts
index email
update status "blocked" email "jim@x.io"
only status == "blocked"
print

backups and checksums

$ cp work/orders.dcdb backups/orders-$(date +%F).dcdb
$ sha256sum backups/orders-*.dcdb
0b31f7fdcc5c179407873f2a8b22e3ae46e6ac8027a4ff444838cd7fac9d6a94  backups/orders-2026-09-23.dcdb

and on load, aes-gcm authenticates the whole file anyway, so a corrupt or wrong-key backup fails loudly instead of loading garbage.

what the files look like

deliberately a small binary format, not json:

DCDB | 03 | salt(16) | iv(12) | aes-256-gcm payload + tag(16)

a real one of mine, the top of it:

$ od -A d -t x1 data/users.dcdb | head -3
0000000 44 43 44 42 03 e0 ad 3b 40 01 ca b1 12 60 cf a6
0000016 7f d4 5f 53 d5 50 36 59 af 67 2a b0 7a d9 37 db
0000032 13 37 e6 99 00 cf 9d b0 59 34 0d 73 e6 fc 93 b8

$ strings data/users.dcdb
DCDB
6^&K
iu-Lu;
XN3>

no names, cities, or json survive — just the magic string and ciphertext noise. old-format files (v1 plaintext, v2 aes-128-ctr + crc32) still load; saves always write v3. and writes go through a .tmp file plus a rename, so a crash mid-save doesn't leave you with half a file.

security, such as it is

• key derivation: pbkdf2-hmac-sha256, fresh 16-byte salt per write, 65536 iterations, 32-byte key
• cipher: aes-256-gcm — authenticated, so tampering and wrong keys fail instead of surprising you:

$ printf 'load vault\n' | ./dcdb -k wrong
> Runtime Error: wrong key or corrupt database file

perspective: this keeps the files from being casually readable, and the kdf makes cheap brute force pointless-ish. it is not a hardware vault. if somebody has your laptop and your passphrase on a sticky note, all bets are off. standard advice applies: long passphrase, don't reuse it.

where it gets weird

scripts are preflighted — the whole file is parsed before anything runs. a parse error means nothing executes, but you get every error in one pass, so it's one fix-up trip instead of whack-a-mole:

$ cat broken.dc
loda accounts
spin invoice
$ ./dcdb broken.dc
Parser Error: line 1: unknown command "loda"; Parser Error: line 2: unknown command "spin"

• runtime errors stop the script; inside a transaction they roll back to begin
• ordering against a non-number is an error — cannot order string against int — not a silent false
• exit stops the script cleanly and keeps the changes made so far
• past about 1024 rows, filters and field rewrites run across your cores, order-preserving
• things i did not build and probably won't: csv import, network access, an on-disk index, concurrency, a query planner. it's a scripting tool; the dataset lives in memory anyway.

the files, with receipts

grabbed this repo from somewhere and want to be sure it's the same thing i'm looking at? here are the sha256s:

    src/main.go       bada596c1825c61d43e93750dc547efbb024e70468c39bbfe72b49d98e9bebcb
    src/lexer.go      32149d3591f2fe6d292141b5dc65fb47f917581ada02f1cc6bdd176daeb429e0
    src/value.go      464bc9729a492578a17cb2da705550e5e8cf4363dc0f714fe7ecb0d66069f17f
    src/interp.go     220bbf90af3be129b63bb92e0101e8482e1101ac55de803ade485b8d352f5706
    src/store.go      9a66a2c0481dff9633cb0364d2a8461f33ca9b5d37317b440c2e2636fe5eafee
    src/db_test.go    cfbbbd842ec4545e13ba2352e88a5e5bbdae4f184e682997effd2435c927fded
    README.md         0e24554139e2d59ee88e868d68c1696655a5c4bd3c6f9f6d334254279b8bc657