Koko Data Access — Quickstart

About

Platform databases (MongoDB, Redis, Neo4j) run inside the Kubernetes cluster and are not reachable from localhost. The `koko` CLI proxies all operations through the Koko API — authenticated, read-safe, and type-preserving.

Skill Content

This is the raw markdown that gets installed as a Claude Code rule.

# Koko Data Access — Quickstart

Platform databases (MongoDB, Redis, Neo4j) run inside the Kubernetes cluster and are not reachable from localhost. The `koko` CLI proxies all operations through the Koko API — authenticated, read-safe, and type-preserving.

## Koko First

**koko is the default for every read and write on a Tawa service database** — any database declared in a service's `catalog-info.yaml` (MongoDB, Redis, Neo4j), in prod, sandbox or uat.

Before you tell a user you have no database access, or ask them to change data by hand (for example a setting in the app's settings collection), try it:

```bash
koko databases -s <service>
koko collections -s <service> -e prod
```

Writes still follow [Editing Prod Data](#editing-prod-data--key-on-_id-and-assert-the-count): back up first, key on `_id`, `--expect` the count, and report the write.

## Setup

```bash
# Install
npm install -g koko-db

# Authenticate (shares your tawa login session — run tawa login first)
tawa login
koko --version   # should be 0.5.0+
```

## The One Rule

**Never use curl, kubectl port-forward, or raw API calls to access databases.**

| Wrong | Right |
|-------|-------|
| `curl http://localhost:3001/api/db/...` | `koko query users -s my-api -e prod` |
| `kubectl port-forward ... && mongosh` | `koko copy users --from prod --to sandbox` |
| Direct MongoDB connection strings | `koko export → koko import` |

The koko CLI handles auth, safety guards, EJSON type preservation, and pagination automatically.

> Writing to prod? Read [Editing Prod Data](#editing-prod-data--key-on-_id-and-assert-the-count)
> first. The guards above do not catch a filter that matched more rows than you meant — only
> `--expect` does.

## Service Name

koko resolves your service name automatically from the project directory. You can also pass it explicitly:

```bash
# From inside a service directory (reads catalog-info.yaml or .tawa.yaml)
koko query products -e prod

# Explicit service name
koko query products -s fairshare-cms -e prod
```

## Inspect

```bash
# List databases provisioned for a service
koko databases -s my-api

# Collection stats
koko stats -s my-api -e prod
koko collections -s my-api -e prod

# Sample a few documents (random)
koko sample orders -s my-api -e prod --limit 5
```

### One row per database type

A service that declares `mongodb`, `redis`, and `neo4j` in `catalog-info.yaml`
shows **all three** in `koko databases` — each as its own row. They share the
default name `{service}-{env}` but are distinguished by **type**, so
`koko tunnel` / `koko query` can reach the MongoDB (and Neo4j) even when a Redis
binding is also present. If you only see one row (e.g. just `redis`) after a
deploy, your Koko predates the type-keyed registry (iec-builder#38) — redeploy
after the platform is updated and all declared databases will register.

## Query

```bash
# Find with filter
koko query users -s my-api -e prod \
  --filter '{"status":"active"}' \
  --limit 20 \
  --sort '{"createdAt":-1}'

# With projection
koko query products -s my-api -e prod \
  --filter '{"active":true}' \
  --projection '{"name":1,"slug":1,"tiers":1}'

# Aggregate
koko query orders -s my-api -e prod \
  --aggregate '[{"$group":{"_id":"$paymentStatus","count":{"$sum":1}}}]'

# JSON output (for piping/scripting)
koko query users -s my-api -e prod --filter '{"email":"[email protected]"}' --json
```

## Export / Import — EJSON Format

`koko export` outputs **EJSON** (MongoDB Extended JSON). ObjectIds are serialized as `{"$oid":"..."}` and Dates as `{"$date":"..."}` — not plain strings. This means the NDJSON file can be re-imported with correct BSON types and no corruption.

```bash
# Export a collection to file
koko export products -s my-api -e prod -o /tmp/products.ndjson

# Export with a filter
koko export orders -s my-api -e prod \
  --filter '{"createdAt":{"$gte":"2026-01-01"}}' \
  -o /tmp/recent-orders.ndjson
```

Import via `koko import` — EJSON is handled natively, no flags needed:

```bash
koko import products -s my-api -e sandbox -i /tmp/products.ndjson --upsert
```

If you must use `mongoimport` (into local MongoDB), add `--legacy` so it recognises `$oid`/`$date` markers:

```bash
mongoimport --db my-api-dev --collection products \
  --file /tmp/products.ndjson --legacy
```

> **Prefer `koko import` over `mongoimport`** — no flags to remember, round-trip is always correct.

## Copy Between Environments

Server-side copy — no download, no EJSON conversion, fastest option:

```bash
# Preview count first
koko copy products -s my-api --from prod --to sandbox --dry-run

# Execute
koko copy products -s my-api --from prod --to sandbox
```

## Sync Prod to Local — Full Workflow

When you need your local MongoDB to exactly match production:

```bash
# Step 1: Export all collections from prod
koko export products -s my-api -e prod -o /tmp/products.ndjson
koko export users    -s my-api -e prod -o /tmp/users.ndjson
koko export orders   -s my-api -e prod -o /tmp/orders.ndjson

# Step 2: Import into local MongoDB (--legacy handles $oid/$date markers)
mongoimport --db my-api-dev --collection products --file /tmp/products.ndjson --legacy --drop
mongoimport --db my-api-dev --collection users    --file /tmp/users.ndjson    --legacy --drop
mongoimport --db my-api-dev --collection orders   --file /tmp/orders.ndjson   --legacy --drop

# OR: sync to sandbox via koko (skip --legacy, it's handled server-side)
koko import products -s my-api -e sandbox -i /tmp/products.ndjson --upsert
koko import users    -s my-api -e sandbox -i /tmp/users.ndjson    --upsert
```

After importing to local, restart your dev server to clear any in-memory caches:

```bash
# If using Next.js
rm -rf .next && npm run dev
```

## Seed Collections

For development seeds committed to the repo:

```bash
# Seeds live in ./seeds/<collection>.ndjson
koko seed -s my-api -e sandbox

# Single collection
koko seed -s my-api -e sandbox --collection products

# Dry-run
koko seed -s my-api -e sandbox --dry-run
```

## Redis

```bash
koko keys      -s my-api -e prod --pattern 'session:*' --limit 100
koko get       -s my-api -e prod 'my-api:some-key'
koko redis-info -s my-api -e prod
```

## Neo4j

```bash
koko cypher "MATCH (n) RETURN labels(n), count(n) LIMIT 10" -s my-api -e prod
koko neo4j-stats -s my-api -e prod --json
```

Koko connects to Neo4j as the service's own `svc_*` user. The builder registers the
database with an authenticated `bolt://user:pass@host:port` URI on every deploy, and Koko
passes those credentials to the driver as an auth token (the Neo4j driver ignores URI
userinfo by itself).

> **Correction (2026-10-03):** before iec-builder#228 / iec-koko#47, the builder registered
> Neo4j as a bare `bolt://host:port` and Koko opened the driver with no auth token, so both
> commands failed with `An unexpected error occurred` (server log: `Unsupported
> authentication token, scheme 'none'`) on every service. **Redeploy** a service that
> declares `neo4j` to re-register it with credentials. Nothing is backfilled.

## Safety Guards

All koko operations are read-only by default:

- MongoDB: `$out` and `$merge` aggregate stages are blocked
- Cypher: `CREATE`, `MERGE`, `DELETE`, `SET`, `REMOVE`, `DROP` are blocked
- Max 1,000 documents per query response
- 30-second query timeout

Write operations (`koko import`, `koko copy`, `koko seed`) require explicit `--upsert` or `--allow-prod` confirmation for production targets.

## Editing Prod Data — Key on `_id`, and Assert the Count

The export → modify → import loop is how prod data actually gets fixed, and it has one
failure mode that none of the guards above catch: **a filter that quietly matches more
rows than you meant.** `koko import --upsert` replaces whole documents keyed by `_id`, so
every extra row your filter caught gets rewritten too.

A real instance: an export filtered on `{"ticketNumber":7}` returned **9** documents, not
1 — ticket numbers were unique per *space*, not globally. Importing that file back would
have replaced 8 unrelated tickets in other spaces. Nothing was broken; every step printed
its count correctly. Nobody compared those counts to an expectation, because nothing asked
for one.

### Assert the count with `--expect` (koko 0.8.0+)

```bash
koko import tickets -s my-api -e prod -i fixed.ndjson --upsert --allow-prod --expect 1
# ✗ Refusing to import: the input holds 9 document(s), but --expect 1 was given — more than intended.
```

Pass it on every prod import. It costs one integer and turns "I assume this is one row"
into a claim the tool can refuse. Rehearse first — `--dry-run --expect n` performs no write
and works against prod, so you can check the count against the real data before touching it.

Check your version with `koko --version`; upgrade with `npm install -g kokopelli`.

### Prefer `_id`, and know that `--filter` cannot take one

`koko export --filter` does **not** accept `{"$oid": "..."}`. To pull an exact document,
match on the string form of `_id` through an aggregate:

```bash
koko query tickets -s my-api -e prod --json \
  --aggregate '[{"$match":{"$expr":{"$eq":[{"$toString":"$_id"},"6a5eba929c378de086202d91"]}}}]'
```

A field that *looks* unique often is not — it may be unique only within a parent (per
space, per org). Check the model's compound indexes before trusting one as a key.

### The loop, in full

```bash
# 1. Back up the untouched rows FIRST — this is your undo.
koko export <collection> -s <svc> -e prod --filter '<filter>' -o rows.backup.ndjson

# 2. Read the count it printed. Is it the number you expected? If not, STOP —
#    your filter is wrong, and no later step will tell you.

# 3. Modify a copy, never the backup. Diff each document field-by-field against
#    its original so only the fields you intend actually move.

# 4. Import, asserting the count.
koko import <collection> -s <svc> -e prod -i rows.new.ndjson --upsert --allow-prod --expect <n>

# 5. Verify with a query that is CAPABLE OF FAILING (see below).
```

### Make sure your verification can fail

A check that cannot fail looks exactly like a check that passed. The common trap is
comparing a date against a **string** in an aggregate — it matches nothing, every time,
and reads as a clean result:

```bash
# ✗ WRONG — matches nothing regardless of the data
--aggregate '[{"$match":{"archivedAt":{"$gte":"2026-09-03T22:44:00Z"}}}]'

# ✅ RIGHT
--aggregate '[{"$match":{"$expr":{"$gte":["$archivedAt",{"$dateFromString":{"dateString":"2026-09-03T22:44:00Z"}}]}}}]'
```

Before trusting a verification, ask what it would print if the change had gone wrong. If
the answer is "the same thing", it is not a verification.

## EJSON — Why It Matters

MongoDB stores ObjectIds and Dates as special BSON types (not plain strings). When exported as plain JSON, `ObjectId("abc123")` becomes the string `"abc123"` — which looks valid but fails as a MongoDB `_id` on re-import because the type is wrong.

EJSON preserves the type information:

```json
{ "_id": {"$oid": "abc123..."}, "createdAt": {"$date": "2026-01-15T..."} }
```

`koko export` always outputs EJSON. `koko import` always deserializes it. The round-trip is type-safe by default — no manual conversion scripts needed.

Install

Copy the skill content and save it to:

~/.claude/rules/tawa-koko-data-access.md

Details

Format

Rule

Category

provision

Version

1.0.32034

Tokens

~2,802

Updated

2026-10-13