System shape
MLabChain deliberately splits into two programs, each with a narrow responsibility and no overlap. The split exists because the two halves answer to different requirements:
- The scientific layer must be easy to change. New datasets, new verifiers, new training loops, new metrics are all normal evolution of a research tool. It should be written in the language most scientists already use: Python.
- The consensus layer must be stable. Every node has to agree on the exact same state transition for the same block. It should be written in a language with explicit control over integer arithmetic, memory layout, and error handling: C++.
Nothing in the C++ core knows what a "model" is. Nothing in the Python layer knows what a Merkle tree is. That is the whole design.
The Python layer
mlabchain.py is a single-file command-line tool. It
owns everything that touches scientific artifacts.
model.pkl, model.txt, and
proof.json to disk. Streams SHA-256 over any
file in 1 MiB chunks.
save_training_artifacts() · hash_file()
verify-ml command re-executes the recorded
training, recomputes metrics, and compares them against the
transaction. This is the audit path, not a consensus rule.
command_verify_ml()
The C++ core
mera_core is a single executable. It is the
authoritative implementation of every state transition on the
chain.
quality_ppm, delta_q, and the
final reward. Uses boost::multiprecision::uint128_t
for intermediate products.
quality_ppm() · work_reward_for_quality()
The boundary
The two programs communicate through a subprocess.
The Python layer invokes mera_core with command-line
arguments, and parses a simple KEY=VALUE response.
# Python calls the core
result = subprocess.run(
[core_path, "submit-ml",
"--sender", wallet.address,
"--challenge-id", challenge_id,
"--manifest-sha256", manifest_hash,
"--model-sha256", model_hash,
# ... 20+ more arguments ...
"--signature", signature],
text=True, capture_output=True, timeout=60.0,
)
# C++ replies in a simple key=value format
TXID=6a8b3c...
STATUS=accepted
Why a subprocess
- No FFI layer. A shared library would need a C ABI, symbol versioning, and careful memory management across the boundary. A subprocess needs none of that.
- Crash isolation. If the C++ core segfaults, the Python session survives with a clear error. If they were linked in-process, a bug in either side would take down both.
- Auditable. Every state transition is expressible as a shell command. Any reader can inspect exactly what the core was asked to do.
Costs of the subprocess
- Process startup. Each invocation spawns a new process and opens the SQLite database. For a single command this is a few milliseconds. For a tight loop it would be too slow — which is why the tool is not designed for tight loops.
- Argument limits. Very large transactions must be passed through temporary files or environment variables if they exceed the OS argument size limit. In practice the current transaction sizes are small enough that this has not been an issue.
-
Error propagation. The C++ core returns
non-zero on failure and writes the error message to stderr.
The Python side raises
RuntimeErrorwith the same message.
Storage model
State lives in a single SQLite database at
mera_data/mera.sqlite. SQLite is configured with
WAL mode, foreign keys enabled, and synchronous FULL.
Tables
| Table | Contents |
|---|---|
| meta | Network name, genesis hash, schema version, supply, parameters, governance config |
| accounts | Address, balance, nonce, current authorization public key |
| blocks | Block headers with producer signature and cumulative chain work |
| txs | Confirmed transactions, indexed by height and order |
| pending | Mempool, ordered by fee priority |
| challenges | Registered challenge state including best quality and budget remaining |
| work_claims | Reserved (challenge, dataset, model) tuples |
| gov_updates | Scheduled parameter changes with effective heights |
Indexes
Five indexes are maintained for the hot paths:
pending(sender, nonce)— for pending-aware nonce checkspending(fee DESC, seq ASC)— for fee-priority selectiontxs(height, ord)— for block replaytxs(sender, nonce)— for account history queriesblocks(height)— for chain traversal
Atomicity
Block commitment is a single SQLite transaction. If any part fails — replay, insertion, state update — the whole transaction rolls back and the block is not committed. This is what makes the database always consistent with the canonical chain.
Migration from v0.2
v0.3 does not silently interpret a v0.2 database. When the core
opens a database whose schema_version does not match
PROTOCOL_VERSION, it refuses with a clear error.
The migration path is a separate tool:
python tools/migrate_v2.py \
--input ./old/mera_data/mera.sqlite \
--output ./migration/v2_snapshot.json
The output is an explicit snapshot containing:
- Source schema version and network name.
- Old genesis hash and tip hash.
- Block and transaction counts.
- Per-account balance and nonce.
- Total balance.
- A snapshot SHA-256 over the canonicalised JSON.
Why an explicit snapshot
Silently replaying v0.2 history under changed consensus rules would be less safe than an explicit migration boundary. The snapshot documents exactly what was in the old chain, so a production migration can commit to it in a new genesis and import balances under human review. The old chain remains valid as an archival record.
Build system
The C++ core uses CMake. The Python side uses standard
pyproject.toml. On Linux and macOS the build is
driven by build.sh; on Windows by
build.ps1.
Dependencies
- OpenSSL — Ed25519 and SHA-256
- SQLite3 — state storage
- Boost — Asio for TCP, cpp_int for 128-bit arithmetic
- cryptography (Python) — Ed25519, Scrypt, AES-GCM
The build produces a single binary:
build/mera_core on Unix,
build/Release/mera_core.exe on Windows.
CMake presets
A CMakePresets.json provides a
release preset:
cmake --preset release
cmake --build --preset release
ctest --preset release
Container build
A Dockerfile is provided for reproducible builds
and CI. The container image includes the compiler toolchain,
all libraries, and the Python runtime.
Trust boundaries
Two programs means two sets of assumptions. The architecture makes both explicit.
| Component | Trusts | Is trusted by |
|---|---|---|
| Python client | The C++ core to enforce consensus | Nobody — it produces transactions that the core validates |
| C++ core | OpenSSL, SQLite, Boost | The Python client, and every other node on the network |
| Network peers | Nothing — all messages are untrusted input | Nothing — peers are anonymous |
| Verifier class | The node binary itself | The whole monetary system |
| Challenge author | Nothing beyond what the registry enforces | Nothing — a challenge is just a record |
The most important row is the verifier class. The entire
monetary mechanism depends on the correctness of
MLabChain-linear-v4, which is compiled into the
node binary. A bug there would allow invalid rewards. This is
why the core is small, and why the reward arithmetic uses integer
arithmetic only.
What is not in the trust boundary
The Python client's local reward estimate is not authoritative. It is a preview. The core recomputes and either accepts or rejects. If they disagree, the core wins — the Python side prints the core's error message and exits.
Take-away
Two programs, one boundary, no overlap. Python owns science. C++ owns state. The subprocess boundary keeps them honest with each other and keeps a bug in one from silently corrupting the other.