MoonMQ
Documentation menu

Introduction

MoonMQ is a log-structured, partitioned message broker written in pure Lua 5.4. Clients speak a compact binary protocol over TCP to produce, fetch or subscribe, manage topics and commit consumer offsets. Topics are partitioned append-only logs with CRC-checked on-disk records.

What you get

StorageSegmented log with time + offset indexes, group-commit fsync, retention, crash recovery; jocko-style backend with key compaction
DeliveryPull (FETCH) or push (SUBSCRIBE), record headers, consumer groups with durable offsets, DLQ with NACK, batching on both sides
CorrectnessIdempotent producer, multi-partition transactions, read_committed
ReplicationRaft-elected leader, in-sync replica set for acks=all, high watermark, epoch-based truncation, client leader-following
ClusterStatic peers, autobalancer, live partition migration, cluster-wide groups, optional Raft metadata controller
SecurityTLS/mTLS with SIGHUP reload, per-user ACLs, SCRAM-SHA-256 with channel binding, SASL EXTERNAL, quotas
OpsPrometheus /metrics, JSON /stats, per-IP lockout, SQL-like console (MQL)

Quick start

sudo apt install lua5.4 liblua5.4-dev luarocks lua-socket libssl-dev zlib1g-dev
make deps          # Lua dependencies
make run           # or: lua5.4 main.lua  (--repl for the MQL console)
local Client = require("src.client")
local c = assert(Client.new{ host = "127.0.0.1", port = 9092,
                             username = "admin", password = "admin" })

assert(c:produce("orders", "key-1", "payload", { ["trace-id"] = "abc123" }))
local res = assert(c:join_group("billing", { "orders" }))

With no credential configured the broker starts open, every listener is plaintext until a Tls block says otherwise, and the metrics endpoints are unauthenticated by default. See Security.

Testing

make check         # luacheck + busted + storage smoke test — what CI runs
make test          # busted only

Guides

  • Installation — Dependencies, fallbacks, the luaposix build and Windows.
  • Configuration — appsettings.json, connection limits, password hashes, cluster and replication blocks.
  • Client API — The reference Lua client: produce, fetch, groups, lag, seeking by time, admin calls.
  • Record headers — Header API, limits, on-disk and wire encoding.
  • Batching — Batch formats on the produce and fetch sides.
  • Dead-letter queue — NACK, retries and dead-letter topics.
  • Transactions — Idempotent producers, multi-partition transactions and read_committed.
  • MQL console — The SQL-like interactive console and its grammar.
  • Security — TLS, users, ACLs, SCRAM, quotas and metrics auth.
  • Replication — Replication, failover and in-sync replicas.
  • Clustering — Clustering, reassignment, the autobalancer and the Raft controller.
  • Architecture — How the broker is put together, and where to start reading.
  • TLS & consensus notes — Design notes behind TLS and controller consensus.
  • Contributing — Setting up, testing and sending changes.
Next Installation