Pure Lua 5.4 · binary protocol over TCP · MIT licensed

A message broker, written in Lua.

MoonMQ is a log-structured, partitioned broker. Produce, fetch or subscribe, run consumer groups and transactions, and replicate across a cluster — all in a codebase you can read in an afternoon.

$git clone https://github.com/HilthonTT/MoonMQ && cd MoonMQ && make deps && make run

Get started Star on GitHub

How it works

Topics are partitioned logs.

Producers append records to a partition chosen by key. Each partition is an ordered, durable log. Consumer groups split partitions between members and commit how far they have read, so a restart picks up where it left off.

Producers

checkout

payments

Topic orders · 4 partitions

Append-only, CRC-checked segments with offset and time indexes.

Group billing

member-a

p0 · p1

member-b

p2 · p3

Features

The parts of Kafka you actually use, without the JVM.

Durable storage, exactly-once building blocks, replication with failover and a real security model, each covered by its own spec suite.

Storage

Segmented log with time and offset indexes, group-commit fsync, retention and crash recovery. A jocko-style backend adds key compaction.

Delivery

Pull with FETCH or push with SUBSCRIBE. Record headers, consumer groups with durable offsets, a DLQ with NACK, and batching on both sides.

Correctness

Idempotent producers, multi-partition transactions and read_committed consumers.

Replication

Raft-elected leaders, an in-sync replica set for acks=all, a high watermark, epoch-based truncation and clients that follow the leader.

Cluster

Static peers, an autobalancer, live partition migration, cluster-wide groups and an optional Raft metadata controller.

Security

TLS and mTLS with SIGHUP reload, per-user ACLs, SCRAM-SHA-256 with channel binding, SASL EXTERNAL and quotas.

Operations

Prometheus /metrics, JSON /stats, per-IP lockout and an SQL-like console for poking at a live broker.

Read how it fits together

Architecture

Use it

From Lua, or from the console.

The reference client covers produce, fetch, subscribe, groups, transactions and admin calls. The MQL console speaks a small SQL-like language to a running broker.

client.lua

local Client = require("src.client")
local c = assert(Client.new{ host = "127.0.0.1", port = 9092,
                             username = "admin", password = "admin" })

assert(c:produce("orders", "key-1", "payload", { ["trace-id"] = "abc123" }))
local res = assert(c:join_group("billing", { "orders" }))

lua5.4 main.lua --repl

mq> CREATE TOPIC orders PARTITIONS 4;
OK topic 'orders' created (4 partitions)

mq> PRODUCE INTO orders KEY 'k1' VALUE 'hello world';
OK produced to orders → partition 2, offset 0

mq> COMMIT orders PARTITION 2 OFFSET 1;
OK committed orders[2] @ offset 1

Client API MQL reference

Run a broker in under a minute.

Install Lua 5.4 and a few rocks, then start it on port 9092. Turn on TLS and credentials before it leaves your machine.

$git clone https://github.com/HilthonTT/MoonMQ && cd MoonMQ && make deps && make run