Quell: Winning Night Shift with Moderation That Learns

42 hours. 500 builders. 1st place.

Last weekend Shakoora Rajaee and I spent two nights on the ninth floor of a Canary Wharf office at Night Shift, London's biggest hackathon, built around the launch of Mubit. We walked out with first place and $15,000. This is what we built and how it works.

The Brief

Mubit sits between your app and its LLM calls. You change one base URL, keep calling OpenAI as usual, and Mubit records the calls, groups the repeated ones into tasks, and trains small open models on them. The brief was simple: build a product where your corrections turn into model updates for a task you run again and again.

We wanted a task that runs thousands of times an hour, where the right answer depends on who you ask. Chat moderation fits perfectly.

The Problem

Generic moderation models don't know house rules. One community bans selling in chat, another is fine with it. One treats "kill -9 the intern's process" as a joke, another doesn't. So moderators end up fixing the same mistakes every stream, and the model never gets any better.

What We Built

Quell is live-chat moderation that learns a community's own rules from its moderators. Moderators get a review queue, the live chat and the model's stats in one view. Viewers join from their phone by scanning a QR code.

Every correction does two things:

• It becomes a rule that applies from the very next message.
• It becomes training data, which Mubit turns into a small model of that community's own.

How It Works

1. One frozen task

Every message is judged with the same model, the same system prompt and the same JSON schema: allow, flag, remove or redact, with a reason. Mubit groups calls by that fingerprint, so all of Quell's moderation is one task to train on.

That makes the fingerprint sacred. The API refuses to start if the prompt or schema changes, because that would quietly create a new task and split the training data in two.

2. Only uncertain calls reach a human

Allowed messages, redactions and confident removals are settled automatically. Phone numbers and addresses are always redacted by a pattern check before the model even sees the message. Everything else goes to the review queue, so moderators only look where the model isn't sure.

3. A correction becomes a rule, instantly

When a moderator overturns a decision, they write the rule in their own words, like "no buying or selling in chat". That rule is sent with every later call, so the fix works from the next message. No waiting for a retrain.

4. A correction becomes training data

One correction is one example, which is too few to train on. So Quell finds recent messages that are similar to the corrected one using embeddings, and judges them again with the new rule. Mubit keeps those calls as training examples. One click from a moderator becomes a handful of good examples.

To keep the data clean, everything that isn't moderation (the simulated chat, embeddings, benchmark runs) is sent with recording turned off. Repeated messages, like a raid of 150 identical messages, are answered from a cache, so a raid costs a few calls instead of 150.

5. A model of your own

Once enough examples are kept, Mubit trains a Qwen3 adapter on them, tests it silently next to the main model, and only lets it serve live chat once it passes. By Sunday morning we had around 200 kept examples and our own adapter in testing, trained overnight.

Does It Work?

We tested it on 114 held-out messages that were never used for training:

• Four corrections took agreement with moderators from 83% to 90%.
• Four messages breaking house rules got through in every run before correction. After one correction each, new look-alikes (a GPU for sale, a job ad) were caught without a moderator.
• Scams caught: 14 out of 14.
• 300 concurrent requests through Mubit, with no rate-limit errors.

On the viewer side, a removed message disappears for everyone else, and only the sender sees why.

The Stack

• API: NestJS, with a bounded moderation queue, a duplicate cache and the correction replay.
• Web: Next.js 15, Tailwind CSS 4 and Motion, updated live over server-sent events.
• Models: every LLM call goes through Mubit's OpenAI-compatible endpoint. The trained adapter is Qwen3.

What I Learned

Pick a task that repeats. The whole idea only works because moderation runs on every single message. A task you call once a day gives you nothing to learn from.

Design for the training data from the start. Freezing the prompt, turning off recording for side calls and caching duplicates felt like overkill on Friday night. By Sunday they were the reason our data was clean enough to train on.

A correction is worth more than one example. Replaying similar messages with the new rule was the trick that turned a handful of clicks into something a model can learn from.

Huge thanks to Anil Peri, Eldar Utiushev, Lily Jahan and the whole Mubit team for running such a good event, and congratulations on the launch. And to Shakoora: thank you for the sleepless nights. Best teammate I could have asked for.

Mubit features Quell on its launch showcase, and the code is on GitHub.

Every correction should make the model better. Now it does.