fronk
A 125B mixture-of-experts model, running on the computers you already own.
Every expert, cut into twenty slices of 32 neurons.
The film
It starts silently when you reach it: turn the sound on for the music. Installer, console and chat are real captures; the chat plays at its original speed.
Install Fronk
Fronk comes with an installation procedure for your coding agent. You need no networking skills: just read the short guide on how to connect your computers.
Each expert in the model is a small network of 640 neurons. fronk doesn't hand whole experts to machines. It cuts every one of them.
Twenty slices.
The 640 neurons become twenty groups of 32. One group, highlighted, is slice 7: neurons 224 to 255.
One slice runs through the whole model.
32 neurons × 512 experts × 48 layers = 1 slice · 3.35 GB
Every node holds one or more.
Twenty slices make the whole model. A node's share of the work is simply how many slices it holds.
The head is a Mac that runs the dense part of the model on its GPU. At every MoE layer it hands the expert work to the cluster in a single call, waits for the partial sums and carries on.
A node is any Linux or macOS machine running fronk node. It keeps its slices in memory, or uses a measured RAM + SSD split with Metal, and answers each request with a partial sum.
Every start of the head runs the same seven phases. Scroll through them: one cluster, one scene, each phase building on the last.
The head announces itself on every interface. Each node answers, proves it knows the cluster secret, and every path to it is timed. The fastest one wins.
Each node reports its hardware and saved compute profile. Installation measures the chosen backend; compatible profiles are reused when reconnecting. Network paths are measured again.
The head fits the twenty slices to measured speed and memory, shows each node's neuron ranges and link, and waits for you to proceed or rescan.
The head cuts the model file into slices and streams to each node only the ones it doesn't already hold. The node checks every byte before it loads a slice into memory.
A distributed expert calculation is compared with a scalar reference read from the model file. If the error is too large, generation does not start.
For every layer of every token the head sends one request and sums the partial answers. Every assigned slice must reply before a complete result can reach the dense engine.
The head collects every node's counters and reports tokens per second, call times, the deadline and the bottleneck. Move a cable or add a machine, and the next run starts from phase one.
The same request goes to every node. Each node answers with its partial sum. The expert calculation is complete when all assigned partial sums arrive. Missing work is an error, not an approximate answer.
per token each way. Activations and partial sums travel in fp16.
verifies drafted tokens together. Useful speed depends on acceptance and the cost of that verification.
UDP retransmissions. Late replies are counted; incomplete sums cannot become a chat answer.
A request for one layer is about 11 KB. The wire carries it in well under 100 µs. What took longer was asking the operating system to send it, once per packet.
Every step but the last is paid again for each packet. Sending eight packets pays it eight times.
| How it is sent | Send time |
|---|---|
| 8 packets, unconnected socket before | 315–460 µs |
| 1 datagram, unconnected socket | 80–130 µs |
| 1 datagram, connected socket now | 36–54 µs |
If one of the frames the kernel cut is lost, the whole message is lost. Ordinary chat requires complete sums: a missing reply is reported as a failure. A dedicated wired network reduces this risk.
Every path to every node is timed with packets the size of real requests. The measured path wins. Separate Thunderbolt links and wired Ethernet usually give more predictable results than Wi-Fi.
While the model generates, threads wait without sleeping, because a wake-up costs more than the work. After 50 ms of silence they park.
Large models can be spread across machines in several ways. A mixed cluster serving one conversation has different constraints from a datacenter serving many users. Here is where the waiting comes from.
A token visits the machines one after another. At any moment one works and the rest wait, so more machines add memory but not speed.
Common tensor-parallel layouts exchange intermediate results several times per layer. Fast links and balanced participants help; on a mixed home network, synchronization can dominate.
The router decides who works. With one user, a few machines hold the chosen experts while the others idle. The busiest sets the pace, and each must fit entire experts.
Every token reads gigabytes of experts. A disk delivers a few gigabytes per second, so the disk, not the processor, sets the speed.
fronk uses a narrower split: expert neurons, while the dense part stays on the head. Every machine holds the same 32 neurons of every expert, computes its slice of whichever experts the router picks, and sends back one partial sum.
Like tensor parallel, all machines compute at once and their speeds add up. Unlike expert parallel, the load follows the plan, not the router, so every assigned node contributes. The slowest completion still sets the pace.
One request out, one partial sum back, about 5 KB each way per token. The head adds the partial sums. Separate Thunderbolt links or an asymmetric Ethernet switch can make these repeated returns affordable.
A 3.35 GB slice gives each backend a small, regular piece of every expert to compute. The plan sizes each machine's share to its memory, speed and link.
A Linux tower, an older Intel iMac and an Apple silicon laptop can serve the same model side by side. Each holds as many slices as its memory allows.
Room for 4 slices after the system and a safety margin.
Room for 13 slices. The same kernels, just more space.
Room for 3 slices. macOS and the GPU keep more for themselves.
Memory sets how many slices a machine can hold.
The cable matters as much as the memory. Every call goes out and comes back once per layer, so the time a link adds is paid on every token. fronk measures each one with requests of the real size.
Any desktop or adapter. One multicast send can reach the nodes on a shared switch. Replies still share the head link: prefer a faster head uplink, for example 10 Gbit/s to 1 Gbit/s nodes.
Use a separate head Thunderbolt port for each direct Mac connection. Each node gets its own path; mixed Thunderbolt and Ethernet are supported.
An illustrative low-latency path for the planning example below. Fronk does not implement RDMA; this is not a measured result or a supported installation option.
Round trip of one request and its reply. Ethernet and Thunderbolt ranges describe the test cluster; the RDMA value is a hypothetical planning input.
A call lasts as long as its slowest node, and each node spends network time plus compute time. fronk times every link during Connection and plans with both numbers, to balance completion times within the available memory. Metal nodes can also report a measured RAM + SSD cost.
Illustrative timings, not a prediction for your cluster. The light segment is network, the solid one is compute.
This is the first public version. The mixed test cluster reached 15–16 tok/s on short warm responses; your machines and workload will differ. Bugs and useful surprises are both worth sharing.
Start with the troubleshooting guide. If you want help from a frontier model, give it the ready-made brief for the step that failed: installation, networking, weights or runtime.
Read the project map, add a backend module, or connect your chat client or agent. Each area has a matching brief for a coding model.
See the actual cluster and measurements, share your configuration and results, or send a focused fix using the contribution guide.