# tinyforge
**Repository Path**: computerhong/tinyforge
## Basic Information
- **Project Name**: tinyforge
- **Description**: No description available
- **Primary Language**: Unknown
- **License**: Not specified
- **Default Branch**: main
- **Homepage**: None
- **GVP Project**: No
## Statistics
- **Stars**: 0
- **Forks**: 0
- **Created**: 2026-10-15
- **Last Updated**: 2026-10-15
## Categories & Tags
**Categories**: Uncategorized
**Tags**: None
## README
# ๐ฅ tinyforge
**A tiny model that teaches itself to get better โ at anything with a test.**
**On your laptop. 6GB RAM. No cloud. No teacher model. No human feedback.**
[](https://python.org)
[](https://support.apple.com/en-us/HT211814)
[](https://github.com/ml-explore/mlx)
[](LICENSE)
[]()
*We discovered that a **0.8B parameter model** can meaningfully improve itself by learning from its own failures โ using only **6GB of RAM** on a MacBook Air. We proved it on code. The technique works anywhere you can automatically verify the output.*
```bash
pip install -e .
tinyforge --model models/mlx-q4-qwen35-08b --quick
```
## ๐ง What this actually does
The model tries to solve coding problems. It fails. It sees *exactly* what failed (which test, what input, what it expected vs what it got). It tries again. When it finds a better solution, we extract the weak-to-strong pair and train the model on it.
> **This is self-play for code. The "game" is passing tests.**
## ๐ Key Results
> All results are on **fresh holdout slices** the model never saw during training.
### ๐ง The repair training effect
*Fresh HumanEval slice 40-47:*
| Setup | Public Tests | Hidden HumanEval+ |
|:---|---:|---:|
| Base model, single-pass | 16/50 | 2/8 |
| **Repair-trained, single-pass** | **28/50** | **4/8** |
| Base model + evolutionary search | 42/50 | 5/8 |
| **Repair-trained + evolutionary search** | **44/50** | **6/8** |
> ๐ **Single-pass improved 75%** (16 โ 28) just from training on 13 self-generated repair pairs.
### ๐ The feedback loop effect
*Fresh HumanEval slice 56-63 (overnight run):*
| Setup | Public Tests | Hidden HumanEval+ |
|:---|---:|---:|
| Base adapter + feedback loop | 42/58 | 4/8 |
| **Repair-trained + feedback loop** | **47/58** | **5/8** |
> ๐ก The trained model doesn't just write better code โ it becomes a **better participant in the repair loop**. It learns how to fix code when told what's wrong.
### ๐๏ธ Hard problems (slice 72-79)
| Setup | Public Tests | Hidden HumanEval+ |
|:---|---:|---:|
| Base model, single-pass | 28/70 | 0/8 |
| **Repair-trained + search + feedback** | **53/70** | **3/8** |
> From **0/8 to 3/8** on the hardest unseen problems. Not magic. But real.
## ๐ฌ What We Discovered
| # | Discovery | Details |
|:---:|:---|:---|
| ๐ฅ๏ธ | **Fine-tune on a laptop** | QLoRA + LoRA rank 4 on 4-bit Qwen3.5-0.8B. Peak: 6.5-13GB. Runs on a MacBook Air M4. |
| ๐ง | **Repair > final-answer training** | "Here's broken code, fix it" works better than "here's the problem, here's the answer." Small models learn fixing patterns, not memorized solutions. |
| ๐งฌ | **Evolutionary search = biggest jump** | Generate multiple attempts, test them, keep the best, mutate using failure feedback. This is where most gains come from. |
| ๐ฏ | **Specific failure feedback matters** | `input=[15] expected='FizzBuzz' got='Fizz'` helps more than `3/6 tests passed`. Evidence > scores. |
| ๐ค | **Model learns to be a repair partner** | Biggest overnight finding: the trained model writes much better code *inside the feedback loop*. It learned how to respond to test failures. |
| โ ๏ธ | **More compute โ better results** | Wider search, colder sampling, extra generalization โ we tried them. They didn't reliably help. |
## โก Quick Start
### Requirements
- ๐ macOS with Apple Silicon (M1/M2/M3/M4)
- ๐พ 8GB+ RAM (16GB+ recommended)
- ๐ Python 3.10+
### Install
```bash
git clone https://github.com/ranausmanai/tinyforge.git
cd tinyforge
pip install -e .
pip install mlx-lm pyyaml
```
### Download a base model
```bash
# Quantized Qwen3.5-0.8B (smallest, ~400MB)
python -c "from mlx_lm import load; load('mlx-community/Qwen2.5-Coder-0.5B-Instruct-4bit')"
```
Or convert the model we trained on:
```bash
python -m mlx_lm.convert \
--hf-path Qwen/Qwen3.5-0.8B \
--mlx-path models/mlx-q4-qwen35-08b \
-q --q-bits 4
```
### Run
```bash
# โก Quick demo (~5-10 minutes, 5 tasks)
tinyforge --model models/mlx-q4-qwen35-08b --quick
# ๐ฅ Full run (~45-60 minutes, 20 tasks)
tinyforge --model models/mlx-q4-qwen35-08b
```
### What you'll see
```
๐ฅ tinyforge
A model teaching itself to code better.
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
๐ BEFORE (single-pass baseline)
[1/5] fizzbuzz ............ 4/6
[2/5] valid_parens ........ 2/5
...
๐งฌ EVOLVING (search + feedback)
[1/5] fizzbuzz ............ 6/6 โ (gen=2)
...
๐ TRAINING on 8 self-generated repair pairs
Mixing with 40 rehearsal samples...
Training for 20 iterations...
๐ AFTER (trained model + search)
[1/5] fizzbuzz ............ 6/6 โ
...
Before: 12/26 โโโโโโโโโโ 46%
Search: 22/26 โโโโโโโโโโ 85%
After: 24/26 โโโโโโโโโโ 92%
```
### Use with your own adapter
```bash
tinyforge --model models/mlx-q4-qwen35-08b --adapter outputs/biolora6g/grow
```
## โ๏ธ How It Works
### ๐งฌ The evolutionary loop
For each coding task:
1. **๐ฑ Seed** โ Generate `population_size` candidate solutions
2. **๐งช Evaluate** โ Run each candidate against test cases in a sandboxed subprocess
3. **๐ Select** โ Keep the top `elites` unique solutions
4. **๐ Mutate** โ Generate new candidates using failure feedback from the best parents
5. **๐ Repeat** for `generations` rounds
6. **โ
Return** the best solution found
### ๐ง Repair pair extraction
After evolution, we pair weak solutions with strong ones:
```
Weak (3/6 tests): Strong (6/6 tests):
def fizzbuzz(n): def fizzbuzz(n):
if n % 3 == 0: return "Fizz" if n % 15 == 0: return "FizzBuzz"
if n % 5 == 0: return "Buzz" if n % 3 == 0: return "Fizz"
return str(n) if n % 5 == 0: return "Buzz"
return str(n)
```
The training prompt includes the weak code + test failures. The training target is the strong code. The model learns: *"when you see this kind of failure, fix it like this."*
### ๐ Training
| Parameter | Value |
|:---|:---|
| Method | MLX LoRA (4-bit quantized base + low-rank adapters) |
| LoRA rank | 4 (extremely small โ adds minimal parameters) |
| Optimizer | Adafactor (memory-efficient) |
| Data | Self-generated repair pairs + rehearsal data (prevents forgetting) |
| Iterations | 40 (~2-3 minutes) |
| Peak memory | 6.5-13GB depending on sequence length |
## ๐ Custom Tasks
Create a JSONL file with your own tasks:
```jsonl
{"id": "add", "type": "code", "prompt": "Write a function add(a, b) that returns the sum", "entry_point": "add", "tests": [{"input": [1, 2], "expected": 3}, {"input": [-1, 1], "expected": 0}]}
```
```bash
tinyforge --model my-model --tasks my_tasks.jsonl
```
See `examples/` for more task formats.
## ๐ Project Structure
```
src/finetuneqwen/
tinyforge/ # ๐ฅ The self-improvement engine
cli.py # Main CLI & demo experience
evolve.py # Evolutionary search
verify.py # Test verification (sandboxed)
extract.py # Repair pair extraction
builtin_tasks.py # 20 built-in coding problems
task.py # Task definition
# ๐ฌ Original research pipeline
humaneval_evolutionary_eval.py # HumanEval-specific evolutionary eval
export_humaneval_repair_pairs.py # HumanEval repair data extraction
train_qlora.py # PyTorch QLoRA training
prepare_data.py # Dataset preparation
configs/ # MLX training configs
scripts/ # Shell scripts for experiments
data/ # Training data (repair pairs, rehearsal)
examples/ # Example task files
results/ # Evaluation summaries
```
## ๐ Reproduce Our Results
### Best result (repair-v1, slice 40-47)
```bash
# Evolutionary search with base adapter
bash scripts/50_humaneval_evo.sh \
models/mlx-q4-qwen35-08b \
outputs/biolora6g/grow \
results/humaneval_repair_v1 \
8 40 4 3 2 0.6 1337 base_evo_40_47 1
# With repair adapter
bash scripts/50_humaneval_evo.sh \
models/mlx-q4-qwen35-08b \
outputs/humaneval_repair_v1/adapter \
results/humaneval_repair_v1 \
8 40 4 3 2 0.6 1337 repair_evo_40_47 1
```
### Build repair training data
```bash
bash scripts/60_humaneval_build_repair_v1.sh
```
### Train repair adapter
```bash
python -m mlx_lm.lora \
--model models/mlx-q4-qwen35-08b \
--data data/humaneval_repair_v1/mlx \
--config configs/biolora6g/humaneval_repair_v1_qwen35_08b.yaml \
--adapter-path outputs/humaneval_repair_v1/adapter \
--resume-adapter-file outputs/biolora6g/grow/adapters.safetensors
```
## ๐ค Honest Limitations
- ๐ซ This is **not** a magic one-shot model. The gains come from the system (model + search + feedback), not from the model alone.
- ๐ Tested on HumanEval slices, not full benchmarks. Results are real but scoped.
- ๐ Transfer is stronger on harder problems and weaker on easy ones (where the base model is already decent).
- ๐ A 0.8B model will never match GPT-4. **That's not the point.**
## ๐ Why This Matters
> If a 0.8B model can teach itself to code better on a laptop with 6GB RAM โ what happens when you run this technique on a 7B model? A 70B model? With a real GPU cluster?
The **technique** is the contribution, not the model size.
Self-play works for Go *(AlphaGo)*. Self-play works for math *(AlphaProof)*. We're showing it works for **code**, on hardware anyone can afford.
## ๐ Beyond Code โ Where This Technique Applies
The core loop is simple: **generate something โ check if it's good โ learn from what went wrong โ try again.** We proved it on code. But it works anywhere you can measure whether the output is right or wrong.
### For engineers & researchers
| Domain | What's the "test"? |
|:---|:---|
| ๐๏ธ **SQL queries** | Run the query, check if the results match |
| ๐ข **Math proofs** | Verify the proof mechanically (Lean, Coq, SymPy) |
| โก **Chip design (Verilog/VHDL)** | Simulate the circuit against test benches |
| ๐งช **Drug molecules** | Check properties via physics simulation |
| ๐ **Security testing** | Did the generated input crash the software? |
| ๐ค **Robotics** | Run the policy in simulation, measure the score |
| ๐ **Data pipelines / ETL** | Input data in, expected data out โ compare |
### For marketers, creators & everyone else
| Domain | What's the "test"? |
|:---|:---|
| ๐ฃ **Ad copy** | A/B test it โ did people click? Click rate is automatic feedback |
| ๐ง **Email outreach** | Did they open? Did they reply? Open/reply rate = the verifier |
| ๐ **SEO content** | Did it rank? Did it get traffic? Google tells you |
| ๐ **Product descriptions** | Conversion rate โ did people buy? |
| ๐ฑ **Social media posts** | Engagement. Likes, shares, saves โ all measurable |
| ๐ **Teaching / tutoring** | Did the student get the next question right? |
| โ๏ธ **Headlines & hooks** | Scroll-stop rate. You already track this |
> **The idea:** if you can score the output โ even roughly โ the model can learn from its own mistakes. No labeled dataset needed. No expert in the loop. The model makes its own training data by failing and fixing.
### ๐ What you need
1. **A task** โ what you want the model to produce
2. **A way to check** โ any script/metric that says "good" or "bad" (and ideally, why)
3. **A small model** โ even 0.8B parameters is enough to start
That's it. The model teaches itself. On your laptop. Your data never leaves your machine.
## ๐ฆ Resource Envelope
| Phase | Peak Memory | Time (MacBook Air M4) |
|:---|:---|:---|
| ๐งฌ Evolutionary search (8 tasks) | ~4GB | ~10 min |
| ๐ง Repair pair extraction | ~100MB | seconds |
| ๐ LoRA training (40 iters) | 6.5-13GB | ~3 min |
| ๐ฅ Full tinyforge demo (5 tasks) | ~10GB | ~8 min |
## ๐ License
MIT โ do whatever you want with it.
## ๐ Citation
If you use this technique in your work:
```bibtex
@misc{tinyforge2025,
title = {tinyforge: Self-improving tiny language models through test-driven repair training},
author = {Usman Muhammad},
year = {2025},
url = {https://github.com/ranausmanai/tinyforge}
}
```
---
**Built with ๐ฅ on a MacBook Air M4 with 24GB RAM**
*No GPUs were harmed in the making of this project.*