OPEN_SOURCE ↗
REDDIT · REDDIT// 28d agoBENCHMARK RESULT
Rnj-1 beats 30B models on Kotlin benchmark
A community benchmark of 11 local LLMs on JetBrains' Kotlin HumanEval shows EssentialAI's 8B RNJ-1 (8.8 GB) taking third place overall with 75% pass@1, outperforming models two to three times its size. GPT-OSS 20B leads at 85% pass@1, while Qwen3.5-35B-a3b places second at 77% after an 18-point jump in seven months.
// ANALYSIS
Small, specialized models are punching above their weight class — RNJ-1's top-3 finish against 30B+ models signals a meaningful efficiency shift for local Kotlin coding.
- –EssentialAI RNJ-1 hits 75%/81% pass@1/pass@3 at just 8.8 GB, well within single-GPU consumer setups (16 GB VRAM)
- –GPT-OSS 20B still dominates at 85%/95% despite a 12 GB footprint, cementing its position as the local coding model to beat
- –Qwen3.5-35B jumped 18 points in seven months, showing rapid iteration in the open-weights coding space
- –JetBrains Kotlin HumanEval uses 161 expert-written problems verified by Kotlin specialists — a more signal-rich eval than generic HumanEval ports
- –For developers targeting Kotlin specifically, the results suggest capable local inference no longer requires high-end multi-GPU hardware
// TAGS
benchmarkllmopen-sourceai-codingrnj-1open-weights
DISCOVERED
28d ago
2026-03-15
PUBLISHED
28d ago
2026-03-15
RELEVANCE
7/ 10
AUTHOR
codeforlyfe