BACK_TO_FEEDAICRIER_2
Rnj-1 beats 30B models on Kotlin benchmark
OPEN_SOURCE ↗
REDDIT · REDDIT// 28d agoBENCHMARK RESULT

Rnj-1 beats 30B models on Kotlin benchmark

A community benchmark of 11 local LLMs on JetBrains' Kotlin HumanEval shows EssentialAI's 8B RNJ-1 (8.8 GB) taking third place overall with 75% pass@1, outperforming models two to three times its size. GPT-OSS 20B leads at 85% pass@1, while Qwen3.5-35B-a3b places second at 77% after an 18-point jump in seven months.

// ANALYSIS

Small, specialized models are punching above their weight class — RNJ-1's top-3 finish against 30B+ models signals a meaningful efficiency shift for local Kotlin coding.

  • EssentialAI RNJ-1 hits 75%/81% pass@1/pass@3 at just 8.8 GB, well within single-GPU consumer setups (16 GB VRAM)
  • GPT-OSS 20B still dominates at 85%/95% despite a 12 GB footprint, cementing its position as the local coding model to beat
  • Qwen3.5-35B jumped 18 points in seven months, showing rapid iteration in the open-weights coding space
  • JetBrains Kotlin HumanEval uses 161 expert-written problems verified by Kotlin specialists — a more signal-rich eval than generic HumanEval ports
  • For developers targeting Kotlin specifically, the results suggest capable local inference no longer requires high-end multi-GPU hardware
// TAGS
benchmarkllmopen-sourceai-codingrnj-1open-weights

DISCOVERED

28d ago

2026-03-15

PUBLISHED

28d ago

2026-03-15

RELEVANCE

7/ 10

AUTHOR

codeforlyfe