jevchat turns Jev decision model into chatbot
jevchat is an experimental open-source tool created by Kyle Pena that turns TypeSafe AI's Jev—a fast decision model intended for discrete classification, routing, and ranking rather than text generation—into an autoregressive conversational chatbot. Because Jev only scores discrete candidate options rather than emitting token streams, jevchat iteratively asks the model to choose the next character or token given the question and partial reply so far, stopping once a designated STOP symbol is drawn.
Repurposing a discrete decision and classification engine into an autoregressive chatbot is economically and computationally absurd, but it serves as a brilliant, transparent masterclass in language model decoding mechanics.
- –Demonstrates that virtually any probabilistic ranking or decision system can act as a generative language model through iterative discrete sampling.
- –The "hypothesis presentation" technique—ranking complete candidate suffix strings rather than isolated symbols—triples top-1 accuracy by aligning with the model's natural inductive bias toward evaluating full texts.
- –Solves strict API constraints (like a 255-option limit) via clever multi-stage bisection, bucketing, and nucleus refinement algorithms.
- –Though deliberately impractical and slow for real-world deployments, the project provides a rich diagnostic sandbox and live visual readout of next-token probability distributions.
DISCOVERED
1h ago
2026-09-20
PUBLISHED
3h ago
2026-09-20
RELEVANCE
AUTHOR
kp1197