Assistant Pepe tops base on 4chan data

This is a strong reminder that “dirty” human data can still move the needle in ways synthetic or overfiltered data may miss, especially on conversational behavior.

• –The result is interesting precisely because it cuts against expectations: a controversial data source apparently improved both a small and a large model, not just one lucky checkpoint.
• –The model cards imply the gains are behavioral, not merely benchmark theater, with stronger banter and more idiosyncratic reasoning showing up in examples.
• –The tradeoff is obvious: datasets like this can also push models toward toxicity, abrasiveness, or odd edge-case behavior, so source quality is not the only variable that matters.
• –For builders, the practical lesson is to test data mixtures empirically against the behaviors you care about, instead of assuming “clean” data is always better.