Top AI Product

Role-playing AI is usually judged on consistency: does it remember the character’s facts and stay on persona? ArcANE, a new arXiv benchmark, argues that’s the wrong bar. A good character isn’t fixed — its values and behavior should evolve as the story moves, and the real question is whether the AI tracks that arc at the right moment.

## How it measures

ArcANE — Arc-Aware Narrative Evaluation — spans 17 novels and 80 principal characters. It segments each narrative into phases along a psychological axis (the “Character Arc”), then poses the same scenario across phases — including situations the source text never actually explores, to test real generalization rather than recall.

## What it found

Across six models and six context modes, feeding the model the Character Arc beat every other context strategy, by a wide margin. Fine-tuning open-weight models on the same data (ArcANE-8B/32B) widened that lead further on out-of-text scenarios. The implication for anyone building companion or character AI: a persona isn’t a fixed string, it’s a trajectory — and most setups aren’t conditioning on it.