If you know what a user is gonna do next, you can prefill your KV cache with candidate contexts and significantly improve your time-to-first-token. The Interaction I was revisiting an interaction idea I had a bit ago- if we have a great model of user context, then we can effectively build an...
