Skip to content

Preemptive caching for code suggestions

Description

This feature proposal aims to improve the perceived performance of code suggestions by introducing preemptive requests and client-side caching.

The overall flow would be:

  • When the user starts a new line and types the first character, fire a code suggestion request preemptively and cache response
  • If the user pauses typing for >300ms, first check if the cached suggestions match the current line content
  • If there is a match, show the cached suggestions immediately instead of making a new network request
  • If no match, clear cache and fetch updated suggestions

Benefits:

  • Perceived performance boost by leveraging caching
  • Avoid network round trip for unchanged suggestions
  • Clear stale cache automatically on content change

To implement:

  • Fire background prefetch request on line start
  • Cache response
  • Check cache first when suggestions triggered
  • Invalidate cache entry if line content changes