Making search answer in 50 milliseconds
Search in Tern got ten times faster in 2.11. What we tried first, why it did not work, and the index that did.
Before 2.11, search in Tern took about 400ms for a typical workspace and much longer for big ones. That is fast enough to use and slow enough to notice, and in a tool that is meant to feel instant, noticing is the problem.
What we tried first
We started by tuning the database's full-text search. It got us to around 250ms, and then every gain cost us something: stale results, missed accents, or a query planner that changed its mind under load.
What worked
We moved search into a small index per workspace that lives next to the app server, kept current from the same change stream that powers real-time updates. A query never leaves the machine that serves it.
- Median query time went from 400ms to 31ms
- The slowest one percent went from 2.1s to 90ms
- Index updates land within 200ms of a change
What we gave up
Memory, mostly. Each index costs a few megabytes per thousand issues. For us it was an easy trade: search is the second most used thing in Tern, after the command menu.