Over the past week, we worked on making small improvements to make the sorting of the results more relevant.
We ran an AB test to tweak the search algorithm. The idea was to see if small changes in how we match queries with search results could positively impact search success metrics.
The changes we made:
- The matches inside topic titles, tags, topic and category titles now all have the same weight. Previously matches at the beginning of the topic titles scored higher in the search result.
- Proximity of words in matches now have more importance in the result ranking than the order of attributes where the match was found (title, opening post post, best answer, replies ...)
For example:
A end user makes the search query "frontend developer" and the two below topics exist:
Topic A
Title "Backend developer"
Opening post "Backend engineering is more fun than frontend engineering"
Topic B
Title "Looking for work"
Opening post "I'm a frontend developer looking for work"
Our new algorithm will prioritise topic B over topic A in the search results because the two words "frontend" and "developer" are close together in the content. The algorithm will see the fact that the two words are close together as more important the fact the mach is in the opening post rather than the title.
The results we got:
We run an AB test with the two above tweaks over 8 communities for a month and got the following results on average:
- Click Through Rate
The Click Through Rate is the percent of searches where at least one result was clicked on by the end user.
- The Conversion rate
We consider the search to be successful (aka to lead to a conversion) when the user spends at least 30 seconds on the clicked topic.
Those two changes are now implemented in our new algorithm, We hope to make further improvements like this along the way to our search algorithm.
Please share any feedback or ideas you make have related to this :)
