Evaluate docs.gitlab search implementation for potential improvement
Summary
Based on the gathered information, the one known result is that our current search implementation is insufficient, and that it will require development efforts and monthly recurring costs to improve our search implementation.
From research of the available options, it would appear that based exclusively on monthly cost, both Algolia and Swiftype would be about the same cost. Development-wise, I imagine the level of work would be similar for each implementation, either to implement (for Swiftype) or to improve the implementation (for Algolia). Based on what's available, though, I would be inclined to prefer Swiftype as the recommended solution, for several reasons:
- Less complexity, as it crawls your site for you, and doesn't have to have a maintained JSON export of site content.
- User interface to allow non-developers to set aliases (looking up "X" can display "Y") and sort page results (ensuring that the preferred page result is the first result, just by dragging results in the interface).
- Similar potential for future expansion, including autocomplete and faceted results.
- Potential for integration with about.gitlab.com results, if the need or desire occurs in the future.
Process
Evaluate the current search implementation for docs.gitlab.com, including mechanisms and opportunities for the search to be improved.
This includes the following:
-
Gather information about the current Algolia implementation, including cost and configuration capabilities; also current issues/problems/limitations - Technical contact: none
- Not sure yet when integration happened
- Using a free plan, and a single shared account for search usage
- JSON is passed to Algolia, which is used to build search index
- Simple tagging is in place, for search results by repo
- Shared account allows only for manual reindex and reporting
-
Gather information about Swiftype usage on about.gitlab.com, including costs, configuration capabilities, and expanding user permissions to allow for TW input for search results. - Technical contact: Brandon Lyon, Marketing Web Developer/Designer
- Integration with Swiftype happened about 3 years ago
- Chosen due to ease of use and nearly zero configuration. Essentially you add a search input and bind some javascript to it and Swiftype does the rest by crawling the associated site.
- Currently on a legacy plan that automatically crawls and updates search results every 24h and new/deleted entries once every 7 days, though documents can be manually submitted too.
- Legacy plan allows 100,000 documents to be indexed.
- Looks like we only have around 5,700 pages indexed at the moment.
- It's possible to separate into different search buckets and then search across multiple or individual buckets. For example, handbook search is limited to the handbook but site search will currently search both the about.gitlab site and the handbook. We may end up creating a separate bucket to search the blog too.
- Can provide TW one (maybe two) Editor role accounts to control search result ordering, reindexing.
- User research search comments:
-
Find preliminary percentage of number of customers using search / number of total site visitors. - For Sept 2020:
- For any comparisons between
docs.gitlab.com
andabout.gitlab.com
usage in the following stats:- Approx 3.5 million pageviews for
docs.gitlab.com
pages (63% of total pageviews) - Approx 2.1 million pageviews for
about.gitlab.com
pages (37% of total pageviews)
- Approx 3.5 million pageviews for
- Google Analytics logged 33,000 visitors as using search on any
*.gitlab.com
website - Algolia search's weekly reporting logged 28,320 users for Aug 31-Sep 27 (search results are tracked by week, and may not be directly comparable with Google Analytics overall results)
- From Google Analytics, 8,500 visitors that used search actually clicked a link in the search results (26% of the total)
- Of the 8,500 visitors that clicked a link, 2,500 visitors clicked a
docs.gitlab.com
search result (29% of all clicked search results) - Of the 8,500 visitors that clicked a link, 6,000 visitors clicked an
about.gitlab.com
search result (71% of all clicked search results)
- For any comparisons between
- For Sept 2020:
-
Research cost for upgrading Algolia to higher, more feature-rich levels. - Algolia pricing is found on their pricing page, and would seem to be $30-45/month or so, depending on the tier selection (knowing we're paying $0/month currently). Standard tier is $1/month/unit (1,000 search requests) and premium tier is $1.50/month/unit.
It's notable that although 2/3 of total pageviews for GitLab are for docs.gitlab.com
, only 1/3 of the search result click events are for docs.gitlab.com
pages. It could be noted that although people are attempting to view search results within the interface (28K visitors for docs.gitlab.com
searches), only a few visitors (2.5K visitors) are actually clicking link results, which is a 9% click-thru rate for search. (Separately, it's notable that the search mechanism in use for about.gitlab.com
is getting a higher percentage of search clicks from results. It may be worth evaluating their methods and possibly their search engine selection as mechanisms to improve the docs.gitlab.com
search performance.)
Mentioning @susantacker for awareness.