Convert AST to gRPC request

What does this MR do and why?

Adds support for parsing AST queries and translating them to complex gRPC queries that are sent to zoekt.

Rails can send a v1 request (which it is doing currently) or it can send an AST request and they will produce the same results. Both v1 and AST search requests are functionally identical.

Note: this does not add a handler for meta queries yet, since that hasn't been merged.

AI Summary

> This merge request enhances the search functionality by improving the API response format and adding a new query processing system. The changes include: > > 1. Added a new `watch-web` command to the Makefile for watching and running the webserver > 2. Created an example search request JSON file showing the new query format > 3. Improved error handling in the webserver API by standardizing JSON error responses > 4. Completely refactored the search query processing system to support a more flexible query structure > 5. Added support for various query types (substring, regexp, and, or, not) in the new query system > 6. Increased the gRPC message size limit to 16MB to handle larger search results > 7. Updated tests to work with the new query structure > 8. Added header forwarding security by only allowing specific headers to be forwarded > > These changes make the search API more robust, with better error handling and a more powerful query system that can handle complex search conditions.

How to set up and validate locally

1. Start in webserver mode: make watch-web listen=:8080 index_dir=~/gdk/zoekt-data/development/index-2/
2. Do a search with v1 format and write the results to a file

curl --request POST \
  --url http://localhost:8080/webserver/api/v2/search \
  --header 'Content-Type: application/json' \
  --header 'User-Agent: insomnia/11.1.0' \
  --data '{
        "Q": "test",
        "Opts": {
                "NumContextLines": 20,
                "MaxFileMatchWindow": 1000,
                "MaxFileMatchResults": 5,
                "MaxLineMatchWindow": 500,
                "MaxLineMatchResults": 10,
                "MaxLineMatchResultsPerFile": 3
        },
        "ForwardTo": [
                { "Endpoint": "http://localhost:6090/api/search", "RepoIds": [8] },
                { "Endpoint": "http://localhost:6091/api/search", "RepoIds": [1, 2, 19] }
        ]
}' > /tmp/v1.json

3. Do a search with v2 format and write the results to a file

curl --request POST \
  --url http://localhost:8080/webserver/api/v2/search \
  --header 'Content-Type: application/json' \
  --header 'User-Agent: insomnia/11.1.0' \
  --data '{
	"version": 2,
	"timeout": "30s",
	"num_context_lines": 20,
	"max_file_match_window": 1000,
	"max_file_match_results": 5,
	"max_line_match_window": 500,
	"max_line_match_results": 10,
	"max_line_match_results_per_file": 3,
	"forward_to": [
		{
			"endpoint": "http://localhost:6090/api/search",
			"query": {
				"and": {
					"children": [
						{
							"repo_ids": [
								8
							]
						},
						{
							"substring": {
								"pattern": "test",
								"case_sensitive": false,
								"content": true
							}
						}
					]
				}
			}
		},
		{
			"endpoint": "http://localhost:6091/api/search",
			"query": {
				"and": {
					"children": [
						{
							"repo_ids": [
								1,
								2,
								19
							]
						},
						{
							"substring": {
								"pattern": "test",
								"case_sensitive": false,
								"content": true
							}
						}
					]
				}
			}
		}
	]
}' > /tmp/v2.json

4. Run this command to verify that the md5 of both files are identical

if [ "$(md5sum /tmp/v1.json | cut -d' ' -f1)" = "$(md5sum /tmp/v2.json | cut -d' ' -f1)" ]; then echo "Files are identical"; else echo "Files are different"; fi