Skip to content

Add limit and offset to SearchFilesByNameRequest (second take)

For Add offset/pagination support to SearchFilesByN... (#4449 - closed). This MR is identical to Add limit and offset to SearchFilesByName RPC (!4911 - merged).

After Add limit and offset to SearchFilesByName RPC (!4911 - merged) was merged, we noticed one failed test case in auto-update MR (corresponding pipeline). Examining test failures, I noticed that SearchFilesByName client recieves the following errors:

13:SearchFilesByName: cmd failed: exit status 128. debug_error_string:{"created":"@1665510713.577980051","description":"Error received from peer unix:tmp/tests/gitaly/praefect.socket","file":"src/core/lib/surface/call.cc","file_line":1063,"grpc_message":"SearchFilesByName: cmd failed: exit status 128","grpc_status":13}

At Gitaly side, the error message is more useful:

time="2022-10-12T04:23:21.573Z" level=error msg="fatal: Not a valid object name master\n" args="[/gitlab-development-kit/gitlab/tmp/run/gitaly-59659/git-exec-4221073102.d/git --git-dir /gitlab-development-kit/gitlab/tmp/tests/repositories/@hashed/2f/ac/2fac394011e7d326f9c7ff5e532316be43ce2e7d88b4f1377f585e8c8c083672.wiki.git -c gc.auto=0 -c core.autocrlf=input -c core.useReplaceRefs=false -c commitGraph.generationVersion=1 -c core.fsync=objects,derived-metadata,reference -c core.fsyncMethod=fsync ls-tree --full-tree --name-status -r -z --end-of-options master .]" command.cpu_time_ms=8.026 command.exitCode=128 command.inblock=0 command.maxrss=4308992 command.oublock=0 command.real_time_ms=297.763667 command.system_time_ms=3.9370000000000003 command.user_time_ms=4.0889999999999995 correlation_id=2508e908d5871cf53659f9be9eeeecfe grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=unknown grpc.meta.method_type=server_stream grpc.method=SearchFilesByName grpc.request.deadline="2022-10-12T11:23:31.004" grpc.request.fullMethod=/gitaly.RepositoryService/SearchFilesByName grpc.request.glProjectPath=frontend-fixtures/branches-project.wiki grpc.request.glRepository=wiki-3260 grpc.request.repoPath=@hashed/2f/ac/2fac394011e7d326f9c7ff5e532316be43ce2e7d88b4f1377f585e8c8c083672.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2022-10-12T11:23:21.274" path=/gitlab-development-kit/gitlab/tmp/run/gitaly-59659/git-exec-4221073102.d/git peer.address= pid=60131 remote_ip=0.0.0.0 span.kind=server system=grpc username=frontend-fixtures

The above line of logs tell us that Gitaly fails to handle git sub-command. Tracing down a bit deeper, in one test, the Wiki repository is not initialized properly. It does not contain any refs nor objects, even though the bare minimal directory was created. The other test fails because it performs a search on a non-existent ref. This could be considered as a known case and handled by the clients.

In the prior version of SearchFilesByName, the handler does not capture the status code of git sub command (source). As a result, search requests to valid but non-existent ref return an empty array. This actually makes sense, though. I observe the same behavior with some listing RPC, such as commit.ListFiles. I'm not sure whether it's a feature or a bug.

That said, the purpose of this issue is to paginate the results from the source. Any semantic change may break clients unexpectedly. Therefore, I decided to keep the same behavior. We can deal with uncaught errors in later MRs.

Edited by Quang-Minh Nguyen

Merge request reports