Backend: Investigate the root cause and fix Internal server error: undefined method `bsearch'
Summary
This typebug was encountered on gitlab.com, and then on GDK locally, where it was identified where it was coming from.
When using a remote include
like in the description, we get an error: Internal server error: undefined method
bsearch' for nil:NilClass", which comes from [the
exists` keyword](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/build/rules/rule/clause/exists.rb#L37).
The config shows up like this as well, which isn't very helpful:
NoMethodError (undefined method `bsearch' for nil:NilClass):
lib/gitlab/ci/build/rules/rule/clause/exists.rb:37:in `block in exact_matches?'
lib/gitlab/ci/build/rules/rule/clause/exists.rb:37:in `any?'
lib/gitlab/ci/build/rules/rule/clause/exists.rb:37:in `exact_matches?'
lib/gitlab/ci/build/rules/rule/clause/exists.rb:21:in `satisfied_by?'
lib/gitlab/ci/build/rules/rule.rb:27:in `block in matches?'
lib/gitlab/ci/build/rules/rule.rb:27:in `all?'
lib/gitlab/ci/build/rules/rule.rb:27:in `matches?'
lib/gitlab/ci/config/external/rules.rb:25:in `block in match_rule'
lib/gitlab/ci/config/external/rules.rb:25:in `each'
lib/gitlab/ci/config/external/rules.rb:25:in `find'
lib/gitlab/ci/config/external/rules.rb:25:in `match_rule'
lib/gitlab/ci/config/external/rules.rb:19:in `evaluate'
lib/gitlab/ci/config/external/mapper.rb:76:in `verify_rules_without_instrumentation'
lib/gitlab/ci/config/external/mapper.rb:71:in `block in verify_rules'
lib/gitlab/ci/pipeline/logger.rb:33:in `instrument'
lib/gitlab/ci/config/external/mapper.rb:70:in `verify_rules'
lib/gitlab/ci/config/external/mapper.rb:45:in `each'
lib/gitlab/ci/config/external/mapper.rb:45:in `filter_map'
lib/gitlab/ci/config/external/mapper.rb:45:in `process_without_instrumentation'
lib/gitlab/ci/config/external/mapper.rb:31:in `block in process'
lib/gitlab/ci/pipeline/logger.rb:33:in `instrument'
lib/gitlab/ci/config/external/mapper.rb:30:in `process'
lib/gitlab/ci/config/external/processor.rb:14:in `initialize'
lib/gitlab/ci/config/external/file/base.rb:116:in `new'
lib/gitlab/ci/config/external/file/base.rb:116:in `expand_includes'
lib/gitlab/ci/config/external/file/base.rb:79:in `block in expanded_content_hash'
lib/gitlab/utils/strong_memoize.rb:44:in `strong_memoize'
lib/gitlab/ci/config/external/file/base.rb:78:in `expanded_content_hash'
lib/gitlab/ci/config/external/file/base.rb:46:in `to_hash'
lib/gitlab/ci/config/external/file/base.rb:110:in `validate_hash!'
lib/gitlab/ci/config/external/file/base.rb:54:in `block in validate!'
lib/gitlab/ci/pipeline/logger.rb:33:in `instrument'
lib/gitlab/ci/config/external/file/base.rb:50:in `validate!'
lib/gitlab/ci/config/external/mapper.rb:130:in `verify!'
lib/gitlab/ci/config/external/mapper.rb:50:in `each'
lib/gitlab/ci/config/external/mapper.rb:50:in `process_without_instrumentation'
lib/gitlab/ci/config/external/mapper.rb:31:in `block in process'
lib/gitlab/ci/pipeline/logger.rb:33:in `instrument'
lib/gitlab/ci/config/external/mapper.rb:30:in `process'
lib/gitlab/ci/config/external/processor.rb:14:in `initialize'
Reproduce
# .gitlab-ci.yml of the project
include:
- remote: http://example.com/random-address/include.yml
# http://example.com/random-address/include.yml
include:
- local: sub-include.yml
rules:
- exists:
- "Dockerfile"
This bug only occurs when the remote include has an include
with an exist
rule.
Problem
However, if the remote file has rules
inside of their include
s, we cannot process them because we don't have enough context to know file changes, variables, etc.
Currently, when you define;
# http://example.com/random-address/include.yml
include:
- remote: http://example.com/random-address/builds.yml
rules:
- if: $CI_COMMIT_BRANCH == "master"
the rule is always evaluated as false
and the file will not be included.
However, when you define the rule with exists
;
# http://example.com/random-address/include.yml
include:
- remote: http://example.com/random-address/builds.yml
rules:
- exists: [Dockerfile]
we get an error.
We have this error because in lib/gitlab/ci/build/rules/rule/clause/exists.rb
;
def satisfied_by?(_pipeline, context)
paths = worktree_paths(context)
exact_matches?(paths) || pattern_matches?(paths)
end
private
def worktree_paths(context)
return unless context.project
if @top_level_only
context.top_level_worktree_paths
else
context.all_worktree_paths
end
end
def exact_matches?(paths)
@exact_globs.any? { |glob| paths.bsearch { |path| glob <=> path } }
end
- we don't have
context
, so we don't havecontext.project
-
worktree_paths
returnsnil
- and we try to call methods on
paths
, which is nil.
Proposal
I think what we need to do here is to return []
from worktree_paths
instead of nil
, so that exists
will also behave as if
for the include
s.