Flaky master: Time-dependent test in obsolete_ignored_columns_spec

https://gitlab.com/gitlab-org/gitlab/-/jobs/381262438
Failures:
5944   1) Gitlab::Database::ObsoleteIgnoredColumns#execute returns a list of class names and columns pairs
5945      Failure/Error:
5946        expect(subject.execute).to eq([
5947          ['Testing::A', {
5948            'unused' => IgnorableColumns::ColumnIgnore.new(Date.parse('2019-01-01'), '12.0'),
5949            'also_unused' => IgnorableColumns::ColumnIgnore.new(Date.parse('2019-02-01'), '12.1')
5950          }],
5951          ['Testing::B', {
5952            'other' => IgnorableColumns::ColumnIgnore.new(Date.parse('2019-01-01'), '12.0')
5953          }]
5954        ])
5955        expected: [["Testing::A", {"also_unused"=>#<struct IgnorableColumns::ColumnIgnore remove_after=Fri, 01 Feb 2019...ther"=>#<struct IgnorableColumns::ColumnIgnore remove_after=Tue, 01 Jan 2019, remove_with="12.0">}]]
5956             got: [["Testing::A", {"also_unused"=>#<struct IgnorableColumns::ColumnIgnore remove_after=Fri, 01 Feb 2019...ther"=>#<struct IgnorableColumns::ColumnIgnore remove_after=Tue, 01 Jan 2019, remove_with="12.0">}]]
5957        (compared using ==)
5958        Diff:
5959        @@ -1,9 +1,13 @@
5960         [["Testing::A",
5961           {"also_unused"=>
5962             "#<struct IgnorableColumns::ColumnIgnore remove_after=Fri, 01 Feb 2019, remove_with=\"12.1\">",
5963        +   "not_used_but_still_ignored"=>
5964        +    "#<struct IgnorableColumns::ColumnIgnore remove_after=Mon, 16 Dec 2019, remove_with=\"12.1\">",
5965            "unused"=>
5966             "#<struct IgnorableColumns::ColumnIgnore remove_after=Tue, 01 Jan 2019, remove_with=\"12.0\">"}],
5967          ["Testing::B",
5968        -  {"other"=>
5969        +  {"not_used_but_still_ignored"=>
5970        +    "#<struct IgnorableColumns::ColumnIgnore remove_after=Mon, 16 Dec 2019, remove_with=\"12.1\">",
5971        +   "other"=>
5972             "#<struct IgnorableColumns::ColumnIgnore remove_after=Tue, 01 Jan 2019, remove_with=\"12.0\">"}]]
5973      # ./vendor/ruby/2.6.0/gems/rspec-support-3.8.2/lib/rspec/support.rb:97:in `block in <module:Support>'
5974      # ./vendor/ruby/2.6.0/gems/rspec-support-3.8.2/lib/rspec/support.rb:106:in `notify_failure'
5975      # ./vendor/ruby/2.6.0/gems/rspec-expectations-3.8.4/lib/rspec/expectations/fail_with.rb:35:in `fail_with'
5976      # ./vendor/ruby/2.6.0/gems/rspec-expectations-3.8.4/lib/rspec/expectations/handler.rb:38:in `handle_failure'
5977      # ./vendor/ruby/2.6.0/gems/rspec-expectations-3.8.4/lib/rspec/expectations/handler.rb:50:in `block in handle_matcher'
5978      # ./vendor/ruby/2.6.0/gems/rspec-expectations-3.8.4/lib/rspec/expectations/handler.rb:27:in `with_matcher'
5979      # ./vendor/ruby/2.6.0/gems/rspec-expectations-3.8.4/lib/rspec/expectations/handler.rb:48:in `handle_matcher'
5980      # ./vendor/ruby/2.6.0/gems/rspec-expectations-3.8.4/lib/rspec/expectations/expectation_target.rb:65:in `to'
5981      # ./spec/lib/gitlab/database/obsolete_ignored_columns_spec.rb:43:in `block (3 levels) in <top (required)>'
5982      # ./vendor/ruby/2.6.0/gems/rspec-core-3.8.2/lib/rspec/core/example.rb:257:in `instance_exec'
5983      # ./vendor/ruby/2.6.0/gems/rspec-core-3.8.2/lib/rspec/core/example.rb:257:in `block in run'
5984      # ./vendor/ruby/2.6.0/gems/rspec-core-3.8.2/lib/rspec/core/example.rb:503:in `block in with_around_and_singleton_context_hooks'
5985      # ./vendor/ruby/2.6.0/gems/rspec-core-3.8.2/lib/rspec/core/example.rb:460:in `block in with_around_example_hooks'
5986      # ./vendor/ruby/2.6.0/gems/rspec-core-3.8.2/lib/rspec/core/hooks.rb:464:in `block in run'
5987      # ./vendor/ruby/2.6.0/gems/rspec-core-3.8.2/lib/rspec/core/hooks.rb:604:in `block in run_around_example_hooks_for'
5988      # ./vendor/ruby/2.6.0/gems/rspec-core-3.8.2/lib/rspec/core/example.rb:345:in `call'
5989      # ./vendor/ruby/2.6.0/gems/rspec-rails-4.0.0.beta3/lib/rspec/rails/adapters.rb:128:in `block (2 levels) in <module:MinitestLifecycleAdapter>'
5990      # ./vendor/ruby/2.6.0/gems/rspec-core-3.8.2/lib/rspec/core/example.rb:450:in `instance_exec'
5991      # ./vendor/ruby/2.6.0/gems/rspec-core-3.8.2/lib/rspec/core/example.rb:450:in `instance_exec'
5992      # ./vendor/ruby/2.6.0/gems/rspec-core-3.8.2/lib/rspec/core/hooks.rb:373:in `execute_with'
5993      # ./vendor/ruby/2.6.0/gems/rspec-core-3.8.2/lib/rspec/core/hooks.rb:606:in `block (2 levels) in run_around_example_hooks_for'
5994      # ./vendor/ruby/2.6.0/gems/rspec-core-3.8.2/lib/rspec/core/example.rb:345:in `call'
5995      # ./vendor/ruby/2.6.0/gems/rspec-retry-0.6.1/lib/rspec/retry.rb:123:in `block in run'
5996      # ./vendor/ruby/2.6.0/gems/rspec-retry-0.6.1/lib/rspec/retry.rb:110:in `loop'
5997      # ./vendor/ruby/2.6.0/gems/rspec-retry-0.6.1/lib/rspec/retry.rb:110:in `run'
5998      # ./vendor/ruby/2.6.0/gems/rspec-retry-0.6.1/lib/rspec_ext/rspec_ext.rb:12:in `run_with_retry'
5999      # ./vendor/ruby/2.6.0/gems/rspec-retry-0.6.1/lib/rspec/retry.rb:37:in `block (2 levels) in setup'
6000      # ./vendor/ruby/2.6.0/gems/rspec-core-3.8.2/lib/rspec/core/example.rb:450:in `instance_exec'
6001      # ./vendor/ruby/2.6.0/gems/rspec-core-3.8.2/lib/rspec/core/example.rb:450:in `instance_exec'
6002      # ./vendor/ruby/2.6.0/gems/rspec-core-3.8.2/lib/rspec/core/hooks.rb:373:in `execute_with'
6003      # ./vendor/ruby/2.6.0/gems/rspec-core-3.8.2/lib/rspec/core/hooks.rb:606:in `block (2 levels) in run_around_example_hooks_for'
6004      # ./vendor/ruby/2.6.0/gems/rspec-core-3.8.2/lib/rspec/core/example.rb:345:in `call'
6005      # ./vendor/ruby/2.6.0/gems/rspec-core-3.8.2/lib/rspec/core/hooks.rb:607:in `run_around_example_hooks_for'
6006      # ./vendor/ruby/2.6.0/gems/rspec-core-3.8.2/lib/rspec/core/hooks.rb:464:in `run'
6007      # ./vendor/ruby/2.6.0/gems/rspec-core-3.8.2/lib/rspec/core/example.rb:460:in `with_around_example_hooks'
6008      # ./vendor/ruby/2.6.0/gems/rspec-core-3.8.2/lib/rspec/core/example.rb:503:in `with_around_and_singleton_context_hooks'
6009      # ./vendor/ruby/2.6.0/gems/rspec-core-3.8.2/lib/rspec/core/example.rb:254:in `run'
6010      # ./vendor/ruby/2.6.0/gems/rspec-core-3.8.2/lib/rspec/core/example_group.rb:633:in `block in run_examples'
6011      # ./vendor/ruby/2.6.0/gems/rspec-core-3.8.2/lib/rspec/core/example_group.rb:629:in `map'
6012      # ./vendor/ruby/2.6.0/gems/rspec-core-3.8.2/lib/rspec/core/example_group.rb:629:in `run_examples'
6013      # ./vendor/ruby/2.6.0/gems/rspec-core-3.8.2/lib/rspec/core/example_group.rb:595:in `run'
6014      # ./vendor/ruby/2.6.0/gems/rspec-core-3.8.2/lib/rspec/core/example_group.rb:596:in `block in run'
6015      # ./vendor/ruby/2.6.0/gems/rspec-core-3.8.2/lib/rspec/core/example_group.rb:596:in `map'
6016      # ./vendor/ruby/2.6.0/gems/rspec-core-3.8.2/lib/rspec/core/example_group.rb:596:in `run'
6017      # ./vendor/ruby/2.6.0/gems/rspec-core-3.8.2/lib/rspec/core/runner.rb:116:in `block (3 levels) in run_specs'
6018      # ./vendor/ruby/2.6.0/gems/rspec-core-3.8.2/lib/rspec/core/runner.rb:116:in `map'
6019      # ./vendor/ruby/2.6.0/gems/rspec-core-3.8.2/lib/rspec/core/runner.rb:116:in `block (2 levels) in run_specs'
6020      # ./vendor/ruby/2.6.0/gems/rspec-core-3.8.2/lib/rspec/core/configuration.rb:2008:in `with_suite_hooks'
6021      # ./vendor/ruby/2.6.0/gems/rspec-core-3.8.2/lib/rspec/core/runner.rb:111:in `block in run_specs'
6022      # ./vendor/ruby/2.6.0/gems/rspec-core-3.8.2/lib/rspec/core/reporter.rb:74:in `report'
6023      # ./vendor/ruby/2.6.0/gems/rspec-core-3.8.2/lib/rspec/core/runner.rb:110:in `run_specs'
6024      # ./vendor/ruby/2.6.0/gems/rspec-core-3.8.2/lib/rspec/core/runner.rb:87:in `run'
6025      # ./vendor/ruby/2.6.0/gems/rspec-core-3.8.2/lib/rspec/core/runner.rb:71:in `run'
6026      # ./vendor/ruby/2.6.0/gems/rspec-core-3.8.2/lib/rspec/core/runner.rb:45:in `invoke'
6027      # ./vendor/ruby/2.6.0/gems/rspec-core-3.8.2/exe/rspec:4:in `<top (required)>'
6028      # ./vendor/ruby/2.6.0/bin/rspec:23:in `load'
6029      # ./vendor/ruby/2.6.0/bin/rspec:23:in `<top (required)>'
6030      # /usr/local/lib/ruby/2.6.0/bundler/cli/exec.rb:74:in `load'
6031      # /usr/local/lib/ruby/2.6.0/bundler/cli/exec.rb:74:in `kernel_load'
6032      # /usr/local/lib/ruby/2.6.0/bundler/cli/exec.rb:28:in `run'
6033      # /usr/local/lib/ruby/2.6.0/bundler/cli.rb:463:in `exec'
6034      # /usr/local/lib/ruby/2.6.0/bundler/vendor/thor/lib/thor/command.rb:27:in `run'
6035      # /usr/local/lib/ruby/2.6.0/bundler/vendor/thor/lib/thor/invocation.rb:126:in `invoke_command'
6036      # /usr/local/lib/ruby/2.6.0/bundler/vendor/thor/lib/thor.rb:387:in `dispatch'
6037      # /usr/local/lib/ruby/2.6.0/bundler/cli.rb:27:in `dispatch'
6038      # /usr/local/lib/ruby/2.6.0/bundler/vendor/thor/lib/thor/base.rb:466:in `start'
6039      # /usr/local/lib/ruby/2.6.0/bundler/cli.rb:18:in `start'
6040      # /usr/local/lib/ruby/gems/2.6.0/gems/bundler-1.17.2/exe/bundle:30:in `block in <top (required)>'
6041      # /usr/local/lib/ruby/2.6.0/bundler/friendly_errors.rb:124:in `with_friendly_errors'
6042      # /usr/local/lib/ruby/gems/2.6.0/gems/bundler-1.17.2/exe/bundle:22:in `<top (required)>'
6043      # /usr/local/bin/bundle:23:in `load'
6044      # /usr/local/bin/bundle:23:in `<main>'
6045 Finished in 12 minutes 15 seconds (files took 39.45 seconds to load)
6046 2048 examples, 1 failure, 4 pending
6047 Failed examples:
6048 rspec ./spec/lib/gitlab/database/obsolete_ignored_columns_spec.rb:42 # Gitlab::Database::ObsoleteIgnoredColumns#execute returns a list of class names and columns pairs
Possible fix

@stanhu commented !21882 (comment 261388576):
Actually, I wonder if this problem happens when the test crosses a date boundary during the test:

2019-12-16 11:42 pm: Date.today runs at https://gitlab.com/gitlab-org/gitlab/blob/46af11ae00cddbcfe9a3971758b48735a2b56195/spec/lib/gitlab/database/obsolete_ignored_columns_spec.rb#L26`

2019-12-16 00:00: https://gitlab.com/gitlab-org/gitlab/blob/46af11ae00cddbcfe9a3971758b48735a2b56195/spec/lib/gitlab/database/obsolete_ignored_columns_spec.rb#L43 runs, and Date.today is now ahead of the time at 1, so the ignored column should be listed in the output.

To fix this, I think we should just use a fixed date.
This looks right. The classes run Date.today on load, and then the it block eventually runs Date.today to compare against remove_after.
Edited Dec 17, 2019 by Michael Kozono