Skip to content

fix(tests): import time correctly to match mocking patch

Igor Drozdov requested to merge id-import-time-correctly-to-fix-flaky-test into main

What does this merge request do and why?

The time.time call is patched as time.time in the tests:

        with patch("time.time", return_value=time_now):
            response = mock_client.post(
                "/search/gitlab-docs",
                headers={
                    "Authorization": "Bearer 12345",
                    "X-Gitlab-Authentication-Type": "oidc",
                },
                json=request_body,
            )

And the function must be called the same way in the code.

How to test it

  1. To test that the example is not flaky anymore we can add a sleep before and after the call:
    time_now = time.time()
    time.sleep(1)
    container = ContainerApplication()
    with container.searches.vertex_search.override(mock_llm_model):
        with patch("time.time", return_value=time_now):
            response = mock_client.post(
                "/search/gitlab-docs",
                headers={
                    "Authorization": "Bearer 12345",
                    "X-Gitlab-Authentication-Type": "oidc",
                },
                json=request_body,
            )
    time.sleep(1)

In this case, the test must still pass because the race condition is eliminated.

  1. Another way is to run the test multiple times. It can be done using pytest-repeat library:
poetry add pytest-repeat
poetry run pytest --count=10 -v tests/api/v1/test_v1_search_docs.py::test_success

The test fails before the fix and passes after the fix

Merge request reports