Change: frontend test fixtures

Change pattern proposal: static fixtures to dynamic fixtures

Old Pattern 💀

Using static HTML or JSON fixtures—typically copied from the application.

Examples

    setHTMLFixture(`
      <div id="one" class="gfm-merge_request" data-mr-title="title" data-iid="1" data-project-path="group/project">
        MR1
      </div>
      <div id="two" class="gfm-merge_request" data-mr-title="title" data-iid="1" data-project-path="group/project">
        MR2
      </div>
      <div id="three" class="gfm-merge_request">
        MR3
      </div>
    `);

(https://gitlab.com/gitlab-org/gitlab-ce/blob/9c3dfd2085369005305260bdaec872cb078a758e/spec/frontend/mr_popover/index_spec.js#L13-23)

export default () => ({
  id: 1,
  iid: 1,
  state: 'opened',
  upvotes: 1,
  userNotesCount: 2,
  closedAt: getDate(1),
  createdAt: getDate(3),
  updatedAt: getDate(2),
  confidential: false,
  webUrl: `${gl.TEST_HOST}/test/issue/1`,
  title: 'Test issue',
  author: {
    avatarUrl: `${gl.TEST_HOST}/avatar`,
    name: 'Author Name',
    username: 'author.username',
    webUrl: `${gl.TEST_HOST}/author`,
  },
});

(https://gitlab.com/gitlab-org/gitlab-ce/blob/9c3dfd2085369005305260bdaec872cb078a758e/spec/frontend/issuable_suggestions/mock_data.js)

import mockData from '../mock_data';

(https://gitlab.com/gitlab-org/gitlab-ce/blob/9c3dfd2085369005305260bdaec872cb078a758e/spec/frontend/issuable_suggestions/components/item_spec.js#L6)

Existing Pattern - Option A 🍎

Write new RSpec tests that

• make requests to backend endpoints
• store the resulting HTML or JSON in files

This approach was originally introduced in https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/6059 for HAML only and later extended to JSON in https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/9211.

Examples

  it 'commit/show.html' do
    params = {
      namespace_id: project.namespace,
      project_id: project,
      id: commit.id
    }

    get :show, params: params

    expect(response).to be_success
  end

(https://gitlab.com/gitlab-org/gitlab-ce/blob/9c3dfd2085369005305260bdaec872cb078a758e/spec/javascripts/fixtures/commit.rb)

  it 'autocomplete_sources/labels.json' do
    issue.labels << create(:label, project: project, title: 'bug')
    issue.labels << create(:label, project: project, title: 'critical')

    create(:label, project: project, title: 'feature')
    create(:label, project: project, title: 'documentation')

    get :labels,
        format: :json,
        params: {
            namespace_id: group.path,
            project_id: project.path,
            type: issue.class.name,
            type_id: issue.id
        }

    expect(response).to be_success
  end

(https://gitlab.com/gitlab-org/gitlab-ce/blob/9c3dfd2085369005305260bdaec872cb078a758e/spec/javascripts/fixtures/autocomplete_sources.rb)

loadFixtures('issues/issue_with_comment.html');

(https://gitlab.com/gitlab-org/gitlab-ee/blob/e7a5fd1b2b25b7aad7400046beec3588bb8dc1a3/spec/javascripts/lazy_loader_spec.js#L18)

const labelsFixture = getJSONFixture('autocomplete_sources/labels.json');

(https://gitlab.com/gitlab-org/gitlab-ee/blob/e7a5fd1b2b25b7aad7400046beec3588bb8dc1a3/spec/frontend/gfm_auto_complete_spec.js#L12)

Disadvantages:

• Frontend developers need to write RSpec tests which is not necessarily their core competency.
• A lot of duplication between RSpec unit tests and fixture generators.

New Pattern - Option B 🍌

Reuse existing RSpec tests (for example unit tests of controllers) by using tags.

Examples

    it 'shows "Open" if an issue has been moved', :frontend_fixture: 'issues/show.html' do
      render

      expect(rendered).to have_selector('.status-box-open:not(.hidden)', text: 'Open')
    end

(https://gitlab.com/gitlab-org/gitlab-ce/blob/c11eb0c3a42dba550764e96426dd9bf21347b917/spec/views/projects/issues/show.html.haml_spec.rb#L72-76)

      it 'returns discussion json', :frontend_fixture: 'issues/discussions.json' do
        get :discussions, params: { namespace_id: project.namespace, project_id: project, id: issue.iid }

        expect(json_response.first.keys).to match_array(%w[id reply_id expanded notes diff_discussion discussion_path individual_note resolvable resolved resolved_at resolved_by resolved_by_push commit_id for_commit project_id])
      end

(https://gitlab.com/gitlab-org/gitlab-ce/blob/c11eb0c3a42dba550764e96426dd9bf21347b917/spec/controllers/projects/issues_controller_spec.rb#L1263-1267)

loadFixtures('issues/issue_with_comment.html');

(https://gitlab.com/gitlab-org/gitlab-ee/blob/e7a5fd1b2b25b7aad7400046beec3588bb8dc1a3/spec/javascripts/lazy_loader_spec.js#L18)

const labelsFixture = getJSONFixture('autocomplete_sources/labels.json');

(https://gitlab.com/gitlab-org/gitlab-ee/blob/e7a5fd1b2b25b7aad7400046beec3588bb8dc1a3/spec/frontend/gfm_auto_complete_spec.js#L12)

Disadvantages:

• We need to replace the existing dynamic fixtures that follow pattern A.

Advantages of switching patterns

• Less duplication between application code and tests.
• Tests don't run on outdated input (and therefore hide bugs).
• Intention is more clear in test setup.

Disadvantages of switching patterns

see above

What is the impact on our existing codebase?

Better maintainability of our frontend tests.