Skip to content
Snippets Groups Projects

Manual Deployment QA

Last edited by Greg Austic

Ideally, we would develop true automated unit tests on our features. However, as a small company and team, our initial step is to have manual tests prior to every deployment to the production server.

These tests are not exhaustive! Additional testing is required on a feature-by-feature basis.

Adding a test to this list

When creating a new feature, consider adding to this list. Remember this list gets run every time the production server is updated - so don't bloat it! But if you can create an efficient but expansive test (a survey with many functions, creation / deletion / modification of orgs or members, etc.) please add it here. When adding a test, make sure to:

  • Describe the test
  • Describe the success state
  • Describe possibly fail states

List of Tested Devices

Where testing on multiple devices is required (as noted below), the testing list is:

  • Android
    • Chrome
    • Firefox
  • iPhone (mobile)
    • Safari
    • Chrome
    • Firefox
  • iOS (desktop/laptop)
    • Safari
    • Chrome
    • Firefox
  • PC
    • Chrome
    • Firefox

Pre-Deployment test list

Farm Deployment Survey

  1. tests: survey submission, scripts, API calls in scripts, many question types (multiple choice, single choice, ontology lists, scripts), 'relevance' feature, DB acceptance of submissions.
  2. Go here: https://app.surveystack.io/surveys/5e9dbcf0c9ea7900018cddc0
  3. Create a new entry and submit it.
    • Use US-based address
    • Farm Information script should produce an output. Check console for errors.
    • Sample Plan script should produce an output consistent with your entries. Check console for errors.
    • confirm: Survey (logic, transitions, input, etc.) should work as expected.
    • confirm: Submission should be successful.
  4. Check survey results to confirm output.
    • confirm: Submission should appear + all expected outputs should appear, including outputs from scripts. Compare against previous entries for expected outputs.
  5. Go to 'Results' and delete new entry.
  6. Repeat for all items on the Tested Devices list

Group functions / Create survey

  1. tests: create / modify groups, add/invite members, create survey in group, submit data in group, group/membership UI
  2. Group creation
    • Create a new group
    • Create a subgroup in that group
    • confirm: group/subgroup creation worked
  3. Invite an email you can access to the subgroup as member, find + accept invite, see new group in user settings (dropdown in upper left)
    • confirm: email DIDN'T go into spam, link worked, group exists when logged in as new user
  4. Create survey
    • Select new group as active group
    • Create new survey
    • confirm: active group is set as group for new survey
    • Add at least 1 required question and 1 private question, save when complete.
    • confirm: survey + question creation worked
    • Go to 'browse'.
    • confirm: Review available screens, ensure that the correct surveys are in the right place ('my surveys' contains the right ones, group surveys contains the right ones, etc.)
  5. Create submission, clean up
    • Select the new survey, fill out submission + submit.
    • Go to 'Results', click to see metadata
    • Click on the new data and delete it, then delete the survey, then delete the group / subgroup.
    • confirm: submissions are marked in the metadata as being associated with the new active group, and that delete worked for submission, survey, and group.

farmOS Push Survey

this will definitely change by 2022, if it's out of date request an update

  1. tests: survey submission, farmOS question types, 'API Compose' feature, push-to-farmOS feature (success and failure)
  2. Prerequisites - you will need to have farmOS login access to ourscitest.farmos.net and oursci.farmos.net. Ask Dan, Greg, or someone else to get that before starting.
  3. Find aggregator information
    • Go to surveystack.io (production site) --> groups --> Our Sci LLC --> admin --> manage farmOS farms --> click SurveyStack Aggregator.
  4. Ensure farmOS integration is set up correctly on your instance
    • Go to your review/testing/dev surveyStack instance --> groups. Create group called our-sci-review-reserved.
    • In that group, go to manage farmOS farms --> New Aggregator.
    • NOTE! we're using the production aggregator even for testing - understand that creating farmOS accounts on groups which may have overlapping names with those in production could affect production. Ensure you are using the group our-sci-review-reserved to avoid this.
    • Copy information from SurveyStack aggregator to New Aggregator
    • Delete the tags. Enter the tag as the group reference: /our-sci-review-reserved/ <-- must be EXACTLY this.
    • Hit Save.
    • confirm: new '+ New Farm' button appears under New Aggregator.
  5. Test farm creation
    • Select '+ New Farm'.
    • Create new farm with 'ourscitest<##>` where ## is some number which isn't already taken. Don't do this often as it creates dead servers for Mike to take care of.
    • Assign yourself as member.
    • Create a field using the field creation tool.
    • Register instance
    • After 10 minutes, find email from Farmier confirmation creation w/ login.
    • Login to farmOS and identify field created in SurveyStack.
    • confirm: field creation (in SurveyStack and presence in farmOS), instance registration, email confirmation, ability to log in.
  6. Test farm assignment
    • Go to our-sci-review-reserved group --> admin --> manage farmOS farms --> SurveyStack Aggregator
    • confirm: there should be 2 our sci farms (ourscitest.farmos.net, oursci.farmos.net).
    • Select both farms, and assign yourself to the farms (if you are not already). Hit 'Save Changes' each time.
    • In a new tab, go to the Builder --> create new survey. Add a Farm, Field, and Planting question (in that order) to the builder.
    • Go to the last question added (should be Planting), go to advanced --> apiCompose. Paste in the following. Note that this has references to the questions you just created, so it's important you don't change names (or if you do you also change them here):
function apiCompose(submission, survey, parent) {
    const activities = [];
    const url = parent.farmos_farm_1.value[0].url;
    const fieldId = parent.farmos_field_2.value[0].fieldId;
    const fieldName = parent.farmos_field_2.value[0].name;
    const plantingId = parent.farmos_planting_3.value[0].assetId;
    const plantingName = parent.farmos_planting_3.value[0].name;
    const now = new Date(Date.now());
    const dateReadable = "Date: "+now.getDate()+
          "/"+(now.getMonth()+1)+
          "/"+now.getFullYear()+
          " "+now.getHours()+
          ":"+now.getMinutes()+
          ":"+now.getSeconds();

    let activity = {
        url: url,
        type: 'farmos',
        farmosType: 'log',
        body: {
            timestamp: `${Math.round(Date.now() / 1000, 1)}`,
            name: `Test: ${dateReadable}, field ${fieldId}: ${fieldName}, planting ${plantingId}: ${plantingName}`,
            type: 'farm_activity',
            log_category: [{ name: 'tillage' }], // required, taxonomy
            area: [{ id: fieldId }], // optional
            done: 1,
        },
    };
    activity.body.asset = [{ id: plantingId }]; // if track by planting, add reference to planting
    activities.push(activity);
    return activities
}
  • You can also try to just import this file as your test survey, it contains the questions and apiCompose. See file here: farmOS_test_survey.json
  • Now review in the submission pane of the builder view and confirm:
    • farmOS Farm --> dropdown contains ourscitest and oursci instances
    • farmOS Field --> dropdown contains any fields pulled from those instances
    • farmOS Planting --> dropdown contains any plantings from these instances
  • confirm: - log into the respective instances using the testing login to confirm the presence of the plantings and fields.
  • open a new tab and complete this survey (not in the builder)
  • confirm: upon submission, you see a farmOS link to a log created. Also, log into the actual farmOS instance and confirm the presence and correct formatting of the submitted data.
  • possible failures
    • if the data is incorrectly formed, the data won't appear in farmOS (no error is thrown so you need to actively check this).
    • if the Our Sci backend is not packaging the data correctly, the data may not appear in farmOS also (same, no error is thrown so you need to check this).
    • if the Our Sci Planting, Farm, or Planting question types arne't outputting the correct structure, you should get a failure during referencing the api compose.

Comments

    Please register or sign in to add a comment.