Automated Test Execution

Owned by Nick
Last updated: Apr 01, 2024
2 min read

AssertThat - BDD & Cucumber for Jira displays the execution test status against each User Story (or any Jira issue type) for manual and automated tests.  The manual test status for the scenario can be updated on the user story itself. 

Automated Test Execution

AssertThat provides a number of options for integrating with Test Automation Frameworks. The typical workflow would be

  1. Scenarios developed in AssertThat

  2. Scenarios downloaded using APIs or Integrations into the Test Automation framework

  3. Tests executed and results imported back into AssertThat

Following the import of the Test Automation results back into AssertThat the execution status for the scenarios will be updated for all Jira linked issues. Note it is possible to update specific and independent linked scenarios on import - see https://assertthat.atlassian.net/wiki/spaces/ABTM/pages/2365555064/Manual+Test+Execution#Independent-manual-scenario-execution-status

image-20240401-093446.png
Test Execution status is updated in AssertThat for Automated test scenarios

For Scenario Outlines where multiple examples can be executed under the same scenario are currently counted as a single execution. This is in the roadmap to change so the executions are counted independently.

Independent automated scenario execution status

Automated tests linked to the story will have the status updated following https://assertthat.atlassian.net/wiki/spaces/ABTM/pages/728105165

By default scenario status will be updated across all linked user stories

A JQL filter can be supplied as a parameter with upload report as part of the Integrations or API call, so that only scenarios linked to filtered issues will be updated. The below example of the API is given below and this feature is also available in all the Automation Integrations.

image-20240229-121023.png
Pass JQL to update specific issues only on Report import in API or Test Automation Integration

More documentation on API usage can be found inhttps://assertthat.atlassian.net/wiki/spaces/ABTM/pages/766738433/REST+API+for+Jira+Cloud#Uploading-report and

Automated Results filtering

Test Automation will be loaded by default even if the scenario isn’t matched in the AssertThat Plugin. To filter and only load results for matched BDD Scenarios enable the Automated results filtering toggle in AssertThat configuration project settings.