Overview

This guide shows how to integrate Azure DevOps Pipelines with TestCollab using the TestCollab CLI (@testcollab/cli). After your tests run in Azure Pipelines, the CLI automatically uploads results to TestCollab so your team can track test execution without manual effort.

How It Works

Azure Pipelines runs your tests and generates a report file (JUnit XML, Mochawesome JSON, etc.)
The TestCollab CLI creates a test plan in your project using tc createTestPlan
The CLI uploads results to TestCollab using tc report, matching test cases by ID

Prerequisites

A TestCollab account with test cases created in your project
A TestCollab API key (how to generate one)
An Azure DevOps project with a pipeline configured

Step 1: Add Variables to Your Pipeline

Store your TestCollab credentials as pipeline variables in Azure DevOps. Go to your pipeline > Edit > Variables and add:

TESTCOLLAB_TOKEN — your TestCollab API key (mark as secret)
TC_PROJECT_ID — your TestCollab project ID
TC_CI_TAG_ID — the tag ID used to filter test cases for CI
TC_ASSIGNEE_ID — the user ID to assign test executions to

Step 2: Configure Your Pipeline YAML

Below are complete azure-pipelines.yml examples for popular test frameworks.

Playwright (JUnit XML)

trigger:
  - main
pool:
  vmImage: 'ubuntu-latest'
steps:
  - task: NodeTool@0
    inputs:
      versionSpec: '20.x'
    displayName: 'Install Node.js'
  - script: npm ci
    displayName: 'Install dependencies'
  - script: npx playwright install --with-deps
    displayName: 'Install Playwright browsers'
  - script: |
      npm install -g @testcollab/cli
      tc createTestPlan --project $(TC_PROJECT_ID) --ci-tag-id $(TC_CI_TAG_ID) --assignee-id $(TC_ASSIGNEE_ID)
    displayName: 'Create test plan'
    env:
      TESTCOLLAB_TOKEN: $(TESTCOLLAB_TOKEN)
  - script: |
      source tmp/tc_test_plan
      echo "##vso[task.setvariable variable=TESTCOLLAB_TEST_PLAN_ID]$TESTCOLLAB_TEST_PLAN_ID"
    displayName: 'Load test plan ID'
  - script: npx playwright test --reporter=junit || true
    displayName: 'Run Playwright tests'
    env:
      PLAYWRIGHT_JUNIT_OUTPUT_NAME: results/junit-report.xml
  - script: |
      tc report --project $(TC_PROJECT_ID) --test-plan-id $(TESTCOLLAB_TEST_PLAN_ID) --format junit --result-file results/junit-report.xml
    displayName: 'Upload results to TestCollab'
    condition: always()
    env:
      TESTCOLLAB_TOKEN: $(TESTCOLLAB_TOKEN)

Cypress (Mochawesome JSON)

trigger:
  - main
pool:
  vmImage: 'ubuntu-latest'
steps:
  - task: NodeTool@0
    inputs:
      versionSpec: '20.x'
    displayName: 'Install Node.js'
  - script: npm ci
    displayName: 'Install dependencies'
  - script: |
      npm install -g @testcollab/cli
      tc createTestPlan --project $(TC_PROJECT_ID) --ci-tag-id $(TC_CI_TAG_ID) --assignee-id $(TC_ASSIGNEE_ID)
    displayName: 'Create test plan'
    env:
      TESTCOLLAB_TOKEN: $(TESTCOLLAB_TOKEN)
  - script: |
      source tmp/tc_test_plan
      echo "##vso[task.setvariable variable=TESTCOLLAB_TEST_PLAN_ID]$TESTCOLLAB_TEST_PLAN_ID"
    displayName: 'Load test plan ID'
  - script: npx cypress run --reporter mochawesome --reporter-options reportDir=results,overwrite=false,html=false,json=true || true
    displayName: 'Run Cypress tests'
  - script: |
      tc report --project $(TC_PROJECT_ID) --test-plan-id $(TESTCOLLAB_TEST_PLAN_ID) --format mochawesome --result-file results/mochawesome.json
    displayName: 'Upload results to TestCollab'
    condition: always()
    env:
      TESTCOLLAB_TOKEN: $(TESTCOLLAB_TOKEN)

pytest (JUnit XML)

trigger:
  - main
pool:
  vmImage: 'ubuntu-latest'
steps:
  - task: UsePythonVersion@0
    inputs:
      versionSpec: '3.11'
    displayName: 'Use Python 3.11'
  - script: pip install -r requirements.txt
    displayName: 'Install dependencies'
  - script: pytest --junitxml=results/junit-report.xml || true
    displayName: 'Run pytest'
  - task: NodeTool@0
    inputs:
      versionSpec: '20.x'
    displayName: 'Install Node.js'
  - script: |
      npm install -g @testcollab/cli
      tc createTestPlan --project $(TC_PROJECT_ID) --ci-tag-id $(TC_CI_TAG_ID) --assignee-id $(TC_ASSIGNEE_ID)
    displayName: 'Create test plan'
    env:
      TESTCOLLAB_TOKEN: $(TESTCOLLAB_TOKEN)
  - script: |
      source tmp/tc_test_plan
      echo "##vso[task.setvariable variable=TESTCOLLAB_TEST_PLAN_ID]$TESTCOLLAB_TEST_PLAN_ID"
    displayName: 'Load test plan ID'
  - script: |
      tc report --project $(TC_PROJECT_ID) --test-plan-id $(TESTCOLLAB_TEST_PLAN_ID) --format junit --result-file results/junit-report.xml
    displayName: 'Upload results to TestCollab'
    condition: always()
    env:
      TESTCOLLAB_TOKEN: $(TESTCOLLAB_TOKEN)

Java / Maven (JUnit XML)

trigger:
  - main
pool:
  vmImage: 'ubuntu-latest'
steps:
  - task: Maven@4
    inputs:
      mavenPomFile: 'pom.xml'
      goals: 'test'
      options: '-Dmaven.test.failure.ignore=true'
    displayName: 'Run Maven tests'
  - task: NodeTool@0
    inputs:
      versionSpec: '20.x'
    displayName: 'Install Node.js'
  - script: |
      npm install -g @testcollab/cli
      tc createTestPlan --project $(TC_PROJECT_ID) --ci-tag-id $(TC_CI_TAG_ID) --assignee-id $(TC_ASSIGNEE_ID)
    displayName: 'Create test plan'
    env:
      TESTCOLLAB_TOKEN: $(TESTCOLLAB_TOKEN)
  - script: |
      source tmp/tc_test_plan
      echo "##vso[task.setvariable variable=TESTCOLLAB_TEST_PLAN_ID]$TESTCOLLAB_TEST_PLAN_ID"
    displayName: 'Load test plan ID'
  - script: |
      tc report --project $(TC_PROJECT_ID) --test-plan-id $(TESTCOLLAB_TEST_PLAN_ID) --format junit --result-file target/surefire-reports/TEST-*.xml
    displayName: 'Upload results to TestCollab'
    condition: always()
    env:
      TESTCOLLAB_TOKEN: $(TESTCOLLAB_TOKEN)

How the Pipeline Works

|| true (or -Dmaven.test.failure.ignore=true for Maven) ensures the pipeline continues to the upload step even when tests fail — this is important so that failed results are still reported to TestCollab.
condition: always() on the upload step ensures it runs regardless of whether previous steps passed or failed.
tc createTestPlan creates a new test plan in TestCollab and writes the plan ID to tmp/tc_test_plan.
source tmp/tc_test_plan loads the TESTCOLLAB_TEST_PLAN_ID variable, which is then set as an Azure pipeline variable using ##vso[task.setvariable].
tc report parses the test report file and updates the test case results in TestCollab.

Custom API URL

By default the CLI connects to https://api.testcollab.io. If your TestCollab instance uses a different API URL, add the --api-url flag to both tc createTestPlan and tc report commands:

tc createTestPlan --project $(TC_PROJECT_ID) --ci-tag-id $(TC_CI_TAG_ID) --assignee-id $(TC_ASSIGNEE_ID) --api-url https://your-api-url.example.com
tc report --project $(TC_PROJECT_ID) --test-plan-id $(TESTCOLLAB_TEST_PLAN_ID) --format junit --result-file results/junit-report.xml --api-url https://your-api-url.example.com

Mapping Test Case IDs

For the CLI to match test results with TestCollab test cases, include the test case ID in your test names. Supported patterns:

TC-123
id-123
testcase-123

For example, in Playwright:

test('TC-1045 Login with valid credentials', async ({ page }) => {
  // test code
});

Troubleshooting

Pipeline fails at upload step — verify that your pipeline variables are set correctly and that the API key has not expired.
No results updated in TestCollab — ensure your test names include a recognized test case ID pattern (TC-123, id-123, or testcase-123).
Authentication error — check that TESTCOLLAB_TOKEN is a valid API key. If using a custom API URL, make sure to include the --api-url flag.

For the full CLI reference, see Uploading Test Results Using TestCollab CLI.

For any queries or feedback, contact us at [email protected].

GitLab CI/CD Integration with TestCollab

Jenkins Integration with TestCollab for CI/CD Pipelines

Uploading Test Results Using TestCollab CLI

GitHub Actions Integration with TestCollab for CI/CD pipelines

Bitbucket Pipelines Integration with TestCollab for CI/CD pipelines