Developer Experience

Guide: Integrating Argos Visual Testing in Vercel Preview

February 1, 2024

|

Jeremy Sfez

Argos now seamlessly integrates with Vercel Preview for visual testing, offering a streamlined approach to catch visual regressions in a production-like environment — without the hassle of complex setups. This guide will walk you through implementing Argos with Vercel Preview via GitHub Actions, complete with source code insights.

Why Visual Testing with Vercel Preview? Key Advantages

• Direct Visual Verification: Allows for immediate identification of visual changes in pull requests, ensuring what you see is what you get in production.
• Cost Reduction: Reduces CI expenses by eliminating the need to duplicate test environments.
• Production-like Environment Testing: Offers a closer approximation to the live site, increasing test reliability.

Note: This method is optimal for repeatable tests that don't significantly alter the environment, as certain tests (e.g. those involving user deletion) may not be applicable.

Getting Started with Argos: Setup Guide

Prerequisites

Before integrating Argos into Vercel Preview for visual testing, I ensured:

1. Set up a Next.js project with Playwright.
2. Integrated Playwright tests into CI/CD.
3. Deployed the project on Vercel.

1. Project Importation to Argos

I signed up into Argos and imported my project. Once done, Argos display an ARGOS_TOKEN for this repository. I saved it for the next step.

I signed up into Argos on Argos and added my project. After this, Argos generated an ARGOS_TOKEN for my repository. I saved it for later.

2. Package Installation

npm install --save-dev @argos-ci/playwright

3. Setup Argos in Playwright config file

Update the playwright.config.ts file:

import { defineConfig } from "@playwright/test";

export default defineConfig({
  reporter: [
    process.env.CI ? ["dot"] : ["list"],
    [
      "@argos-ci/playwright/reporter",
      {
        uploadToArgos: !!process.env.CI,
        token: process.env.ARGOS_TOKEN,
      },
    ],
  ],
  use: {
    trace: "on-first-retry",
    screenshot: "only-on-failure",
  },
});

4. Screenshot my app

I utilized the Screenshot pages script from Argos's documentation for capturing my application's homepage. This script is flexible for adding more pages in the future.

import { argosScreenshot } from "@argos-ci/playwright";
import { test } from "@playwright/test";

const baseUrl = "http://localhost:3000";
const pages = [
  { name: "homepage", path: "/" },
  // { name: "faq", path: "/faq" },
  // { name: "contact", path: "/contact-us" },
  // { name: "pricing", path: "/pricing" },
];

test("screenshot pages", async ({ page }, workerInfo) => {
  for (const { name, path } of pages) {
    const browserName = workerInfo.project.name;
    await page.goto(`${baseUrl}${path}`);
    await argosScreenshot(page, `${name}-${browserName}`);
  }
});

5. Set up ARGOS_TOKEN and test integration

First, I set up ARGOS_TOKEN as a repository secret on GitHub. Then, I pushed my branch to Github and monitored the status checks to ensure everything was functioning as expected: Playwright, Vercel, and Argos.

Run Argos on Vercel Preview

To run my tests on Vercel Preview, I updated my CI workflow file:

name: Playwright + Argos Tests
# 👉 Trigger on deployment event
on: deployment_status:
jobs:
  test:
    # 👉 Run tests only if the deployment is successful
    if: github.event_name == 'deployment_status' && github.event.deployment_status.state == 'success'
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v3
      - name: Install dependencies
        run: npm ci && npx playwright install --with-deps
      - name: Run Playwright tests
        run: npx playwright test
        # 👉 Add this variables
        env:
          BASE_URL: ${{ github.event.deployment_status.environment_url }}
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
          ARGOS_BRANCH: ${{ github.event.deployment_status.environment == 'Production' && 'main' || '' }}

For a comprehensive explanation of each step, read Argos documentation.

Then, I updated my Playwright's config file:

export default defineConfig({
  use: {
    // URL generated for the preview
    baseURL: process.env.BASE_URL,
  },
  extraHTTPHeaders: {
    // Hide vercel toolbar in tests
    "x-vercel-skip-toolbar": "0",
  },
});

After this my are merged, upon a successful deploy, the E2E workflow start on the preview:

Explore the Source Code

Explore my GitHub Repository to read to code source of this integration. In the Pull Request #2 where I migrate the tests to Vercel Preview deployment:

• Github Repository
• Pull Request #2 - test: run argos on vercel preview
• Deployed Project

Conclusion

Vercel Preview and Argos serve complementary roles in ensuring web development excellence. Vercel Preview focuses on verifying expected outcomes, while Argos excels at detecting visual regressions across various viewports. Together, they strengthen the development workflow, allowing developers to harness the best of both tools for unparalleled visual consistency and reliability.