End-to-End Testing Plugins with Playwright

Unit tests prove a plugin’s functions work; they say nothing about whether the tab actually renders, the API call actually returns, and the button actually does something when a real user clicks it. Playwright drives a real browser against a running Backstage, catching the integration breaks unit tests miss. This how-to sets up end-to-end tests for a portal plugin. It extends Plugin Testing & Publishing within Plugin Ecosystem & Custom Extensions.

Prerequisites

  • Backstage 1.20+ that runs locally or in CI with a seedable catalog.
  • Playwright installed with its browsers, and a way to start the app for tests.
  • Stable selectors on the elements you assert against — data-testid beats brittle text or CSS paths.
  • A strategy for backend data: seed fixtures or mock external calls so runs are deterministic.
Prerequisites for Playwright E2E A runnable app, Playwright, stable selectors, and deterministic data enable reliable end-to-end tests. runnable app Playwright + browsers stable selectors deterministic data Real-browser confidence
Determinism is the hard part — a test that depends on live external data will flake, so seed fixtures or mock the calls the plugin makes.

Exact Configuration

  1. Configure Playwright to start the app and run against it.

    // playwright.config.ts
// Requires @playwright/test >= 1.44
import { defineConfig } from '@playwright/test';
export default defineConfig({
  webServer: { command: 'yarn start', url: 'http://localhost:3000', timeout: 120_000 },
  use: { baseURL: 'http://localhost:3000', trace: 'on-first-retry' },
});

  2. Add stable selectors to the plugin’s key elements.

    // in the plugin component
<button data-testid="cost-refresh">Refresh</button>

  3. Write a test that drives the plugin like a user and asserts on the result.

    // e2e/cost-tab.spec.ts
import { test, expect } from '@playwright/test';

test('cost tab renders a series for a service', async ({ page }) => {
  await page.goto('/catalog/default/component/payments-api/cost');
  await page.getByTestId('cost-refresh').click();
  await expect(page.getByTestId('cost-chart')).toBeVisible();
  await expect(page.getByText('USD')).toBeVisible();
});
The E2E test loop Playwright starts the app, drives the plugin in a real browser, and asserts the rendered result. start app webServer drive plugin click, navigate assert result visible, text pass / fail + trace
The trace on first retry is your best debugging asset — it captures the DOM, network, and console at the moment a flaky test failed.

Validation

# Requires playwright installed
# 1. The suite runs green locally
npx playwright test
# Expected: all specs pass

# 2. Selectors are stable (no reliance on brittle text/CSS)
grep -rc "getByTestId" e2e/ | tail -1
# Expected: assertions favor testids over deep CSS paths

# 3. Runs are deterministic across repeats
npx playwright test --repeat-each=3
# Expected: consistently green, no flakes
Playwright validation checks The suite passes, selectors are stable, and repeated runs stay green. Suite green all specs pass Selectors stable testids used No flakes repeat-each green
The --repeat-each pass is the flake detector — a test that only fails one run in ten is worse than useless in CI.

Edge Cases & Troubleshooting

Symptom Root Cause Resolution
Test flakes intermittently Waiting on timing, not state Use expect(...).toBeVisible() auto-waits, not sleep
Passes locally, fails in CI Different data or timing Seed the same fixtures; raise the server timeout
Selector breaks on refactor Asserting on text or deep CSS Switch to data-testid
External call makes it slow Hitting a real upstream Mock the route with Playwright’s request interception
Auth blocks the test No test login Use a storage-state login fixture once, reuse it
Playwright pitfalls by class Flakes and CI-only failures are determinism issues; brittle selectors and slow external calls are test-design issues. Determinism flaky CI-only fail → auto-wait + seed Test design brittle selector slow external → testid + mock route
Auto-waiting assertions eliminate most flakes — reaching for a fixed sleep is the single most common cause of a brittle suite.

Frequently Asked Questions

How many E2E tests should a plugin have?

Few, but high-value. E2E is slow and expensive, so reserve it for the critical user journeys — the tab loads, the primary action works, an error state shows — and lean on unit tests for the branchy logic. A handful of solid E2E flows per plugin catches the integration breaks without turning CI into a half-hour wait.

Should E2E hit real backends?

Mock the external ones, keep the portal’s own backend real. The value of E2E is exercising the actual plugin-to-backend wiring, so stubbing that away defeats the point; but a live third-party API makes runs slow and flaky, so intercept those calls with deterministic responses. Real portal, mocked outside world.