TechDocs Documentation Pipelines

Shipping accurate documentation alongside every service requires a docs-as-code pipeline that generates, builds, and publishes TechDocs directly from the same repository that holds the code. This page targets platform engineers and tech leads who need to wire TechDocs into the Backstage service catalog so that documentation is versioned, validated, and rendered next to each entity without manual upload steps.

TechDocs is the documentation system built into Backstage. It treats Markdown as a first-class artifact: authors write Markdown in their service repo, a generator (MkDocs) renders it to static HTML, a publisher pushes the output to object storage, and the Backstage frontend reads it back and renders it on the entity page. Decoupling these four stages — generate, build, store, serve — is what lets documentation scale to thousands of repositories without overwhelming the portal backend.

Prerequisites & Environment Baseline

Before wiring a pipeline, establish a stable toolchain. The TechDocs generator runs MkDocs under the hood, so you need Python 3.11+ with mkdocs and the mkdocs-techdocs-core package, plus Node.js 20+ to run @techdocs/cli. Your Backstage instance must be on @backstage/plugin-techdocs-backend 1.10.0 or later. For object storage, provision an S3 bucket (or GCS equivalent) with a dedicated IAM role rather than long-lived keys. This page assumes you have already completed a Backstage Architecture Deep Dive and have a running catalog, and that authors are comfortable with the Markdown conventions covered in MkDocs for Internal Docs.

# Requires Python >= 3.11, Node.js >= 20.0.0
pip install mkdocs-techdocs-core==1.3.3
npm install -g @techdocs/[email protected]

# Verify the toolchain
mkdocs --version            # mkdocs, version 1.6.1
techdocs-cli --version      # 1.8.13

Set the documentation build strategy to external so heavy MkDocs renders run in CI, not inside the portal backend. The backend then only reads pre-built HTML from storage.

# app-config.yaml — requires @backstage/plugin-techdocs-backend >= 1.10.0
techdocs:
  builder: 'external'        # CI builds; backend never invokes MkDocs
  generator:
    runIn: 'local'
  publisher:
    type: 'awsS3'
    awsS3:
      bucketName: ${TECHDOCS_S3_BUCKET}
      region: ${AWS_REGION}

Step-by-Step Configuration & Plugin Architecture

1. Annotate the catalog entity. TechDocs locates the docs source via the backstage.io/techdocs-ref annotation. The dir:. value tells the generator the mkdocs.yml lives at the repo root.

   # catalog-info.yaml — requires Backstage >= 1.20.0
   apiVersion: backstage.io/v1alpha1
   kind: Component
   metadata:
     name: payments-api
     annotations:
       backstage.io/techdocs-ref: dir:.
   spec:
     type: service
     owner: group:default/payments
     lifecycle: production
   

2. Add an mkdocs.yml and docs/ tree to the service repo. The techdocs-core plugin bundles the theme, admonitions, and syntax highlighting the Backstage reader expects.

   # mkdocs.yml — requires mkdocs-techdocs-core >= 1.3.0
   site_name: payments-api
   plugins:
     - techdocs-core
   nav:
     - Home: index.md
     - Runbook: runbook.md
   

3. Build and publish in CI. The generator renders Markdown to HTML; the publisher uploads it under an entity-namespaced prefix (default/component/payments-api) that the backend reads on demand. The end-to-end CI recipe is covered in How to Publish TechDocs to S3 with CI.

   # Requires @techdocs/cli >= 1.8.0
   techdocs-cli generate --no-docker --source-dir . --output-dir ./site
   techdocs-cli publish \
     --publisher-type awsS3 \
     --storage-name ${TECHDOCS_S3_BUCKET} \
     --entity default/component/payments-api \
     --directory ./site
   

4. Wire the frontend reader. In packages/app/src/App.tsx, the TechDocsReaderPage route renders stored HTML inside the entity layout. No backend MkDocs invocation occurs at read time, which keeps the portal responsive even with thousands of doc sites.

Validation & Health Checks

Validate each stage independently. A strict local build catches broken links and missing nav entries before anything reaches storage.

# Fail the build on broken references — requires mkdocs >= 1.6
mkdocs build --strict
# Expected: "INFO - Documentation built in 0.84 seconds" with exit code 0

# Confirm the published artifact landed at the expected prefix
aws s3 ls s3://${TECHDOCS_S3_BUCKET}/default/component/payments-api/
# Expected: index.html, techdocs_metadata.json, search/search_index.json

# Verify the backend can serve it
curl -sf "http://localhost:7007/api/techdocs/static/docs/default/component/payments-api/index.html" \
  | head -n 5
# Expected: HTML <!doctype html> output, HTTP 200

The presence of techdocs_metadata.json is the canonical signal that a publish succeeded — the backend uses its build_timestamp to decide whether cached content is stale.

Maintenance & Lifecycle Management

Pin mkdocs-techdocs-core and @techdocs/cli to exact patch versions in CI; theme regressions between minor releases are the most common source of silent rendering drift. Run an upgrade in a feature branch and diff the rendered site/ output before merging. For rollback, S3 object versioning lets you restore a prior index.html set without rebuilding. Monitor the backend’s techdocs_cache_hit_ratio metric — a falling ratio usually means entities are being rebuilt too often or the storage prefix is misconfigured. As your catalog grows, front the storage bucket with a CDN and set long-lived immutable cache headers on hashed assets while keeping techdocs_metadata.json uncached so freshness checks stay accurate.

# Inspect freshness for one entity
aws s3api head-object --bucket ${TECHDOCS_S3_BUCKET} \
  --key default/component/payments-api/techdocs_metadata.json \
  --query 'LastModified'

Common Pitfalls & Mitigation Strategies

• Leaving builder: 'local' in production: Forces the backend to run MkDocs on every cache miss, exhausting CPU under load. Mitigation: Set builder: 'external' and build exclusively in CI.
• Mismatched entity reference and storage prefix: Causes 404s on the entity docs tab even though the build succeeded. Mitigation: Always publish with --entity <namespace>/<kind>/<name> matching the catalog entity exactly.
• Skipping --strict builds: Broken internal links ship silently to readers. Mitigation: Gate CI on mkdocs build --strict.
• Embedding diagrams as binary images: Bloats repos and breaks dark mode. Mitigation: Author diagrams as code, as shown in Writing TechDocs with MkDocs and Mermaid Diagrams.
• Hardcoding AWS keys in CI: Violates least-privilege and complicates rotation. Mitigation: Use OIDC-federated short-lived roles via aws-actions/configure-aws-credentials.

Frequently Asked Questions

Should documentation be built by the Backstage backend or in CI?

For any production catalog, build in CI using the external strategy. The local builder is convenient for evaluation but makes the backend responsible for CPU-heavy MkDocs renders, which does not scale past a handful of entities and couples doc latency to portal availability.

Can I use a framework other than MkDocs for TechDocs?

The TechDocs reader expects the HTML structure and techdocs_metadata.json that the techdocs-core MkDocs plugin produces. You can render from any tool that emits that exact contract, but in practice teams either use MkDocs for catalog-integrated docs or run a standalone site such as Docusaurus for marketing-style portals. Compare the trade-offs in the framework decision guidance below.

How do I migrate existing Markdown docs into a TechDocs pipeline?

Move the Markdown under a docs/ directory, add an mkdocs.yml with the techdocs-core plugin, add the techdocs-ref annotation to catalog-info.yaml, and run the generate-and-publish steps in CI. Existing relative links between Markdown files are preserved as long as the directory layout is kept.

Related

• Up: Developer Portal Architecture & Frameworks
• How to Publish TechDocs to S3 with CI
• Writing TechDocs with MkDocs and Mermaid Diagrams
• Docusaurus Setup & Customization