Documentation as Infrastructure: Scaling Knowledge Across Engineering Teams

Documentation debt slows engineering teams in ways that look fine until the wrong person leaves. The cost surfaces during a payment incident, a migration, or an audit: the written procedure exists but the deep system understanding doesn’t. The documentation that survives that moment has three layers: decisions captured as RFCs and ADRs, system reference generated from the systems themselves, and process docs written from real examples. Everything else is tooling and the review habits that keep those three layers current.

Wiki Graveyards and README Roulette

The same shapes keep showing up, whatever the tool. A Confluence space grows to thousands of pages and most of them go stale, so a search for “authentication” returns hundreds of hits with no way to tell which one is current. README quality runs from excellent to absent because every repository sets its own standard, and new engineers end up guessing which file to trust. The architectural decision that would settle the guess usually lives in a #general thread nobody can find, or in a DM nobody else can read.

Then there is the billing system that exactly one person understands. They answer questions about it all day, and the knowledge walks out with them. What does get written down often lands in hundreds of Google Docs with no shared format, which turns “why was this designed this way” into an archaeology problem.

The tool in use matters far less than how a team thinks about information architecture. Adding more pages to Confluence or Notion won’t fix it; without ownership and a review habit, even a well-structured wiki goes stale.

Documentation Debt vs Technical Debt

We spend a lot of time discussing technical debt, but documentation debt can be even trickier to spot. Technical debt usually shows up in slower deployments or harder maintenance. Documentation debt shows up when teams start second-guessing decisions they made six months ago because no one remembers the reasoning.

The cost lands in three places. Day to day, it lands on onboarding time, on the interruptions senior engineers absorb, and on two teams solving the same problem without knowing it. Underneath that sit the quieter costs: repeating mistakes the organization already made, and hesitating to change systems nobody can explain. In a crisis it becomes an incident that runs long because the person who understands the system is unreachable, or an audit where nobody can reconstruct why a control was designed that way.

A Three-Layer Documentation Approach

Each layer has its own update rhythm, which is the reason to keep them apart:

The Decision Record

This is where the reasoning behind choices lives. Not what you built, but why you built it that way.

/docs
  /decisions  # ADRs - architecture decisions made
  /proposals  # RFCs - future changes being considered  
  /discussions  # RFDs - open problems being explored

Size the document to the blast radius. A reversible, single-team call fits in a one to two page mini-RFC turned around in a week. A multi-team change with real investment behind it justifies five to ten pages and a two to four week review. Company-wide architecture shifts run longer on both counts.

System Reference That Regenerates Itself

This describes your current reality. What exists, how it connects, who owns it.

/systems
  /service-catalog  # What services exist, who owns them
  /architecture  # How systems connect and communicate
  /runbooks  # How to operate and troubleshoot
  /dependencies  # What depends on what

This layer works best when it is mostly automated. Hand-written system docs go stale the moment they are finished. Documentation derived from OpenAPI schemas, Terraform state, or CI pipelines updates alongside the change that made it stale.

Process Docs Built From Examples

How you work, how you make decisions, how you handle incidents.

/processes
  /engineering  # How we design, build, and review
  /oncall  # How we respond to incidents
  /releases  # How we deploy and rollback
  /hiring  # How we evaluate and onboard

Concrete examples work better here than abstract guidelines. A walkthrough of how one incident was actually handled gives a new engineer more to copy from than a list of principles does.

The Design Doc Format

Google’s design doc culture is the version of this with published guidance behind it: a collaborative technical document that goes through peer review, with the argument in the text and the discussion in the comments. The elements that carry the weight:

• Context and Scope - What are we solving?
• Goals and Non-Goals - What success looks like
• Design - How we’ll solve it
• Alternatives - What we considered and rejected
• Cross-cutting Concerns - Security, performance, monitoring

Alternatives is the section teams skip. Without it, the next reader cannot tell whether the obvious option was considered and rejected, or never considered at all.

Validating Docs in CI

The checks that keep a docs tree honest belong in the same pipeline as the code:

# .github/workflows/docs.yml
name: Documentation Infrastructure
on:
  pull_request:
    paths: ['docs/**', 'adr/**', 'rfcs/**']

jobs:
  validate-documentation:
    runs-on: ubuntu-latest
    steps:
      - name: Validate RFC format
        run: |
          # Check required sections exist
          # Validate YAML frontmatter
          # Ensure decision status is valid
      
      - name: Check broken links
        run: |
          # Scan for dead internal links
          # Verify external links return 200
          # Flag links to deprecated services
          
      - name: Generate architecture diagrams
        run: |
          # Auto-generate from PlantUML source
          # Update system dependency graphs
          # Create visual service maps
          
      - name: Update search index
        run: |
          # Index new content for searchability
          # Tag documents with metadata
          # Update recommendation engine

Tool stack that works well in practice:

• MkDocs Material - Beautiful, searchable documentation sites
• PlantUML/Mermaid - Version-controlled architecture diagrams
• ADR-tools - Command-line decision record management
• GitHub Actions - Automated validation and publishing

The DACI Framework for Documentation Decisions

For any significant technical decision, the DACI framework (Driver, Approver, Contributors, Informed) keeps the roles clear:

# RFC-042: Database Migration Strategy

## DACI Matrix
- **Driver:** Database Team Lead
  - Responsible for gathering input and driving to decision
  - Owns the timeline and process
  
- **Approver:** VP Engineering  
  - Makes the final call
  - Accountable for the outcome
  
- **Contributors:** Backend Teams, SRE, Security, Data Engineering
  - Provide input and expertise
  - Will be impacted by the decision
  
- **Informed:** All Engineering, Product, Finance
  - Need to know the outcome
  - May need to adjust their plans

## Decision Timeline
- **Week 1:** Stakeholder interviews and requirements gathering
- **Week 2:** Technical evaluation and proof of concepts
- **Week 3:** Cost analysis and migration planning
- **Week 4:** Final decision and communication

This framework helps avoid the “too many cooks” situation while still making sure people feel heard. Getting the balance right takes some trial and error.

Documentation Champions

Documentation culture is hard to mandate from above; it grows or it doesn’t. What can be arranged are the conditions. One arrangement that works is naming a documentation champion per team: someone who runs the RFC reviews, checks that new systems ship with docs, flags gaps and stale pages, and coaches teammates on the templates. It is a few hours a week at most, and the role should rotate before it turns into a permanent second job.

Metrics That Predict Trouble

Many teams track things that don’t necessarily correlate with documentation health. Here is what tends to be more useful to measure:

interface DocumentationHealth {
  // Leading indicators (predict future problems)
  rfcParticipation: number;  // % engineers participating in RFC reviews
  docUpdateFrequency: number;  // Average days since last update
  knowledgeDistribution: number;  // % of systems with >1 expert
  
  // Lagging indicators (measure current state)
  onboardingVelocity: number;  // Days from hire to first commit
  crossTeamQuestions: number;  // Questions requiring cross-team knowledge
  
  // Quality indicators (measure documentation value)
  documentRelevance: number;  // % of docs accessed in last 90 days
  linkHealth: number;  // % of internal links that work
  searchSuccess: number;  // % of searches that find answers
}

In a monthly review, the questions worth asking are which knowledge gaps caused delays, which questions came up more than once, which documents are going stale, and where people are looking for answers outside the documentation system.

Where Documentation Changes the Outcome

The on-call engineer is often someone who has never run the rollback procedure, and the person who built the system may be unreachable that night. A runbook earns its place when it has been rehearsed and updated often enough to still work under that pressure.

Integrations after an acquisition or a reorg put the same problem on a slower clock. Most of the schedule goes to reconstructing why the other side built things the way they did. Where that reasoning already exists as ADRs and design docs, the reconstruction step drops out of the plan and the migration work starts earlier.

Audits ask the question in writing. Auditors want to know why access controls and data handling were designed the way they are, and the current configuration alone does not carry that reasoning; a trail of ADRs is what carries it. Without one, the team rebuilds its own rationale from memory while the audit clock runs.

Picking a Documentation Tool

The choice usually comes down to constraints. A hard Jira dependency or a compliance requirement pins you to Confluence. A security policy that blocks third-party SaaS pins you to SharePoint. What is left comes down to who writes and who reads: product specs go where the PMs already work, API docs go where they can sync from OpenAPI specs, and engineering decisions go next to the code that implements them.

Making a Corporate Wiki Survivable

When Confluence or SharePoint is mandatory, most of the pain comes from how the space is organized. Split spaces by audience so engineering RFCs and product PRDs never share a search index. Put the date in the page title ([2024-01-22] Database Migration RFC), because chronology survives a bad search result. Template every page type, archive old pages with labels so they stop competing with current ones, and keep OneNote out of the wiki role; its search will not hold up. On SharePoint, checkout/checkin covers version control and Power Automate covers approvals.

Service Catalog Entries

Backstage starts earning its keep when “which service is this and who owns it” no longer has an obvious answer. The catalog entry becomes the documentation, and it lives in the service repository:

apiVersion: backstage.io/v1alpha1
kind: Component
metadata:
  name: payment-service
  description: Handles payment processing
  annotations:
    docs: https://docs.internal/payment
    pagerduty: PD123
spec:
  type: service
  owner: platform-team
  lifecycle: production

Tool Selection Matrix

Use Case	First Choice	Alternative	Avoid
Engineering RFCs	GitHub + MkDocs	GitBook	SharePoint
Product Documentation	Notion	Confluence	Word Docs
API Docs	GitBook	Backstage	Wiki
Runbooks	MkDocs	Confluence	OneNote
Knowledge Base	Obsidian	Notion	Folders
Service Catalog	Backstage	Custom	Excel

Leaving Confluence Without Breaking Links

The redirect map is the step teams skip. Without it, every bookmark and every link ever pasted into a Slack thread dies on migration day:

# 1. Export Confluence space
confluence-export --space ENG --format markdown

# 2. Transform to MkDocs structure
python transform_confluence.py --input export/ --output docs/

# 3. Setup redirects for old URLs
# mkdocs.yml
plugins:
  - redirects:
      redirect_maps:
        'old-page.md': 'new-structure/page.md'

The Stack Most Teams End Up With

Most organizations use multiple tools:

documentation_stack:
  decisions:
    tool: GitHub + ADR-tools
    reason: "Version control and code review"
  
  product_specs:
    tool: Notion
    reason: "Easy for PMs, rich formats"
  
  runbooks:
    tool: Confluence
    reason: "On-call engineers are familiar"
  
  api_docs:
    tool: GitBook
    reason: "Auto-sync with OpenAPI specs"
  
  knowledge_base:
    tool: Obsidian
    reason: "Connected knowledge graph"

A split stack only works when everyone knows which tool holds what. The question “Where’s the RFC?” should have a single answer that any team member can give without searching several systems first.

Consistency and Freshness

Shared templates and a shared review path scale better than letting every team invent its own format. A reader who knows the shape of one RFC can find their way through any of them, including ones written by teams they have never worked with.

Freshness matters more than polish. A decent document that gets updated beats a perfect one that has gone stale, so the effort belongs in making updates cheap: a review slot inside a meeting that already exists, a link checker in the pipeline, an owner named on the page.

When This Approach Pays Off

Documentation-as-infrastructure pays off when knowledge turnover is high, systems are complex enough that no single person holds the full picture, or onboarding friction regularly slows delivery. It is a poor fit for single-engineer projects or short-lived prototypes where the cost of maintenance outweighs the benefit. A reasonable first step is a single ADR covering the decision that caused the most confusion this quarter.

References

• Diátaxis Framework - A systematic approach to technical documentation that distinguishes tutorials, how-to guides, reference, and explanation by user need
• Software Documentation Guide - Write the Docs - Community-maintained guide covering documentation process, tooling, and best practices from practitioners
• Architectural Decision Records - Reference hub for ADR formats, templates (including MADR), and tooling for capturing and tracking architectural decisions
• Documentation Best Practices - Google Style Guides - Google’s engineering documentation guidelines, including docs-as-code principles and freshness practices
• Software Engineering at Google - Documentation Chapter - In-depth chapter from the Google SWE book on how documentation is treated as code at scale