⌘ The Architecture Reference

Fnd evolution · Foundations · Advanced

Incremental Change and Data

How deployment pipelines, feature toggles, and database evolution patterns let an architecture change in small, safe, reversible steps — including the hardest dimension, data.

Fnd evolution Advanced ⏱ 5 min read Complete

🧱 Analogy

Renovating a house you still live in, you don’t demolish every wall at once. You build the new bathroom, switch over when it works, then remove the old one. Incremental change — especially to data — is exactly this: stand up the new state beside the old, transition while both exist, and only then tear down the old.

Two aspects of incremental change

Building Evolutionary Architectures distinguishes development (how teams build) from operational (how teams deploy) incremental change, both building on Continuous Delivery. Services are operationally isolated in a share-nothing architecture, each deployed in its own container. When a new service version ships, a temporary proxy routes callers to the version they request, so old callers needn’t change and new callers gain capability immediately. When monitoring shows a version is no longer routed to, operations garbage-collects it.

The deployment pipeline

The core mechanism is the deployment pipeline (distinct from a CI server): it listens for changes and runs a series of increasingly sophisticated, separable stages — environment provisioning, multiple levels of testing, and crucially fitness functions.

graph LR
S1["1. Replicate CI<br/>unit + functional"] --> S2["2. Containerize + deploy"]
S2 --> S3["3. Atomic fitness functions<br/>scalability · security · metrics"]
S3 --> S4["4. Holistic fitness functions<br/>contract tests"]
S4 --> S5["5. Manual stages<br/>security review · audit"]
S5 --> S6["6. Automated deploy"]

Feature toggles (an if-statement on config) reconcile continuous deployment with staged “big bang” releases and enable QA in production (routing only QA users to a hidden feature). Fan-out / fan-in lets the pipeline verify in parallel that a change didn’t break production and that the change itself succeeds, then consolidate. Consumer-driven contracts (e.g. Pact) act as an engineering safety net — the consumer hands the provider a test suite the provider promises to keep passing, so the provider can evolve freely.

Data: the hardest dimension

Data is a first-class evolutionary dimension and often more problematic than architectural coupling — databases become integration points, and data teams historically lagged on engineering practices. Evolutionary database design rests on three axes: tested (fitness functions keep ORM mappings in sync), versioned (source and schema are symbiotic), and incremental (automated migration scripts, not manual edits).

⚠️ Migrations are immutable

Modeled on double-entry bookkeeping, once a migration/delta script has run it is immutable — to reverse a change you add a new migration. Most teams have abandoned undo migrations entirely: you can always rebuild forward from script 1, and maintaining both directions doubles testing while making dropped-column reversals daunting.

Expand/Contract for schema change

Shared Database Integration fossilizes a schema across every application that touches it. The remedy is the Expand/Contract pattern (a subset of Parallel Change) with a transition phase that maintains old and new states simultaneously.

graph LR
E["Expand<br/>add new columns/state<br/>(old still works)"] --> T["Transition<br/>both states live<br/>code migrates at its pace"]
T --> C["Contract<br/>drop the old state<br/>once nothing reads it"]
The worked example — splitting name into firstname/lastname — culminates in the hardest case (legacy data and integration points) handled with a bidirectional database trigger plus refinements like SET UNUSED, exploiting native database facilities to evolve safely.

When microservices give each bounded context its own database, native guarantees are replaced by fitness functions: referential integrity becomes event-driven deletion propagation; data duplication distinguishes write ownership from read access (owners publish changes, readers cache on startup); and triggers/stored procedures are retired via the database Strangler Fig (“Migrate Method from Database”), with a fitness function verifying all dependencies have migrated before the procedure is dropped.

Engineering maturity is a prerequisite

Incremental change assumes a certain engineering maturity — for example, not leaving builds broken for days. Without it, the pipeline and parallel-change patterns can’t deliver their safety.

💡 Key insight

Make decisions reversible (blue/green deploys, toggles, canary releasing, service routing) and remove needless variability (immutable infrastructure). The $400M “Knightmare” feature-flag disaster shows what happens when a snowflake server drifts from the rest — reversibility and uniformity are what make fast change safe.

See also

When to use it — and when not

✅ Reach for it when

  • When you need to deploy architectural change in small, reversible increments
  • When evolving a database schema that other applications or services depend on
  • When replacing triggers, stored procedures, or shared-database integration

⛔ Think twice when

  • When engineering maturity is too low (builds left broken for days)
  • When a fossilized schema or COTS package makes incremental change impractical
fnd-evolutionEvolutionary Architecture

Architecture that supports guided, incremental change across multiple dimensions — and why evolvability is the meta-characteristic that protects all the others.

fnd-evolutionFitness Functions

Objective, automatable integrity checks for architecture characteristics — the categories they fall into, and how they turn governance from aspiration into enforcement.

fnd-evolutionArchitecture Decision Records

How to capture, justify, and communicate architecture decisions as ADRs — overcoming the three decision anti-patterns by recording the why.

Check your understanding

Score: 0 / 4

1. What is the Expand/Contract pattern used for?

Expand/Contract (a subset of Parallel Change) adds the new state, transitions while both exist, then contracts away the old — letting code and schema evolve without a breaking big-bang change.

2. Once a migration script has run, what is the rule?

Modeled on double-entry bookkeeping, a run migration is immutable; you rebuild forward from script 1, so most teams have abandoned undo migrations.

3. What is the chief role of a deployment pipeline in evolutionary architecture?

The pipeline runs separable stages (provisioning, testing levels, atomic and holistic fitness functions) on every change, automating governance.

4. How do feature toggles support incremental change?

A toggle is an if-statement on config that enables/disables a feature, letting teams deploy continuously, release on a schedule, and route only QA users to a hidden feature.

Comments

Sign in with GitHub to join the discussion.