Strategy, Migration & Complexity

Migrating a System to the Cloud

Inventory, dependency mapping, strategy, pilot, gradual move, validation. Migrations do not fail at the cutover — they fail at the inventory, on the undocumented cron job running on the server nobody owns.

The question this answers

Infrastructure question

How do we move an existing production system to the cloud without discovering its real dependencies during the cutover?

Application requirement

A twelve-year-old order platform runs on nine servers in a leased facility. The lease ends in fourteen months. Nobody currently employed set it up, the original architect left in 2021, and the runbook is a wiki page last edited three years ago.

What it provides

A sequenced move where each step is reversible, where the surprises are found during discovery rather than during the cutover window, and where "done" is defined by validation criteria agreed before anything moves.

Application RequirementInfrastructure RequirementComputeNetworkStorageIdentityDeploymentScalingReliabilityObservabilitySecurityCostTrade-offs

Six phases, and the two that decide the outcome

The phases are unglamorous and the order matters. Inventory answers what actually exists. Dependency mapping answers what talks to what, including the things nobody mentioned. Strategy assigns each component a disposition — the five choices in The Five Dispositions. Pilot moves one low-risk, representative component end to end and proves the whole toolchain. Gradual move works through the rest in dependency order, keeping a rollback available at each step. Validation confirms, against pre-agreed criteria, that the thing running in the new environment is the thing that used to work.

Everyone plans this correctly and then compresses the first two phases, because they produce no visible progress. That is precisely where migrations fail. Cutover weekends do not blow up because the servers were the wrong size; they blow up because at 03:00 someone discovers that settlement runs from a cron job on app-03 that writes to an SFTP share the migration plan never mentioned, owned by a team that was never consulted, and depended on by a bank.

A useful discipline is to date the discovery. Every unknown found during inventory costs an hour. The same unknown found during the pilot costs a day. Found during cutover, it costs the weekend and a rollback. Found after cutover, it costs an incident with customer impact and no rollback available, because the old environment has already been decommissioned. The entire value of the first two phases is moving discoveries leftward on that scale.

The six phases, what happens at each, and what actually goes wrong there.PROVIDER-NEUTRAL
  1. 1Inventoryweeks, and longer than anyone budgets — ILLUSTRATIVE

    Enumerate every host, service, scheduled job, certificate, credential, data store, DNS record, licence and integration — from reality, not from documentation.

    Trusting the wiki. Anything not discovered here is discovered later, at ten to a thousand times the cost.

  2. 2Dependency mappingweeks, overlapping with inventory — ILLUSTRATIVE

    Map what calls what, in both directions: inbound integrations, outbound calls, shared file paths, database links, hard-coded IP addresses and firewall rules that encode a relationship.

    Missing the inbound direction. You know what your system calls; you frequently do not know who calls it, or from which fixed source address they were allow-listed.

  3. 3Strategy per componentdays once the map exists — ILLUSTRATIVE

    Assign each component a disposition — rehost, replatform, refactor, retain, retire — with the reason and the accepted cost recorded.

    Applying one disposition to the whole estate. The correct answer is almost always a mix, and "retire" is frequently the most valuable.

  4. 4Pilota few weeks — ILLUSTRATIVE

    Move one component end to end: build, deploy, network, identity, secrets, monitoring, backup, rollback. Choose something real but low-blast-radius.

    Piloting something atypical, so the toolchain is proven against a case that resembles nothing else in the estate.

  5. 5Gradual movemonths — ILLUSTRATIVE

    Migrate in dependency order, running old and new in parallel where possible, shifting traffic incrementally and keeping the old path warm.

    Data divergence during the parallel window, and the cost of paying for both environments for longer than planned.

  6. 6Validation and decommissionweeks, plus one full business cycle — ILLUSTRATIVE

    Check against criteria agreed before the move — correctness, latency, throughput, batch completion, restore from backup — then decommission the old environment deliberately.

    Declaring success on "the site loads". The old environment is then switched off, and the first month-end batch fails with no rollback.

The cron job on the server nobody owns

Inventory has to be built from the running system, not from documentation. Documentation records what someone intended in the past; the running system records what is true now. That means enumerating processes, listening ports, scheduled tasks, outbound connections observed over a full business cycle, certificates and their expiries, credentials in use, DNS records pointing at the estate, and the firewall rules that reveal integrations nobody remembers agreeing to.

A full business cycle matters more than any tool. A month-end job, a quarterly regulatory export or an annual renewal will not appear in a week of observation, and each is the kind of thing that fails silently for six weeks after cutover and then produces a compliance incident. If the lease clock allows only one thing beyond the basics, make it observing one complete monthly cycle.

The inbound direction is the one teams systematically miss. You can discover what your system calls by watching its outbound connections. Discovering who calls *your* system requires reading load balancer and firewall logs, and the answers are frequently uncomfortable: a partner integration hard-coded to a specific IP address, an internal tool nobody maintains, a monitoring system from a decommissioned project still polling an endpoint. Every one of those becomes an outage the moment the address changes, and IP addresses always change in a migration.

host      finding                                              owner        in plan?
--------  ---------------------------------------------------  -----------  --------
app-01    order-api (documented)                               orders       yes
app-02    order-api replica (documented)                       orders       yes
app-03    CRON 0 2 * * *  /opt/scripts/settle_batch.sh         <unknown>    NO
          -> writes /mnt/sftp-out/settlement-YYYYMMDD.csv
          -> file collected nightly by an external bank SFTP
          -> last modified 2019-04-11, no source control
app-03    CRON 0 4 1 * *  /opt/scripts/month_end_reconcile.sh  <unknown>    NO
          -> only runs on the 1st; invisible in a 2-week study
db-01     inbound 5432 from 198.51.100.44 (partner, allow-listed)  <unknown>  NO
          -> hard-coded source IP; breaks on any address change
util-01   TLS cert CN=orders.internal expires in 63 days       <unknown>    NO
util-01   process: legacy-fax-gateway, listening, 0 conns/90d  <unknown>    RETIRE?

7 findings. 5 not in the migration plan. 1 candidate for retirement.
The two crons are the ones that would have failed the cutover.
What a discovery pass turns up that the documentation did not. ILLUSTRATIVE.

Move gradually, and define "done" before you start

A big-bang cutover is attractive because it is short and it ends the dual-running cost. It is also the option with no rollback once data has been written on the new side. Prefer a gradual move: migrate in dependency order (things with no dependents first, or read-only replicas ahead of writers), run both environments in parallel, and shift traffic incrementally so a problem affects 5% of users rather than all of them. The techniques are the ones from Canary: Let 5% of Traffic Find the Bug and Blue/Green: Two Environments, One Switch, applied at the level of an estate rather than a deploy.

The parallel window is where the money goes and where the data risk lives. You are paying for both environments simultaneously, which is the single largest cost of any migration and the reason schedules slip into budget problems. And if both sides can accept writes, you have a consistency problem that must be solved deliberately — usually by keeping exactly one authoritative writer at any moment and replicating one way, rather than by attempting bidirectional synchronization, which is far harder than it appears in the plan.

Finally, agree the validation criteria before anything moves, and make them specific. Not "the system works" but: order creation p99 under X, the nightly settlement file present and byte-comparable for five consecutive days, the month-end batch completing within its window, a restore from backup performed successfully in the new environment, and the audit trail intact. Then decommission the old environment only after one full business cycle has passed on the new one — because the migration is not finished when traffic moves, it is finished when the quarterly job has run.

What a migration costs while it is in progress. Relative weights, not currency.COST-VARIES
Dual running fixed
driven by both environments live × months of overlap · The dominant cost, and it grows with every week the schedule slips.
Data transfer out of the old environment · surprisespiky
driven by TB moved, plus re-syncs · Initial bulk load plus ongoing replication for the whole parallel window.
Engineering time fixed
driven by discovery, tooling, migration, validation · Never on the cloud bill; usually the largest number in the programme.
Over-provisioning on arrival · surprisefixed
driven by like-for-like sizing of old physical hardware · Rehosting a machine bought for 2019 peak keeps paying for 2019 peak. Right-size after stabilizing.
Rollback capacity fixed
driven by keeping the old environment warm and restorable · The cost of being able to change your mind. Cheap relative to not being able to.

Bars are relative weights, not currency. Real rates depend on provider, region, commitment and volume.

Key points

  • Six phases: inventory, dependency mapping, strategy, pilot, gradual move, validation — and the first two decide the outcome.
  • Build the inventory from the running system over a full business cycle; monthly and quarterly jobs are invisible in a two-week study.
  • Map dependencies in both directions — who calls you, from which hard-coded address, matters more than what you call.
  • Pilot something real but low-blast-radius, and prove the whole toolchain including rollback and restore.
  • Move gradually with one authoritative writer at a time; bidirectional synchronization is much harder than the plan assumes.
  • Agree specific validation criteria before moving, and decommission only after a full business cycle has passed on the new environment.

The loop, answered

Every field is required, which is why no lesson here can recommend something without saying what it costs and what simpler thing to consider first.

How it works
  • Discovery agents, network flow logs, scheduler dumps and firewall rules produce an inventory of what actually runs, listens, connects and expires.
  • Observed traffic over a complete business cycle is turned into a dependency graph, annotated with direction, protocol and whether the source address is fixed.
  • Each component receives a disposition and an owner; the graph then determines a migration order in which nothing moves before its dependencies are resolved.
  • A pilot component is migrated end to end, exercising build, network, identity, secrets, monitoring, backup and a deliberate rollback.
  • Data is bulk-loaded then kept in sync one-way, with the authoritative writer explicit and switched at a defined moment.
  • Traffic shifts incrementally by DNS or load balancer weight, with the old path warm until validation criteria are met.
What you still own
  • You own the inventory as a living artifact for the duration; it goes stale within weeks if nobody maintains it.
  • You own both environments during the parallel window, including patching, monitoring and on-call for the one you are leaving.
  • You own the data synchronization and its lag, and you own knowing which side is authoritative at any moment.
  • You own rollback readiness at each step — a step with no rollback is a step that needs a much better reason.
  • You own right-sizing after stabilization, or you keep paying cloud prices for hardware you sized in 2019. See Right-Sizing Without Causing an Outage.
  • You own the decommission: switching off the old environment deliberately, after validation, rather than letting it linger and bill.
How it fails
  • An undocumented scheduled job discovered after cutover, when the external party it fed reports a missing file.
  • A partner integration allow-listed by source IP that breaks the moment addresses change, presenting as an unexplained partner outage.
  • Bidirectional write divergence during the parallel window, producing records that exist on one side only and reconcile to nothing.
  • A pilot chosen for being easy rather than representative, so the toolchain is proven against a case unlike the rest of the estate.
  • Like-for-like rehosting of oversized physical hardware, turning a capital asset into a permanently oversized monthly bill.
  • Decommissioning the old environment before the first month-end, removing the rollback exactly when it turns out to be needed.
How it scales
  • Migration effort scales with the number of dependencies, not the number of servers — a nine-server estate with forty integrations is harder than forty servers with three.
  • The parallel window scales cost linearly with time, which is what turns a schedule slip into a budget conversation.
  • Data volume determines the cutover mechanics: below a threshold you can copy in a window, above it you must replicate continuously.
  • Team attention is the scarce resource; running discovery, migration and business-as-usual simultaneously is where quality drops.
Security
  • Migration is when credentials proliferate: the new environment needs every secret the old one had, and copies made "temporarily" during a cutover survive for years.
  • Take the opportunity to replace static keys with workload identity rather than lifting the old credential model wholesale. See Roles vs Static Keys.
  • Firewall rules encode business relationships; re-implementing them as cloud security groups is a re-authorization decision, not a translation exercise.
  • The old environment stays a live attack surface while it runs, and an unpatched one once attention has moved to the new estate.
  • Data in transit during bulk migration is a genuine exposure — encrypt it and account for the copies made along the way.
Cost shape
  • Dual running dominates and grows with every week of slippage.
  • Egress out of the old environment is a real, spiky line item, including every re-sync you did not plan for.
  • Engineering time is the largest cost and never appears on a cloud bill.
  • Like-for-like sizing carries yesterday's peak into a monthly charge; right-sizing after stabilization is where the savings actually appear.
  • Retiring components during discovery is the highest-return activity in the whole programme — nothing is cheaper to run than something you deleted.
What to watch
  • Discovery coverage: percentage of hosts, jobs, certificates and integrations with a named owner. This is the leading indicator of cutover risk.
  • Data synchronization lag and divergence counts throughout the parallel window.
  • Side-by-side correctness comparisons on the same input, not just health checks on the new side.
  • Batch job completion in the new environment across a full cycle, including the monthly and quarterly ones.
  • The signal that lies: "the new environment is healthy". It says nothing about the nightly file nobody has checked, and that is what fails first.
Simpler alternatives
  • Do not migrate. If the lease is not ending, the hardware is healthy and the workload is steady, staying put is a legitimate and often cheaper answer. See On-Premises vs Cloud.
  • Retire instead of migrate. Every component you delete during discovery is one you never have to move, test, validate or run — and estates of this age typically contain several.
  • Migrate a subset. Move the components with a real reason and leave the rest, accepting hybrid deliberately with an owner and a budget. See Hybrid Cloud.
  • Replace with a purchased product. For commodity functions — email, file exchange, identity — buying is often cheaper than migrating something you built.
  • Rehost first, improve later. Modernizing during a move multiplies the variables, and a failed migration is far more expensive than an unmodernized one.
What adopting this costs
  • Thorough discovery buys cheap surprises; costs weeks with no visible progress, which is politically hard to defend.
  • A gradual move buys reversibility; costs a longer parallel window and therefore a bigger bill.
  • Rehosting first buys a shorter, safer migration; costs an inefficient shape that must be improved afterwards or forever paid for.
  • A big-bang cutover buys a short dual-running period; costs the ability to roll back once writes have landed on the new side.

What people believe, and what is true

Claim

Migration is mostly a capacity-planning exercise.

Reality

Sizing is the easy part and is adjustable afterwards. The hard part is knowing what exists and what depends on it, which no amount of capacity fixes.

Claim

We will modernize as we migrate — it is the natural moment.

Reality

It doubles the variables, so a failure has two possible causes. Move first, verify equivalence, then improve with a working baseline to compare against.

Claim

The migration is done when traffic is on the new environment.

Reality

It is done when a full business cycle has completed there — month-end, quarter-end, the annual renewal — and a restore has been proven. Decommissioning before that removes your rollback.

Apply it