In most organisations Active Directory is backed up by virtue of the domain controllers being virtual machines covered by the hypervisor backup. That is enough for one scenario and dangerous for the rest. The directory has its own consistency logic between controllers, and restoring it requires a procedure, not a click on “restore machine”.
Why a snapshot is not a backup
Every controller tracks changes with a sequence number and exchanges them with the others. Restoring a controller to yesterday’s state without the directory knowing means the controller has an old sequence number while the others remember having seen a higher one. Changes made on the restored controller may never replicate, and the directory drifts apart silently. Newer hypervisors and operating systems mitigate this with a virtual machine generation identifier, but that mechanism protects against an accidental snapshot rollback; it does not replace a directory backup.
A proper backup is a system state backup taken by a directory-aware tool: the built-in Windows backup or an application-aware backup system that uses the volume shadow copy interface and registers the backup with the directory.
Four scenarios, four procedures
A deleted object. An account, group or organisational unit removed by mistake. The Active Directory Recycle Bin, enabled in advance, restores the object with all its attributes and memberships in seconds. Without the Recycle Bin an authoritative restore from backup is needed, a far longer procedure. The Recycle Bin is enabled once, irreversibly, and should be enabled in every environment.
Failure of a single controller. A controller does not come back after a disk or host failure. It is not restored from backup. Its metadata is removed from the directory, the roles it held are seized, and a new controller is built that replicates the directory from the remaining ones. No backup is needed here; a procedure and a second working controller are.
Directory corruption propagating through replication. A bad bulk change, a malicious script, schema damage. All controllers hold the same bad state. One controller has to be restored from a backup taken before the event as authoritative and replicated back to the others.
Loss of the whole forest. Ransomware encrypting all controllers, a data centre failure, a compromise that requires assuming every controller is owned. This is the forest recovery scenario.
Forest recovery in outline
- An isolated network. Recovery happens without a connection to the rest of the environment until the directory is known to be clean.
- Restore one controller per domain from a system state backup taken before the event, in directory services restore mode.
- Seize all operations master roles on the restored controller and remove the metadata of the other controllers.
- Reset the ticket-granting service account password twice, reset inter-domain trust passwords and restore mode passwords, invalidate existing tickets.
- Clear DNS caches and fix records pointing at controllers that no longer exist.
- Verify: logon, intra-domain replication, queries, ticket issuance.
- Build the remaining controllers as new ones, and only then connect to the production network.
Each of these steps has details that must be written down before the event, with commands and expected results. A procedure invented during an incident usually ends in a second incident.
Backup rules
- System state backup from at least two controllers in every domain, taken daily.
- A backup no older than the directory’s deleted object lifetime, because an older one cannot be used for a restore.
- At least one copy beyond the reach of domain accounts: ransomware with domain administrator rights deletes the backups it can reach.
- The directory services restore mode password of every controller known and stored outside the directory. Without it a restored controller is inaccessible.
- Installation media, documentation and the procedure available without a working domain.
The rehearsal
Once a year, in an isolated virtual environment: restore a controller from a production backup, seize the roles, verify logon. Measure the time. The rehearsal reveals missing passwords, outdated steps and dependencies nobody remembered. A forest recovery nobody has rehearsed takes days instead of hours.
Summary
A virtual machine snapshot protects against one scenario and harms the rest. The directory needs system state backups, an enabled Recycle Bin, a procedure for a single controller failure that does not involve a restore, a written forest recovery walkthrough, and a yearly rehearsal. It is one of the few procedures in IT nobody wants to perform, and whose absence stops an organisation for weeks.