Incremental Backup

The Redstor backup reference describes the operational pattern: “An incremental backup saves only the data changed since the most recent backup, full or incremental. This minimises backup windows and storage by capturing smaller sets of changes. Restores require the last full backup plus every subsequent incremental taken up to the recovery point, which can make recovery slower.”¹

The fundamental concept

An incremental backup is a backup type defined by what it captures and what it depends on:

• What it captures: only data that has changed since the most recent prior backup of any type.
• What “changed” means: files modified, created, or deleted since the previous backup completed.
• What it depends on: the chain of all preceding backups going back to the last full backup.
• What it produces: the smallest backup file of any backup type for the same time period.
• What it requires for restore: the full backup plus every incremental in sequence up to the desired recovery point.

The classic Sunday-Monday-Tuesday example

The AWS backup reference describes the classic pattern: “An incremental backup only copies modified data since the last backup. For example, if you took a full backup on Sunday, your incremental backup on Monday would only copy changes since the Sunday backup. On Tuesday, it would only copy changes to the backup image file since the Monday backup.”² The Unitrends photo example illustrates the pattern with concrete numbers:

• Monday: 100 photos exist; full backup creates 100-photo backup file.
• Tuesday: 100 new photos added (200 total); incremental captures only the 100 new photos.
• Wednesday: 50 new photos added (250 total); incremental captures only those 50 new photos.
• Thursday: 30 new photos added (280 total); incremental captures only those 30.
• Restoration of Thursday state: requires Monday full + Tuesday incremental + Wednesday incremental + Thursday incremental; total 280 photos restored.

The contrast with full and differential backups

Three backup types differ in what they capture and how they’re restored:

• Full backup: copies all selected data every time; self-contained for restoration; largest storage requirement.
• Incremental backup: copies changes since the last backup of any type; smallest storage; chain-dependent restoration.
• Differential backup: copies all changes since the last full backup; intermediate storage; restoration needs full + latest differential only.
• Restore time progression: full (fastest) < differential (intermediate) < incremental (slowest, depends on chain length).
• Backup time progression: incremental (fastest) < differential (grows with time since full) < full (slowest).
• Storage requirement progression: incremental (smallest) < differential (grows with time since full) < full (largest).

Why use incremental backups

Incremental backups address several specific operational requirements:

• Short backup windows: production systems with limited maintenance time benefit from incremental’s small backup size.
• Limited bandwidth: remote sites and cloud destinations benefit from minimal data transfer.
• Storage cost optimization: incremental’s smaller footprint reduces backup storage requirements.
• Frequent backup intervals: hourly or sub-hourly backups become feasible with incremental’s small size.
• Cloud backup destinations: reduced bandwidth and storage costs make cloud backup affordable at scale.
• VM environments: Changed Block Tracking enables very efficient incrementals at the hypervisor level.

The Macrium storage math

The Macrium incremental reference illustrates the storage advantage with specific numbers: “two 500GB full images use 1TB of sectors to provide just two restore points. One 500GB full image followed by ten 5GB incremental images uses only 550GB of sectors while providing eleven restore points. Fewer sectors in use means a smaller surface area for potential corruption, incrementals actually reduce the risk at a fundamental level.”³ The implication:

• Backup storage scales with change rate, not data size.
• More restore points become possible for same storage budget.
• Smaller storage footprint means smaller surface area for corruption.
• Long retention is feasible with incrementals when full-only retention would be prohibitive.

Incremental backups rely on change-detection mechanisms to identify which data has been modified since the previous backup. The specific mechanism depends on the backup software, the file system, and the operating system.

Change detection mechanisms

Several mechanisms enable incremental backup software to identify changed data:

• Archive bit (Windows): file attribute set when a file is modified; backup software clears the bit after backing up; subsequent backups check this bit.
• Modification time (mtime): Unix file attribute updated when content changes; backup software compares against the previous backup’s recorded mtime.
• Change time (ctime): Unix attribute updated for any inode change including permission changes; more comprehensive than mtime.
• File system snapshots: ZFS, Btrfs, NTFS Volume Shadow Copy provide before/after states for differential analysis.
• Hash-based comparison: tools like restic, BorgBackup, and rsync compute content hashes to detect actual changes regardless of timestamps.
• Block-level change tracking: Changed Block Tracking (CBT) at the hypervisor level identifies modified blocks without scanning files.
• Database transaction logs: database systems (Oracle, SQL Server, PostgreSQL) provide log-based incremental backup at the transaction level.

The Volume Shadow Copy Service (VSS) integration

On Windows, incremental backups typically integrate with the Volume Shadow Copy Service (VSS) for consistency:

• VSS creates a point-in-time snapshot of the volume at backup start.
• The snapshot provides a consistent view of the file system during the backup operation.
• Open files (databases, locked Office documents) are captured in their VSS snapshot state.
• VSS writers in applications coordinate quiescing for application-consistent snapshots.
• The backup software reads from the VSS snapshot rather than the live volume; backup completion releases the snapshot.
• Application-aware backups (SQL Server, Exchange) leverage VSS writers for transaction-consistent backup.

VMware Changed Block Tracking (CBT)

For virtualized environments, CBT is the foundation of efficient incremental backup:

• Origin: VMware introduced CBT in vSphere 4 (2009) for vStorage APIs for Data Protection (VADP).
• Mechanism: hypervisor maintains a bitmap of modified blocks for each virtual disk.
• Backup software query: backup software calls the QueryChangedDiskAreas API to retrieve the changed-block list.
• Result: backup software copies only the changed blocks rather than scanning entire VMDK files.
• Performance impact: 1 TB virtual disk with 100 MB of changes results in 100 MB backup transfer instead of 1 TB scan.
• Fallback: if CBT becomes inconsistent (rare), backup software performs full VM scan as recovery.

Hyper-V Resilient Change Tracking (RCT)

Microsoft Hyper-V implements equivalent functionality starting with Windows Server 2016:

• Equivalent capability: tracks changed blocks for incremental backup of Hyper-V VHDX files.
• Improvement over earlier: Hyper-V before 2016 required differential disks for incremental tracking, less efficient than RCT.
• API integration: backup software queries RCT through Hyper-V WMI providers.
• Cross-platform implication: Veeam, Veritas, and other backup vendors support both VMware CBT and Hyper-V RCT through unified VM backup engines.

Hash-based change detection

Modern backup tools use content hashing for change detection independent of file system metadata:

• restic: chunks files and computes SHA-256 hashes per chunk; stores deduplicated chunks in repository.
• BorgBackup: similar approach with content-defined chunking and HMAC-SHA-256 chunk identification.
• rsync: rolling checksum comparison detects byte-level differences; transfers only changed portions.
• duplicity: uses rsync algorithm but stores in tar volumes for compatibility.
• Advantages over timestamp-based: detects actual content changes regardless of timestamp manipulation; deduplicates identical content across files; resilient to filesystem metadata corruption.
• Trade-offs: hash computation requires reading entire file; slower than timestamp comparison for very large files.

Database incremental specifics

Database systems implement incremental backup using transaction log mechanisms:

• SQL Server: transaction log backups capture all log records since the previous log backup; restoration applies log backups in sequence.
• Oracle: archive log mode preserves redo logs; RMAN incremental backups use Block Change Tracking (BCT) file for efficient block-level backup.
• PostgreSQL: Write-Ahead Log (WAL) shipping provides continuous incremental backup capability.
• MySQL: binary log replication and Percona XtraBackup provide incremental backup capability.
• Common pattern: full database backup weekly + differential or transaction log backups daily/hourly.
• Recovery model: database systems typically use point-in-time recovery via log application rather than file-level restore.

Several variants of incremental backup address different operational requirements and trade-offs.

Five incremental variants

Variant	Pattern	Source-Side Fulls	Use Case
Classic incremental	Full + Inc + Inc + Inc + new Full	Periodic (weekly/monthly)	Standard SMB/enterprise pattern
Incremental forever	Initial Full + Inc + Inc + Inc… indefinitely	Single initial full only	Cloud backup, tape libraries
Synthetic full	Full + Inc + Inc… + (synthesized Full)	None after initial	Bandwidth-constrained environments
Forever forward incremental	Full + Inc + Inc + (merge oldest into Full)	None after initial	Long-retention with bounded chain length
Reverse incremental	Synthesized current Full + reverse Incs for older versions	None after initial	Veeam approach; fast restore of recent state

Classic incremental

The classic incremental pattern is the most-widely deployed strategy:

• Weekly full backup (Sunday) establishes baseline.
• Daily incremental backups (Monday-Saturday) capture changes since previous backup.
• Following Sunday: new full backup resets the chain.
• Retention: keep multiple weekly fulls plus their associated incremental chains.
• Common variant: daily incrementals + weekly differential + monthly full for hybrid restore characteristics.

Incremental forever and synthetic full

The TechTarget backup types reference describes the synthetic approach: “A synthetic full backup compares the data that has changed at the source with the original full backup and all incremental backups to create the next full synthesized backup. In a synthetic full backup, you don’t save on storage space but you do save on network bandwidth. You only send incremental changes to the server instead of sending all your data. The server uses the data it already has to create the full backup copy.”⁴ Implementation:

• Initial full backup: taken once at backup deployment.
• Subsequent incrementals: capture changes since previous backup indefinitely.
• Synthetic full: backup server consolidates full + incrementals into virtual full without re-reading source.
• Source bandwidth: only incremental changes ever transmitted; no repeated full transfers.
• Storage: typically deduplicated across the backup chain to minimize space.
• Restoration: backup server presents the most-recent synthetic full as restore target.

Forever forward incremental

The Acronis backup reference describes forever forward incremental: “Forever Forward incremental backups rely on creating a single full backup, followed by subsequent incremental backups. This backup method only retains one backup generation in the backup server. Here, the backup system regularly renews the full backup copy by merging the latest relevant increments. Once the new full backup is compiled, the previous full and incremental backup copies are deleted according to a set retention policy to optimize storage space usage.”⁵ Implementation:

• Single full backup: only one full backup retained at any time.
• Forward merging: as new incrementals are added, the oldest incrementals are merged into the full backup.
• Bounded chain length: chain length stays constant; old data is consolidated forward.
• Trade-off: older recovery points are lost as they merge forward; long retention not supported by this pattern alone.
• Use case: environments needing recent restore points only with minimal storage overhead.

Reverse incremental (Veeam pattern)

Reverse incremental is Veeam’s signature backup pattern:

• Most-recent backup is full: the active full backup represents current state.
• Reverse incrementals: older versions are stored as differences from the current full going backward.
• Latest version restore: single full backup file; fastest possible restore.
• Older version restore: apply reverse incrementals in reverse chronological order from current full.
• New backup operation: new incremental against current full; promotes incremental data into full while archiving old version as reverse incremental.
• Advantages: fastest restore for most-common case (recent state); retention chain managed automatically.
• Limitations: backup operation more complex than forward incremental; requires more sophisticated backup software.

Block-level vs file-level vs byte-level incremental

The Acronis backup reference describes the granularity options: “Byte-level incremental backup perceives the file system as individual bytes and only backs up bytes that have changed since the last backup process. Since so much less data is comprised in a byte-level copy, it can yield the smallest available backups.”⁶ The granularity progression:

• File-level incremental: entire file copied if any byte changed; simplest implementation; works on any file system.
• Block-level incremental: only changed blocks copied within modified files; significantly more efficient for large files with small changes (databases, VHDX, large logs).
• Byte-level incremental: only changed byte sequences captured; finest granularity; produces smallest backups but requires sophisticated detection and reconstruction.
• Sub-file deduplication: content-defined chunking (restic, BorgBackup) achieves block-level efficiency without explicit block-level support.
• VM block-level: CBT/RCT operate at virtual disk block level natively.

Multilevel incremental

Multilevel incremental introduces incremental hierarchy for granular recovery point management:

• Level 0: full backup serves as the baseline.
• Level 1: first incremental after Level 0; captures all changes since the full.
• Level 2: second-tier incremental; captures changes since Level 1.
• Higher levels: further granularity for very-frequent backup schedules.
• Restoration: apply Level 0 + Level 1 + Level 2 in order to reach desired recovery point.
• Use case: environments needing very-frequent recovery points (hourly or sub-hourly).

Incremental backup implementations vary across vendor products, but several common patterns and considerations apply.

Backup software supporting incremental

Most backup software supports incremental backup with various sophistication levels:

• Veeam Backup & Replication: reverse incremental, forever incremental, synthetic full; CBT/RCT integration for VMs.
• Veritas NetBackup: forward incremental, synthetic full, accelerator for incremental optimization.
• Commvault: incremental, synthetic full, IntelliSnap for snapshot-based incrementals.
• Acronis Cyber Protect: forever forward incremental with byte-level capability.
• restic (open source): hash-based deduplicated incremental with point-in-time snapshots.
• BorgBackup (open source): deduplicated content-defined chunking; effective incremental.
• duplicity (open source): rsync-based incremental with encryption; standard Linux backup tool.
• Bacula (open source): traditional incremental with extensive scheduling control.
• Apple Time Machine: hourly incremental with hard-link consolidation; the standard macOS backup.
• Windows File History: simple incremental for user data; preceded modern Windows Backup.

Scheduling considerations

Incremental backup scheduling balances recovery point objectives with operational constraints:

• Backup frequency: daily incrementals for typical environments; hourly for active databases; sub-hourly for high-change-rate systems.
• Full backup frequency: weekly full + daily incrementals is the canonical pattern; monthly fulls acceptable for stable systems.
• Backup window: incrementals fit in shorter windows than fulls; matters for production systems with limited maintenance time.
• Verification frequency: chains should be verified periodically; some software performs automatic verification on synthetic full creation.
• Retention period: longer retention requires more incrementals retained; balance against storage cost.
• Off-site rotation: traditional tape rotation problematic with long incremental chains; synthetic full enables off-site rotation of consolidated backups.

Storage considerations

Incremental backup storage planning differs from full backup:

• Initial sizing: first full backup sets baseline; subsequent incrementals scale with change rate.
• Change rate estimation: typical 1-5% daily change rate for many environments; databases and active systems may have 10%+ daily change.
• Deduplication: inline or post-process deduplication can dramatically reduce incremental storage requirements.
• Compression: backup software compression typically achieves 2-3x compression for typical data.
• Encryption: at-rest encryption applied after deduplication and compression to maximize efficiency.
• Cloud storage tiering: recent incrementals on hot tier; older chains on cool tier; very old on archive tier.

The 3-2-1 rule applied

The 3-2-1 backup rule applies to incremental backups with chain-specific considerations:

• 3 copies: original + 2 backup copies; the chain (full + incrementals) counts as one copy.
• 2 different media: different storage technologies (local disk + cloud, or NAS + tape).
• 1 offsite: chain replicated to offsite location; bandwidth-efficient with incremental forever.
• Chain integrity: all media must contain complete chains; partial chains are useless.
• Verification: hash verification confirms chain integrity periodically.
• 3-2-1-1-0 extension: add immutable copy and zero verified errors for ransomware protection.

Cloud backup considerations

Cloud destinations particularly favor incremental backup:

• Bandwidth efficiency: incremental’s small size minimizes monthly cloud transfer.
• Storage cost: incremental’s smaller footprint reduces monthly storage charges.
• Egress fees: some providers charge for retrieval; chain restoration triggers fees for entire chain.
• Sync limitations: cloud backup tools must handle partial uploads, retries, network interruptions during incremental transfer.
• Provider features: AWS Backup, Azure Backup, Google Cloud Backup handle incremental natively with synthetic full consolidation.
• Hybrid approach: local incremental + periodic cloud sync balances speed and offsite protection.

The chain dependency of incremental backups creates specific recovery characteristics that affect both successful and failed recovery scenarios.

Standard restoration workflow

Restoring from incremental backups follows a specific sequence:

1. Identify recovery point: determine which incremental contains the desired state (typically by date/time).
2. Locate the chain: find the full backup that anchors the chain plus all incrementals up to recovery point.
3. Verify chain integrity: confirm all required incrementals are present and readable; verify hashes if available.
4. Restore the full backup: apply the baseline full backup to recovery destination.
5. Apply incrementals in order: each incremental applied in chronological sequence on top of restored full.
6. Verify restored state: confirm restoration matches expected recovery point state.
7. Resume operations: bring restored system online or extract specific files as needed.

Recovery time analysis

Incremental backup restoration time scales with chain length:

• Time to read full backup: proportional to full backup size and storage read speed.
• Time per incremental: proportional to incremental size; small individually but cumulative.
• Sequential application: incrementals must be applied in order; cannot be parallelized in most implementations.
• Total time: approximately full restore time + sum of incremental restore times.
• 30-day chain example: daily incrementals for 30 days = 30 sequential application steps after full restore.
• Recovery time objective (RTO) implications: longer chains mean longer RTO; if RTO is short, fewer increments between fulls is required.

Chain corruption scenarios

The Veeam backup reference describes the dependency property: “Restoring data from incremental backups requires the presence and integrity of all preceding backups in the sequence. A failure in any incremental backup in the chain could impact the restoration process.”⁷ Specific failure modes:

• Missing incremental: chain broken at the gap; recovery limited to incrementals before the gap.
• Corrupted incremental: chain may be unrecoverable from that point forward; partial recovery may be possible from prior increments.
• Tape failure mid-chain: single bad tape can break recovery for many subsequent recovery points.
• Cloud retrieval failure: incremental retrieval from cold archive may fail or time out; entire chain affected.
• Software version mismatch: backup software incompatibility may prevent reading older incrementals.
• Format obsolescence: incremental backup format may become unreadable as backup software evolves.

The off-site rotation problem

The TechTarget backup reference identifies a specific incremental backup limitation: “If backups are shipped off-site, incremental backups are a bad idea because you must retrieve all the tapes before you can begin a restoration.”⁸ The implications:

• Traditional tape rotation requires retrieving all tapes in chain for restoration.
• Geographic separation of tapes means longer recovery time during disasters.
• Tape library coordination must ensure complete chain availability.
• Mitigation: synthetic full or incremental forever consolidation creates single restore point on the most-recent media.
• Alternative: off-site full backups (less frequent) plus on-site incrementals; off-site full sufficient for disaster recovery.
• Modern alternative: cloud-based incremental forever eliminates the tape retrieval problem.

Recovery time vs storage trade-off

The fundamental incremental backup trade-off in recovery contexts:

Strategy	Storage	Recovery Time	Chain Dependency Risk
Full only (daily)	Highest	Fastest (single restore)	None
Full weekly + Differential daily	Medium	Fast (full + 1 differential)	Low
Full weekly + Incremental daily	Lower	Slower (full + up to 6 increments)	Medium
Full monthly + Incremental daily	Lowest	Slowest (full + up to 30 increments)	High
Incremental forever + synthetic	Optimized	Fast (synthetic full)	Managed by software

When incremental is the right choice

Incremental backups are the appropriate strategy in specific scenarios:

• Large datasets with low change rates: 10 TB system with 50 GB daily change benefits from incremental’s bandwidth efficiency.
• Cloud backup destinations: bandwidth and storage costs favor incremental’s smaller transfers.
• Remote sites with limited connectivity: nightly incremental fits in available bandwidth window.
• Frequent backup intervals: hourly backups feasible only with incremental’s small footprint.
• VM environments: CBT/RCT make incremental highly efficient at virtual disk level.
• Long retention requirements: incremental’s storage efficiency enables practical long-term retention.

When incremental is the wrong choice

Several scenarios make incremental backup inappropriate:

• Tight RTO requirements: chain reconstruction takes too long when business continuity demands fast restoration.
• Off-site tape rotation without synthetic full: retrieving all tapes for restoration impractical for disaster recovery.
• High change rate workloads: when daily changes approach full backup size, incremental loses its advantages.
• Critical systems requiring simple recovery: when chain complexity creates unacceptable recovery risk.
• Older backup software lacking synthetic full: traditional incremental without consolidation becomes unmanageable over time.
• Compliance scenarios requiring audit-friendly backups: some compliance frameworks prefer self-contained full backups.