Zuzia.app logo
Get started
Features
How It Works
Pricing
About
Faq
Sign up Log In
← All guides

Terraform Drift Monitoring: Detect IaC Changes Early

Updated:

Terraform Drift Monitoring: Detect IaC Changes Early

Infrastructure as Code (IaC) monitoring is essential for maintaining infrastructure consistency, detecting configuration drift, and ensuring reliable deployments. This comprehensive guide covers everything you need to know about monitoring Terraform-managed infrastructure, tracking changes, and detecting drift.

For related infrastructure topics, see Cloud Resources Monitoring. For troubleshooting IaC issues, see Infrastructure as Code Drift Failures.

Why Infrastructure as Code Monitoring Matters

Infrastructure as Code monitoring helps you track infrastructure changes, detect configuration drift, ensure infrastructure consistency, monitor deployment status, and maintain infrastructure reliability. Without proper monitoring, infrastructure drift can cause inconsistencies and deployment failures.

Effective IaC monitoring enables you to:

  • Track infrastructure changes and deployments
  • Detect configuration drift between code and actual infrastructure
  • Monitor Terraform state and execution
  • Ensure infrastructure consistency across environments
  • Plan infrastructure updates proactively
  • Maintain infrastructure reliability

Understanding Infrastructure as Code Metrics

Before diving into monitoring methods, it's important to understand key IaC metrics:

Deployment Metrics

Deployment frequency shows how often infrastructure is deployed. Deployment success rate indicates successful deployment percentage. Deployment duration shows deployment execution time. Deployment changes indicates infrastructure modifications.

Drift Detection Metrics

Drift detected shows configuration inconsistencies. Drift count indicates number of drifted resources. Drift severity shows drift impact level. Drift resolution indicates drift fix completion.

State Metrics

State file size shows Terraform state size. State lock status indicates state file locks. State consistency shows state accuracy. State backup indicates backup status.

Key Metrics to Monitor

  • Deployment status: Successful deployments, failures, duration
  • Configuration drift: Detected drift, drifted resources, drift resolution
  • State health: State file status, locks, consistency
  • Resource changes: Planned changes, applied changes, failed changes

Method 1: Monitor Terraform Executions

Track Terraform plan and apply operations:

Check Terraform Execution Status

# Check if Terraform is installed
which terraform || echo "Terraform not found"

# List Terraform workspaces
terraform workspace list

# Check current workspace
terraform workspace show

# Check Terraform version
terraform version

# Check last Terraform execution (if logged)
if [ -f /var/log/terraform-executions.log ]; then
  tail -20 /var/log/terraform-executions.log
else
  echo "No Terraform execution log found"
fi

Terraform execution monitoring shows deployment status.

Monitor Terraform Plans

# Run Terraform plan (dry-run)
cd /path/to/terraform && terraform plan -out=tfplan

# Check plan output for changes
terraform plan -out=tfplan 2>&1 | grep -E "No changes|will be created|will be destroyed|will be updated"

# Count planned changes
terraform plan -out=tfplan 2>&1 | grep -c "will be"

# Save plan for analysis
terraform plan -out=tfplan
terraform show -json tfplan > /tmp/terraform-plan.json

Terraform plan monitoring shows proposed infrastructure changes.

Track Terraform Applies

# Run Terraform apply
cd /path/to/terraform && terraform apply tfplan

# Check apply output
terraform apply tfplan 2>&1 | tee /var/log/terraform-apply.log

# Verify apply success
if terraform apply tfplan 2>&1 | grep -q "Apply complete"; then
  echo "Terraform apply successful"
  echo "$(date +%s),terraform-apply,success" >> /var/log/terraform-executions.log
else
  echo "Terraform apply failed"
  echo "$(date +%s),terraform-apply,failed" >> /var/log/terraform-executions.log
fi

Terraform apply monitoring tracks deployment execution.

Method 2: Monitor Terraform State

Track Terraform state file health and consistency:

Check State File Status

# Check state file exists
if [ -f terraform.tfstate ]; then
  echo "State file found: terraform.tfstate"
  ls -lh terraform.tfstate
else
  echo "State file not found"
fi

# Check state file size
if [ -f terraform.tfstate ]; then
  STATE_SIZE=$(stat -f%z terraform.tfstate 2>/dev/null || stat -c%s terraform.tfstate)
  echo "State file size: ${STATE_SIZE} bytes"
fi

# Check state file backup
if [ -f terraform.tfstate.backup ]; then
  echo "State backup found"
  ls -lh terraform.tfstate.backup
fi

State file monitoring shows Terraform state health.

Check State Locks

# Check for state locks
terraform force-unlock -force LOCK_ID 2>&1 | grep -i "lock" || echo "No locks detected"

# Check state lock status (if using remote state)
# Check backend configuration for lock status
terraform show -json | jq '.values.root_module.resources[] | select(.type == "terraform_remote_state")'

State lock monitoring prevents concurrent modifications.

Validate State Consistency

# Validate Terraform configuration
terraform validate

# Check state consistency
terraform state list

# Verify state matches infrastructure
terraform plan -detailed-exitcode
EXIT_CODE=$?
if [ $EXIT_CODE -eq 0 ]; then
  echo "State matches infrastructure"
elif [ $EXIT_CODE -eq 2 ]; then
  echo "Drift detected - state differs from infrastructure"
else
  echo "Terraform plan failed"
fi

State consistency validation detects configuration drift.

Method 3: Detect Infrastructure Drift

Identify discrepancies between code and actual infrastructure:

Run Drift Detection

# Check for drift (plan should show no changes if no drift)
terraform plan -detailed-exitcode
DRIFT_EXIT_CODE=$?

if [ $DRIFT_EXIT_CODE -eq 0 ]; then
  echo "No drift detected"
  echo "$(date +%s),drift-detection,no-drift" >> /var/log/terraform-drift.log
elif [ $DRIFT_EXIT_CODE -eq 2 ]; then
  echo "Drift detected"
  echo "$(date +%s),drift-detection,drift-detected" >> /var/log/terraform-drift.log
  terraform plan > /var/log/terraform-drift-details.log
fi

# Count drifted resources
DRIFTED_COUNT=$(terraform plan 2>&1 | grep -c "must be replaced\|must be recreated")
echo "Drifted resources: $DRIFTED_COUNT"

Drift detection identifies infrastructure inconsistencies.

Track Drift History

# Log drift detection results
if [ -f /var/log/terraform-drift.log ]; then
  echo "Drift Detection History:"
  tail -20 /var/log/terraform-drift.log
fi

# Count drift detections
DRIFT_DETECTIONS=$(grep -c "drift-detected" /var/log/terraform-drift.log 2>/dev/null || echo "0")
echo "Total drift detections: $DRIFT_DETECTIONS"

# Calculate drift frequency
DAYS_ACTIVE=$(echo "($(date +%s) - $(stat -f %m /var/log/terraform-drift.log 2>/dev/null || stat -c %Y /var/log/terraform-drift.log)) / 86400" | bc)
if [ $DAYS_ACTIVE -gt 0 ] && [ $DRIFT_DETECTIONS -gt 0 ]; then
  DRIFT_FREQUENCY=$(echo "scale=2; $DRIFT_DETECTIONS / $DAYS_ACTIVE" | bc)
  echo "Drift frequency: ${DRIFT_FREQUENCY} detections per day"
fi

Drift history tracking shows drift trends over time.

Monitor Resource Changes

# Track resource creation
CREATED=$(terraform plan 2>&1 | grep -c "will be created")
echo "Resources to be created: $CREATED"

# Track resource updates
UPDATED=$(terraform plan 2>&1 | grep -c "will be updated")
echo "Resources to be updated: $UPDATED"

# Track resource destruction
DESTROYED=$(terraform plan 2>&1 | grep -c "will be destroyed")
echo "Resources to be destroyed: $DESTROYED"

# Log resource changes
echo "$(date +%s),resource-changes,created:$CREATED,updated:$UPDATED,destroyed:$DESTROYED" >> /var/log/terraform-changes.log

Resource change tracking shows infrastructure modifications.

Method 4: Automated Infrastructure as Code Monitoring with Zuzia.app

Manual checks work while you are looking. To keep running them without you, Zuzia uses its agent and scheduled tasks.

How it works in Zuzia

  • Install the Zuzia agent on the server with one command. CPU, RAM, disk, load and network metrics start reporting on their own, with alerts on a threshold, on drift from the server's normal, or on a disk fill-up forecast.
  • Save the commands from this guide as a scheduled task and choose how often it runs, from every minute to once a year. The agent runs it on every host you pick.
  • If a command fails, you get an alert right away by email, Slack, Discord, push (through your Pushover account) or SMS (through your Vonage account).
  • The output of every run is kept, so you can look back at what changed. On paid plans, AI can read the output and alert you only when it spots a problem.

Setting it up

  1. Add the server in the Zuzia dashboard and install the agent.
  2. Create a scheduled task with the check commands below, then choose the hosts and how often it runs.
  3. Choose your notification channels. Turn on maintenance mode during planned work so it doesn't trigger alerts.

Custom Infrastructure as Code Monitoring Commands

You can also add custom commands for detailed IaC analysis:

# Check for drift
terraform plan -detailed-exitcode

# Validate configuration
terraform validate

# Check state
terraform state list

# Monitor Terraform executions
tail -20 /var/log/terraform-executions.log

Add these commands as scheduled tasks in Zuzia.app to monitor Infrastructure as Code continuously and receive alerts when issues are detected.

Best Practices for Infrastructure as Code Monitoring

1. Monitor Infrastructure as Code Continuously

Don't wait for problems to occur:

  • Use Zuzia.app for continuous IaC monitoring
  • Set up alerts before infrastructure drift becomes critical
  • Review infrastructure changes regularly (daily or weekly)
  • Plan infrastructure updates based on monitoring data

2. Set Appropriate Alert Thresholds

Configure alerts based on your infrastructure requirements:

  • Warning: Drift detected, plan shows changes
  • Critical: Apply failed, state inconsistent
  • Emergency: Multiple resources drifted, state lock issues

Adjust thresholds based on your infrastructure criticality and change frequency.

3. Monitor Both Code and State

Monitor at multiple levels:

  • Code level: Configuration validation, plan execution
  • State level: State file health, locks, consistency
  • Infrastructure level: Actual resource state, drift detection

Comprehensive monitoring ensures early detection of issues.

4. Correlate IaC Monitoring with Other Metrics

Infrastructure as Code monitoring doesn't exist in isolation:

  • Compare infrastructure changes with application deployments
  • Correlate drift with manual infrastructure changes
  • Monitor IaC alongside cloud resource usage
  • On paid plans, let AI read the task output and alert you only when it spots a problem

5. Plan Infrastructure Updates Proactively

Use monitoring data for planning:

  • Analyze infrastructure change trends
  • Plan infrastructure updates based on drift detection
  • Optimize infrastructure configuration
  • Review and improve IaC practices

Troubleshooting Infrastructure as Code Issues

Step 1: Identify IaC Problems

When Infrastructure as Code issues are detected:

  1. Check Current IaC Status:

    • View Zuzia.app dashboard for current IaC status
    • Check Terraform execution status
    • Review drift detection results
    • Check state file health

  2. Identify Infrastructure Issues:

    • Review Terraform plans and applies
    • Check for configuration drift
    • Verify state consistency
    • Identify failed deployments or state issues

Step 2: Investigate Root Cause

Once you identify IaC problems:

  1. Review IaC History:

    • Check historical IaC data in Zuzia.app
    • Identify when drift or failures started
    • Correlate IaC problems with infrastructure changes

  2. Check Terraform Configuration:

    • Verify Terraform configuration validity
    • Check state file integrity
    • Review backend configuration
    • Identify configuration errors or inconsistencies

  3. Analyze Infrastructure Changes:

    • Review planned vs applied changes
    • Check for manual infrastructure modifications
    • Identify drift causes and patterns
    • Analyze deployment failures

Step 3: Take Action

Based on investigation:

  1. Immediate Actions:

    • Resolve drift by applying Terraform changes
    • Fix state inconsistencies
    • Resolve state locks if blocking
    • Address failed deployments

  2. Long-Term Solutions:

    • Implement better IaC monitoring
    • Optimize Terraform workflows
    • Plan infrastructure improvements
    • Review and improve IaC practices

FAQ: Common Questions About Infrastructure as Code Monitoring

What is considered healthy Infrastructure as Code status?

Healthy IaC status means Terraform configurations are valid, state files are consistent, no drift is detected, deployments succeed, state locks are managed properly, and infrastructure matches code definitions. State should be backed up regularly.

How often should I check for infrastructure drift?

For production infrastructure, daily or weekly drift checks are recommended. Critical infrastructure may require more frequent checks. Drift detection frequency depends on your infrastructure change rate and criticality. Zuzia.app can monitor drift continuously and alert you when drift is detected.

What's the difference between Terraform plan and apply?

Terraform plan shows proposed infrastructure changes without making changes. Terraform apply executes the plan and modifies infrastructure. Plan is used for review and validation, while apply performs actual deployments.

Can infrastructure drift cause deployment failures?

Yes, infrastructure drift can cause deployment failures when Terraform tries to modify resources that have been changed manually. Drift can also cause inconsistencies between environments. Early detection through monitoring allows you to resolve drift before deployments fail.

How do I identify which resources have drifted?

Run terraform plan to see proposed changes. Resources showing changes when no code changes were made indicate drift. Terraform plan output shows which resources will be created, updated, or destroyed. Zuzia.app tracks drift detection and can help identify drifted resources.

Should I be concerned about Terraform state locks?

Yes, state locks prevent concurrent Terraform executions that could corrupt state. However, stale locks can block legitimate operations. Monitor state locks and resolve stale locks promptly. Set up alerts in Zuzia.app to be notified when locks are detected.

How can I prevent infrastructure drift?

Prevent infrastructure drift by using Infrastructure as Code exclusively, monitoring drift continuously, detecting and resolving drift promptly, preventing manual infrastructure changes, using state locking, backing up state files regularly, and reviewing infrastructure changes regularly. Regular drift detection helps maintain consistency.

Related guides, recipes, and problems

We use cookies to ensure the proper functioning of our website.