How to Monitor Log File Sizes and Rotation on Linux Server

How to Monitor Log File Sizes and Rotation on Linux Server

Monitor log file sizes and rotation to track log growth, detect rotation failures, and prevent disk space issues. This guide shows you how to monitor log files and set up automated monitoring.

For comprehensive log monitoring, see Centralized Log Monitoring. For troubleshooting log issues, see Log Files Consuming Disk Space.

Why Monitoring Log Files Matters

Log files can grow indefinitely and consume disk space, causing disk full errors and service failures. Monitoring log file sizes and rotation ensures logs are managed properly and disk space is maintained.

Method 1: Check Log File Sizes

Find Large Log Files

# Find large log files in /var/log
find /var/log -type f -size +100M -exec ls -lh {} \;

# Find largest log files
find /var/log -type f -exec du -h {} + | sort -rh | head -10

# Check specific log file size
ls -lh /var/log/syslog
ls -lh /var/log/nginx/access.log

Monitor Log File Growth

# Monitor log file size over time
while true; do
  SIZE=$(du -h /var/log/syslog | awk '{print $1}')
  echo "$(date): $SIZE"
  sleep 60
done

# Check log file sizes in directory
du -sh /var/log/* | sort -rh

Method 2: Check Log Rotation Status

Check Logrotate Configuration

# View logrotate configuration
cat /etc/logrotate.conf

# Check logrotate status
ls -la /var/lib/logrotate/status

# Test logrotate configuration
sudo logrotate -d /etc/logrotate.conf

# Force logrotate execution
sudo logrotate -f /etc/logrotate.conf

Verify Log Rotation

# Check rotated log files
ls -lh /var/log/*.log.*.gz

# Check log rotation dates
ls -lth /var/log/*.log.* | head -10

# Count rotated log files
find /var/log -name "*.log.*.gz" | wc -l

Method 3: Automated Log File Monitoring with Zuzia.app

Set up automated monitoring to track log file sizes continuously and receive alerts when log files exceed thresholds or rotation fails.

Step 1: Add Log File Size Monitoring Command

1. Log in to Zuzia.app Dashboard

   • Access your Zuzia.app account
   • Navigate to your server
   • Click "Add Scheduled Task"

2. Configure Log Size Check Command

   find /var/log -type f -size +100M -exec ls -lh {} \; | wc -l
   

   • Set execution frequency (every 30-60 minutes)
   • Configure alerts when large log files are found

Step 2: Configure Alert Thresholds

• Warning: Log file > 100MB
• Critical: Log file > 500MB
• Emergency: Log file > 1GB

Step 3: Monitor Log Rotation

Add command to check log rotation status:

# Check if logrotate ran recently
stat /var/lib/logrotate/status | grep Modify

Best Practices for Log File Monitoring

1. Monitor Log File Sizes Continuously

• Track log file sizes regularly
• Alert when log files exceed thresholds
• Monitor log growth rates
• Prevent disk space issues

2. Verify Log Rotation

• Check log rotation configuration
• Monitor rotation execution
• Verify rotated logs are compressed
• Ensure old logs are deleted

3. Set Appropriate Rotation Policies

• Configure rotation based on log volume
• Set appropriate retention periods
• Compress old logs to save space
• Delete logs after retention period

4. Monitor Log Growth Trends

• Track log growth over time
• Identify log growth patterns
• Plan log storage capacity
• Optimize log generation

Troubleshooting Log File Issues

Step 1: Identify Log Problems

When log files are consuming disk space:

# Find largest log files
find /var/log -type f -exec du -h {} + | sort -rh | head -10

# Check log file sizes
du -sh /var/log/*

# Check disk space
df -h /var/log

Step 2: Resolve Log Issues

Based on investigation:

1. Rotate Log Files:

   # Force log rotation
   sudo logrotate -f /etc/logrotate.conf
   
   # Manually rotate specific log
   sudo logrotate -f /etc/logrotate.d/nginx
   

2. Clean Up Old Logs:

   # Remove old rotated logs
   find /var/log -name "*.log.*.gz" -mtime +30 -delete
   
   # Clean up old logs
   sudo journalctl --vacuum-time=30d
   

3. Optimize Log Configuration:

   • Adjust log rotation frequency
   • Reduce log retention period
   • Optimize log verbosity

FAQ: Common Questions About Log File Monitoring

How often should I check log file sizes?

For production servers, continuous automated monitoring is essential. Zuzia.app can check log file sizes every 30-60 minutes, alerting you when log files exceed thresholds.

What is considered a large log file?

Large log file depends on your disk capacity and log volume. Generally, log files over 100MB should be rotated, files over 500MB require immediate attention, and files over 1GB indicate rotation failures.

How do I prevent log files from consuming disk space?

Prevent log file issues by configuring proper log rotation, setting appropriate retention periods, monitoring log file sizes continuously, and cleaning up old logs regularly.

Can log monitoring impact server performance?

Log monitoring commands have minimal impact on server performance when done correctly. Use appropriate monitoring frequency and avoid monitoring during peak log generation periods.

Related guides, recipes, and problems

• Related guides

  • Centralized Log Monitoring and Analysis - Best Practices for Linux Servers
  • Disk Monitoring Deep Dive - Understanding Storage Metrics

• Related recipes

  • Quick Log Commands - View Recent Entries Right Now
  • Continuous Log Monitoring - Detect Errors Before Users Report
  • Find Files Over a Size Threshold on Linux

• Related problems

  • Log Files Consuming Disk Space - How to Fix
  • Disk Full Emergency - Free Up Space in 5 Minutes