Log Files Consuming Disk Space - How to Fix
Troubleshooting guide for log files consuming excessive disk space. Learn how to identify large log files, implement log rotation, and prevent disk space issues from logs.
Log Files Consuming Disk Space - How to Fix
Log files can grow indefinitely and consume disk space, causing disk full errors and service failures. This troubleshooting guide helps you identify large log files, implement log rotation, and prevent disk space issues.
For comprehensive log monitoring, see Centralized Log Monitoring. For monitoring log sizes, see Monitor Log File Sizes.
Symptoms of Log Files Consuming Disk Space
- Disk space warnings or errors
- Large log files in /var/log
- Services failing due to disk full
- Slow system performance
- Log rotation not working
Step 1: Identify Large Log Files
Find Large Log Files
# Find large log files in /var/log
find /var/log -type f -size +100M -exec ls -lh {} \;
# Find largest log files
find /var/log -type f -exec du -h {} + | sort -rh | head -10
# Check disk usage by directory
du -sh /var/log/* | sort -rh | head -10
# Check total log directory size
du -sh /var/log
Check Disk Space Usage
# Check disk space
df -h
# Check disk usage for /var/log
df -h /var/log
# Check inode usage
df -i
# Find files consuming space
du -h --max-depth=1 /var/log | sort -rh
Step 2: Check Log Rotation Status
Verify Logrotate Configuration
# Check logrotate configuration
cat /etc/logrotate.conf
# Check application-specific logrotate configs
ls -la /etc/logrotate.d/
# Test logrotate configuration
sudo logrotate -d /etc/logrotate.conf
# Check logrotate status
ls -la /var/lib/logrotate/status
Check Log Rotation Execution
# Check last logrotate execution
stat /var/lib/logrotate/status | grep Modify
# Force logrotate execution
sudo logrotate -f /etc/logrotate.conf
# Check logrotate logs
grep logrotate /var/log/syslog | tail -20
Step 3: Identify Root Causes
Common Causes of Large Log Files
-
Log Rotation Not Working:
- Logrotate not configured
- Logrotate not running
- Incorrect logrotate configuration
- Permission issues
-
High Log Volume:
- Excessive logging
- Debug logging enabled
- Error loops generating logs
- High application activity
-
Log Retention Issues:
- Logs not being deleted
- Retention period too long
- Compressed logs accumulating
- Archive logs not cleaned
Step 4: Fix Log File Issues
Implement Log Rotation
-
Configure Logrotate:
# Edit logrotate configuration sudo nano /etc/logrotate.d/application-logs # Add configuration /var/log/application/*.log { daily rotate 7 compress delaycompress missingok notifempty create 0644 user group } -
Test Logrotate Configuration:
# Test configuration sudo logrotate -d /etc/logrotate.d/application-logs # Force rotation sudo logrotate -f /etc/logrotate.d/application-logs
Clean Up Large Log Files
-
Archive Old Logs:
# Compress old logs find /var/log -name "*.log" -mtime +7 -exec gzip {} \; # Move to archive find /var/log -name "*.log.gz" -mtime +30 -exec mv {} /archive/logs/ \; -
Delete Old Logs:
# Delete logs older than 30 days find /var/log -name "*.log" -mtime +30 -delete # Delete compressed logs older than 90 days find /var/log -name "*.log.gz" -mtime +90 -delete -
Clean Up Specific Logs:
# Truncate large log file sudo truncate -s 0 /var/log/large-log.log # Or remove and recreate sudo rm /var/log/large-log.log sudo touch /var/log/large-log.log
Reduce Log Volume
-
Adjust Log Levels:
- Change debug logging to info/warning
- Reduce verbose logging
- Disable unnecessary log statements
- Update application logging configuration
-
Fix Error Loops:
- Identify and fix error loops
- Reduce error logging frequency
- Implement log throttling
- Fix application bugs causing errors
Step 5: Prevent Future Log Issues
Implement Proper Monitoring
Set up automated monitoring with Zuzia.app to track log file sizes continuously:
-
Add Log Size Monitoring:
find /var/log -type f -size +100M | wc -l- Monitor log file sizes
- Alert when large logs detected
-
Monitor Disk Space:
df -h /var/log | awk 'NR==2 {print $5}' | sed 's/%//'- Track disk usage
- Alert when disk space low
Best Practices
-
Configure Log Rotation:
- Set up logrotate for all applications
- Configure appropriate retention periods
- Test logrotate configuration regularly
- Monitor log rotation execution
-
Monitor Log Sizes:
- Track log file sizes continuously
- Alert when logs exceed thresholds
- Review log growth patterns
- Optimize logging configuration
-
Implement Log Retention:
- Set appropriate retention periods
- Archive old logs before deletion
- Compress old logs to save space
- Clean up archived logs regularly
-
Optimize Logging:
- Use appropriate log levels
- Reduce verbose logging
- Implement log throttling
- Fix error loops
FAQ: Common Questions About Log Files
How do I prevent log files from consuming disk space?
Prevent log file issues by configuring proper log rotation, setting appropriate retention periods, monitoring log file sizes continuously, and cleaning up old logs regularly.
How often should I rotate logs?
Rotate logs daily for high-volume applications, weekly for moderate volume, and monthly for low volume. Adjust based on your log volume and disk capacity.
Can I delete log files safely?
You can delete old log files safely if they're rotated and archived. Never delete active log files that applications are writing to. Use logrotate to manage log files properly.
How do I reduce log file size?
Reduce log file size by adjusting log levels, reducing verbose logging, fixing error loops, implementing log throttling, and configuring proper log rotation.
Related guides, recipes, and problems
-
Related guides
-
Related recipes
-
Related problems