Zuzia.app logo
Get started
Features
How It Works
Pricing
About
Faq
Sign up Log In
← All guides

Centralized Log Monitoring: Collect, Search & Alert on Logs

Updated:

Centralized Log Monitoring: Collect, Search & Alert on Logs

Centralized log monitoring enables you to collect, aggregate, and analyze logs from multiple servers in one place, making troubleshooting and security monitoring more efficient. This comprehensive guide covers everything you need to know about centralized log monitoring and analysis.

For related log monitoring topics, see Log Files Consuming Disk Space. For monitoring specific log files, see Monitor Log File Sizes.

Why Centralized Log Monitoring Matters

Centralized log monitoring provides a unified view of system activity across all servers, making it easier to troubleshoot issues, detect security threats, and maintain compliance. Without centralized logging, you must check logs on each server individually, making it difficult to correlate events and identify patterns.

Effective centralized log monitoring enables you to:

  • Collect logs from all servers in one location
  • Search and analyze logs across multiple servers
  • Correlate events across different systems
  • Detect security threats and anomalies
  • Maintain compliance with log retention requirements
  • Troubleshoot issues more efficiently

Key Log Types to Monitor

System Logs

  • syslog: System messages and events
  • auth.log: Authentication and authorization events
  • kern.log: Kernel messages
  • messages: General system messages

Application Logs

  • Web server logs: Nginx, Apache access and error logs
  • Application logs: Custom application log files
  • Database logs: MySQL, PostgreSQL logs
  • Service logs: Systemd service logs

Security Logs

  • Failed login attempts: Authentication failures
  • SSH access: SSH connection logs
  • Firewall logs: iptables, firewalld logs
  • Intrusion detection: Fail2ban, security alerts

Method 1: Centralized Logging with rsyslog

Configure rsyslog Server

# Install rsyslog
sudo apt-get install rsyslog  # Debian/Ubuntu
sudo yum install rsyslog      # CentOS/RHEL

# Configure rsyslog to receive remote logs
sudo nano /etc/rsyslog.conf

# Add these lines:
module(load="imudp")
input(type="imudp" port="514")

module(load="imtcp")
input(type="imtcp" port="514")

# Restart rsyslog
sudo systemctl restart rsyslog

Configure rsyslog Clients

# Configure client to send logs to central server
sudo nano /etc/rsyslog.conf

# Add this line (replace with your log server IP):
*.* @192.168.1.100:514

# Restart rsyslog
sudo systemctl restart rsyslog

Method 2: Centralized Logging with syslog-ng

Configure syslog-ng Server

# Install syslog-ng
sudo apt-get install syslog-ng  # Debian/Ubuntu
sudo yum install syslog-ng      # CentOS/RHEL

# Configure syslog-ng
sudo nano /etc/syslog-ng/syslog-ng.conf

# Add source for remote logs:
source s_net { udp(ip(0.0.0.0) port(514)); };
destination d_central { file("/var/log/central/${HOST}/${YEAR}-${MONTH}-${DAY}.log"); };
log { source(s_net); destination(d_central); };

# Restart syslog-ng
sudo systemctl restart syslog-ng

Method 3: Automated Log Monitoring with Zuzia.app

Running these checks by hand works once. Zuzia's agent lets you repeat them on every server without logging in to each one.

How it works in Zuzia

  • Install the Zuzia agent on the server with one command.
  • Zuzia runs a basic security audit on the host and lists what it found. On paid plans, AI summarises the findings in plain language.
  • The agent collects system logs, so you can read them in the panel instead of over SSH.
  • Save the commands from this guide as a scheduled task and choose how often they run. If a command fails, you get an alert right away by email, Slack, Discord, push (through your Pushover account) or SMS (through your Vonage account). On paid plans, AI can read the output and alert you only when it spots a problem.

Setting it up

  1. Add the server in the Zuzia dashboard and install the agent.
  2. Open the host's security audit and logs.
  3. Create a scheduled task with the check commands below, choose the hosts and frequency, and pick your notification channels.

Best Practices for Centralized Log Monitoring

1. Implement Log Rotation

  • Configure log rotation to prevent disk space issues
  • Set appropriate retention periods
  • Compress old logs to save space
  • Monitor log rotation status

2. Monitor Log File Sizes

  • Track log file sizes continuously
  • Alert when logs exceed size thresholds
  • Implement automatic log rotation
  • Clean up old logs regularly

3. Analyze Log Patterns

  • Search logs for error patterns
  • Correlate events across servers
  • Identify security threats
  • Track application errors

4. Set Up Automated Alerts

  • Configure alerts for critical errors
  • Set up alerts for security events
  • Monitor log file sizes
  • Alert on log rotation failures

5. Maintain Log Retention

  • Implement appropriate retention policies
  • Archive old logs for compliance
  • Delete logs after retention period
  • Monitor log storage usage

Troubleshooting Log Monitoring Issues

Step 1: Identify Log Problems

When log monitoring indicates issues:

  1. Check Log File Sizes:

    • Review log file sizes
    • Identify large log files
    • Check log rotation status

  2. Review Log Errors:

    • Search logs for errors
    • Identify error patterns
    • Review recent log entries

  3. Check Log Collection:

    • Verify logs are being collected
    • Check network connectivity
    • Review log server status

Step 2: Resolve Log Issues

Based on investigation:

  1. Fix Log Rotation:

    • Configure log rotation properly
    • Fix rotation failures
    • Clean up old logs

  2. Optimize Log Storage:

    • Compress old logs
    • Archive logs to external storage
    • Delete unnecessary logs

  3. Fix Log Collection:

    • Fix network connectivity issues
    • Restart log collection services
    • Verify log server configuration

FAQ: Common Questions About Centralized Log Monitoring

How do I set up centralized logging?

Set up centralized logging by configuring a log server (rsyslog or syslog-ng), configuring clients to send logs to the server, and implementing log rotation and retention policies. Zuzia.app can help monitor the entire process.

What logs should I monitor?

Monitor system logs (syslog, auth.log), application logs (web server, application logs), security logs (failed logins, SSH access), and service logs (systemd logs). Focus on logs relevant to your infrastructure.

How long should I retain logs?

Log retention depends on compliance requirements and storage capacity. Typically, retain logs for 30-90 days for operational purposes, and longer for compliance. Implement log rotation and archiving to manage storage.

Can log monitoring impact server performance?

Log monitoring has minimal impact when done correctly. Use efficient log collection methods, implement log rotation, and monitor log file sizes to prevent performance issues.

Related guides, recipes, and problems

  • Related guides

    • guides--server-performance-monitoring-best-practices.md
    • guides--monitoring-application-performance-linux.md

  • Related recipes

    • recipes--monitor-log-file-sizes-rotation.md
    • recipes--monitor-recent-system-logs-last-minutes.md
    • recipes--monitor-system-log-entries-linux.md
    • recipes--check-nginx-error-logs-recent.md
    • recipes--monitor-apache-error-logs-recent.md

  • Related problems

    • problems--log-files-consuming-disk-space.md
    • problems--server-slow-performance-issues.md

We use cookies to ensure the proper functioning of our website.