Monitor Smarter, Debug Faster: 15 Logging Scripts Every DevOps Pro Needs


Introduction
 Effective logging is the key to maintaining high-performing systems in the fast-paced world of DevOps. Whether you're debugging an issue or trying to spot patterns in production, the proper logging scripts can significantly boost your productivity. In this post, we'll cover 15 essential logging scripts every DevOps professional should have in their toolkit for more intelligent monitoring and faster debugging.

1. Basic Log Tail for Real-Time Monitoring


tail -f /var/log/syslog


This script allows you to view the most recent entries in the log file. It’s perfect for real-time monitoring when investigating live systems.

2. Grep for Filtering Logs by Keywords


grep "error" /var/log/syslog


Search through your logs for specific errors or keywords to pinpoint issues faster.

3. Log Rotation Script for Disk Space Management


logrotate /etc/logrotate.conf


Automatically rotate logs to ensure that old logs don't take up too much disk space.

4. Check the Last 100 Log Entries


tail -n 100 /var/log/syslog


Quickly check the last 100 log entries to see recent activities or errors.

5. Log Backup Script for Disaster Recovery


cp /var/log/syslog /backup/logs/syslog-$(date +%F).log


Automatically back up logs daily to prevent data loss during unexpected events.

6. Monitor Log Size to Prevent Overflows


du -sh /var/log/*


Check the log file sizes and ensure no files are taking up too much space.

7. Error Log Notification Script (Email Alert)


grep "error" /var/log/syslog | mail -s "Error Logs Detected" admin@company.com


Send an email alert whenever an error is logged in your system.

8. Combine Logs from Multiple Servers


ssh user@server 'cat /var/log/syslog' >> combined_logs.log


Aggregate logs from different servers into a single file for centralized debugging.

9. Summarize Log Entries by Date


awk '{print $1}' /var/log/syslog | sort | uniq -c


Quickly summarize how many log entries there are for each date.

10. Extract Logs by Date Range


awk '$0 >= "2024-10-01" && $0 <= "2024-10-15"' /var/log/syslog


Extract log entries between specific dates, which is helpful for pinpointing incidents over a given period.

11. Monitor System Performance with Logs


sar -u 1 3 >> /var/log/sar.log


Monitor system performance metrics like CPU usage and log them for analysis.

12. Log Analytics for Apache Access Logs


awk '{print $9}' /var/log/apache2/access.log | sort | uniq -c | sort -nr


Analyze Apache logs to find the most requested URLs or identify unusual traffic.

13. Filter Specific User Activity in Logs


grep "user1" /var/log/auth.log


Search for all logins and activities associated with a specific user.

14. Check Log Severity Levels


grep -i "critical" /var/log/syslog


Identify critical logs that need immediate attention to ensure system stability.

15. Archive Logs for Long-Term Storage


tar -czvf logs-$(date +%F).tar.gz /var/log/*


Compress and archive logs regularly for long-term storage and compliance purposes.

Conclusion
 Effective logging is essential for any DevOps team. With the right logging scripts in place, you can ensure that your systems run smoothly and that you can debug issues quickly. These 15 scripts are just the beginning—customize them to fit your needs and level up your monitoring and troubleshooting game.

Comments

Post a Comment

Popular posts from this blog

HTTP Basic vs API Key Auth: Best Practices for Secure API Development

Image
Picture this: You’ve spent weeks building your API, and now your security approach boils down to a coin flip between HTTP Basic and API Keys. Choose wrong, and your data’s basically wearing a “hack me” sign. Every developer faces this exact decision, yet most guides leave you with more questions than answers. When implementing authentication for your API, the choice between HTTP Basic Authentication and API Key Authentication can significantly impact your security posture and user experience. So what makes one better than the other? When should you use HTTP Basic over API Keys? Is there ever a scenario where the “simpler” option is actually more secure? The answers might surprise you – and they definitely aren’t what most Stack Overflow threads would have you believe. Understanding API Authentication Fundamentals Why API Security Matters in Modern Development API security isn’t just some technical checkbox—it’s the fortress protecting your digital kingdom. With businesses exposing crit...
Read more

ECS Deployment Best Practices: Blue/Green with CodePipeline and CodeDeploy

Image
DevOps engineers and AWS cloud architects looking to improve their container deployment processes will benefit from implementing blue/green deployments with Amazon ECS. This guide walks through setting up reliable, zero-downtime deployments using AWS CodePipeline and CodeDeploy for your containerized applications. We’ll cover how to configure your ECS environment properly, create automated deployment pipelines, and implement blue/green deployment strategies that minimize risk during updates. Understanding ECS Deployment Strategies What is Amazon ECS and why it matters Amazon Elastic Container Service (ECS) isn’t just another tool in AWS’s massive catalog—it’s the backbone of modern containerized applications. At its core, ECS is a fully managed container orchestration service that handles all the complex tasks of running, stopping, and managing Docker containers. Think of ECS as the conductor of an orchestra where each container is an instrument. Without proper coordination, you’d just...
Read more

AWS Console Not Loading? Here’s How to Fix It Fast

Image
When the AWS Management Console refuses to load, it can be incredibly frustrating, especially when trying to deploy, debug, or manage production infrastructure. Fortunately, quick and effective fixes get you back on track. This guide outlines common causes and step-by-step solutions to resolve the issue quickly. 1. Check AWS Service Status Start by verifying if AWS itself is experiencing issues: Visit the AWS Health Dashboard to check for regional outages. Look for service-specific issues affecting IAM , EC2 , S3 , or the Console itself . Tip: Subscribe to AWS status updates on Twitter for real-time alerts. 2. Clear Browser Cache and Cookies Corrupted cache or cookies often prevent pages from rendering correctly. Steps: Open browser settings. Clear cache , cookies , and site data for aws.amazon.com . Reload the page. Try logging in using Incognito/Private Mode to rule out browser extensions interfering. 3. Try a Different Browser or Device Sometimes, browser-specific quirks b...
Read more

YouTube Channel