Logging plays an important role in software development. Whenever there is any problem with the system, it is always advised to trace Logs of the system.
In this post, we will be learning about how logs can be invaluable for the development as well as for Analysis of the system.
The ELK Stack
If you are not yet familiar with this stack, then you should at least give it a try. ElasticSearch, Logstash and Kibana are the open source products provided by Elastic.
- ElasticSearch: a search engine based on Lucene. It provides a distributed, multitenant-capable full-text search engine with an HTTP web interface and schema-free JSON documents.
- Logstash: a pipeline for managing events and logs, which come from variety of systems. It takes Input from some source, Processes it and then Outputs it to some destination.
- Kibana: a powerful tool for exploring and visualizing the data from ElasticSearch. It analyze the data more intelligently and provides better understanding with the help of Bar Charts, Histograms, Pie Charts, Maps etc.
Our Objective
In its most brief explanation, our objective is to give meaning to Logs. The objective can be achieved in given 3 steps:
- Collect Logs from various different possible Sources and send them to Logstash
- In Logstash, Process and Filter them according to your needs and send it to ElasticSearch
- Fetch Log Data from ElasticSearch and visualize logs data
Other Tools
Apart from the ELK Stack, we will be going to use some more tools in order to setup our complete arrangement for Logging. You can read a brief about all the tools below.
Syslog
Syslog is a standard for message logging. It permits separation of the software that generates messages, the system that stores them, and the software that reports and analyzes them. Each message is labeled with a facility code, indicating the software type generating the message, and assigned a severity label.
– Wikipedia
Syslog-ng
Syslog-ng extends the basic syslog protocol with various outstanding features, which makes it Next-Generation. It provides the ability to collect log events from various possible sources, process them in near real-time and deliver them to variety of destinations.
Read more about syslog-ng at: https://syslog-ng.org/
Log4J
Log4J is a reliable, fast and flexible logging framework (APIs) written in Java, which is distributed under the Apache Software License. Log4J is a popular logging package written in Java. Log4J has been ported to the C, C++, C#, Perl, Python, Ruby, and Eiffel languages.
– TutorialsPoint
So, after we are introduced enough with all the tools, that we can use for setting up Logging, we will be learning about how we can use these tools in order to have a good Logging Management in our system.
In the Next part of this blog, we will learn about installing and setup of these tools.
Tags: elasticsearch, kibana, log4j, logging, logstash, syslog, syslog-ng
Leave a Reply