Piyush Tech Blog

Goal: In these tutorial we gonna cover deletion of old logs in ELK Stack. We gonna achive these by deleting old indices created by Logstash while dumping logs in Elasticsearch. Prerequisites: Old logs to delete... 😜😜 Let's Begin the exercise: Install curator Curator is a package in Elasticsearch  repository to delete old indices. Create a file sudo vi /etc/yum.repos.d/curator.repo paste following lines Save and Exit file Run yum install sudo yum install elasticsearch-curator Configure Curator Create a directory mkdir ~/.curator/ Open a file sudo vi ~/.curator/curator.yml paste following code Save and Exit file Deletion pattern Create file to define delete pattern in Elasticesearch sudo vi ~/.curator/delete_indices.yml paste following lines in file Create a log file for curator on the location you defined in configuration, and assign permission to right into file. sudo touch /var/log/curator #to assign permission to write l

Goal Trigger alert on elastic search stream.   Install Elastalert sudo yum install gcc sudo pip install elastalert sudo yum install git Just to get basic elastalert rules reference clone following git repository. git clone https://github.com/Yelp/elastalert.git example_elastalert Create your own rules sudo mkdir -p /etc/elastalert/rules_folder/ sudo cp ~/example_elastalert/example_rules/example_frequency.yaml /etc/elastalert/rules_folder/frequency.yaml sudo cp ~/example_elastalert/config.yaml.example /etc/elastalert/config.yaml Configure Elastalert sudo vi /etc/elastalert/config.yaml Search for 'rules_folder', 'es_host' and change values in file rules_folder: "/etc/elastalert/rules_folder" es_host: localhost Now Open File  and change conf sudo vi /etc/elastalert/rules_folder/frequency.yaml filter: - term: _type: "apache-error" - query: query_string: query: "host:HostName" alert: - "email" include: [