Elastalert

Goal
Trigger alert on elastic search stream. 

Install Elastalert

sudo yum install gcc
sudo pip install elastalert
sudo yum install git

Just to get basic elastalert rules reference clone following git repository.

git clone https://github.com/Yelp/elastalert.git example_elastalert

Create your own rules

sudo mkdir -p /etc/elastalert/rules_folder/
sudo cp ~/example_elastalert/example_rules/example_frequency.yaml /etc/elastalert/rules_folder/frequency.yaml
sudo cp ~/example_elastalert/config.yaml.example /etc/elastalert/config.yaml

Configure Elastalert

sudo vi /etc/elastalert/config.yaml

Search for 'rules_folder', 'es_host' and change values in file

rules_folder: "/etc/elastalert/rules_folder" 

es_host: localhost 

Now Open File  and change conf

sudo vi /etc/elastalert/rules_folder/frequency.yaml

filter:
- term:
  _type: "apache-error"
- query:
  query_string:
  query: "host:HostName"

alert:
 - "email"
include: ["host", "message", "errormessage"]
top_count_keys: ["errormessage"]
alert_subject: 'Apache Error rate increased for hostname {0} at {1}'
alert_subject_args: ["host", "@timestamp"]
alert_text_type: exclude_fields
email: ["Email-1", "Email-2"]
from_addr: 'prod-elk@hostname'

To test rule

elastalert-test-rule --config /etc/elastalert/config.yaml /etc/elastalert/rules_folder/frequency.yaml

If any conflict occur

sudo pip install pip-conflict-checker
pipconflictchecker

Generally following package have issue so upgrade package version

sudo pip install "six>=1.9"
sudo pip install "requests-oauthlib(>=0.8.0)"

Test again

elastalert-test-rule --config /etc/elastalert/config.yaml /etc/elastalert/rules_folder/frequency.yaml

Now all thing should be okay

To run elastalert as service

sudo vi /etc/init.d/elastalert

Gist
To make it executable

sudo chmod +x /etc/init.d/elastalert

Further Reference
https://github.com/Yelp/elastalert

Hope You will be able to get basic alert from your elastic search stream, If any issue occur please lemme know in comments.