Logstash schedule problem.

Currently, the logstash.conf is set to repeat after 10 hrs.

Generating indices requires high memory. According to this, we will need to set up the schedule in logstash.conf

The suggested way to regenerate indices is to add a timestamp field in the respective table which is linked to specific indices. So logstash will rewrite the respective index if there are changes in fields which are included in SQL query.

Advantages :

1. We can set schedule to minutes rather than hours.
2. Will take less CPU if regeneration of indices includes only records where data is updated.
3. As the database expands this is the most appropriate way to generate indices. (According to my research)

During deployment, we can set schedule to 1 hr and once we add the timestamp column to track the updation status of record we can further reduce the schedule to 5-10 minutes as required.