How to monitor a log file that updates the same line?

[leon24]

Hi all,

I have a log file that briefly logs file in this pattern.

For e.g.
Available 12-01-2014 03:03:44

So if there is no change in the status, the timestamp gets updated every minute (it doesn't creates a new line)

The previous sample might be changed to the following.

For e.g.
Available 12-01-2014 03:04:44

If there is a change in the status, a new line will be created (and if no change in status, only the timestamp gets updated).

For e.g.
Available 12-01-2014 03:04:44
Unavailable 12-01-2014 03:04:50

I am able to create dashboards for logs that appends new line at the end of file, however, I'm unable to monitor the above mentioned logs that does not append new lines but instead updates the existing line.

My search string is as follow (it works for logs that appends new lines)

index="index_name" | stats latest(status), latest(availability_date), latest (availability_time)

The dashboard always shows the latest line at first (which is what I want), however when the dashboard gets updated again, it will automatically show the earliest result in the log file instead.

Anybody able to help?

[dwaddle]

This is less solution and more personal opinion. If you have control of this application's source code - change it to do something more sane 🙂

[leon24]

Hi dwaddle,

Unfortunately I do not have control of the application's source code, apparently it is SAP log files and the SAP application belongs to another team. 😞

[gkanapathy]

You would need to use the CHECK_METHOD setting in props.conf. If you set it to entire_md5 or modtime, changes anywhere in the file will index the whole file.

[leon24]

My dashboard search string is as follow:

index="my_index" sourcetype="sap_availability" | stats latest(sap_availability_status) as "Status", latest(sap_availability_date) as "Date" by host | eval host=lower(host) | sort host ASC | rename host as "Host"

[leon24]

Hi gkanapathy and all,

I've made the changes in the server which has the forwarder to the file ../etc/system/local/props.conf.

This is the new line that I've created.

[]
CHECK_METHOD = modtime

In my case, it is

[sap_availability]
CHECK_METHOD = modtime

I'm still seeing the earliest entry in the dashboard. Am I configuring the wrong file or using the wrong format?

Prior to this change, my dashboard actually loads with the latest event and when the dashboard refreshes, it will load back earliest event. Further refreshes also reflects to the earliest event.