topic Re: How to make Splunk read the file when only date stamp of the file changes. in Splunk Dev

How to make Splunk read the file when only date stamp of the file changes.

lakromani — Mon, 25 Sep 2017 07:15:13 GMT

I have a system where I use SSH to pull out status data from a remote system
This is then stored to a file that Splunk i set to monitor.
My problem is that the files is read from the system every 5 minutes, but Splunk only shows indexed data when file content is changed.
I would like Splunk to show all the content every time the file changes date (5 min cron job), even if nothing has changed within the file.
Is this possible?

Example first run:
red=1
yellow=2
time stamp of file 09:05
Splunk now show two events.

Second run:
red=1
yellow=2
time stamp of file 09:10
Splunk now shows no events.
I need to show both every 5 min, even if they do not change.

Third run:
red=1
yellow=3
time stamp of file 09:15
Splunk now shows all event again, since content of file has change.

Re: How to make Splunk read the file when only date stamp of the file changes.

gcusello — Mon, 25 Sep 2017 07:30:37 GMT

Hi lakromani,
you don't need to write a file and then read it, you could use a scripted input (see https://docs.splunk.com/Documentation/Splunk/latest/AdvancedDev/ScriptedInputsIntro ).
In other words, you should create a script with the command you have to run and then schedule and run it in from Splunk.
In this way script output will be indexed with the indexing time and you have your result to search.
To schedule and run a script you have to put your script in $SPLUNK_HOME/bin or $SPLUNK_HOME/etc/apps/your_app/bin folder and then follow the web gui procedure [Settings -- Data Inputs -- Scripts -- New].

Bye.
Giuseppe

Re: How to make Splunk read the file when only date stamp of the file changes.

inventsekar — Mon, 25 Sep 2017 07:35:04 GMT

the file is read by splunk and getting indexed.
but Splunk only shows indexed data when file content is changed.
I would like Splunk to show all the content every time the file changes date (data?!?!?!) , even if nothing has changed within the file.

little bit confusing. is this is the real issue ?

when you search, splunk shows only the recent changed data, not whole data.
when you search, splunk should show the whole content of the file, even if there was no recent updates

what query you are using

Re: How to make Splunk read the file when only date stamp of the file changes.

gcusello — Mon, 25 Sep 2017 07:42:54 GMT

In search you can show all the indexed data or filter them as you like.
The problem is to take logs only when changed or always.
Using your solution, you index only changes, using scripted inputs, you index script output at every run.
Based on the solution you choose you have to build you search.
What is your need: an alert when there's a change? or to show always situation?
Bye.
Giuseppe

Re: How to make Splunk read the file when only date stamp of the file changes.

lakromani — Mon, 25 Sep 2017 07:47:05 GMT

It should state "date" in the title, so:
I would like Splunk to show all the content every time the file changes date stamp.

See updated post.

Re: How to make Splunk read the file when only date stamp of the file changes.

lakromani — Mon, 25 Sep 2017 07:48:22 GMT

Would this then give me all the different status for all the event in the file, or only the event that do change?

See updated post.

Re: How to make Splunk read the file when only date stamp of the file changes.

gcusello — Mon, 25 Sep 2017 07:57:43 GMT

try to use scripted input, it's the solution.
Bye.
Giuseppe

Re: How to make Splunk read the file when only date stamp of the file changes.

lakromani — Mon, 25 Sep 2017 08:07:01 GMT

Will try, thanks.

Re: How to make Splunk read the file when only date stamp of the file changes.

lakromani — Mon, 25 Sep 2017 09:01:53 GMT

Can confirm its working.
Learning some every day, thanks.