topic How do I know and change at what time is Splunk indexing the data from local files? in Getting Data In

How do I know and change at what time is Splunk indexing the data from local files?

marina_rovira — Tue, 12 Jan 2016 09:48:46 GMT

Hello all,

I have a question. Every night, between 00:00 and 01:30 at night, the data is being actualized by scripts I've done for exporting and getting the data in the host. When It does the exports, it also changes some part of the data. For example, Priority firstly appears as "1 - Critical" and the scripts change this to "1", but, my problem is that Splunk collects the data early or in some way that I cannot figure out, as it is indexing this as "1 - Critical".

When I check the data in the morning, in the file is correctly changed, but not in the index, so I have priorities "1","3"... and priorities "1-Critical", "3- Low".... in the same index and I would like to it be indexed just as "1", "2", "3"...

Could someone help me on this? Until now, I'm deleting the indexes and creating them once a week, but it's like 10 indexes and the files are correct. I think I just need Splunk to index it later. How can I configure it?

Thank you!

Re: How do I know and change at what time is Splunk indexing the data from local files?

renjith_nair — Tue, 12 Jan 2016 09:59:12 GMT

It looks like splunk indexes both , the original file and changed file. Easiest solution would be separating the locations , ie splunk's input should be pointing to the location where the changed files are available not the original files

Re: How do I know and change at what time is Splunk indexing the data from local files?

marina_rovira — Tue, 12 Jan 2016 10:14:10 GMT

There are not two different locations.

I have my files in... we say for example /home/marina/ and every night, the cron executes scripts that export data from the system and actualise this files. Also, these are the files splunk is indexing every night.

So for example, it's been a week since I've changed manually the indexes, so now, if i look up the data, I have the priorities well setted ("1") since a week ago (when I changed it), but since last week, the priority appears the other way ("1-Critcal") . If i look the file in the host, they are ok, just the index is wrong.

So for this, I would like to know if I can set splunk look for the data for indexes later or someway that I hadn't to renew the indexes to get them right.

(All the cron tasks ends at 1:30 am more or less, I need to splunk collect the data later than that)

Re: How do I know and change at what time is Splunk indexing the data from local files?

renjith_nair — Tue, 12 Jan 2016 10:28:48 GMT

Is it possible for you to change the locations ? For eg: The actual files will be in /home/marina and once you change the files with your script, copy to some other location say /home/marina1 and configure splunk input to watch /home/marina1 location.

Or another dirty work around is to schedule a cron to activate/inactivate the input configuration of splunk or instead of monitor stanza , introduce scripted input which will read and push data at specific time.

Re: How do I know and change at what time is Splunk indexing the data from local files?

marina_rovira — Tue, 12 Jan 2016 14:53:37 GMT

mmm... maybe is better to try first the changing location thing.

I will do it soon and maybe if It doesn't work I will ask for the second option, because I haven't understand well now.

Re: How do I know and change at what time is Splunk indexing the data from local files?

marina_rovira — Fri, 15 Jan 2016 08:39:13 GMT

I did the location thing and It seems is working! Thanks! 🙂