Getting Data In

Unable to monitor and index a log file

royimad
Builder

Splunk is unable to monitor a local file - and a search query is not returning any values - No events is indexed, How to troubleshoot this?

Search: sourcetype="online_error_test1" >>> No results for any time.

inputs.conf

[monitor:///home/splunk/error_delta.log]
disabled = false
followTail = 0
sourcetype = online_error_test1

props.conf

[online_error_test1]
TIME_FORMAT = %a %b %e %Y %k:%M:%S,%3 %Z

1 Solution

Ayn
Legend

One thing to do is troubleshoot the input using amrit's excellent script:

http://blogs.splunk.com/2011/01/02/did-i-miss-christmas-2/

View solution in original post

jbsplunk
Splunk Employee
Splunk Employee

First, check splunkd.log for messages from the WatchedFile and TailingProcessor components looking for anything related to error_delta.log. Hopefully this tells you what is happening, but it might not tell you anything at all. If this solves the problem, great! If not, then

Second,From $SPLUNK_HOME/bin you can run 'splunk _internal call /services/admin/inputstatus/TailingProcessor:FileStatus > output.txt'

look at output.txt for the error_delta.log, if it read the file, it'll tell you how far into the file we read and the size at the time of reading. If it ignored the file it'll say why we ignored it.

If it says we read the file and it isn't showing up, try searching in a very non specific way for something in particular which you'd expected to see but didn't, with a search like this:
'index=* <uniquedata>' over all-time via the search app. It's possible the timestamp is being misinterpreted or the metadata isn't matching for some reason.

jbsplunk
Splunk Employee
Splunk Employee

That means we didn't read the file because there is another file that has the same crc. This indicates the first 256 bytes of the file are the same as another file already read. In this input stanza you can put in this option to force splunk to include the source name as well as the crc:
crcSalt =

0 Karma

royimad
Builder

Look what i got in output.txt



ignored file (crc conflict, needs crcSalt)/s:key
/s:dict
/s:key

What does it mean ? Ignored file ( crc conflict, needs crcSalt )

0 Karma

Ayn
Legend

One thing to do is troubleshoot the input using amrit's excellent script:

http://blogs.splunk.com/2011/01/02/did-i-miss-christmas-2/

royimad
Builder

I have added crcSalt= into props.conf and that's have fixed the problem. Thanks.

0 Karma

ThomasControlw1
Explorer

where do you have added this "crcSalt" ?
could you please give me more details about this case?
thanks in advance

0 Karma

royimad
Builder

what does this mean?

0 Karma

royimad
Builder

Using this script is showing exactly the same:
ignored file (crc conflict, needs crcSalt)

0 Karma
Get Updates on the Splunk Community!

Introducing the 2024 SplunkTrust!

Hello, Splunk Community! We are beyond thrilled to announce our newest group of SplunkTrust members!  The ...

Introducing the 2024 Splunk MVPs!

We are excited to announce the 2024 cohort of the Splunk MVP program. Splunk MVPs are passionate members of ...

Splunk Custom Visualizations App End of Life

The Splunk Custom Visualizations apps End of Life for SimpleXML will reach end of support on Dec 21, 2024, ...