- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I have set up an inputs.conf monitor stanza to index a few .dat.gz files but do not see them getting indexed.
There is no blacklist or null queue routing going on.
Also, there are no error messages in splunkd.log.
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
There are two issues going on here.
- Why are the files not being moitored and how to actually monitor them
- Why are there no errors in splunkd.log
First, Splunk will not index binary files. Even if you have a file.dat.gz that is not binary, and you specify NO_BINARY_CHECK, Splunk will still fail to index the file.
This is because there is stanza in /etc/system/default/props.conf that tells splunk not to.
[source::....(0t|a|ali|asa|au|bmp|cg|cgi|class|d|dat|deb|del|dot|dvi|dylib|elc|eps|exe|ftn|gif|hlp|hqx|hs|icns|ico|inc|iso|jame|jin|jpeg|jpg|kml|la|lhs|lib|lo|lock|mcp|mid|mp3|mpg|msf|nib|o|obj|odt|ogg|ook|opt|os|pal|pbm|pdf|pem|pgm|plo|png|po|pod|pp|ppd|ppm|ppt|prc|ps|psd|psym|pyc|pyd|rast|rb|rde|rdf|rdr|rgb|ro|rpm|rsrc|so|ss|stg|strings|tdt|tif|tiff|tk|uue|vhd|xbm|xlb|xls|xlw)]
sourcetype = known_binary
Note that this is not just happening for .dat files but happens for all the above extensions..
In order to actually have the files indexed then we need to create (or edit) a props.conf within /etc/system/local/ with the following stanza:
[source::....(dat)]
sourcetype = mysourcetype
priority = 20
This will overwrite the default/props.conf and will tell splunk to use a different sourcetype for the .dat.gz files, and hence allow them to actually be indexed.
The reason why there was no message in splunkd.log is because of a bug, which has been filed and our friendly devs will soon be working on it.
Hope this helps someone out there..
.gz
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
There are two issues going on here.
- Why are the files not being moitored and how to actually monitor them
- Why are there no errors in splunkd.log
First, Splunk will not index binary files. Even if you have a file.dat.gz that is not binary, and you specify NO_BINARY_CHECK, Splunk will still fail to index the file.
This is because there is stanza in /etc/system/default/props.conf that tells splunk not to.
[source::....(0t|a|ali|asa|au|bmp|cg|cgi|class|d|dat|deb|del|dot|dvi|dylib|elc|eps|exe|ftn|gif|hlp|hqx|hs|icns|ico|inc|iso|jame|jin|jpeg|jpg|kml|la|lhs|lib|lo|lock|mcp|mid|mp3|mpg|msf|nib|o|obj|odt|ogg|ook|opt|os|pal|pbm|pdf|pem|pgm|plo|png|po|pod|pp|ppd|ppm|ppt|prc|ps|psd|psym|pyc|pyd|rast|rb|rde|rdf|rdr|rgb|ro|rpm|rsrc|so|ss|stg|strings|tdt|tif|tiff|tk|uue|vhd|xbm|xlb|xls|xlw)]
sourcetype = known_binary
Note that this is not just happening for .dat files but happens for all the above extensions..
In order to actually have the files indexed then we need to create (or edit) a props.conf within /etc/system/local/ with the following stanza:
[source::....(dat)]
sourcetype = mysourcetype
priority = 20
This will overwrite the default/props.conf and will tell splunk to use a different sourcetype for the .dat.gz files, and hence allow them to actually be indexed.
The reason why there was no message in splunkd.log is because of a bug, which has been filed and our friendly devs will soon be working on it.
Hope this helps someone out there..
.gz
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi ..i am facing the similar issue for indexing dat files..i have removed the dat extension in system/default/props.conf file ..but then its not indexing...pls help