How can Splunk index a file without a file extension?

[mookiie2005]

I have several servers with SQL logs that are in the format:

sqlerror
sqlerror.1
sqlerror.2

I have tried all kinds of wildcard variations, but I cannot get Splunk to index these files. I can see from the internal logs that Splunk says it is watching the file, yet it never indexes any data from it. Anyone face a similar issue?

[mookiie2005]

Wow I feel really embarrassed, turns out the logfile path provided had "Microsoft" spelled wrong! Always the little things. Thanks to all that replied with assistance!

[lycollicott]

No worries, dude! We've all had those experiences!
Glad you found it.

[mookiie2005]

I gave everyone who responded 25 points! Thanks again guys!

[lycollicott]

Please post your inputs.conf stanza and short sample of what the sqlerror file looks like.

[sowings]

It sounds like Splunk believes it has already indexed the file (as in, it read it when it was still called just 'sqlerror'). You can query the Splunk fishbucket (the record of what's been indexed) on the local forwarder, or the TailingProcessor's File Status endpoint to see why Splunk won't index it.

The easiest way is splunk _internal call /admin/inputstatus/TailingProcessor:FileStatus and redirect that to a file. You'll find an entry in the XML like <s:key name="<YOUR_FILENAME_HERE>"; following that (indented, so that it's clearly part of the details for that entry) would be <s:key name="type">finished reading</s:key>. Other options are possible, of course, but I suspect that's the reason.

[eking]

I had exactly the same problem, the tailing processor led me to the answer... Splunk was ignoring the ERRORLOG file because I had recursion disabled. Thank you!

[mookiie2005]

I should point out that teh logs in question are SQL server logs if that makes a difference.

[dwaddle]

Point of clarification: When you say "SQL Server Logs" - are we referring to plain text log files about the running of the SQL Server? Or are we talking about Transaction log files?

[Nicolo_Figiani · Accepted solution]

Hi,
Splunk definitely doesn't care about file extension in general. You should be able to collect logs with no extension without troubles (I did it many times exactly with sql related files).

I guess your problem is somewhere else, aka the watch on path is correct but there is something else wrong going on. Take a look at this page of the Splunk wiki for troubleshooting the input monitoring process: http://wiki.splunk.com/Community:Troubleshooting_Monitor_Inputs

Hope it helps,
regards

[jkat54]

Does the Splunk user have permission to read the files? Maybe if you check the windows security and application logs, you will find more details such as an access denied, etc.

[mookiie2005]

It should, it is running under my user and I am able to open the files in question.

[lycollicott]

I would think that if the internal logs indicate that it is being watched then permissions "should" be correct - I really hate the word "should" - but Windows permissions could be very funky.

You need some special privileges to run a Windows service as your own userid (and not really recommended anyway), so perhaps those are not all set up. Can you run the service a Local System (or an MSA) to test for different behaviour?

[jkat54]

I hate to ask again, but did you check the windows event logs? I know it's cliche but they've seriously saved my butt on more than one occasion only after I said... hmphh there wont be anything in THOSE logs about THIS issue.

[mookiie2005]

Nothing in the windows logs.

[somesoni2]

Can you try something like this (assuming files with extension .1/.2 are rollover files and doesn't have to be monitored)

[monitor://D:\Your Path\To Log File\Till TheFolder\Containing The\SQL Logs]
whitelist = sqlerror$
index = yourindex
sourcetype = yoursourcetype