Can you use regex in file_path

[nhicks]

Is it possible to use regex in the file_path setting for the File/Directory Information Input app.

Here is what I am trying to get to

• E:\Folder\Folder2\20160808\InvalidFile\*.cdi_Error1
• E:\Folder\Folder2\20160809\InvalidFile\*.cdi_Error1
• E:\Folder\Folder2\20160810\InvalidFile\*.cdi_Error1 etc.

I have tried

• file_path = E:\Folder\Folder2\*\InvalidFiles\*.cdi_Error1
• file_path = E:\Folder\Folder2\...\InvalidFiles\*.cdi_Error1

I have also tried several different regex options for *.cdi_Error1. To many to list.

When I try the above options I am receiving this message in the file_meta_data_modular_input.log

• 2016-08-26 10:34:45,864 WARNING Unable to access path="E:\Folder\Folder2\*\InvalidFiles\*.cdi_Error1", reason="[Error 123] The filename, directory name, or volume label syntax is incorrect: 'E:\Folder\Folder2\*\InvalidFiles\*.cdi_Error1'"
• 2016-08-26 10:34:45,864 INFO Completed retrieval of file data, count=0, path=E:\Folder\Folder2\*\InvalidFiles\*.cdi_Error1

Not sure why the 2nd message shows it was complete but it definitely did not pull in the information.

I also tried using whitelist

• file_path = E:\Folder\Folder2
• recurse = 1
• whitelist = *.cdi_Error1

But then I get this message

• 2016-08-26 12:54:28,592 ERROR The input stanza 'file_meta_data://APPNAME' is invalid: The parameter 'whitelist' is not a valid argument

I know that I can set the file_path setting to E:\Folder\Folder2 and set recurse = 1 but this then pulls in some 50000 files and I only need the .cdi_Error1 files.

I also know that if I pull in the 50000 files I can just use logic in the search parameters to filter out only the .cdi_Error1 files but this server is already heavily used and I do not want to put more stress on it by grabbing metadata for 50000 files. Plus its just a lot of data that I do not need to index.

I did try restarting splunk on both the indexer, search head and forwarder many times but it did not help.

Any help is appreciated. Thank you

[LukeMurphey · Accepted solution]

Regular expressions and wild-cards are not currently supported. That is a good idea though. I created an enhancement request: http://lukemurphey.net/issues/1453 for it.

[nhicks]

Thank you Luke for the reply and the enhancement request. This will be very helpful for us if it is implemented.

[nhicks]

It was a typo. Sorry, I was trying to make sure I typed it correctly and missed that. The file name and the error in the log file are the exact same.

[inventsekar]

ohk.. that File/Directory Information Input was built by Luke Murphey.
https://splunkbase.splunk.com/app/2776/

as per the above reply from Luke Murphey, Regular expressions and wild-cards are not currently supported.

[inventsekar]

The actual filename says ".....\InvalidFile\" and the error msg says ..\InvalidFiles\
was it a typo?!?!