Splunk Search

How to extract a file name up to a specific character

sheloaha
Path Finder

I have a list of files similar to this list:

FileObjMgr_01235_567.log
EIM_0080123_45.log
EIM_01031234_56.log
EIM_01272345_67.log
FINSObjMgr_56789_1234.log
File3Svc.log
SCFile_123456_1345.log
SRFile_5794_5879.log
ServerMgr_54525852_452.log
SvrTaskPersist_857494_58674.log
WfProcBatch_748490_5857.log
WfProcMgr_2971_9928.log
XMLPReporter_8751_35485.log

I am trying to extract the first part of the file name up to the "" character. Some files do not have an "" so I need the name up to .log. So far I've only been able to successfully pull names using this:

index= sourcetype=:* | rex field=source "./(?[A-Z]\w\w\w\w\w\w\w)..log" | inputlookup append=T filetypes_prod.csv| stats count by filetype | eval Status=if(count<2, "Missing", "OK") | sort filetype| outputlookup filetypes_prod.csv

However I am getting too many characters as below. How to I extract just to the "_" and, if that is not present to .log?

EAIObjMg
EIM_0080
EIM_0103
EIM_0127
EIM_0151
EIM_0171
EIM_0191
EIM_0223
FINSObjM
MASL3Svc
SCBroker
SRBroker
ServerMg
SvrTaskP
WfProcBa
WfProcMg
XMLPRepo

0 Karma
1 Solution

elliotproebstel
Champion

I believe you need this regex:

| rex field=source "^(?<mything>[^_\.]*)(_[^\.]*)?\.log"

The one @cusello posted above is close but will not extract the lines that do not contain underscores. He's also right, though, that you can't have an inputlookup in the middle of your search. Perhaps you meant to append the contents of the lookup file to the existing search results, in which case you'd want ...| append [ | inputlookup append=T filetypes_prod.csv ] | stats count by filetype...

View solution in original post

0 Karma

elliotproebstel
Champion

I believe you need this regex:

| rex field=source "^(?<mything>[^_\.]*)(_[^\.]*)?\.log"

The one @cusello posted above is close but will not extract the lines that do not contain underscores. He's also right, though, that you can't have an inputlookup in the middle of your search. Perhaps you meant to append the contents of the lookup file to the existing search results, in which case you'd want ...| append [ | inputlookup append=T filetypes_prod.csv ] | stats count by filetype...

View solution in original post

0 Karma

sheloaha
Path Finder

This worked great! And thanks everyone for correcting my inputlookup error.

0 Karma

kunalmao
Communicator

you can try | rex field=source "(?\w*)_ "

0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi sheloaha,
If I correctly understand, do you like to extract only chars before the first "_" if present, correct?
if this is your need use this regex

| rex field=source "^(?<myfield>[^_]*)"

Viewing you search I see an error: you cannot use | inputlookup in the middle of a search, only at starting point.

Bye.
Giuseppe

0 Karma

gcusello
SplunkTrust
SplunkTrust

Correct suggestion:

| rex field=source "^(?<myfield>[^_\.]*)"

Bye.
Giuseppe

0 Karma

sheloaha
Path Finder

Sorry. That should read "up to the "_" character."

0 Karma
Register for .conf21 Now! Go Vegas or Go Virtual!

How will you .conf21? You decide! Go in-person in Las Vegas, 10/18-10/21, or go online with .conf21 Virtual, 10/19-10/20.