Splunk Search

search time rex works but simple field extraction searching does not


Basically, when I try to search for mf4 values on their own, index="sean-testing" mf4=w, the data found is zero or blank but if I use the full search with rex as seen below then click the mf4 value from the table view, the full event data will come up. I use regex101.com to drop the rex into the expression line and the data sample, one at a time, into the test sample box to test things out.
*Q: I'm thinking that I need to place this field extraction in a promps as a report or extract on the indexers? *
Any advice, examples or links to a better understanding would be great! Thank you. -Sean

full Search with rex

index="sean-testing" sourcetype="mfsource1" 
| rex "\s{2}(?P<mf1>\d+)\s(?P<mf2>.{8})\s(?P<mf3>\S{10}){1}(?P<mf4>\S+)\s(?P<mf5>.+)[0-2][0-9]:[0-9][0-9]:[0-9][0-9]$"
| search mf4=W OR mf4=I 
| table _time mf1 mf2 mf3 mf4 mf5

Event Data Sample 1

200401 07595444      17476 CARDS5   EXSFJM1083I EJM1: Using LWASP provider module "LWASP32.DLL", from API version 2009 07:59:54

Event Data Sample 2

200331 18250270       1764 CARDS1   CASHO00200I AB2P XA interface loaded. Name(AB2 for WINDOWS), Registration Mode(Dynamic) 18:25:02

Event Data Sample 3

200331 18250131       6508 CARDS3   CASZS50110W Failed to open port 21661 for TCPIPSERVICE ZTGIPP1  18:25:01

Basic stuff - break at everyline, no regex added

0 Karma


So it turns out this old blog post fixed my issue. https://www.splunk.com/en_us/blog/tips-and-tricks/cannot-search-based-on-an-extracted-field.html entitled "Cannot search based on an extracted field" There seems to be an issue when I try to create the extraction for MF3 and MF4 ...

I created a fields.conf file within the app folder on the search head ... (ie ...\apps\MF-Servers\local\fields.conf ... and added the stanza for the fields giving issues and it worked

========== The fields.conf file: ========



I also found some specific field lengths so I expanded on my rex - "(^(?P<mf01>\d+\s\d+).{6}(?P<mf02>.{5}).(?P<mf02a>.{8}).(?P<mf3>.{9})(?<mf4>.).(?<mf5>.+)(?P<mf6>[0-2][0-9]:[0-9][0-9]:[0-9][0-9])$)"

0 Karma

Ultra Champion


EXTRACT-mf = ^\S+\s\S+\s+(?P<mf1>\d+)\s(?P<mf2>\w+)\s+(?P<mf3>\S{10})(?P<mf4>\S)\s(?P<mf5>.+)\s+\S+$

It is better to extract with REGEX from the beginning.


Thanks for the input, i'll try the props.conf suggestion
| search mf_status=W OR mf_status=I ... this should read | search mf4=W OR mf4=I
I've updated the original post.

0 Karma
Get Updates on the Splunk Community!

Streamline Data Ingestion With Deployment Server Essentials

REGISTER NOW!Every day the list of sources Admins are responsible for gets bigger and bigger, often making the ...

Remediate Threats Faster and Simplify Investigations With Splunk Enterprise Security ...

REGISTER NOW!Join us for a Tech Talk around our latest release of Splunk Enterprise Security 7.2! We’ll walk ...

Introduction to Splunk AI

WATCH NOWHow are you using AI in Splunk? Whether you see AI as a threat or opportunity, AI is here to stay. ...