topic Re: search time rex works but simple field extraction searching does not in Splunk Search

search time rex works but simple field extraction searching does not

rewritex — Tue, 07 Apr 2020 22:34:41 GMT

Basically, when I try to search for mf4 values on their own, index="sean-testing" mf4=w, the data found is zero or blank but if I use the full search with rex as seen below then click the mf4 value from the table view, the full event data will come up. I use regex101.com to drop the rex into the expression line and the data sample, one at a time, into the test sample box to test things out.
*Q: I'm thinking that I need to place this field extraction in a promps as a report or extract on the indexers? *
Any advice, examples or links to a better understanding would be great! Thank you. -Sean

full Search with rex

index="sean-testing" sourcetype="mfsource1" 
| rex "\s{2}(?P<mf1>\d+)\s(?P<mf2>.{8})\s(?P<mf3>\S{10}){1}(?P<mf4>\S+)\s(?P<mf5>.+)[0-2][0-9]:[0-9][0-9]:[0-9][0-9]$"
| search mf4=W OR mf4=I 
| table _time mf1 mf2 mf3 mf4 mf5

Event Data Sample 1

200401 07595444      17476 CARDS5   EXSFJM1083I EJM1: Using LWASP provider module "LWASP32.DLL", from API version 2009 07:59:54

Event Data Sample 2

200331 18250270       1764 CARDS1   CASHO00200I AB2P XA interface loaded. Name(AB2 for WINDOWS), Registration Mode(Dynamic) 18:25:02

Event Data Sample 3

200331 18250131       6508 CARDS3   CASZS50110W Failed to open port 21661 for TCPIPSERVICE ZTGIPP1  18:25:01

sourcetype
Basic stuff - break at everyline, no regex added

Re: search time rex works but simple field extraction searching does not

to4kawa — Wed, 08 Apr 2020 01:25:05 GMT

props.conf

[mfsource1]
EXTRACT-mf = ^\S+\s\S+\s+(?P<mf1>\d+)\s(?P<mf2>\w+)\s+(?P<mf3>\S{10})(?P<mf4>\S)\s(?P<mf5>.+)\s+\S+$

It is better to extract with REGEX from the beginning.

Re: search time rex works but simple field extraction searching does not

rewritex — Wed, 30 Sep 2020 04:53:52 GMT

Thanks for the input, i'll try the props.conf suggestion
| search mf_status=W OR mf_status=I ... this should read | search mf4=W OR mf4=I
I've updated the original post.

Re: search time rex works but simple field extraction searching does not

rewritex — Thu, 09 Apr 2020 00:16:12 GMT

So it turns out this old blog post fixed my issue. https://www.splunk.com/en_us/blog/tips-and-tricks/cannot-search-based-on-an-extracted-field.html entitled "Cannot search based on an extracted field" There seems to be an issue when I try to create the extraction for MF3 and MF4 ...

I created a fields.conf file within the app folder on the search head ... (ie ...\apps\MF-Servers\local\fields.conf ... and added the stanza for the fields giving issues and it worked

========== The fields.conf file: ========
[mf3]
INDEXED_VALUE = False

[mf4]

INDEXED_VALUE = False

I also found some specific field lengths so I expanded on my rex - "(^(?P<mf01>\d+\s\d+).{6}(?P<mf02>.{5}).(?P<mf02a>.{8}).(?P<mf3>.{9})(?<mf4>.).(?<mf5>.+)(?P<mf6>[0-2][0-9]:[0-9][0-9]:[0-9][0-9])$)"