topic Re: Cannot search for value in extracted field in Splunk Search

Cannot search for value in extracted field

jrfrost — Tue, 29 Sep 2020 16:49:02 GMT

I have a field extraction that gets the message number from the raw message string

.{22}\s0-9

The message string is in the format of

2017-11-15T13:32:53,915 4790018 299939553102122275000175000000000022 6834527000103_0_007500002610100_100850055_00045010000010000_1___________________

The field is available and has values of 01, 02, 09, 11, 12, 19, 51, 52, 79, 90, 91 etc. but I cannot search for all values.

If I search for message number 51 I get results
index=main msg_number=51

If I search for message number 52 no results are returned.
index=main msg_number=52

If I use the following search index=main | eval msg_number=msg_number*1 |search msg_number=52, I get results

I have no idea why search for some numbers does not work.

Re: Cannot search for value in extracted field

kamlesh_vaghela — Thu, 16 Nov 2017 17:11:18 GMT

Hi @jrfrost,

can you please try this search?

| index=main
| rex max_match=0 field=_raw ".{22}\s[0-9](?<msg_number>\d{2})" 
| search msg_number=YOUR_NUMBER

You can Set up your transforms.conf and props.conf files to configure multivalue extraction.

In transforms.conf, add the following.

[mv-type]
REGEX = .{22}\s[0-9](?<msg_number>\d{2})
MV_ADD = true

In props.conf for your sourcetype or source, set the following.

REPORT-type = mv-type

Thanks

Re: Cannot search for value in extracted field

rphillips_splk — Tue, 29 Sep 2020 16:49:27 GMT

@jrfrost My thoughts are that the value you are searching for in "msg_number" is not a separate token in the raw event text but part of a token string in the raw message string from which it is extracted. Events are tokenized based on rules from segmenters.conf, and the value (msg_number) is probably not its own token but just part of a token. Try setting the INDEXED_VALUE in fields.conf on the SH to account for the field-extraction settings.

drop this exact configuration onto your search heads to fix the issue:
$SPLUNK_HOME/etc/system/local/fields.conf
[msg_number]
INDEXED_VALUE=*<VALUE>*

restart splunk

The lispy search will then look something like *msg_number*

Its likely that the events being returned when you search msg_number=51 is a false positive and contain some other tokens in the raw event text containing 51.

Re: Cannot search for value in extracted field

jrfrost — Fri, 17 Nov 2017 09:00:46 GMT

Thanks for this, you were right the 51 was a false positive.