topic Re: Cannot search for field but field is shown in field list in Splunk Search

Cannot search for field but field is shown in field list

afx — Fri, 14 Jun 2019 12:53:38 GMT

I have a totally weird case...
I have field extractions defined in props.conf either individually or all in one extraction, no difference.
The fields show up in the "Interesting Fields" list.

When I then select a field and pick a value, the result is empty.
For example

index=myindex sourcetype=mysrc myfield=A

Shows nothing even though the popup told me there are 100 entries of myfield with contents A.
But if I then flip the search to exclude A I get results...

index=myindex sourcetype=mysrc myfield!=A

This happens for several fields defined for that index/source type but not all and I see no pattern why some work and some do not (position seems irrelvant).
If I use the fieldsummary filter, I find things just fine.
Splunk 7.4.2, 2 indexers, one SH

So why can't I select field contents that Splunk clearly knows about?

I am totally lost.

thx
afx

Re: Cannot search for field but field is shown in field list

jnudell_2 — Fri, 14 Jun 2019 12:57:11 GMT

What results do you see when you table the events:

index=myindex sourcetype=mysrc 

| table index sourcetype myfield

| sort myfield

Do you see values in myfield?

Re: Cannot search for field but field is shown in field list

afx — Fri, 14 Jun 2019 13:14:54 GMT

Yes, they show up there.
I also did this:

 index=myindex sourcetype=mysource 
    | fieldsummary 
    | search field="myfield" 
    | table field count distinct_count values

And the result is just perfectly fine.

Re: Cannot search for field but field is shown in field list

jnudell_2 — Fri, 14 Jun 2019 13:20:00 GMT

And if you do:

index=myindex sourcetype=mysrc 

| table index sourcetype myfield

| search myfield="A"

What do you see?

Re: Cannot search for field but field is shown in field list

FrankVl — Fri, 14 Jun 2019 13:24:42 GMT

Can you share the props/transforms for how that specific field is being extracted?

Re: Cannot search for field but field is shown in field list

afx — Fri, 14 Jun 2019 13:26:05 GMT

"No results found"
is all I get.

Re: Cannot search for field but field is shown in field list

afx — Fri, 14 Jun 2019 13:33:46 GMT

This is the props.conf file:

[sap:sal]
EXTRACT-sal = ^(?<message_id>.{3})(?<date>.{8})(?<time>.{6})(\w\w)(?<process_id>.{5})(?<task>.{5})(?<proctype>.{2})(?<term>.{8})(?<user>.{12})(?<transaction>.{20})(?<app>.{40})(?<client>.{3})(?<message>.{64})(?<src>.{20})

LOOKUP-auto_sap_sm20 = sap_sm20 message_id AS message_id OUTPUTNEW audit_class AS sap_audit_class event_class AS sap_event_class message AS sap_message new_in_release AS sap_new_in_release

transforms.conf:

[sap_sm20]
batch_index_query = 0
case_sensitive_match = 1
filename = SAP_SM20.csv

Re: Cannot search for field but field is shown in field list

FrankVl — Fri, 14 Jun 2019 13:41:25 GMT

And what field(s) do you have issues with? The ones from the lookup perhaps?

Re: Cannot search for field but field is shown in field list

afx — Wed, 30 Sep 2020 00:56:03 GMT

That would be easy 😉
The ones that work are user, app,src, sap_audit_class.
They don't seem to follow a pattern.
Even weirder, sap_audit_class is based on message_id and works, but message_id itself does not...

Re: Cannot search for field but field is shown in field list

FrankVl — Fri, 14 Jun 2019 13:52:21 GMT

And you're not overlooking whitespace in the field values or so?

Re: Cannot search for field but field is shown in field list

jnudell_2 — Fri, 14 Jun 2019 13:55:05 GMT

Then everything is working fine, and you're not searching for the right value.

Can you provide a sample event where it contains the value you're looking for?

Re: Cannot search for field but field is shown in field list

jnudell_2 — Fri, 14 Jun 2019 13:57:11 GMT

Hi @afx ,
Based upon your answers in the comments, you are probably not using the right search term(s). Can you provide a sample of what an event looks like that contains the data you're looking for? Highlight the value that is supposed to be the myfield value you're looking for.

Here's another thing you might try (seems to match your case):
https://www.splunk.com/blog/2011/10/07/cannot-search-based-on-an-extracted-field.html

Re: Cannot search for field but field is shown in field list

afx — Fri, 14 Jun 2019 14:01:09 GMT

Nope.
I select values from the dropdown menus that splunk offers me for the fields.

Re: Cannot search for field but field is shown in field list

FrankVl — Fri, 14 Jun 2019 14:06:04 GMT

Yeah, I thought that's what you meant with how you described your testing in the question post, just wanted to double check.

Well then I'm running out of ideas...

Re: Cannot search for field but field is shown in field list

afx — Fri, 14 Jun 2019 14:09:55 GMT

Why would I not use the right search terms?
Splunk is offering me the terms in the popup menus for the fields.
For example

AUK20190614155300000387400019D110.42.24S_ESA_ANG SAPMSSY1 1001ARFC&&ARFC_DEST_CONFIRM 10.42.242.200

Should be found when I select message_id=AUK from the interesting fields sidebar.
But I get no results
(The copy and paste did collapse spaces in this record. The extract definition above shows where things are. Just Imagine plenty of spaces to fill the fixed size records where you see a single space, this line is always exactly 200 characters wide).

Re: Cannot search for field but field is shown in field list

jnudell_2 — Fri, 14 Jun 2019 14:13:19 GMT

What happens when you try message_id="*AUK*"?

Re: Cannot search for field but field is shown in field list

afx — Fri, 14 Jun 2019 14:16:16 GMT

No results found ;-(

Re: Cannot search for field but field is shown in field list

FrankVl — Fri, 14 Jun 2019 14:24:22 GMT

And when you search for "AUK" or "AUK*" (so without the field name). And if you get results, what part of the event is getting highlighted?

I'm guessing this has something to do with bloom filters failing due to your event being one long string.

Re: Cannot search for field but field is shown in field list

jnudell_2 — Wed, 30 Sep 2020 00:56:14 GMT

What do you get when you try:

| ... [ your search without message_id= ] ...

| stats dc(message_id) as message_id_count values(message_id) as message_id_values

Do you get a numeric value for the count and a list of message_id values?

Re: Cannot search for field but field is shown in field list

afx — Fri, 14 Jun 2019 14:30:34 GMT

It is not the one long string, as I wrote in the first post, this also happens with individual definitions.

Ok, tried just the string and I do not get any results...
Which is weird.

I do get results for AUK* or "AUK*" which then highlights everything until the first space.