topic Re: Why is the following search that contains "field=" not retuning results unless I use a wildcard? in Splunk Search

Why is the following search that contains "field=" not retuning results unless I use a wildcard?

cjmckenna — Tue, 16 Oct 2018 18:23:56 GMT

Running into a strange issue that I, nor my Splunk admins, can figure out. We have a filed extraction called "Service" that holds the name of our SOA services. When I do a search using:
index=blah Service="examplename" , i get no results.

If I do this, I get results:

index=blah Service="*examplename"

We can not figure out why we have to add the asterisk for it to work. Even if I do index=blah and select a value for Service from the interesting fields, and let Splunk pop that in the search, I get no results. As soon as I add the *, BAM there are the results.

I hope someone can help as we are stumped. This happens on several fields while other extracted fields work fine.

Re: Why is the following search that contains "field=" not retuning results unless I use a wildcard?

DalJeanis — Tue, 16 Oct 2018 20:59:09 GMT

Please post the extraction code.

Re: Why is the following search that contains "field=" not retuning results unless I use a wildcard?

cjmckenna — Wed, 17 Oct 2018 13:31:02 GMT

[layer7:sisyslog]
EXTRACT-Service = ^(?:[^~\n]*~){2}(?P\w+)

Re: Why is the following search that contains "field=" not retuning results unless I use a wildcard?

skoelpin — Wed, 17 Oct 2018 13:38:14 GMT

Post some sample data too. This will make it easy to recreate

Re: Why is the following search that contains "field=" not retuning results unless I use a wildcard?

cjmckenna — Wed, 17 Oct 2018 13:48:14 GMT

Oct 17 09:40:48 info <14>Oct 17 09:40:48 hostname123 SSG[2,651]: [L7Metrics-PRD] INFO com.l7tech.log.custom.si.metrics : -4: ~obfuscateddomainname~obfuscatedservicename~obfuscatedoperationname~http://obfuscatedWSDLURL/1/~obfuscatedWSSid~63~73~

obfuscatedservicename is where the service name would be and what the extraction is pulling out.

Re: Why is the following search that contains "field=" not retuning results unless I use a wildcard?

skoelpin — Wed, 17 Oct 2018 13:55:19 GMT

That regex has a lot of steps and could be refactored..

Try this in your search. Field name will be called servicename. If this works then you could save this as a search time extraction

| rex ^(?:[^~\n]*~){2}(?<servicename>\w+)

Re: Why is the following search that contains "field=" not retuning results unless I use a wildcard?

cjmckenna — Wed, 17 Oct 2018 14:06:08 GMT

The extraction is working fine. Please see my original post. In "Interesting Fields" Service is there and has a list of values. The issue is whan we do Service="something" we get no results. Service="*something" does.

We will look at cleaning up the regex but for now its actually extracting data

Re: Why is the following search that contains "field=" not retuning results unless I use a wildcard?

skoelpin — Wed, 17 Oct 2018 14:19:35 GMT

Yes, I'm thinking it has something to do with your extraction. Keep it simple

Use the extraction I provided you and save it as a search time extraction so it appears in interesting fields without having to explicitly call the rex command. Also, please accept/upvote the answer since it helped solved your problem

Re: Why is the following search that contains "field=" not retuning results unless I use a wildcard?

cjmckenna — Wed, 17 Oct 2018 14:22:02 GMT

it has not helped my issue. Cant upvote something that is not a solution

Re: Why is the following search that contains "field=" not retuning results unless I use a wildcard?

skoelpin — Wed, 17 Oct 2018 14:25:16 GMT

You just said this worked as expected... Why waste time fixing a poorly written regular expression when I just provided you with a working solution? Your logic doesn't make sense..

Re: Why is the following search that contains "field=" not retuning results unless I use a wildcard?

cjmckenna — Wed, 17 Oct 2018 15:05:54 GMT

No... the EXISTING extraction is working fine. We see the service names listed in "Service" the "Interesting Fields" with the existing extraction. The problem is when that field is used in a search we always have to add an *

Re: Why is the following search that contains "field=" not retuning results unless I use a wildcard?

skoelpin — Wed, 17 Oct 2018 15:09:03 GMT

If your existing solution is working fine, then why ask for help on Answers? Your regex is bad and you can prove it by using an inline regex like I provided above to test against..

Re: Why is the following search that contains "field=" not retuning results unless I use a wildcard?

cjmckenna — Wed, 17 Oct 2018 15:12:56 GMT

So help me understand exactly what is wrong in our existing regex when it finds the data and extracts it and it shows up in interesting fields. Please explain to me why using that data from the existing extraction will not allow us to use index=blah Service="servicename" in a search. You are saying do this, do that but not providing any insight as to why.

Maybe its me and I am not explaining that actual issue clear enough

Re: Why is the following search that contains "field=" not retuning results unless I use a wildcard?

skoelpin — Wed, 17 Oct 2018 15:19:13 GMT

I gave you a working solution to test against and told you your regex is bad which is causing the issue. I'm not going to troubleshoot your buggy regex and give you a play by play of why its bad. I gave you the solution and an easy way to test against it. You easily have enough to troubleshoot the problem on your own

Re: Why is the following search that contains "field=" not retuning results unless I use a wildcard?

brannonrad — Wed, 15 Jan 2020 19:49:56 GMT

I know what the issue is. It is explained here:

https://www.splunk.com/en_us/blog/tips-and-tricks/cannot-search-based-on-an-extracted-field.html

Here is the way I understand it. Splunk automatically indexes word tokens, which are detected using "standard" delimiters like spaces, tabs, commas, etc.

Here is what most people don't know or understand: even though you specify a search like so (in your example):

index=blah Service="examplename"

What Splunk actually does initially is a search like this:

index=blah "examplename"

Under normal circumstances, this works fine - and it normally returns a "super set" of the results you are looking for. Then, Splunk refines the results further with a search like this:

| search Service="examplename"

Which should reduce the data set down to the specific results you are looking for.

What went wrong here? Well, your data doesn't have the standard word boundary delimiters Splunk expects. In your case, there are tilde characters around the word you want to search on, so the "examplename" string doesn't match, but the "*examplename" does.

How can you fix this? I think you have two choices:

1) Don't use tildes as delimiters! Actually, don't use any non-standard delimiters. But we don't always have control over that, so...
2) If you know you have this issue with your data, create a fields.conf file in the same app that contains your props.conf, and set it like this (NOTE: replace "fieldname" below with the actual fieldname!):

[fieldname]
INDEXED_VALUE = false

But don't arbitrarily do this for all of your fields - as it can make your searches less efficient.

Re: Why is the following search that contains "field=" not retuning results unless I use a wildcard?

payl_chdhry — Thu, 12 Mar 2020 16:16:44 GMT

We too are facing this issue and the only solution we have been able to come up change delimiter before upload.
We are using ~ as delimiter in csv file. Using FIELD_NAMES parameter to provide header field names. It is extracting fine but we are not able to search data using extracted field unless we use "*" in search.
Were you able to resolve it.

Adding INDEXED_VALUE = false did not work.