Splunk Search

Why is one field not been extracted as expected?

glpadilla_sol
Path Finder

Hello everyone,

I have an issue with one field let say foo

These are the scenarios:

1. If I run a search just with the index that contains the logs I can see the field foo at the fields bar perfectly and also I can see the values.

2. If I select the field and is added to the search for example index=bar foo="hello" the results are ZERO even though I select that value from the previous search (where I saw the field and the values at the field bar).

3. If I add the sourcetype at the search example index=bar sourcetype=net foo="hello" I can see results but not the expected results, usually I get less than the real number and the number of results are random in the same interval of time.

Configuration:

I am using the automatic key-value field extraction KV_MODE=json to try to extract the fields of a source. The sources is sending the logs in JSON format.  

#props.conf - SH configuration and indexers

[net]
CHARSET=UTF-8
KV_MODE=json
TRUNCATE = 99999
LINE_BREAKER=([\r\n]+)
NO_BINARY_CHECK=true
SHOULD_LINEMERGE=true

#inputs.conf - forwarder

[monitor:///pat/*.json]
sourcetype= net
index = bar
disabled = false
crcSalt = <SOURCE>
ignoreOlderThan = 1d

 

Version 8.2.2

Cluster enviroment.

 

Notes:

Also I tried with another sourcetype and data not using the KV_MODE instead I used the EXTRACT-foo and I had the same results, the field doesn't show results when is added to the search.

The rest all the fields are not having this issue, they work perfectly.

Thank you for the help.

Labels (2)
0 Karma
1 Solution

glpadilla_sol
Path Finder

I found the root cause, updating in case someone else is facing the same issue:

https://www.splunk.com/en_us/blog/tips-and-tricks/cannot-search-based-on-an-extracted-field.html?_gl...

The solution

$SPLUNK_HOME/etc/system/local/fields.conf
[MyField]
INDEXED_VALUE = false

 

View solution in original post

glpadilla_sol
Path Finder

I found the root cause, updating in case someone else is facing the same issue:

https://www.splunk.com/en_us/blog/tips-and-tricks/cannot-search-based-on-an-extracted-field.html?_gl...

The solution

$SPLUNK_HOME/etc/system/local/fields.conf
[MyField]
INDEXED_VALUE = false

 

cpetterborg
SplunkTrust
SplunkTrust

It kind of looks like you might have created an indexed field extraction using EXTRACT-foo. If that is the case, try the following search:

index=bar foo::"*hello*"

 

0 Karma

VatsalJagani
Champion

@glpadilla_sol - Try searching like this:

index=bar foo="*hello*" | where foo="hello"

(added wildcard(*) in the search command.)

 

I hope this helps!!!

 

0 Karma

glpadilla_sol
Path Finder

Thank you again.

I tried with that search and no results, if I add the sourcetype, 

index=bar sourcetype="test" foo="*hello*" | where foo="hello"

 

I can see results but not all the results, it's a partial view, that also happens when I run this 

index=bar sourcetype ="test" foo="hello" 

Both scenarios I got the same results.

 

Also I notice at the search.log that the search is running against some extractions that are not part of that sourcetype. I mean we have the same field been extracted into another sourcetype, but at indexing time using props.conf and transforms.conf.

Can this for some reason be related? 

Event though are different sourcetypes?

 

Thank you for your inputs.

0 Karma

glpadilla_sol
Path Finder

Hello @VatsalJagani thank you for the answer, but the issue is not the search. I mean no matter if I use * or not we are not getting results.

The issue is that I can see the field and the values at the field bar, but no when I add it at the search.

Kind Regards.

0 Karma

isoutamo
SplunkTrust
SplunkTrust
0 Karma

VatsalJagani
Champion

@glpadilla_sol - It's not because of the search you need to add *. 

But there is some concept about minor and major segmentation that sometimes will not allow you to get results even when you search the extracted value.

0 Karma

isoutamo
SplunkTrust
SplunkTrust
Can you post a sample from your json file? Preferably from source as raw events inside <> -block.
0 Karma

glpadilla_sol
Path Finder

Hello,

Unluckily I cannot share that content because is confidential. 

But is a normal JSON like this

{"backendhost": "2.11.27.94", "backendport": 1001, "console_device": "unknown", "console_name": "sdj934", "domain": "domainname", "frontendport": 8347, "frontendprotocol": "tcp"}

Regards.

0 Karma
Get Updates on the Splunk Community!

Improve Your Security Posture

Watch NowImprove Your Security PostureCustomers are at the center of everything we do at Splunk and security ...

Maximize the Value from Microsoft Defender with Splunk

 Watch NowJoin Splunk and Sens Consulting for this Security Edition Tech TalkWho should attend:  Security ...

This Week's Community Digest - Splunk Community Happenings [6.27.22]

Get the latest news and updates from the Splunk Community here! News From Splunk Answers ✍️ Splunk Answers is ...