Splunk Search

Need some clarification on search-time _meta field extraction.

AndreAtNN
New Member

I have got a question about using _meta fields in the /opt/splunkforwarder/etc/system/local/inputs.conf
of a Splunk Universal Forwarder (deployed on an AWS EC2 Instance)

In our inputs.conf of the Splunk Universal Forwarder in developement we use one _meta field, vendor, e.g.

[default] 
host = $decideOnStartup
_meta =  vendor::devops

In our organization the Indexers and Search Heads are managed by a dedicated tooling team. I did NOT requested the tooling team to update The fields.conf on the Search Head with e.g. the following statements

[vendor]
INDEXED=true;

If I execute the following search in the Search & Reporting Splunk App

index=os host="ip-10-106-152-169.aws.misin.zbi" 

I see in Selected Fields the following fields, host, source, sourcetype and vendor. The default fields host, source and sourcetype are index time extracted fields, but I don’t understand how the vendor field is extracted by Splunk because the fields.conf on the Search Head was not updated yet.

Can anybody explain why the vendor _meta field is automatically extracted when I execute the search index=os host="ip-10-106-152-169.aws.misin.zbi", without updating the fields.conf in the Search Head.

I also noticed that the following search

index=os host="ip-10-106-152-169.aws.misin.zbi” vendor=devops 

gives no result, but if I change the search to this

index=os host="ip-10-106-152-169.aws.misin.zbi” vendor::devops 

I get the same result set as the search index=os host="ip-10-106-152-169.aws.misin.zbi"

0 Karma
1 Solution

skalliger
SplunkTrust
SplunkTrust

Hi,

I think you're confusing things here. What you did by adding

_meta = a::b

is an override of the fields.conf setting. This tags all your events and make them searchable by defining "a::b" in your search.
Your search with vendor=devops however will only return results once you have set the "INDEXED = true" setting on your Search Head.

Ah, I knew it was here: https://answers.splunk.com/answers/389567/why-is-a-search-for-fields-added-with-meta-in-inpu.html
Take a look at the end of the post. That should explain it as well. 🙂

Skalli

View solution in original post

AndreAtNN
New Member

Hi Skali

Now it makes a little bit more sense to me.

When you mentioned that the _meta field definition in the /opt/splunkforwarder/etc/system/local/inputs.conf overrides the fields.conf definition on the Search Head I now understand why the _meta field vendor can be searched on.

Just out of curiosity, if I change the fields.conf on the Search Head with the following statements

[vendor]
INDEXED=false;   
INDEXED_VALUE=true;   

Can I perform the following search?

index=os host="ip-10-106-152-169.aws.misin.zbi” vendor=devops

According to what I read for INDEXED_VALUE at http://docs.splunk.com/Documentation/Splunk/latest/Admin/Fieldsconf it should work.
"Setting this (the INDEXED_VALUE) to true expands any search for key=value into a search of value AND key=value (since value is indexed)"

Thanks

André

0 Karma

skalliger
SplunkTrust
SplunkTrust

Should work. 🙂

0 Karma

AndreAtNN
New Member

Hi Skalli

Great, I will instruct our tooling team to change the fields.conf on the Search Head.

Thanks
André

0 Karma

skalliger
SplunkTrust
SplunkTrust

Hi,

I think you're confusing things here. What you did by adding

_meta = a::b

is an override of the fields.conf setting. This tags all your events and make them searchable by defining "a::b" in your search.
Your search with vendor=devops however will only return results once you have set the "INDEXED = true" setting on your Search Head.

Ah, I knew it was here: https://answers.splunk.com/answers/389567/why-is-a-search-for-fields-added-with-meta-in-inpu.html
Take a look at the end of the post. That should explain it as well. 🙂

Skalli

Get Updates on the Splunk Community!

Using Machine Learning for Hunting Security Threats

WATCH NOW Seeing the exponential hike in global cyber threat spectrum, organizations are now striving more for ...

Observability Newsletter Highlights | March 2023

 March 2023 | Check out the latest and greatestSplunk APM's New Tag Filter ExperienceSplunk APM has updated ...

Security Newsletter Updates | March 2023

 March 2023 | Check out the latest and greatestUnify Your Security Operations with Splunk Mission Control The ...