topic Re: Search Head filter data from Backend in Getting Data In

Search Head filter data from Backend

mailtosnsolutio — Fri, 17 Jan 2020 18:37:14 GMT

Hello Team,

I am in New Splunk,

I am have Search head where I am applying Some filter like

index=xyz sourcetype=xyz User=*NYZ*

So this User=NYZ filter , i want splunk do it for myself while getting indexed,so basically i want filter index data rather then complete data of users i am filter data to be shown for index XYZ in backend

expected output without Filter should be seeen all NYZ Users

Re: Search Head filter data from Backend

jscraig2006 — Fri, 17 Jan 2020 19:54:08 GMT

so basically you want to only send the data to the index based upon that user?

Re: Search Head filter data from Backend

mailtosnsolutio — Sat, 18 Jan 2020 07:05:11 GMT

Yes ,please

Re: Search Head filter data from Backend

maciep — Sat, 18 Jan 2020 14:05:08 GMT

are you just a user of splunk or are you the splunk admin? filtering data before indexing is doable and not terribly complicated, but it's not button clicks either. And it's helpful to have an understanding of how Splunk works, which isn't often the case with new users.

I would recommend reading this article: https://wiki.splunk.com/Where_do_I_configure_my_Splunk_settings%3F
And if that makes enough sense, then check out splunk's docs on routing: https://docs.splunk.com/Documentation/Splunk/latest/Forwarding/Routeandfilterdatad

That said...assuming that you do want to filter data before it is indexed, the data is coming from a universal forwarder, going to an indexer, has a sourcetype of "xyz" and contains the literal text "user=NYZ", then try putting these settings on your indexer(s).

props.conf

[xyz]
TRANSFORMS-keep_nyz_only = send_to_null_queue, keep_nyz

transforms.conf

[send_to_null_queue]
REGEX = .
DEST_KEY = queue
FORMAT = nullQueue

[keep_nyz]
REGEX = user=NYZ
DEST_KEY = queue
FORMAT = indexQueue

The concept here is that when data comes to the indexer with a sourcetype of xyz, splunk is configured to transform that data twice. The first thing we do is set all of the data to be destined for the null queue (deleted). Next, we go back and reset any events with that user to be destined for the indexQueue (to be indexed).