Getting Data In

Search Head filter data from Backend

mailtosnsolutio
Explorer

Hello Team,

I am in New Splunk,

I am have Search head where I am applying Some filter like

index=xyz sourcetype=xyz User=*NYZ*

So this User=NYZ filter , i want splunk do it for myself while getting indexed,so basically i want filter index data rather then complete data of users i am filter data to be shown for index XYZ in backend

expected output without Filter should be seeen all NYZ Users

0 Karma
1 Solution

maciep
Champion

are you just a user of splunk or are you the splunk admin? filtering data before indexing is doable and not terribly complicated, but it's not button clicks either. And it's helpful to have an understanding of how Splunk works, which isn't often the case with new users.

I would recommend reading this article: https://wiki.splunk.com/Where_do_I_configure_my_Splunk_settings%3F
And if that makes enough sense, then check out splunk's docs on routing: https://docs.splunk.com/Documentation/Splunk/latest/Forwarding/Routeandfilterdatad

That said...assuming that you do want to filter data before it is indexed, the data is coming from a universal forwarder, going to an indexer, has a sourcetype of "xyz" and contains the literal text "user=NYZ", then try putting these settings on your indexer(s).

props.conf

[xyz]
TRANSFORMS-keep_nyz_only = send_to_null_queue, keep_nyz

transforms.conf

[send_to_null_queue]
REGEX = .
DEST_KEY = queue
FORMAT = nullQueue

[keep_nyz]
REGEX = user=NYZ
DEST_KEY = queue
FORMAT = indexQueue

The concept here is that when data comes to the indexer with a sourcetype of xyz, splunk is configured to transform that data twice. The first thing we do is set all of the data to be destined for the null queue (deleted). Next, we go back and reset any events with that user to be destined for the indexQueue (to be indexed).

View solution in original post

maciep
Champion

are you just a user of splunk or are you the splunk admin? filtering data before indexing is doable and not terribly complicated, but it's not button clicks either. And it's helpful to have an understanding of how Splunk works, which isn't often the case with new users.

I would recommend reading this article: https://wiki.splunk.com/Where_do_I_configure_my_Splunk_settings%3F
And if that makes enough sense, then check out splunk's docs on routing: https://docs.splunk.com/Documentation/Splunk/latest/Forwarding/Routeandfilterdatad

That said...assuming that you do want to filter data before it is indexed, the data is coming from a universal forwarder, going to an indexer, has a sourcetype of "xyz" and contains the literal text "user=NYZ", then try putting these settings on your indexer(s).

props.conf

[xyz]
TRANSFORMS-keep_nyz_only = send_to_null_queue, keep_nyz

transforms.conf

[send_to_null_queue]
REGEX = .
DEST_KEY = queue
FORMAT = nullQueue

[keep_nyz]
REGEX = user=NYZ
DEST_KEY = queue
FORMAT = indexQueue

The concept here is that when data comes to the indexer with a sourcetype of xyz, splunk is configured to transform that data twice. The first thing we do is set all of the data to be destined for the null queue (deleted). Next, we go back and reset any events with that user to be destined for the indexQueue (to be indexed).

jscraig2006
Communicator

so basically you want to only send the data to the index based upon that user?

0 Karma

mailtosnsolutio
Explorer

Yes ,please

0 Karma
Get Updates on the Splunk Community!

Why You Can't Miss .conf25: Unleashing the Power of Agentic AI with Splunk & Cisco

The Defining Technology Movement of Our Lifetime The advent of agentic AI is arguably the defining technology ...

Deep Dive into Federated Analytics: Unlocking the Full Power of Your Security Data

In today’s complex digital landscape, security teams face increasing pressure to protect sprawling data across ...

Your summer travels continue with new course releases

Summer in the Northern hemisphere is in full swing, and is often a time to travel and explore. If your summer ...