Getting Data In

Search Head filter data from Backend

mailtosnsolutio
Explorer

Hello Team,

I am in New Splunk,

I am have Search head where I am applying Some filter like

index=xyz sourcetype=xyz User=*NYZ*

So this User=NYZ filter , i want splunk do it for myself while getting indexed,so basically i want filter index data rather then complete data of users i am filter data to be shown for index XYZ in backend

expected output without Filter should be seeen all NYZ Users

0 Karma
1 Solution

maciep
Champion

are you just a user of splunk or are you the splunk admin? filtering data before indexing is doable and not terribly complicated, but it's not button clicks either. And it's helpful to have an understanding of how Splunk works, which isn't often the case with new users.

I would recommend reading this article: https://wiki.splunk.com/Where_do_I_configure_my_Splunk_settings%3F
And if that makes enough sense, then check out splunk's docs on routing: https://docs.splunk.com/Documentation/Splunk/latest/Forwarding/Routeandfilterdatad

That said...assuming that you do want to filter data before it is indexed, the data is coming from a universal forwarder, going to an indexer, has a sourcetype of "xyz" and contains the literal text "user=NYZ", then try putting these settings on your indexer(s).

props.conf

[xyz]
TRANSFORMS-keep_nyz_only = send_to_null_queue, keep_nyz

transforms.conf

[send_to_null_queue]
REGEX = .
DEST_KEY = queue
FORMAT = nullQueue

[keep_nyz]
REGEX = user=NYZ
DEST_KEY = queue
FORMAT = indexQueue

The concept here is that when data comes to the indexer with a sourcetype of xyz, splunk is configured to transform that data twice. The first thing we do is set all of the data to be destined for the null queue (deleted). Next, we go back and reset any events with that user to be destined for the indexQueue (to be indexed).

View solution in original post

maciep
Champion

are you just a user of splunk or are you the splunk admin? filtering data before indexing is doable and not terribly complicated, but it's not button clicks either. And it's helpful to have an understanding of how Splunk works, which isn't often the case with new users.

I would recommend reading this article: https://wiki.splunk.com/Where_do_I_configure_my_Splunk_settings%3F
And if that makes enough sense, then check out splunk's docs on routing: https://docs.splunk.com/Documentation/Splunk/latest/Forwarding/Routeandfilterdatad

That said...assuming that you do want to filter data before it is indexed, the data is coming from a universal forwarder, going to an indexer, has a sourcetype of "xyz" and contains the literal text "user=NYZ", then try putting these settings on your indexer(s).

props.conf

[xyz]
TRANSFORMS-keep_nyz_only = send_to_null_queue, keep_nyz

transforms.conf

[send_to_null_queue]
REGEX = .
DEST_KEY = queue
FORMAT = nullQueue

[keep_nyz]
REGEX = user=NYZ
DEST_KEY = queue
FORMAT = indexQueue

The concept here is that when data comes to the indexer with a sourcetype of xyz, splunk is configured to transform that data twice. The first thing we do is set all of the data to be destined for the null queue (deleted). Next, we go back and reset any events with that user to be destined for the indexQueue (to be indexed).

jscraig2006
Communicator

so basically you want to only send the data to the index based upon that user?

0 Karma

mailtosnsolutio
Explorer

Yes ,please

0 Karma
*NEW* Splunk Love Promo!
Snag a $25 Visa Gift Card for Giving Your Review!

It's another Splunk Love Special! For a limited time, you can review one of our select Splunk products through Gartner Peer Insights and receive a $25 Visa gift card!

Review:





Or Learn More in Our Blog >>