topic Re: Filter data from showing up in Splunk Search

Filter data from showing up

splunknewbie81 — Thu, 02 Sep 2021 07:32:12 GMT

Hi Guys, I would like to check if it's possible to prevent some data from showing up in the search.

Below is what I want to prevent from showing up.

=============

I understand can use regex or eval. Can someone show me how it's done?

Re: Filter data from showing up

PickleRick — Thu, 02 Sep 2021 07:36:30 GMT

Depends on what you mean by "prevent from showing up". Do you want to filter out whole events that match some criteria? If so, then you can use |search or |where. But this one you would typicaly use much further down the processing pipeline since instead of filtering simple search results you should probably just add another condition to your initial search.

Or maybe you want to limit your processing to only a subset of files - for that you can use |fields. Of course for this you should have parsing rules for given event in place.

Re: Filter data from showing up

splunknewbie81 — Thu, 02 Sep 2021 07:43:59 GMT

Hi @PickleRick,

You're right. I was intending to drop the above log from showing up in the search using some of the data from the above mentioned text.

Text to be used > |httpMethod=POST| | restEndPoint=/v1/insights_transport/transfer_data_to_multicluster|

Re: Filter data from showing up

PickleRick — Thu, 02 Sep 2021 07:49:31 GMT

You might just add another condition to your initial search.

_raw!=*your_text_to_filter_out*

It ain't very pretty nor very efficient but should work. But generally you'd be better off filtering by values of parsed fields. So in this case, assuming that the even parses the fields from the event, you would just filter by "http_method!=POST restEndPoint!=/v1/whatever" and so on.

Re: Filter data from showing up

splunknewbie81 — Thu, 02 Sep 2021 08:15:42 GMT

I got a earlier log that grabs all the necessary info into the following tables - username , defaultmsg , date , time.

=============

Aug 25 17:41:05 IP_ADDR consolidated_audit: {"affectedEntityList":[{"entityType":"vm","name":"HOSTNAME01 | PROGRAMNAME","uuid":"11aef477-6f03-4a22-baa7-23736c8e741c"}],"alertUid":"VmUpdateAudit","classificationList":["UserAction"],"creationTimestampUsecs":"1629874233889978","defaultMsg":"Updated VM HOSTNAME01 | Tenable","opEndTimestampUsecs":"1629874233884373","opStartTimestampU============== secs":"1629874233779524","operationType":"Update","originatingClusterUuid":"0005ad50-f430-a46c-001e-e4434bb76b0e","params":{"is_secure_boot":"false","is_uefi_boot":"true","vm_name":"HOSTNAME01 | PROGRAMNAME"},"recordType":"Audit","severity":"Audit","userName":"ABC@ABC.COM","userUuid":"3f73d040-7516-5caa-a956-91179b2cb1f5","uuid":"254f3952-bf9b-4205-8a14-ad2074ed70ab"}

But the issue now is that in the earlier msg, there is no defaultmsg field in the earlier event and it under table (defualtmsg) , it's empty

Re: Filter data from showing up

PickleRick — Thu, 02 Sep 2021 08:20:19 GMT

Well, this event is in a completely different format.

So I suppose you should first work on your data input - providing proper clasification of events into sourcetypes and parsing rules (extractions).