Filter data from showing up

[splunknewbie81]

Hi Guys, I would like to check if it's possible to prevent some data from showing up in the search. 

Below is what I want to prevent from showing up.

=============

Aug 31 23:59:43 a.b.c.d hey_audit: INFO 2021-08-31 23:59:43 12,345 HelloType=External |userName=zzz| |xxxapiversion=1.0| |httpMethod=POST| | restEndPoint=/v1/insights_transport/transfer_data_to_multicluster| |entityUuid=| |queryParams=| |payload=

I understand can use regex or eval. Can someone show me how it's done?

[splunknewbie81]

Hi @PickleRick,

You're right. I was intending to drop the above log from showing up in the search using some of the data from the above mentioned text.

Text to be used > |httpMethod=POST| | restEndPoint=/v1/insights_transport/transfer_data_to_multicluster| 

[PickleRick]

Depends on what you mean by "prevent from showing up". Do you want to filter out whole events that match some criteria? If so, then you can use |search or |where. But this one you would typicaly use much further down the processing pipeline since instead of filtering simple search results you should probably just add another condition to your initial search.

Or maybe you want to limit your processing to only a subset of files - for that you can use |fields. Of course for this you should have parsing rules for given event in place.

[PickleRick]

You might just add another condition to your initial search.

_raw!=*your_text_to_filter_out*

It ain't very pretty nor very efficient but should work. But generally you'd be better off filtering by values of parsed fields. So in this case, assuming that the even parses the fields from the event, you would just filter by "http_method!=POST restEndPoint!=/v1/whatever" and so on.

[splunknewbie81]

I got a earlier log that grabs all the necessary info into the following tables - username , defaultmsg , date , time.

=============

Aug 25 17:41:05 IP_ADDR consolidated_audit: {"affectedEntityList":[{"entityType":"vm","name":"HOSTNAME01 | PROGRAMNAME","uuid":"11aef477-6f03-4a22-baa7-23736c8e741c"}],"alertUid":"VmUpdateAudit","classificationList":["UserAction"],"creationTimestampUsecs":"1629874233889978","defaultMsg":"Updated VM HOSTNAME01 | Tenable","opEndTimestampUsecs":"1629874233884373","opStartTimestampU==============
secs":"1629874233779524","operationType":"Update","originatingClusterUuid":"0005ad50-f430-a46c-001e-e4434bb76b0e","params":{"is_secure_boot":"false","is_uefi_boot":"true","vm_name":"HOSTNAME01 | PROGRAMNAME"},"recordType":"Audit","severity":"Audit","userName":"ABC@ABC.COM","userUuid":"3f73d040-7516-5caa-a956-91179b2cb1f5","uuid":"254f3952-bf9b-4205-8a14-ad2074ed70ab"}

But the issue now is that in the earlier msg, there is no defaultmsg field in the earlier event and it under table (defualtmsg) , it's empty

[PickleRick]

Well, this event is in a completely different format.

So I suppose you should first work on your data input - providing proper clasification of events into sourcetypes and parsing rules (extractions).