Splunk Search

what's the best way to create a table of the app name, found data, and log snippet

llh
New Member

requirements:
find and save sensitive data fields from logs
Save log snippet around sensitive data field
Remove duplicates for mule apps and sensitive data field
Create table showing mule app name, sensitive data, and log snippet

is there a way to improve the search query so I don't have to duplicate the rex commands every time I need to add a new sensitive data value?

(app_name is an existing custom field)

index="prod"  |rex field=_raw (?i)(?<birthDate>(birthDate))|rex field=_raw (?i)(?<dob>(dob)) |rex field=_raw (?i)(?<birthday>(birthday)) |rex field=_raw (?i)(?<birthDateLog>(birthDate).*?\w\W) |rex field=_raw (?i)(?<dobLog>(dob).*?\w\W) |rex field=_raw (?i)(?<birthdayLog>(birthday).*?\w\W)|eval SENSITIVE_DATA= mvappend(birthDate,dob,birthday) |eval SENSITIVE_DATA_LOWER=lower(SENSITIVE_DATA) | dedup app_name SENSITIVE_DATA_LOWER |eval SENSITIVE_DATA_LOG=mvappend(birthDateLog,dobLog,birthdayLog) |stats list(SENSITIVE_DATA_LOG) as SENSITIVE_DATA_LOG list(SENSITIVE_DATA_LOWER) as SENSITIVE_DATA_LOWER by app_name | table app_name SENSITIVE_DATA_LOWER SENSITIVE_DATA_LOG

 

example output:

app_nameSENSITIVE_DATA_LOWERSENSITIVE_DATA_LOG
s-api

dob

birthdate

dob: 01/01/2024

birthdate:  09-09-1999

p-apibirthdaybirthday: August 23, 2024
Labels (2)
0 Karma

PickleRick
SplunkTrust
SplunkTrust

Well, you can try to make a compound regex containing some alternative branches.

Also you seem to have some XML-like structure there. If it's a valid XML, why not just parse the XML into fields and check for existence of specific fields?

I'm also not sure about the rest of the search but honestly speaking it's too late and I'm too tired at the moment to look into it.

0 Karma
Get Updates on the Splunk Community!

Get Early Access to AI Playbook Authoring: Apply for the Alpha Private Preview ...

Passionate about security automation? Apply now to our AI Playbook Authoring Alpha private preview ...

Reduce and Transform Your Firewall Data with Splunk Data Management

Managing high-volume firewall data has always been a challenge. Noisy events and verbose traffic logs often ...

Automatic Discovery Part 1: What is Automatic Discovery in Splunk Observability Cloud ...

If you’ve ever deployed a new database cluster, spun up a caching layer, or added a load balancer, you know it ...