Splunk Search

Check if latest logs contain IOC

superuser88
Engager

 

INDEX Name generated (10 million new records every day)
INDEX Fields username, secret, key 

Lookup file secrets.csv with fields secret (128 bit string - 1 million static records)

I am creating a report to check if any of secret is found within the secrets.csv list and flag it.

index=generated [| inputlookup secrets.csv | fields secret] | table username, secret, key

How does the check for secret if exists in both index generated is validated in inputlookup in the search string?

Labels (3)
0 Karma

ITWhisperer
SplunkTrust
SplunkTrust

Subsearches are limited to 50,000 events so if you still want to do it this way, you would have to split your lookup into about 20 different lookups.

0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @superuser88 ,

if you want to search IOCs in all your raw log, you could try something like this:

index=generated [| inputlookup secrets.csv | rename secret AS query | fields query ] 
| table username key

In this way you perform a full text search on your raw log.

Ciao.

Giuseppe

0 Karma

bowesmana
SplunkTrust
SplunkTrust

You can't do a subsearch returning a million rows

0 Karma

bowesmana
SplunkTrust
SplunkTrust

You can't do that - because the subsearch will not return 1 million rows and it would take forever anyway.

index=generated 
| lookup secrets.csv secret OUTPUT secret as found
| where isnotnull(found)
| table username, secret, key
0 Karma
Get Updates on the Splunk Community!

3 Ways to Make OpenTelemetry Even Better

My role as an Observability Specialist at Splunk provides me with the opportunity to work with customers of ...

What's New in Splunk Cloud Platform 9.2.2406?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.2.2406 with many ...

Enterprise Security Content Update (ESCU) | New Releases

In August, the Splunk Threat Research Team had 3 releases of new security content via the Enterprise Security ...