Splunk Search

Best way to extract _raw to a host OS of a SH programatically

cybersecnutant
Engager

We have a 3rd party pulling AWS logs as far back as AWS holds onto logs. However, we want to be able to go back further so we are looking at our AWS index in Splunk. We want to extract a full export of _raw for the entire index. We have access to the management port of our searchhead which is pointing to an indexer cluster with all of the aws index data - noting that the index is SmartStore enabled.

What's the best way to export this programmatically? It would not scale to manually run the search in the GUI and export it. We've looked at the oneshot search with js but it seems to be timing out even though we have baked in pagination.

Thanks in advance

Labels (1)
0 Karma

terminaloutcome
Explorer

I've got example code here for running searches - using the latest splunklib and JSONReader you can dump at high speed to disk.

https://github.com/yaleman/splunk-sdk-games/blob/main/write_raw_json.py

Tags (1)
0 Karma
Get Updates on the Splunk Community!

Improve Your Security Posture

Watch NowImprove Your Security PostureCustomers are at the center of everything we do at Splunk and security ...

Maximize the Value from Microsoft Defender with Splunk

 Watch NowJoin Splunk and Sens Consulting for this Security Edition Tech TalkWho should attend:  Security ...

This Week's Community Digest - Splunk Community Happenings [6.27.22]

Get the latest news and updates from the Splunk Community here! News From Splunk Answers ✍️ Splunk Answers is ...