Splunk Search

CLI search with FATAL: The search job terminated unexpectedly

vnguyen46
Contributor

Hello,

I try to export a large log with CLI search below. It works well with a smaller log return, but giving error on large logs, FATAL: The search job terminated unexpectedly.

For instance, this search on Pan_logs terminated:

/opt/splunk/bin/splunk search "index=pan_logs earliest=-7d" -preview 0 -maxout 0 -output rawdata | gzip > pan_logs_7days.gz

Anyone knows how to resolve this issue?

Thanks,

Labels (1)
Tags (2)
0 Karma
1 Solution

vnguyen46
Contributor

@twesty That's another idea.

For small log, I run CLI search directly on the SH: 

/opt/splunk/bin/splunk search "index=small_log earliest=-14d" -preview 0 -maxout 0 -output rawdata | gzip > small_log_14days.gz

I used dump for large logs by running this query on the SH homepage:

index=wineventlog | dump basefilename=WinEventLog rollsize=20000 compress=9 format=raw
the output file saved at this dir: /opt/splunk/var/run/splunk/dispatch/(sid)/dump/

Best,

View solution in original post

0 Karma

twesty
Path Finder

I would suggest running the search in smaller batches. there are many reasons why you could have a failure however the likelihood is that the returning message is just too large to handle.

If you're looking for 7 days worth of data, run 7 separate queries over a day each and then stitch the output together outside of Slpu

0 Karma

vnguyen46
Contributor

@twesty That's another idea.

For small log, I run CLI search directly on the SH: 

/opt/splunk/bin/splunk search "index=small_log earliest=-14d" -preview 0 -maxout 0 -output rawdata | gzip > small_log_14days.gz

I used dump for large logs by running this query on the SH homepage:

index=wineventlog | dump basefilename=WinEventLog rollsize=20000 compress=9 format=raw
the output file saved at this dir: /opt/splunk/var/run/splunk/dispatch/(sid)/dump/

Best,

0 Karma
Get Updates on the Splunk Community!

Splunk Observability Cloud | Unified Identity - Now Available for Existing Splunk ...

Raise your hand if you’ve already forgotten your username or password when logging into an account. (We can’t ...

Index This | How many sides does a circle have?

February 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

Registration for Splunk University is Now Open!

Are you ready for an adventure in learning?   Brace yourselves because Splunk University is back, and it's ...