Getting Data In

Is there a way to get Splunk to do oneshot indexing sequentially?

rf2010
New Member

I have a large number of files (going by the thousands) that I only need to index once, and since I want to know when the indexing has completed, I make use of the oneshot adding command in the CLI to do this.

The problem is, Splunk indexes all the files in its oneshot list at the same time (correct me if I'm wrong on this), which makes the indexing exponentially slower as more files are added to the oneshot list.

Adding in a fixed delay in the CLI script is one way to help, but it's not the best solution here.

Is there any way to alter the behaviour of the oneshot indexing to index only one log file after another instead of all at once?

Tags (2)
0 Karma

ddeighton
Explorer

I just kicked off a BASH for loop to index several files and they are indexing sequentially. You may want to try that.

For example:

for file in file1 file2 file3; do
 splunk add oneshot $file
done
0 Karma

ddeighton
Explorer

OK. I take it back. On a system that is set up as a forwarder, the files are being sent sequentially to the indexer. Not sure if this is by design, but that is what I'm seeing.

On the indexer, all of the files are being indexed at once.

0 Karma
Career Survey
First 500 qualified respondents will receive a $20 gift card! Tell us about your professional Splunk journey.

Can’t make it to .conf25? Join us online!

Get Updates on the Splunk Community!

Can’t Make It to Boston? Stream .conf25 and Learn with Haya Husain

Boston may be buzzing this September with Splunk University and .conf25, but you don’t have to pack a bag to ...

Splunk Lantern’s Guide to The Most Popular .conf25 Sessions

Splunk Lantern is a Splunk customer success center that provides advice from Splunk experts on valuable data ...

Unlock What’s Next: The Splunk Cloud Platform at .conf25

In just a few days, Boston will be buzzing as the Splunk team and thousands of community members come together ...