All Apps and Add-ons

Splunk Stream app: Uploading a large pcap file fails without error


Hi Splunkers,

I'm trying to uploading a large pcap file (3,5 GB). After entering a name and choosing the file I select next. I can see that the file is being uploaded, but after this step nothing happened.
Is someone facing the same issue?

Best regards,

Tags (2)
0 Karma

Splunk Employee
Splunk Employee

As noted here
there are three ways to use Stream to parse PCAP files;

  • Upload pcaps in Splunk Web.
  • Ingest pcaps using command line options.
  • Ingest pcaps using streamfwd.conf.

Splunk Web has a limit in terms of the file size it can accept via HTTP POST. This seems to be a function of Splunk's implementation of CherryPy in Splunk Web, and likely not to be fixed in the near future.

Using the command line to ingest PCAPs into Stream is the most reliable way to accomplish this task for PCAPs of arbitrary size. From the above page:

Read pcap files

Use the -r option to read individual pcap files. For example:

./streamfwd -r my.pcap

Ingest pcap files from a directory

Use the --pcapdir DIR option to monitor and index pcap files in a directory. For example:

./streamfwd --pcapdir ~/test_pcap_dir --afteringest repeat

You can run these on the search head (if the App for Stream is installed there), or on an arbitrary machine hosting the Stream Forwarder.

0 Karma

Splunk Employee
Splunk Employee

Have a look here:

and here:

Specifically, Example 4. I think that may hold the key to resolving things and allow you to replay after you upload the pcap to the correct location.

New Member

has it made your inputs.conf file really big?

I am having the same issue. Each time i add a pcap it seems to complain and if i look on my search head under etc\system\local the inputs file will contain the pcap data. Not sure that's right as it could potentially fill up the search heads drive


0 Karma
.conf21 Now Fully Virtual!
Register for FREE Today!

We've made .conf21 totally virtual and totally FREE! Our completely online experience will run from 10/19 through 10/20 with some additional events, too!