All Apps and Add-ons

Splunk Stream app: Uploading a large pcap file fails without error


Hi Splunkers,

I'm trying to uploading a large pcap file (3,5 GB). After entering a name and choosing the file I select next. I can see that the file is being uploaded, but after this step nothing happened.
Is someone facing the same issue?

Best regards,

Tags (2)
0 Karma

Splunk Employee
Splunk Employee

As noted here
there are three ways to use Stream to parse PCAP files;

  • Upload pcaps in Splunk Web.
  • Ingest pcaps using command line options.
  • Ingest pcaps using streamfwd.conf.

Splunk Web has a limit in terms of the file size it can accept via HTTP POST. This seems to be a function of Splunk's implementation of CherryPy in Splunk Web, and likely not to be fixed in the near future.

Using the command line to ingest PCAPs into Stream is the most reliable way to accomplish this task for PCAPs of arbitrary size. From the above page:

Read pcap files

Use the -r option to read individual pcap files. For example:

./streamfwd -r my.pcap

Ingest pcap files from a directory

Use the --pcapdir DIR option to monitor and index pcap files in a directory. For example:

./streamfwd --pcapdir ~/test_pcap_dir --afteringest repeat

You can run these on the search head (if the App for Stream is installed there), or on an arbitrary machine hosting the Stream Forwarder.

0 Karma

Splunk Employee
Splunk Employee

Have a look here:

and here:

Specifically, Example 4. I think that may hold the key to resolving things and allow you to replay after you upload the pcap to the correct location.


has it made your inputs.conf file really big?

I am having the same issue. Each time i add a pcap it seems to complain and if i look on my search head under etc\system\local the inputs file will contain the pcap data. Not sure that's right as it could potentially fill up the search heads drive


0 Karma
Get Updates on the Splunk Community!

Accelerate Service Onboarding, Decomposition, Troubleshooting - and more with ITSI’s ...

Accelerate Service Onboarding, Decomposition, Troubleshooting - and more!  Faster Time to Value Managing and ...

New Release | Splunk Enterprise 9.3

Admins and Analyst can benefit from:  Seamlessly route data to your local file system to save on storage ...

2024 Splunk Career Impact Survey | Earn a $20 gift card for participating!

Hear ye, hear ye! The time has come again for Splunk's annual Career Impact Survey!  We need your help by ...