All Apps and Add-ons

Splunk Stream app: Uploading a large pcap file fails without error

sebastianstruwe
Explorer

Hi Splunkers,

I'm trying to uploading a large pcap file (3,5 GB). After entering a name and choosing the file I select next. I can see that the file is being uploaded, but after this step nothing happened.
Is someone facing the same issue?

Best regards,
Sebastian

Tags (2)
0 Karma

dcavuto_splunk
Splunk Employee
Splunk Employee

As noted here http://docs.splunk.com/Documentation/StreamApp/7.1.2/DeployStreamApp/UseStreamtoparsePCAPfiles
there are three ways to use Stream to parse PCAP files;

  • Upload pcaps in Splunk Web.
  • Ingest pcaps using command line options.
  • Ingest pcaps using streamfwd.conf.

Splunk Web has a limit in terms of the file size it can accept via HTTP POST. This seems to be a function of Splunk's implementation of CherryPy in Splunk Web, and likely not to be fixed in the near future.

Using the command line to ingest PCAPs into Stream is the most reliable way to accomplish this task for PCAPs of arbitrary size. From the above page:

Read pcap files

Use the -r option to read individual pcap files. For example:

./streamfwd -r my.pcap

Ingest pcap files from a directory

Use the --pcapdir DIR option to monitor and index pcap files in a directory. For example:

./streamfwd --pcapdir ~/test_pcap_dir --afteringest repeat

You can run these on the search head (if the App for Stream is installed there), or on an arbitrary machine hosting the Stream Forwarder.

0 Karma

pdaigle_splunk
Splunk Employee
Splunk Employee

Have a look here:

https://answers.splunk.com/answers/155769/app-for-stream-pcap-replay.html

and here:

http://docs.splunk.com/Documentation/StreamApp/7.1.2/DeployStreamApp/ConfigureStreamForwarder#stream...

Specifically, Example 4. I think that may hold the key to resolving things and allow you to replay after you upload the pcap to the correct location.

damianpadden
Loves-to-Learn

has it made your inputs.conf file really big?

I am having the same issue. Each time i add a pcap it seems to complain and if i look on my search head under etc\system\local the inputs file will contain the pcap data. Not sure that's right as it could potentially fill up the search heads drive

Damain

0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

Agent Mode Engaged! Enchaining Agentic Operations with Splunk AI Assistant 2.0

    Are you ready to transform how your team handles complex data requests? We invite you to our upcoming ...

Announcing Modern Navigation: A New Era of Splunk User Experience

We are excited to introduce the Modern Navigation feature in the Splunk Platform, available to both cloud and ...

Modernize your Splunk Apps – Introducing Python 3.13 in Splunk

We are excited to announce that the upcoming releases of Splunk Enterprise 10.2.x and Splunk Cloud Platform ...