Splunk Search

streamfwd ingestion of data from the pcap searching

ekremikizoglu
Explorer

Hi,

Following the Documentation provided by splunk I triggered streamfwd from the command line for my pcap.
http://docs.splunk.com/Documentation/StreamApp/6.1.1/DeployStreamApp/streamfwdcommandlineoptions

I searched "alltime" but i couldn't see any event about my pcap on search app. How can we search old pcap events?

App : App For Stream 6.6.1

streamfwd logs:
C:\Program Files\Splunk\etc\apps\Splunk_TA_stream\windows_x86_64\bin>streamfwd.exe -r test4.pcap
15:16:17.624 INFO stream.CaptureServer - Found DataDirectory: C:/Program Files\Splunk\etc\apps\Splunk_TA_stream\data
15:16:17.633 INFO stream.CaptureServer - Found UIDirectory: C:/Program Files\Splunk\etc\apps\Splunk_TA_stream\ui
15:16:18.956 INFO stream.StreamSender - Successfully pinged server (config needs update): cb807f08-8b58-42c8-84d3-42a2602a0fc0
15:16:18.969 INFO stream.CaptureServer - Default configuration directory: C:/Program Files\Splunk\etc\apps\Splunk_TA_stream\default
15:16:18.984 INFO stream.CaptureServer - Start sending pcap data
15:16:19.996 INFO stream.StreamSender - Successfully pinged server (config needs update): cb807f08-8b58-42c8-84d3-2a2602a0fc0
15:16:22.058 INFO stream.CaptureServer - Configuring offline capture with pcapfile C:/Program Files\Splunk\etc\apps\Splunk_TA_stream\windows_x86_64\bin\test4.pcap
15:16:22.075 INFO stream.CaptureServer - Starting data capture
15:16:22.089 INFO stream.main - streamfwd has started successfully (version 6.6.1 build 159)
15:16:22.337 INFO stream.CaptureServer - Finished sending pcap data; shutting down
15:16:22.345 INFO stream.main - streamfwd is shutting down
C:\Program Files\Splunk\etc\apps\Splunk_TA_stream\windows_x86_64\bin>

And also i tried below command but ı couldnt see anything...
C:\Program Files\Splunk\etc\apps\Splunk_TA_stream\windows_x86_64\bin>streamfwd.exe -r "C:\Program Files\Splunk\etc\apps\Splunk_TA_stream\windows_x86_64\bin\test4.pcap" -s localhost:8889 -b 1048576 --repeat
15:21:02.301 INFO stream.CaptureServer - Found DataDirectory: C:/Program Files\Splunk\etc\apps\Splunk_TA_stream\data
15:21:02.310 INFO stream.CaptureServer - Found UIDirectory: C:/Program Files\Splunk\etc\apps\Splunk_TA_stream\ui
15:21:03.624 INFO stream.StreamSender - Successfully pinged server (config needs update): f956a995-c651-4d5f-a458-0c1608db0003
15:21:03.634 INFO stream.CaptureServer - Default configuration directory: C:/Program Files\Splunk\etc\apps\Splunk_TA_stream\default
15:21:03.648 INFO stream.CaptureServer - Start sending pcap data
15:21:04.660 INFO stream.StreamSender - Successfully pinged server (config needs update): f956a995-c651-4d5f-a458-0c1608db0003
15:21:06.714 INFO stream.CaptureServer - Configuring offline capture with pcapfile C:/Program Files\Splunk\etc\apps\Splunk_TA_stream\windows_x86_64\bin\test4.pcap
15:21:06.731 INFO stream.CaptureServer - Starting data capture
15:21:06.747 INFO stream.main - streamfwd has started successfully (version 6.6.1 build 159)
15:21:10.246 INFO stream.StreamSender - Successfully pinged server (config up to date): f956a995-c651-4d5f-a458-0c1608db0003
15:21:16.015 INFO stream.StreamSender - Successfully pinged server (config up to date): f956a995-c651-4d5f-a458-0c1608db0003
15:21:21.040 INFO stream.StreamSender - Successfully pinged server (config up to date): f956a995-c651-4d5f-a458-0c1608db0003
15:21:29.143 INFO stream.StreamSender - Successfully pinged server (config up to date): f956a995-c651-4d5f-a458-0c1608db0003

Logs seem good.

0 Karma

vshcherbakov_sp
Splunk Employee
Splunk Employee

Hi @ekremikizoglu,

A few questions:

  • What's in the pcap file and what streams are enabled on this instance? Stream will only generate the events for protocols/streams that are enabled.

  • What's the exact SPL you're using to search for pcap events?

0 Karma

ekremikizoglu
Explorer

Hi vshcherbakov,

In the pcap, there are some https,http traffics which i generated. On the instance http is enabled.

I tried these SPL
index=*
index=* sourcetype="stream:http"
index=default
index=main

0 Karma

vshcherbakov_sp
Splunk Employee
Splunk Employee

@ekremikizoglu,

you can try adding the --systime command line option to override the timestamps in pcap file with system time, so the command line will look like "streamfwd.exe -r test4.pcap --systime"

you can also enable network_interface field in the http protocol in Stream UI, so that you can use

... network_interface="*test4.pcap" in your SPL.

You won't be able to see "stream:http" events from https traffic unless you configure the stream forwarder (Splunk_TA_stream) with the SSL server's private key.

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...