streamfwd ingestion of data from the pcap searching

[ekremikizoglu]

Hi,

Following the Documentation provided by splunk I triggered streamfwd from the command line for my pcap.
http://docs.splunk.com/Documentation/StreamApp/6.1.1/DeployStreamApp/streamfwdcommandlineoptions

I searched "alltime" but i couldn't see any event about my pcap on search app. How can we search old pcap events?

App : App For Stream 6.6.1

streamfwd logs:
C:\Program Files\Splunk\etc\apps\Splunk_TA_stream\windows_x86_64\bin>streamfwd.exe -r test4.pcap
15:16:17.624 INFO stream.CaptureServer - Found DataDirectory: C:/Program Files\Splunk\etc\apps\Splunk_TA_stream\data
15:16:17.633 INFO stream.CaptureServer - Found UIDirectory: C:/Program Files\Splunk\etc\apps\Splunk_TA_stream\ui
15:16:18.956 INFO stream.StreamSender - Successfully pinged server (config needs update): cb807f08-8b58-42c8-84d3-42a2602a0fc0
15:16:18.969 INFO stream.CaptureServer - Default configuration directory: C:/Program Files\Splunk\etc\apps\Splunk_TA_stream\default
15:16:18.984 INFO stream.CaptureServer - Start sending pcap data
15:16:19.996 INFO stream.StreamSender - Successfully pinged server (config needs update): cb807f08-8b58-42c8-84d3-2a2602a0fc0
15:16:22.058 INFO stream.CaptureServer - Configuring offline capture with pcapfile C:/Program Files\Splunk\etc\apps\Splunk_TA_stream\windows_x86_64\bin\test4.pcap
15:16:22.075 INFO stream.CaptureServer - Starting data capture
15:16:22.089 INFO stream.main - streamfwd has started successfully (version 6.6.1 build 159)
15:16:22.337 INFO stream.CaptureServer - Finished sending pcap data; shutting down
15:16:22.345 INFO stream.main - streamfwd is shutting down
C:\Program Files\Splunk\etc\apps\Splunk_TA_stream\windows_x86_64\bin>

And also i tried below command but ı couldnt see anything...
C:\Program Files\Splunk\etc\apps\Splunk_TA_stream\windows_x86_64\bin>streamfwd.exe -r "C:\Program Files\Splunk\etc\apps\Splunk_TA_stream\windows_x86_64\bin\test4.pcap" -s localhost:8889 -b 1048576 --repeat
15:21:02.301 INFO stream.CaptureServer - Found DataDirectory: C:/Program Files\Splunk\etc\apps\Splunk_TA_stream\data
15:21:02.310 INFO stream.CaptureServer - Found UIDirectory: C:/Program Files\Splunk\etc\apps\Splunk_TA_stream\ui
15:21:03.624 INFO stream.StreamSender - Successfully pinged server (config needs update): f956a995-c651-4d5f-a458-0c1608db0003
15:21:03.634 INFO stream.CaptureServer - Default configuration directory: C:/Program Files\Splunk\etc\apps\Splunk_TA_stream\default
15:21:03.648 INFO stream.CaptureServer - Start sending pcap data
15:21:04.660 INFO stream.StreamSender - Successfully pinged server (config needs update): f956a995-c651-4d5f-a458-0c1608db0003
15:21:06.714 INFO stream.CaptureServer - Configuring offline capture with pcapfile C:/Program Files\Splunk\etc\apps\Splunk_TA_stream\windows_x86_64\bin\test4.pcap
15:21:06.731 INFO stream.CaptureServer - Starting data capture
15:21:06.747 INFO stream.main - streamfwd has started successfully (version 6.6.1 build 159)
15:21:10.246 INFO stream.StreamSender - Successfully pinged server (config up to date): f956a995-c651-4d5f-a458-0c1608db0003
15:21:16.015 INFO stream.StreamSender - Successfully pinged server (config up to date): f956a995-c651-4d5f-a458-0c1608db0003
15:21:21.040 INFO stream.StreamSender - Successfully pinged server (config up to date): f956a995-c651-4d5f-a458-0c1608db0003
15:21:29.143 INFO stream.StreamSender - Successfully pinged server (config up to date): f956a995-c651-4d5f-a458-0c1608db0003

Logs seem good.

[vshcherbakov_sp]

Hi @ekremikizoglu,

A few questions:

• What's in the pcap file and what streams are enabled on this instance? Stream will only generate the events for protocols/streams that are enabled.

• What's the exact SPL you're using to search for pcap events?

[ekremikizoglu]

Hi vshcherbakov,

In the pcap, there are some https,http traffics which i generated. On the instance http is enabled.

I tried these SPL
index=*
index=* sourcetype="stream:http"
index=default
index=main

[vshcherbakov_sp]

@ekremikizoglu,

you can try adding the --systime command line option to override the timestamps in pcap file with system time, so the command line will look like "streamfwd.exe -r test4.pcap --systime"

you can also enable network_interface field in the http protocol in Stream UI, so that you can use

... network_interface="*test4.pcap" in your SPL.

You won't be able to see "stream:http" events from https traffic unless you configure the stream forwarder (Splunk_TA_stream) with the SSL server's private key.