Unable to convert pcap file to a csv for indexing and analysis.
I followed the instructions from Daniel; however, the pcap file is not converting to a csv. Therefore, the data is not being indexed.
I gave Full rights to my ID (and all users on my laptop) to
- Wireshark folder and subfolders (for access to tshark.exe)
- SplunkForPCAP folder and subfolders (for access to ../SplunkForPCAP/bin/ folder)
I set SPLUNK_HOME variable. I tried both as a system and as a public variable.
Here is the procedure I followed
- Drop a pcap in the folder I configured for Data Inputs (PCAPanalyzerTEST)
- A few minutes later, the file is processed? and no longer in the PCAPanalyzerTEST folder
- It is in the PCAPConverted folder
- There is also a csv file in the PCAPcsv folder. However, it is zero bytes long.
- Windows 8.1 Enterprise
- Splunk Enterprise 18.104.22.168 - Single instance on laptop
- Splunk Stream 7.1.3
- Splunk PCAP Analyzer 22.214.171.124
Here are the contents of the indexes.conf and input.conf files in the Splunk home folder \etc\apps\SplunkForPCAP\local.
coldPath = $SPLUNK_DB\pcap\colddb
enableDataIntegrityControl = 0
enableTsidxReduction = 0
homePath = $SPLUNK_DB\pcap\db
maxTotalDataSizeMB = 512000
thawedPath = $SPLUNK_DB\pcap\thaweddb
host = GCJPC
index = pcap
path = C:\Users\gcj\Desktop\PCAPanalyzerTEST
Thanks in advance for any direction or advice you can offer.
from what I am reading there could be 2 things which can be the reason:
Since the csv file (0bytes) is already created, something is wrong on the script which either points to tshark or missing Splunk_HOME, %programfiles% variable.
While searching through the events I was unable to find the start (since I have tried several times, even before I started this thread).
However, I did find some events that may be more helpful in resolving this issue.
Since the time I first install the app to the present, this type of event occurs every ~3 minutes (total 257 times).
-0400 ERROR ExecProcessor - message from ""C:\Program Files\Splunk\etc\apps\SplunkForPCAP\bin\pcap2csv.bat"" File Not Found
pcap2csv.bat is located here C:\Program Files\Splunk\etc\apps\SplunkForPCAP\bin
REM Daniel Schwartz
REM This script aims to check which tshark script to execute
REM Version 1.2
REM Created: December 2016
REM Updated: 08.11.2017 - Monitored folders moved to app directory.
for /f "delims=" %%i in ('"%programfiles%\Wireshark\tshark" -v ^| findstr /r (v') do set "TS=%%i"
for /f "delims=" %%a in ('"%programfiles%\Wireshark\tshark" -v ^| findstr /r (v ^|findstr /r v2') do set "V2="%%a""
IF NOT [%V2%] == ELSE (
IF %H% LSS 2 IF %T% LEQ 10 (
) ELSE (
Thanks and God bless,
Actually you can ignore those errors. The script checks every 3minutes if there is new .pcap file in your folder of your choice. So if you don't put a new .pcap file in the folder, there is no file to be found. The new version of the app will exclude those errors. Not in this release though.
Try to search: "index=_internal pcap2csv NOT "File Not Found""
That results with this.
05-20-2019 11:04:33.219 -0400 ERROR ExecProcessor - Couldn't start command ""C:\Program Files\Splunk\etc\apps\SplunkForPCAP\bin\pcap2csv.sh"": FormatMessage was unable to decode error (193), (0xc1)
Thanks and God bless,
I haven't heard back from you since your last reply.
I've attempted to use tshark to create a CSV from the original PCAP file. Then use PCAP Analyzer to search and analyze the data. I used a tshark command I found here in this SANS paper. Unfortunately, the fields the author is extracting do not match with the fields your app is extracting.
Thanks again for your help with this.
Sorry, I was not able to reply earlier.
In your case to troubleshoot better I would concentrate on the 3 bat scripts located in C:\Program Files\Splunk\etc\apps\SplunkForPCAP\bin.
Try to hard code the variables %programfiles%+%SPLUNK_HOME%.
And then execute the script manually via cmd.
Let me know what the script output says when you do it.