Dashboards & Visualizations

Dashboarding a csv file

Naskez
Engager

Hello splunk community,

I'm a newbie on splunk so i this maybe a basic question.

Basically I'm trying to do a piechart containing all the processes currently running. I managed (via powershell script) to generate a csv file containing this:

 

"Values","Count","Group","Name" "System.Collections.ArrayList","1","System.Collections.ObjectModel.Collection`1[System.Management.Automation.PSObject]","ApplicationFrameHost" "System.Collections.ArrayList","1","System.Collections.ObjectModel.Collection`1[System.Management.Automation.PSObject]","conhost" "System.Collections.ArrayList","3","System.Collections.ObjectModel.Collection`1[System.Management.Automation.PSObject]","csrss" "System.Collections.ArrayList","1","System.Collections.ObjectModel.Collection`1[System.Management.Automation.PSObject]","dllhost"

........

.........

When forwarded, splunk couldn't find fields associated with the file, even when i tried to extract fields manually, splunk confused field name with data.

(Objective: Pie chart containing the name of process and the number of its processes.)

Labels (1)
0 Karma
1 Solution

niketn
Legend

Your sample data seem to work out of the box for me. Only issue that I see is that you do not have Time field in your data, which implies you need to set Time to CURRENT for each csv file event.

Following is the props.conf setting for a dummy sourcetype I created to ingest your data.

 

[ sample_data_csv ]
DATETIME_CONFIG=CURRENT
SHOULD_LINEMERGE=false
LINE_BREAKER=([\r\n]+)
BREAK_ONLY_BEFORE_DATE=null
NO_BINARY_CHECK=true
CHARSET=UTF-8
INDEXED_EXTRACTIONS=csv
KV_MODE=none
category=Custom
description=Comma-separated value format. Set header and other settings in "Delimited Settings"
disabled=false
pulldown_type=true
HEADER_FIELD_LINE_NUMBER=1

 

As per the sample data provided in the question following are the extracted fields by default. But I still added HEADER_FIELD_LINE_NUMBER=1 and DATETIME_CONFIG=CURRENT config (you must check and confirm the date information whether it can be the time of file forward or it has to be supplied within the CSV).

Screen Shot 2020-07-18 at 12.54.36 AM.png

 

And following is the query I tried and worked. PS : I had added INDEXED_EXTRACTION = csv in the props.conf for tstats to work.

Screen Shot 2020-07-18 at 12.57.17 AM.png

So in case even after applying props.conf like the one above fields are not getting extracted, you would need to ensure whether your csv is valid UTF8 format CSV with no special characters or not. 

 In case you need further help you might have to share your props.conf or sample CSV file.

____________________________________________
| makeresults | eval message= "Happy Splunking!!!"

View solution in original post

niketn
Legend

Your sample data seem to work out of the box for me. Only issue that I see is that you do not have Time field in your data, which implies you need to set Time to CURRENT for each csv file event.

Following is the props.conf setting for a dummy sourcetype I created to ingest your data.

 

[ sample_data_csv ]
DATETIME_CONFIG=CURRENT
SHOULD_LINEMERGE=false
LINE_BREAKER=([\r\n]+)
BREAK_ONLY_BEFORE_DATE=null
NO_BINARY_CHECK=true
CHARSET=UTF-8
INDEXED_EXTRACTIONS=csv
KV_MODE=none
category=Custom
description=Comma-separated value format. Set header and other settings in "Delimited Settings"
disabled=false
pulldown_type=true
HEADER_FIELD_LINE_NUMBER=1

 

As per the sample data provided in the question following are the extracted fields by default. But I still added HEADER_FIELD_LINE_NUMBER=1 and DATETIME_CONFIG=CURRENT config (you must check and confirm the date information whether it can be the time of file forward or it has to be supplied within the CSV).

Screen Shot 2020-07-18 at 12.54.36 AM.png

 

And following is the query I tried and worked. PS : I had added INDEXED_EXTRACTION = csv in the props.conf for tstats to work.

Screen Shot 2020-07-18 at 12.57.17 AM.png

So in case even after applying props.conf like the one above fields are not getting extracted, you would need to ensure whether your csv is valid UTF8 format CSV with no special characters or not. 

 In case you need further help you might have to share your props.conf or sample CSV file.

____________________________________________
| makeresults | eval message= "Happy Splunking!!!"

Naskez
Engager

Thank you very much dear niketnilay that helped a lot !

i'll proceed on your steps and re-port the results

0 Karma

richgalloway
SplunkTrust
SplunkTrust
Before you can create a chart you need to extract fields. Let's tackle that first. Please share the props.conf settings you are using for the CSV file.
There are existing apps which can ingest process data so you don't have to re-invent the wheel. See Splunk Add-on for Microsoft Windows (https://splunkbase.splunk.com/app/742/) and Splunk Add-on for Infrastructure (https://splunkbase.splunk.com/app/4217/).
---
If this reply helps you, Karma would be appreciated.
Get Updates on the Splunk Community!

Stay Connected: Your Guide to November Tech Talks, Office Hours, and Webinars!

🍂 Fall into November with a fresh lineup of Community Office Hours, Tech Talks, and Webinars we’ve ...

Transform your security operations with Splunk Enterprise Security

Hi Splunk Community, Splunk Platform has set a great foundation for your security operations. With the ...

Splunk Admins and App Developers | Earn a $35 gift card!

Splunk, in collaboration with ESG (Enterprise Strategy Group) by TechTarget, is excited to announce a ...