Dashboards & Visualizations

Dashboarding a csv file

Naskez
Engager

Hello splunk community,

I'm a newbie on splunk so i this maybe a basic question.

Basically I'm trying to do a piechart containing all the processes currently running. I managed (via powershell script) to generate a csv file containing this:

 

"Values","Count","Group","Name" "System.Collections.ArrayList","1","System.Collections.ObjectModel.Collection`1[System.Management.Automation.PSObject]","ApplicationFrameHost" "System.Collections.ArrayList","1","System.Collections.ObjectModel.Collection`1[System.Management.Automation.PSObject]","conhost" "System.Collections.ArrayList","3","System.Collections.ObjectModel.Collection`1[System.Management.Automation.PSObject]","csrss" "System.Collections.ArrayList","1","System.Collections.ObjectModel.Collection`1[System.Management.Automation.PSObject]","dllhost"

........

.........

When forwarded, splunk couldn't find fields associated with the file, even when i tried to extract fields manually, splunk confused field name with data.

(Objective: Pie chart containing the name of process and the number of its processes.)

Labels (1)
0 Karma
1 Solution

niketn
Legend

Your sample data seem to work out of the box for me. Only issue that I see is that you do not have Time field in your data, which implies you need to set Time to CURRENT for each csv file event.

Following is the props.conf setting for a dummy sourcetype I created to ingest your data.

 

[ sample_data_csv ]
DATETIME_CONFIG=CURRENT
SHOULD_LINEMERGE=false
LINE_BREAKER=([\r\n]+)
BREAK_ONLY_BEFORE_DATE=null
NO_BINARY_CHECK=true
CHARSET=UTF-8
INDEXED_EXTRACTIONS=csv
KV_MODE=none
category=Custom
description=Comma-separated value format. Set header and other settings in "Delimited Settings"
disabled=false
pulldown_type=true
HEADER_FIELD_LINE_NUMBER=1

 

As per the sample data provided in the question following are the extracted fields by default. But I still added HEADER_FIELD_LINE_NUMBER=1 and DATETIME_CONFIG=CURRENT config (you must check and confirm the date information whether it can be the time of file forward or it has to be supplied within the CSV).

Screen Shot 2020-07-18 at 12.54.36 AM.png

 

And following is the query I tried and worked. PS : I had added INDEXED_EXTRACTION = csv in the props.conf for tstats to work.

Screen Shot 2020-07-18 at 12.57.17 AM.png

So in case even after applying props.conf like the one above fields are not getting extracted, you would need to ensure whether your csv is valid UTF8 format CSV with no special characters or not. 

 In case you need further help you might have to share your props.conf or sample CSV file.

____________________________________________
| makeresults | eval message= "Happy Splunking!!!"

View solution in original post

niketn
Legend

Your sample data seem to work out of the box for me. Only issue that I see is that you do not have Time field in your data, which implies you need to set Time to CURRENT for each csv file event.

Following is the props.conf setting for a dummy sourcetype I created to ingest your data.

 

[ sample_data_csv ]
DATETIME_CONFIG=CURRENT
SHOULD_LINEMERGE=false
LINE_BREAKER=([\r\n]+)
BREAK_ONLY_BEFORE_DATE=null
NO_BINARY_CHECK=true
CHARSET=UTF-8
INDEXED_EXTRACTIONS=csv
KV_MODE=none
category=Custom
description=Comma-separated value format. Set header and other settings in "Delimited Settings"
disabled=false
pulldown_type=true
HEADER_FIELD_LINE_NUMBER=1

 

As per the sample data provided in the question following are the extracted fields by default. But I still added HEADER_FIELD_LINE_NUMBER=1 and DATETIME_CONFIG=CURRENT config (you must check and confirm the date information whether it can be the time of file forward or it has to be supplied within the CSV).

Screen Shot 2020-07-18 at 12.54.36 AM.png

 

And following is the query I tried and worked. PS : I had added INDEXED_EXTRACTION = csv in the props.conf for tstats to work.

Screen Shot 2020-07-18 at 12.57.17 AM.png

So in case even after applying props.conf like the one above fields are not getting extracted, you would need to ensure whether your csv is valid UTF8 format CSV with no special characters or not. 

 In case you need further help you might have to share your props.conf or sample CSV file.

____________________________________________
| makeresults | eval message= "Happy Splunking!!!"

Naskez
Engager

Thank you very much dear niketnilay that helped a lot !

i'll proceed on your steps and re-port the results

0 Karma

richgalloway
SplunkTrust
SplunkTrust
Before you can create a chart you need to extract fields. Let's tackle that first. Please share the props.conf settings you are using for the CSV file.
There are existing apps which can ingest process data so you don't have to re-invent the wheel. See Splunk Add-on for Microsoft Windows (https://splunkbase.splunk.com/app/742/) and Splunk Add-on for Infrastructure (https://splunkbase.splunk.com/app/4217/).
---
If this reply helps you, Karma would be appreciated.
Get Updates on the Splunk Community!

Enterprise Security Content Update (ESCU) | New Releases

In November, the Splunk Threat Research Team had one release of new security content via the Enterprise ...

Index This | Divide 100 by half. What do you get?

November 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with this ...

Stay Connected: Your Guide to December Tech Talks, Office Hours, and Webinars!

❄️ Celebrate the season with our December lineup of Community Office Hours, Tech Talks, and Webinars! ...