Alerting

How do I correctly index each line of a CSV file as separate events in Splunk, and how do I group similar alert names together?

splunksurekha
Path Finder

1) I have a input file which looks like below:

dc10splunksrch01:/opt/splunk/etc/apps/sfapp_all_zbx/lookups # head ZbxDailyReport.csv 
host,hostname,alertname,alertstatus,alertseverity,alertdate 
"prodmail2a","prodmail2a","Mail queue greater than 6000 on prodmail2a, current queue: 8051","OK","Information","Fri Sep 25 08:06:59 2015" 
"prodmail2a","prodmail2a","Mail queue greater than 8000 on prodmail2a, current queue: 8051","Problem","high","Fri Sep 25 08:06:59 2015" 
"pcwebp01n01.ams2.sf.priv","pcwebp01n01.ams2.sf.priv","pcwebp01n01.ams2.sf.priv /var Space Low (Total=1.94 GB, %Used=70.56 %)","Problem","Information","Fri Sep 25 08:09:32 2015" 
"pcwebp01n01.ams2.sf.priv","pcwebp01n01.ams2.sf.priv","pcwebp01n01.ams2.sf.priv /var Space Low (Total=1.94 GB, %Used=70.56 %)","OK","average","Thu Sep 24 16:03:44 2015" 
"scm001.ams2.sf.priv","scm001.ams2.sf.priv","CPU Load > 90%, Idle < 5% - scm001.ams2.sf.priv: Idle=56.49 %, Load=7.32, CPU.count=1","OK","high","Fri Sep 25 07:05:30 2015" 

2) I am indexing it using the inputs.conf, props.conf, and transforms.conf like below.

dc10splunksrch01:/opt/splunk/etc/apps/sfapp_all_zbx/local # cat inputs.conf

[monitor:///opt/splunk/etc/apps/sfapp_all_zbx/lookups/ZbxDailyReport.csv] 
disabled = false 
followTail = 0 
sourcetype=zbxAlertReport 
index=dc10_oss 

dc10splunksrch01:/opt/splunk/etc/apps/sfapp_all_zbx/local # cat transforms.conf

[zbxAlertReport] 
filename = ZbxDailyReport.csv 

dc10splunksrch01:/opt/splunk/etc/apps/sfapp_all_zbx/local # cat props.conf

[zbxAlertReport] 
TRANSFORMS-index-oss = route-index-oss 

3) But its not getting indexed properly. In one line I am getting multiple records in some cases. Please refer to the screen shot. How to get them indexed correctly each line of CSV file as one line in Splunk?
alt text

4) How can I cluster same alertnames together irrespective of hostname since the alertname has hostname in it first. I need to get rid of hostname in the alert name and then group the alertname together.

Note: hostname doesn't come in a particular place in the alertname.

0 Karma

lguinn2
Legend

First, your props.conf references a stanza that doesn't exist in transforms.conf - so that's not going to work.

You might change it to this:

dc10splunksrch01:/opt/splunk/etc/apps/sfapp_all_zbx/local/transforms.conf

 [zbxAlertReportFields] 
 FIELDS=alerthost,hostname,alertname,alertstatus,alertseverity,alertdate
 DELIMS = ","

dc10splunksrch01:/opt/splunk/etc/apps/sfapp_all_zbx/local/props.conf

 [zbxAlertReport] 
 TRANSFORMS-zar = zbxAlertReportFields
 SHOULD_LINE_MERGE=false

Note that I changed the "host" field to "alerthost". "host" is a default field name in Splunk and you really shouldn't use it in your data. It will just get confusing at best.
Second, you will need to remove the data from the dc10_oss index and re-index the file.

Finally, if you want to group data, you need to use a reporting command in Splunk. For example

index=dc10_oss sourcetype=zbxAlertReport
| sort alertname alertdata

is a very simple report that groups the data. Or,

index=dc10_oss sourcetype=zbxAlertReport
| stats count by alertname, alertstatus,alertseverity,hostname
0 Karma
Register for .conf21 Now! Go Vegas or Go Virtual!

How will you .conf21? You decide! Go in-person in Las Vegas, 10/18-10/21, or go online with .conf21 Virtual, 10/19-10/20.