topic Re: Not automatically recognizing fields in Splunk Search

Not automatically recognizing fields

jamesklassen — Fri, 26 Nov 2010 04:02:58 GMT

I have data that is not being recognized. A PowerShell script outputs data (that I copied to a file for testing) that looks like this:

Identity = MAILHUB2\45191, DeliveryType = SmtpRelayToTiRg, Status = Ready, MessageCount = 0
Identity = MAILHUB2\45210, DeliveryType = MapiDelivery, Status = Ready, MessageCount = 0
Identity = MAILHUB2\45226, DeliveryType = MapiDelivery, Status = Ready, MessageCount = 0
Identity = MAILHUB2\45235, DeliveryType = MapiDelivery, Status = Ready, MessageCount = 0

I formatted the PowerShell output like what I saw in the manual, which described the formatting as "Splunk loves these files. It eats them like jam (or chocolate)", from http://www.splunk.com/base/Documentation/latest/AppManagement/Getdata.

Well it doesn't like that formatting! I've played with this endlessly, changing commas, replacing = with :...no luck.

Splunk idexes it like this:

1
11/25/10 1:56:07.000 PM
geCount = 0 host=localhost Options| sourcetype=access_combined Options| source=script Options

2
11/25/10 1:56:07.000 PM
Identity = MAILHUB1\Submission, DeliveryType = Undefined, Status = Ready, Messa host=localhost Options| sourcetype=access_combined Options| source=script Options

3
11/25/10 1:56:07.000 PM
Count = 0 host=localhost Options| sourcetype=access_combined Options| source=script Options

Please help! Thanks.

Re: Not automatically recognizing fields

jamesklassen — Fri, 26 Nov 2010 04:04:17 GMT

The data has a new line for each piece of data, I'm not sure why it's not formatted like that above. Each line starts with Identity

Re: Not automatically recognizing fields

jamesklassen — Fri, 26 Nov 2010 04:41:58 GMT

I figured it out. I tried doing it as CSV, where the first line contains the column headings. I set the script sourcetype to csv-4. Here's the powershell formatting:

write-host "Identity, DeliveryType, Status, MessageCount"

$output = $queue.identity.ToString() + ", " + $queue.deliverytype + ", " + $queue.status + ", " + $queue.MessageCount

$output

Splunk now recognizes the field names, and I can alert on when my queues fill up...

Re: Not automatically recognizing fields

gkanapathy — Tue, 30 Nov 2010 04:57:21 GMT

You should either put timestamps in your data, or set DATETIME_CONFIG = CURRENT for your sourcetype.