Getting Data In

Why is Powershell generated CSV data that is monitored only getting indexed once and is not indexed again until a Splunk restart?

jamesklassen
Path Finder

I've got an extremely frustrating problem here, at my wit's end and finally coming here.

I've got CSV files being generated by Powershell every 15 minutes. Splunk is set to look at the directory and pull in the CSV files' data. It pulls it in ONCE, and then does not pull the data in again until after Splunk is restarted. What on earth is going on here?

The CSV files look like this, for example:

2016-09-22-13-56-sfbClientConnections.csv

First few lines of the CSV look like this:

"Time","Username","ClientVersion","Fqdn"
"2016-09-22-13:56","user1@domain.com","UCCAPI/15.0.4849.1000 OC/15.0.4849.1000 (Skype for Business)","server1"
"2016-09-22-13:56","user2@domain.com","UCCAPI/15.0.4849.1000 OC/15.0.4849.1000 (Skype for Business)","server2"

inputs.conf looks like this...I've tried with and without the crcSalt:

[monitor://C:\Scripts\SkypeForBusinessClientConnections\logs\*csv]
disabled = false
host = mailmon2
index = sfb
sourcetype = HydroMessaging:SFB:ClientConnections
crcSalt = <SOURCE>

props.conf looks like this:

[HydroMessaging:SFB:ClientConnections]
NO_BINARY_CHECK = 1
pulldown_type = 1
HEADER_MODE = firstline
FIELD_DELIMITER=,
FIELD_QUOTE="
TIME_FORMAT=%Y-%m-%d-%H:%M
TIMESTAMP_FIELDS=Time
0 Karma

lycollicott
Motivator

Try a monitorNoHandle stanza instead of just monitor. (http://docs.splunk.com/Documentation/Splunk/6.4.3/Data/Monitorfilesanddirectories)

0 Karma
Career Survey
First 500 qualified respondents will receive a $20 gift card! Tell us about your professional Splunk journey.

Can’t make it to .conf25? Join us online!

Get Updates on the Splunk Community!

Take Action Automatically on Splunk Alerts with Red Hat Ansible Automation Platform

 Are you ready to revolutionize your IT operations? As digital transformation accelerates, the demand for ...

Calling All Security Pros: Ready to Race Through Boston?

Hey Splunkers, .conf25 is heading to Boston and we’re kicking things off with something bold, competitive, and ...

Beyond Detection: How Splunk and Cisco Integrated Security Platforms Transform ...

Financial services organizations face an impossible equation: maintain 99.9% uptime for mission-critical ...