Getting Data In

Why is Powershell generated CSV data that is monitored only getting indexed once and is not indexed again until a Splunk restart?

jamesklassen
Path Finder

I've got an extremely frustrating problem here, at my wit's end and finally coming here.

I've got CSV files being generated by Powershell every 15 minutes. Splunk is set to look at the directory and pull in the CSV files' data. It pulls it in ONCE, and then does not pull the data in again until after Splunk is restarted. What on earth is going on here?

The CSV files look like this, for example:

2016-09-22-13-56-sfbClientConnections.csv

First few lines of the CSV look like this:

"Time","Username","ClientVersion","Fqdn"
"2016-09-22-13:56","[email protected]","UCCAPI/15.0.4849.1000 OC/15.0.4849.1000 (Skype for Business)","server1"
"2016-09-22-13:56","[email protected]","UCCAPI/15.0.4849.1000 OC/15.0.4849.1000 (Skype for Business)","server2"

inputs.conf looks like this...I've tried with and without the crcSalt:

[monitor://C:\Scripts\SkypeForBusinessClientConnections\logs\*csv]
disabled = false
host = mailmon2
index = sfb
sourcetype = HydroMessaging:SFB:ClientConnections
crcSalt = <SOURCE>

props.conf looks like this:

[HydroMessaging:SFB:ClientConnections]
NO_BINARY_CHECK = 1
pulldown_type = 1
HEADER_MODE = firstline
FIELD_DELIMITER=,
FIELD_QUOTE="
TIME_FORMAT=%Y-%m-%d-%H:%M
TIMESTAMP_FIELDS=Time
0 Karma

lycollicott
Motivator

Try a monitorNoHandle stanza instead of just monitor. (http://docs.splunk.com/Documentation/Splunk/6.4.3/Data/Monitorfilesanddirectories)

0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

Supercharging Windows Security Detection Performance: Introducing Hybrid Field ...

Windows event logs—from Security auditing and Sysmon to PowerShell script blocks—form the operational backbone ...

Ditch the Manual Grind: Building AI Agents with Splunk

Ditch the Manual Grind: Building AI Agents with Splunk Let’s be real: your team’s time is being eaten alive. ...

Cisco Data Fabric from Architecture to Investigation, Better SOC Visibility, and More ...

Splunk Lantern is Splunk’s customer success center that provides practical guidance from Splunk experts on key ...