Why is Eventgen not finding files in my samples directory?

[jbouch03]

Hi Everyone,

I'm having a strange issue in my eventgen. I've created eventgens before, but it's been awhile so I'm probably missing something basic. The problem seems to be that it can't find my samples directory files. Nothing shows up inside of Splunk, so I'm trying to run it through the command line and I get the following output:

WARNING module='config' sample='null': Sample 'hr_data.sample' in config but no matching files

Here is the full output from the command line instance:

python $SPLUNK_HOME/etc/apps/eventgen/bin/eventgen.py -v $SPLUNK_HOME/etc/apps/myapp/default/eventgen.conf 

2016-02-01 08:24:43,945 INFO module='main' sample='null': Starting eventgen
2016-02-01 08:24:44,946 INFO module='config' sample='null': Retrieving eventgen configurations with ConfigParser()
2016-02-01 08:24:44,955 WARNING module='config' sample='null': Sample 'hr_data.sample' in config but no matching files
2016-02-01 08:24:44,958 INFO module='config' sample='null': Starting OutputWorker 0
2016-02-01 08:24:44,958 INFO module='config' sample='null': Starting GeneratorWorker 0
2016-02-01 08:24:45,052 INFO module='main' sample='null': OutputQueueDepth=0  GeneratorQueueDepth=0 GeneratorsPerSec=0 OutputtersPerSec=0
2016-02-01 08:24:45,052 INFO module='main' sample='null': GlobalEventsPerSec=0.0 KilobytesPerSec=0.000000 GigabytesPerDay=0.000000
2016-02-01 08:24:45,053 INFO module='config' sample='null': Caught kill, exiting...
2016-02-01 08:24:45,053 INFO module='OutputRealWorker' sample='null': Stopping OutputThreadWorker 0
2016-02-01 08:24:45,053 INFO module='GeneratorRealWorker' sample='null': Stopping GeneratorThreadWorker 0
2016-02-01 08:24:45,053 INFO module='config' sample='null': Exiting main thread.

My hr_data.sample file is located in $SPLUNK_HOME/etc/apps/myapp/samples/ and my eventgen.conf is the following:

[hr_data.sample]
sampletype = raw
mode = sample
interval = 60
earliest = -60s
latest = now
count = 100
outputMode = splunkstream

## replace timestamp 01:12:01
token.0.token = \d{2}:\d{2}:\d{2}
token.0.replacementType = timestamp
token.0.replacement = %H:%M:%S

## replace HR value
token.1.token =  "value":\s"(\d{2})"
token.1.replacementType = random
token.1.replacement = integer[20:220]

The hr_data.sample file is the following:

{"activities-heart": [{"dateTime": "today", "customHeartRateZones": [], "heartRateZones": [{"max": 93, "caloriesOut": 1.24524, "minutes": 1, "min": 30, "name": "Out of Range"}, {"max": 130, "caloriesOut": 0, "minutes": 0, "min": 93, "name": "Fat Burn"}, {"max": 158, "caloriesOut": 0, "minutes": 0, "min": 130, "name": "Cardio"}, {"max": 220, "caloriesOut": 0, "minutes": 0, "min": 158, "name": "Peak"}], "value": "85"}], "activities-heart-intraday": {"datasetInterval": 1, "datasetType": "minute", "dataset": [{"time": "09:26:00", "value": 85}]}}

Again, I'm probably just missing something obvious, but any help you can provide would be greatly appreciated. Thanks!

[Richfez]

I had this problem as well and it was resolved when I did the following:

Stop Splunk, delete the $SPLUNK_HOME/var/log/splunk/eventgen* files, then start Splunk back up again. When I checked the eventgen.log file immediately afterwards, I found lines like

2016-07-25 09:04:00,866 WARNING module='Sample' sample='sample_network.csv': Can't find a timestamp (using patterns '['\\w{3}\\s\\d{2}\\s\\d{2}:\\d{2}:\\d{2}']') in this event: 'Jul 25 08:34:23 10.128.255.23 :%ASA-session-6-302016 ....

The fix seemed to be to edit my eventgen.conf file and changing the token.X.replacement to be %b %d %H:%ML%S which made it match the original log lines. See, a quick glance through that config made me believe that the token.X.replacement could be anything able to be parsed by Splunk so I had one of the more standard "Month-Day-Year Hours:Minutes:Seconds" style entries there.

But it seems that's not the case, that it has to actually match the formatting of the original lines. So I stopped Splunk again, made the change to have it be like below, then deleted the log files and restarted it. All was well after that.

token.1.token = \w{3}\s\d{2}\s\d{2}:\d{2}:\d{2}
token.1.replacementType = timestamp
token.1.replacement = %b %d %H:%M:%S

As full disclosure, I also made a change around line 630 of $SPLUNK_HOME/etc/apps/eventgen/lib/eventgenconfig.py that changed a log line from "debug" to "info" when it was listing the sample files found, but that shouldn't have changed anything - and indeed I did not even see those log lines in there afterward anyway so I'm pretty sure it did nothing.

[phr0zbyte]

i'm experiencing something similar.. configs should be good on my side..