Eventgen - ERROR ExecProcessor - message from "python /opt/splunk/etc/apps/SA-Eventgen/bin/modinput_eventgen.py"

[astatrial]

Hi all,
For some reason i have this error in splunkd.log and there are no logs being generated from other applications which have eventgen.conf and samples dir.

Did anyone now how to solve this problem.

I suspect that this error is due to permissions but i checked all the permissions and everything is fine.

Here is an more detailed example for the log:

DEBUG    MainProcess {'event': 'Using cached earliest time: 2019-09-15 16:06:20.961619'}
09-15-2019 16:07:20.970 +0300 ERROR ExecProcessor - message from "python /opt/splunk/etc/apps/SA-Eventgen/bin/modinput_eventgen.py" 2019-09-15 16:07:20 eventgen        DEBUG    MainProcess {'event': "Flushing queue for sample 'nessus_singlehost.samples' with size 60"}

09-15-2019 16:27:24.664 +0300 ERROR ExecProcessor - message from "python /opt/splunk/etc/apps/SA-Eventgen/bin/modinput_eventgen.py" 2019-09-15 16:27:24 eventgen        DEBUG    MainProcess {'event': "Flushing queue for sample 'symantec_ep_scm_agent_act.samples' with size 2"}

Thanks in advanced !

[lwu_splunk]

Could you share your eventgen.conf ?

[astatrial]

Of course:

# Copyright (C) 2005-2015 Splunk Inc. All Rights Reserved.
# DO NOT EDIT THIS FILE!
# Please make all changes to files in $SPLUNK_HOME/etc/apps/SA-Eventgen/local.
# To make changes, copy the section/stanza you want to change from $SPLUNK_HOME/etc/apps/SA-Eventgen/default
# into ../local and edit there.
#

## IMPORTANT! Do not specify any settings under a default stanza
## The layering system will not behave appropriately
## Use [global] instead
[default]

[global]
disabled = false
debug = false
verbosity = false
spoolDir = $SPLUNK_HOME/var/spool/splunk
spoolFile = <SAMPLE>
breaker = [^\r\n\s]+
mode = sample
sampletype = raw
interval = 60
delay = 0
timeMultiple = 1
count = -1
earliest = now
latest = now
randomizeEvents = false
outputMode = modinput
fileMaxBytes = 10485760
fileBackupFiles = 5
splunkPort = 8089
splunkMethod = https
index = main
source = eventgen
sourcetype = eventgen
host = 127.0.0.1
generator = default
rater = config
generatorWorkers = 1
outputWorkers = 1
timeField = _raw
threading = thread
profiler = false
maxIntervalsBeforeFlush = 3
maxQueueLength = 0
useOutputQueue = false
autotimestamps = [["\\d{4}-\\d{2}-\\d{2} \\d{2}:\\d{2}:\\d{2}", "%Y-%m-%d %H:%M:%S"], ["\\d{1,2}\\/\\w{3}\\/\\d{4}\\s\\d{2}:\\d{2}:\\d{2}:\\d{1,3}", "%d/%b/%Y %H:%M:%S:%f"], ["\\d{4}-\\d{2}-\\d{2}T\\d{2}:\\d{2}:\\d{2}\\.\\d{3}", "%Y-%m-%dT%H:%M:%S.%f"], ["\\d{1,2}/\\w{3}/\\d{4}\\s\\d{2}:\\d{2}:\\d{2}:\\d{1,3}", "%d/%b/%Y %H:%M:%S:%f"], ["\\d{1,2}/\\d{2}/\\d{2}\\s\\d{1,2}:\\d{2}:\\d{2}", "%m/%d/%y %H:%M:%S"], ["\\d{2}-\\d{2}-\\d{4} \\d{2}:\\d{2}:\\d{2}", "%m-%d-%Y %H:%M:%S"], ["\\w{3} \\w{3} +\\d{1,2} \\d{2}:\\d{2}:\\d{2}", "%a %b %d %H:%M:%S"], ["\\w{3} \\w{3} \\d{2} \\d{4} \\d{2}:\\d{2}:\\d{2}", "%a %b %d %Y %H:%M:%S"], ["^(\\w{3}\\s+\\d{1,2}\\s\\d{2}:\\d{2}:\\d{2})", "%b %d %H:%M:%S"], ["(\\w{3}\\s+\\d{1,2}\\s\\d{1,2}:\\d{1,2}:\\d{1,2})", "%b %d %H:%M:%S"], ["(\\w{3}\\s\\d{1,2}\\s\\d{1,4}\\s\\d{1,2}:\\d{1,2}:\\d{1,2})", "%b %d %Y %H:%M:%S"], ["\\d{4}-\\d{2}-\\d{2} \\d{2}:\\d{2}:\\d{2}\\.\\d{3}", "%Y-%m-%d %H:%M:%S.%f"], ["\\,\\d{2}\\/\\d{2}\\/\\d{2,4}\\s+\\d{2}:\\d{2}:\\d{2}\\s+[AaPp][Mm]\\,", ",%m/%d/%Y %I:%M:%S %p,"], ["^\\w{3}\\s+\\d{2}\\s+\\d{2}:\\d{2}:\\d{2}", "%b %d %H:%M:%S"], ["\\d{2}/\\d{2}/\\d{4} \\d{2}:\\d{2}:\\d{2}", "%m/%d/%Y %H:%M:%S"], ["^\\d{2}\\/\\d{2}\\/\\d{2,4}\\s+\\d{2}:\\d{2}:\\d{2}\\s+[AaPp][Mm]", "%m/%d/%Y %I:%M:%S %p"], ["\\d{2}\\/\\d{2}\\/\\d{4}\\s\\d{2}:\\d{2}:\\d{2}", "%m-%d-%Y %H:%M:%S"], ["\\\"timestamp\\\":\\s\\\"(\\d+)", "%s"], ["\\d{2}\\/\\w+\\/\\d{4}\\s\\d{2}:\\d{2}:\\d{2}:\\d{3}", "%d-%b-%Y %H:%M:%S:%f"], ["\\\"created\\\":\\s(\\d+)", "%s"], ["\\d{4}-\\d{2}-\\d{2}T\\d{2}:\\d{2}:\\d{2}", "%Y-%m-%dT%H:%M:%S"], ["\\d{1,2}/\\w{3}/\\d{4}:\\d{2}:\\d{2}:\\d{2}:\\d{1,3}", "%d/%b/%Y:%H:%M:%S:%f"], ["\\d{1,2}/\\w{3}/\\d{4}:\\d{2}:\\d{2}:\\d{2}", "%d/%b/%Y:%H:%M:%S"]]
autotimestamp = false
httpeventWaitResponse = true
disableLoggingQueue = true

This is the default eventgen.conf of the eventgen app.

The symantec eventgen.conf is also the one shipped with the add on.

Thanks

[lwu_splunk]

I mean the eventgen.conf in your symantec app.

[astatrial]

It is a really long file, it is the default of the "Splunk_TA_symantec-ep".

[lwu_splunk]

Actually the error msg above is DEBUG msg. I could not see any ERROR from the log. I have checked with the eventgen.conf in Splunk_TA_symantec-ep. Seems every config is fine to generate the data. Could you change the time range in Splunk search and check the events?

[lwu_splunk]

It seems by default all the stanzas in eventgen.conf in app Splunk_TA_symantec-ep are disabled. You should manually enable them. Change disabled = 1 to disabled = 0.

[astatrial]

If you will look closely, there is "ERROR ExecProcessor".
There is disabled=1 for specific stanzas.
I did the same process on another machine without any further configurations and it worked fine.
In addition the problem is not just with symantec, but with every other app with eventget.conf

I just need to fix this in the other machine (it is not an option to replace it).

Thanks

[lwu_splunk]

The ERROR ExecProcessor is misleading that we need to fix for Eventgen. But it is not error log actually.

[astatrial]

So do you know what may be the reason that eventgen can't generate events from other apps files ?

On the other machine that works fine i don't get those logs.

[lwu_splunk]

I am not sure since there is not enough info for me. A few key points:
1. Make sure the symantec add-on has permission. http://splunk.github.io/eventgen/SETUP.html
2. Make sure the Eventgen modular input is enabled;
3. Search in Splunk with all time filter;

[astatrial]

I want to correct myself.
I get the same logs on the machine that it does work on.
So the reason is apparently something else.
But i still can't seem to find the problem.

[lwu_splunk]

I can schedule a short meeting with you. Send your available time to me: lwu@splunk.com.
(I am on GMT+8 timezone)