topic Re: DEBUG AggregatorMiningProcessor - Failed to parse timestamp getting this message in splunkd.log in Getting Data In

DEBUG AggregatorMiningProcessor - Failed to parse timestamp getting this message in splunkd.log

Hemnaath — Tue, 29 Sep 2020 11:41:05 GMT

Hi All, I could this message into my Heavy Forwarder instance (Splunkd.log) I am not sure what is the problem why I am getting this information in my splunkd.log. We are using Splunk 6.2.1 version and its running in Linux 64 bit instance VM machine. Kindly guide me on how to fix this issue, as I am very much beginner in splunk.

splunkd.log
11-06-2016 10:58:38.108 -0500 DEBUG AggregatorMiningProcessor - Failed to parse timestamp. Defaulting to time specified by data input. - data_source="/opt/syslogs/proxy/uspxxxx.xxxx.com/bluecoat.log", data_host="uspxxxx.xxxx.com", data_sourcetype="bluecoat_syslog"
11-06-2016 10:58:38.109 -0500 DEBUG AggregatorMiningProcessor - Failed to parse timestamp. Defaulting to time specified by data input. - data_source="/opt/syslogs/proxy/uspxxxx.xxxx.com/bluecoat.log", data_host="uspxxxx.xxxx.com", data_sourcetype="bluecoat_syslog"
11-06-2016 10:58:38.109 -0500 DEBUG AggregatorMiningProcessor - Failed to parse timestamp. Defaulting to time specified by data input. - data_source="/opt/syslogs/proxy/uspxxxx.xxxx.com/bluecoat.log", data_host="uspxxxx.xxxx.com", data_sourcetype="bluecoat_syslog"
11-06-2016 10:58:38.109 -0500 DEBUG AggregatorMiningProcessor - Failed to parse timestamp. Defaulting to time specified by data input. - data_source="/opt/syslogs/proxy/uspxxxx.xxxx.com/bluecoat.log", data_host="uspxxxx.xxxx.com", data_sourcetype="bluecoat_syslog"
11-06-2016 10:58:38.109 -0500 DEBUG AggregatorMiningProcessor - Failed to parse timestamp. Defaulting to time specified by data input. - data_source="/opt/syslogs/proxy/uspxxxx.xxxx.com/bluecoat.log", data_host="uspxxxx.xxxx.com", data_sourcetype="bluecoat_syslog"

thanks in advance.

Re: DEBUG AggregatorMiningProcessor - Failed to parse timestamp getting this message in splunkd.log

mattymo — Sun, 06 Nov 2016 16:59:27 GMT

Hey Hemnaath,

Splunk is just advising you that it cannot auto parse your timestamp in your bluecoat logs and is differing to the sourcetype set for that input.

what does your bluecoat props.conf look like?

Re: DEBUG AggregatorMiningProcessor - Failed to parse timestamp getting this message in splunkd.log

Hemnaath — Tue, 29 Sep 2020 11:41:08 GMT

thanks mmodestino for your quick response on this. I could see two props.conf file for bluecoat_syslog. One is under the app name called TA-Bluecoat and Another app name Admin-HVY-Forwarder.

Under TA-Bluecoat app I do not see any inputs.conf file defined, whereas under Admin-HVY-Forwarder could see inputs.conf defined but props.conf is not defined for bluecoat.

App name Admin-HVY-Forwarder - Props.conf
[host::Tesx*]
TZ = GMT

[host::TESX*]
TZ = GMT

[f5_web_server]
TIME_PREFIX = f5_time="
TRANSFORM-time = f5_syslog_time

Under app name TA-bluecoat, could see this configuration setup
Props.conf detail
[source::....bluecoat]
sourcetype = bluecoat

[bluecoat]
SHOULD_LINEMERGE=false
KV_MODE = none
REPORT-0auto_kv_for_bluecoat = auto_kv_for_bluecoat
LOOKUP-vendor_info_for_bluecoat = bluecoat_vendor_info_lookup sourcetype OUTPUT vendor,product
MAX_TIMESTAMP_LOOKAHEAD = 19
TIME_FORMAT = %Y-%m-%d %T
TRANSFORM-main = nullPound
TRANSFORMS-bluecoat_host_override = bluecoat_host_override
TZ = GMT

[bluecoat_syslog]
SHOULD_LINEMERGE=false
KV_MODE = none
REPORT-0auto_kv_for_bluecoat = auto_kv_for_bluecoat_syslog
LOOKUP-vendor_info_for_bluecoat = bluecoat_vendor_info_lookup sourcetype OUTPUT vendor,product
TIME_PREFIX = \w{3}\s\d{1,2}\s\d{1,2}:\d{1,2}:\d{1,2}\s\w+.\w+.\w+.
TIME_FORMAT = %Y-%m-%d %H:%M:%S
MAX_TIMESTAMP_LOOKAHEAD = 19
TRANSFORM-main = nullPound
TRANSFORMS-bluecoat_host_override = bluecoat_host_override

thanks in advance.

Re: DEBUG AggregatorMiningProcessor - Failed to parse timestamp getting this message in splunkd.log

mattymo — Sun, 06 Nov 2016 18:11:45 GMT

looks like ur all good! This is just a debug message telling you how splunk is setting the timestamp.

Are you running debug log level?

Re: DEBUG AggregatorMiningProcessor - Failed to parse timestamp getting this message in splunkd.log

Hemnaath — Sun, 06 Nov 2016 18:14:18 GMT

thanks mmodestino, but how to figure out whether we are running the debug log level in splunk ?

Re: DEBUG AggregatorMiningProcessor - Failed to parse timestamp getting this message in splunkd.log

mattymo — Sun, 06 Nov 2016 18:47:26 GMT

You likely arent...what does your inputs.conf look like for this heavy forwarder?

Re: DEBUG AggregatorMiningProcessor - Failed to parse timestamp getting this message in splunkd.log

Hemnaath — Tue, 29 Sep 2020 11:41:10 GMT

Taken only particular stanza related to bluecoat_syslogs from Admin-HVY-forwarder app

/opt/splunk/etc/apps/Admin-HVY-forwarder/default

[monitor:///opt/syslogs/proxy/.../*bluecoat.log]
whitelist = .log$
sourcetype = bluecoat_syslog
index = net_proxy
host_segment = 4

Re: DEBUG AggregatorMiningProcessor - Failed to parse timestamp getting this message in splunkd.log

mattymo — Sun, 06 Nov 2016 19:56:37 GMT

you set up looks fine to me...I am pretty sure these messages can be disregarded as they are simply verbose debug logs. Your timestamping is working correctly, right?

Re: DEBUG AggregatorMiningProcessor - Failed to parse timestamp getting this message in splunkd.log

Hemnaath — Mon, 07 Nov 2016 12:33:51 GMT

thanks mmodestino for throwing some lights on this issue.