When running the health check for the monitoring console I am consistently receiving event-processing issues (date_parsing_issues)
I have added a stanza to /opt/splunk/etc/system/local for the props.conf file and the errors are still persistent.
[default]
DEPTH_LIMIT = 100000
maxDist = 300
MAX_TIMESTAMP_LOOKAHEAD = 256
TRUNCATE = 99999999
MAX_EVENTS = 99999999
DATE_CONFIG = CURRENT
[cisco:ios]
DATETIME_CONFIG = CURRENT
TRANSFORMS-force_sourcetype_cisco_traceback = force_sourcetype_cisco_traceback
SHOULD_LINEMERGE=false
KV_MODE = none
SHOULD_LINEMERGE = false
LINE_BREAKER = ([\r\n]+)
TRUNCATE = 99999999
MAX_EVENTS = 99999999
[cisco_syslog]
DATETIME_CONFIG = CURRENT
pulldown_type = 0
MAX_TIMESTAMP_LOOKAHEAD = 128
SHOULD_LINEMERGE = False
TRANSFORMS = syslog-host
TRUNCATE = 99999999
MAX_EVENTS = 99999999
REPORT-syslog = syslog-extractions
any recommendations would be appreciated. thank you
this is an example of the syslog information with time stamp info
024044: Mar 23 11:24:08.831 UTC: %LINK-3-UPDOWN: Interface GigabitEthernet2/0/6, changed state to down
024045: Mar 23 11:24:11.201 UTC: %LINK-3-UPDOWN: Interface GigabitEthernet2/0/6, changed state to up
As per my comments on your question, you don't want to use TRUNCATE or DATE_CONFIG - certainly not in default, and not for these logs.
Try the following props settings - I encourage you to change the default section back to its defaults, but this could have implications if you have logs indexed with current time presently.
[default]
DEPTH_LIMIT = 100000
maxDist = 300
MAX_TIMESTAMP_LOOKAHEAD = 256
#TRUNCATE = 999999999
#MAX_EVENTS = 99999999
#DATE_CONFIG = CURRENT
[cisco:YOUR-ST]
#DATETIME_CONFIG = CURRENT
TRANSFORMS-force_sourcetype_cisco_traceback = force_sourcetype_cisco_traceback
SHOULD_LINEMERGE=false
KV_MODE = none
SHOULD_LINEMERGE = false
LINE_BREAKER = ([\r\n]+)
TIME_PREFIX=^\d+\:\s
TIME_FORMAT=%b %d %H:%M:%S.%N3 %Z
MAX_TIMESTAMP_LOOKAHEAD=23
#TRUNCATE = 99999999
#MAX_EVENTS = 99999999
As per my comments on your question, you don't want to use TRUNCATE or DATE_CONFIG - certainly not in default, and not for these logs.
Try the following props settings - I encourage you to change the default section back to its defaults, but this could have implications if you have logs indexed with current time presently.
[default]
DEPTH_LIMIT = 100000
maxDist = 300
MAX_TIMESTAMP_LOOKAHEAD = 256
#TRUNCATE = 999999999
#MAX_EVENTS = 99999999
#DATE_CONFIG = CURRENT
[cisco:YOUR-ST]
#DATETIME_CONFIG = CURRENT
TRANSFORMS-force_sourcetype_cisco_traceback = force_sourcetype_cisco_traceback
SHOULD_LINEMERGE=false
KV_MODE = none
SHOULD_LINEMERGE = false
LINE_BREAKER = ([\r\n]+)
TIME_PREFIX=^\d+\:\s
TIME_FORMAT=%b %d %H:%M:%S.%N3 %Z
MAX_TIMESTAMP_LOOKAHEAD=23
#TRUNCATE = 99999999
#MAX_EVENTS = 99999999
Truncate=999999999
is a terrible idea. It means you could conceivably end up with events of ~100mb in Splunk (which would be very bad)
The default limit is 10,000 bytes - if you are receiving Truncation warnings at 10k you almost certainly have line breaking issues that setting Truncate higher will simply mask or make worse.
DATE_CONFIG=CURRENT
is also another terrible idea.
It tells Splunk to ignore the timestamps in your event logs, and instead 'approximate' the event time to that which your indexers processed the event. This will make correlating events in chronological difficult, especially if you have multiple indexers. The ONLY time (imho) you should use this is when you have logs which do not provide timestamp information - in these cases you first should kick your vendors/developers (hard) to introduce a time to their logs before using this setting.
appreciate your input!