Addon Props.conf configuration

[imheejin]

Hello!
I ask you to check if the props.conf I wrote is appropriate.

1. Data

{"subscription_id": "ec7d6887-675d-46d6", "maximum": 109133.0, "namespace": "microsoft.dbformariadb/servers", "unit": "Bytes", "_time": "2020-10-29T06:36:00Z", "average": 109133.0, "host": "/subscriptions/ec7d6887-675d-46d6/resourceGroups/RG-T/providers/Microsoft.DBforMariaDB/servers/azure-mariadb", "metric_name": "serverlog_storage_usage", "minimum": 109133.0} 

2. index="_internal" host="VM-KC" log_level!=INFO (*fail* OR *extract*)

ARN DateParserVerbose - Failed to parse timestamp in first MAX_TIMESTAMP_LOOKAHEAD (128) characters of event. Defaulting to timestamp of previous event (Thu Oct 29 14:02:00 2020). Context: source=azure_metrics://MariaDB|host=VM-KC|azure:metrics|

3. Line Breaking Error

ERROR LineBreakingProcessor - Line breaking regex has no capturing groups: \}\} - data_source="/monitoring/scouter/server/ext_plugin_filelog/scouter-counter-javaee.json", data_host="VM-KC", data_sourcetype="scouter_json"

4. Timestamp Parsing Error

A possible timestamp match (Fri Sep 10 00:41:19 2010) is outside of the acceptable time window. If this timestamp is correct, consider adjusting MAX_DAYS_AGO and MAX_DAYS_HENCE.

WARN DateParserVerbose - A possible timestamp match (Fri Sep 10 00:41:19 2010) is outside of the acceptable time window. If this timestamp is correct, consider adjusting MAX_DAYS_AGO and MAX_DAYS_HENCE. Context: source=azure_metrics://MariaDB|host=VM-KC|azure:metrics|

<props.conf Sample>

 index = "azure" source = "azure_metrics : // MariaDB" sourcetype = "azure : metrics"

 [source :: azure_metrics : // MariaDB]

DATETIME_CONFIG = CURRENT

BREAK_ONLY_BEFORE_DATE = true

NO_BINARY_CHECK = true

MAX_TIMESTAMP_LOOKAHEAD=200

[richgalloway]

You don't say what the props.conf settings need to be appropriate for, but my answer is no, they are not because they do not address all of the conditions listed.  Try these settings.

[source :: azure_metrics : // MariaDB]
# Should resolve "Failed to parse timestamp in first 128 characters"
TIME_PREFIX = _time":"
TIME_FORMAT = %Y-%m-%dT%H:%M:%S%Z
# Look-ahead starts at TIME_PREFIX so we only need 20 characters
MAX_TIMESTAMP_LOOKAHEAD = 20
# There is a line breaking error, but no LINE_BREAKER attribute in props so this value is just a guess
LINE_BREAKER = \}\}-()
# Assuming we want to keep events from 2010
MAX_DAYS_AGO = 3650
SHOULD_LINEMERGE = false
TRUNCATE = 10000