Getting Data In

Help in parsing Avamar Logs

ansif
Motivator

Hi All,

Please help me to parse this event into key value pair:

Timestamp Hostname and Field name in angle bracket and values

Jul  8 22:02:05 RXXXXXXX001 MCS:BS:    <Code> 30900 <Type> WARNING    <Severity> PROCESS <Category>    APPLICATION <User> root <HwSource>    RXXXXXXX001 <Summary> Activity failed
   - timed out before completion. <Group> SQL_Transaction_Logs_4_Hours    <Action> Scheduled Backup    <status_code> 30900 <starttime>    2019-07-08 23:00:00 <targetCid>    8exxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx9    8xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx9    <account_name>    rxxxxxxxxxxx.xx.xxxxxxx.com <client>    /clients/rxxxxxxxxxxx.xx.xxxxxxx.com    <bytes_modified_sent> 0 <client_name>    rxxxxxxxxxxx.xx.xxxxxxx.com    <errorcode> 10019 <CID>    8exxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx9    <hard_limit> 0 <retention_policy>    14_Days <bytes_protected> 0 <endtime>    2019-07-09 0 3:02:05 <PID> SQL    <plugin_name> Windows SQL    <snapup_number>  <snapup_label>     <schedule> Transaction log 4 HR    <bytes_scanned> 1 <WID> Transaction    log 4 HR-SQL_Transactio    n_Logs_4_Hours-1562626800005 <domain>    /clients <dataset> /Transactional_Log    _4_Hours <account> /clients/rxxxxxxxxxxx.xx.xxxxxxx.com
0 Karma
1 Solution

FrankVl
Ultra Champion

Assuming field values do not contain <> characters, this should be fairly straightforward using props and transforms like so:

props.conf (make sure to set the sourcetype to what you actually use)

[avamar:syslog]
REPORT-key_value_extract = avamar_key_value_extract

transforms.conf

[avamar_key_value_extract]
REGEX = \<([^>]+)\>\s+([^<]*?)\s*(?=\<|$)
FORMAT = $1::$2

https://regex101.com/r/76VHjG/1

You may want to tune the regex a bit, but it is a bit tricky to get a really simple and fast one, that still strips out the spaces around the field values.

View solution in original post

0 Karma

FrankVl
Ultra Champion

Assuming field values do not contain <> characters, this should be fairly straightforward using props and transforms like so:

props.conf (make sure to set the sourcetype to what you actually use)

[avamar:syslog]
REPORT-key_value_extract = avamar_key_value_extract

transforms.conf

[avamar_key_value_extract]
REGEX = \<([^>]+)\>\s+([^<]*?)\s*(?=\<|$)
FORMAT = $1::$2

https://regex101.com/r/76VHjG/1

You may want to tune the regex a bit, but it is a bit tricky to get a really simple and fast one, that still strips out the spaces around the field values.

0 Karma
Get Updates on the Splunk Community!

Splunk Answers Content Calendar, June Edition

Get ready for this week’s post dedicated to Splunk Dashboards! We're celebrating the power of community by ...

What You Read The Most: Splunk Lantern’s Most Popular Articles!

Splunk Lantern is a Splunk customer success center that provides advice from Splunk experts on valuable data ...

See your relevant APM services, dashboards, and alerts in one place with the updated ...

As a Splunk Observability user, you have a lot of data you have to manage, prioritize, and troubleshoot on a ...