All Apps and Add-ons

Syslog events not matching IOS XR regex to transform

notwrkvz
Explorer

Here is the format of our data coming from Cisco IOS XR NCS 4K platform. I don't think the regex is able to match our data. Running Enterprise 7.0 and Cisco Networks Add-on 2.3.4.
Thank you.

Cisco IOS XR Software, Version 6.1.12
Copyright (c) 2013-2016 by Cisco Systems, Inc.

Sample events:

Oct 2 16:04:57 65.230.192.100 222107: HRSHPAXH-0110013A RP/0/RP0:2017 Oct 2 16:04:57.084 UTC: SSHD_[68398]: %SECURITY-SSHD-6-INFO_GENERAL : Enc name is NULL: client aes128-cbc,blowfish-cbc,3des-cbc server aes128-ctr,aes192-ctr,aes256-ctr

Oct 2 16:04:55 65.230.40.4 24078: FLPKNYFP-0330608A LC/0/LC1:Oct 2 12:04:55.531 : fia_driver[118]: %PLATFORM-CIH-5-ASIC_ERROR_THRESHOLD : fia[18]: A generic-err error has occurred causing performance loss transient. CMIC.CMIC_CMC0_IRQ_STAT4.FCT.Interrupt_Register.UnrchDestEvent Threshold has been exceeded

Oct 2 16:04:20 65.230.165.132 47232: GLBONJGB-0114503A RP/0/RP0:2017 Oct 2 12:04:20.587 EDT: smartlicserver[397]: %LIBRARY-REPLICATOR-3-IDT_FAIL : Failed to complete IDT after several retries: rc 0x0 (Success)

0 Karma
1 Solution

rphillips_splk
Splunk Employee
Splunk Employee

@notwrkvz It looks like the developer wrote the regex to capture an additional slot in the node_id which your system does not have (RP/0/RP0) , original regex written to capture node_id like: (RP/0/RP0/x)

If you add the following configurations to your SHs and Indexers your syslog events coming from ios xr 4k should be forced into the sourcetype cisco:ios and then have the field extractions applied

/opt/splunk/etc/apps/TA-cisco_ios/local/transforms.conf

Cisco IOS XR

[force_sourcetype_for_cisco_ios-xr]
REGEX = ((?<reported_hostname>\S+)\s)?(?<event_id>\d+)\:\s((?<reported_hostname2>\S+)\s)?(?<node_id>(?:[A-Z]+)\/(?:\d+)\/(?:[A-Z0-9]+))\:(?<device_time>.+)\s?\:\s?(?<process_name>[A-Za-z0-9_]+)\[(?<pid>\d+)\]\:\s+%(?<category>[A-Za-z0-9_]+)-(?<facility>[A-Za-z0-9_]+)-((?<subfacility>[A-Za-z12_]*(-?[A-Za-z_][^-]*))-?)?(?<severity_id>[0-7])-(?<mnemonic>[A-Z0-9_]+)\s:\s(?<message_text>.+)

Cisco IOS XR

[extract_cisco_ios-general-xr]
REGEX = ((?<reported_hostname>\S+)\s)?(?<event_id>\d+)\:\s((?<reported_hostname2>\S+)\s)?(?<node_id>(?:[A-Z]+)\/(?:\d+)\/(?:[A-Z0-9]+))\:(?<device_time>.+)\s?\:\s?(?<process_name>[A-Za-z0-9_]+)\[(?<pid>\d+)\]\:\s+%(?<category>[A-Za-z0-9_]+)-(?<facility>[A-Za-z0-9_]+)-((?<subfacility>[A-Za-z12_]*(-?[A-Za-z_][^-]*))-?)?(?<severity_id>[0-7])-(?<mnemonic>[A-Z0-9_]+)\s:\s(?<message_text>.+)

restart both SH and indexers after that .. any new events coming in with original sourcetype=syslog should be caught and look correct.

data flow through props/transforms should look like:

  1. original events come into HF or indexer with sourcetype 'syslog'
  2. hits props.conf stanza for [syslog] [syslog] TRANSFORMS-force_sourcetype_for_cisco_ios = force_sourcetype_for_cisco_ios, force_sourcetype_for_cisco_ios-xr, force_sourcetype_for_cisco_ios-xe
  3. hits transforms.conf stanza [force_sourcetype_for_cisco_ios-xr] and sourcetype is re-written to cisco:ios
  4. props.conf for sourcetype stanza [cisco:ios] defines a REPORT in transforms.conf for the field extractions REPORT-cisco_ios-general = extract_cisco_ios-general-xe, extract_cisco_ios-general, extract_cisco_ios-general-xr, extract_cisco_ios-general-wlc, extract_cisco_ios-general-rfc5424
  5. transforms.conf [extract_cisco_ios-general-xr] does field extractions at search time

View solution in original post

rphillips_splk
Splunk Employee
Splunk Employee

@notwrkvz It looks like the developer wrote the regex to capture an additional slot in the node_id which your system does not have (RP/0/RP0) , original regex written to capture node_id like: (RP/0/RP0/x)

If you add the following configurations to your SHs and Indexers your syslog events coming from ios xr 4k should be forced into the sourcetype cisco:ios and then have the field extractions applied

/opt/splunk/etc/apps/TA-cisco_ios/local/transforms.conf

Cisco IOS XR

[force_sourcetype_for_cisco_ios-xr]
REGEX = ((?<reported_hostname>\S+)\s)?(?<event_id>\d+)\:\s((?<reported_hostname2>\S+)\s)?(?<node_id>(?:[A-Z]+)\/(?:\d+)\/(?:[A-Z0-9]+))\:(?<device_time>.+)\s?\:\s?(?<process_name>[A-Za-z0-9_]+)\[(?<pid>\d+)\]\:\s+%(?<category>[A-Za-z0-9_]+)-(?<facility>[A-Za-z0-9_]+)-((?<subfacility>[A-Za-z12_]*(-?[A-Za-z_][^-]*))-?)?(?<severity_id>[0-7])-(?<mnemonic>[A-Z0-9_]+)\s:\s(?<message_text>.+)

Cisco IOS XR

[extract_cisco_ios-general-xr]
REGEX = ((?<reported_hostname>\S+)\s)?(?<event_id>\d+)\:\s((?<reported_hostname2>\S+)\s)?(?<node_id>(?:[A-Z]+)\/(?:\d+)\/(?:[A-Z0-9]+))\:(?<device_time>.+)\s?\:\s?(?<process_name>[A-Za-z0-9_]+)\[(?<pid>\d+)\]\:\s+%(?<category>[A-Za-z0-9_]+)-(?<facility>[A-Za-z0-9_]+)-((?<subfacility>[A-Za-z12_]*(-?[A-Za-z_][^-]*))-?)?(?<severity_id>[0-7])-(?<mnemonic>[A-Z0-9_]+)\s:\s(?<message_text>.+)

restart both SH and indexers after that .. any new events coming in with original sourcetype=syslog should be caught and look correct.

data flow through props/transforms should look like:

  1. original events come into HF or indexer with sourcetype 'syslog'
  2. hits props.conf stanza for [syslog] [syslog] TRANSFORMS-force_sourcetype_for_cisco_ios = force_sourcetype_for_cisco_ios, force_sourcetype_for_cisco_ios-xr, force_sourcetype_for_cisco_ios-xe
  3. hits transforms.conf stanza [force_sourcetype_for_cisco_ios-xr] and sourcetype is re-written to cisco:ios
  4. props.conf for sourcetype stanza [cisco:ios] defines a REPORT in transforms.conf for the field extractions REPORT-cisco_ios-general = extract_cisco_ios-general-xe, extract_cisco_ios-general, extract_cisco_ios-general-xr, extract_cisco_ios-general-wlc, extract_cisco_ios-general-rfc5424
  5. transforms.conf [extract_cisco_ios-general-xr] does field extractions at search time

notwrkvz
Explorer

Thank you very much for getting that regex correct! That totally fixed it and the data is being transformed now.

Best regards,
Alan

0 Karma

DalJeanis
Legend

What is the regex?

0 Karma
Get Updates on the Splunk Community!

New in Observability - Improvements to Custom Metrics SLOs, Log Observer Connect & ...

The latest enhancements to the Splunk observability portfolio deliver improved SLO management accuracy, better ...

Improve Data Pipelines Using Splunk Data Management

  Register Now   This Tech Talk will explore the pipeline management offerings Edge Processor and Ingest ...

3-2-1 Go! How Fast Can You Debug Microservices with Observability Cloud?

Register Join this Tech Talk to learn how unique features like Service Centric Views, Tag Spotlight, and ...