All Apps and Add-ons

My Cisco UCS syslog events are being overridden, how can I see them in the app?

halr9000
Motivator

(This is from a question received over email.)

I’ve configured everything as in the readme file.
Yet when receiving the syslog messeges the SPLUNK still discovers the data as cisco:ios :

Field   Value
host    x.x.x.x
source  udp:514
sourcetype  cisco:ios
app network
device_time 2014 Dec 15 13:51:06 UTC
dvc x.x.x.x
eventtype   cisco_ios (ciscoiosnetwork)
facility    DAEMON
index   main
linecount   1
message_text    DHCPACK on x.x.x.x to aa:bb:af:af:2f:dd via vlan4044 - dhcpd
mnemonic    SYSTEM_MSG
product IOS
reliable_time   TRUE
reported_hostname   x.x.x.x
severity    informational
severity_description    Informational message only
severity_id 6
severity_id_and_name    6 - informational
severity_name   informational
splunk_server   splunk2
splunk_server_group dmc_group_indexer
tag cisco

Any idea how to solve this issue ?

0 Karma

mikaelbje
Motivator

This will always be a problem when vendors use a common logging format without any identifier to specify the technology type the event is sent from.

Another way to solve this is to create a new UDP input on a different port on your Splunk server with:


sourcetype = ciscoucs:syslog
index = cisco_ucs

Set up your UCS servers to sent their syslogs to this port.

0 Karma

halr9000
Motivator

@mikaelbje afraid that's not possible, UCS does not support alternate ports!

But they could differentiate the events in another way. For example, send all UCS events to a unique IP endpoint, distinct from others. The IP could simply be an additional address for the very same host, but that unique IP would show up in the event somewhere and could be used as a search filter. Or, use a syslog server and a heavy forwarder, etc, etc. Lots of ways.

Source: UCS docs

0 Karma

mikaelbje
Motivator

Ouch, in that case your IP alias tip is a way to go. It might get a little bit messy if you end up with lots of IPs and ports, but I still think it beats customizing downloaded apps. It also saves you CPU cycles if you use using port/IP combinations instead of regex to categorize data 🙂

0 Karma

halr9000
Motivator

This is a challenge with any app which overrides fields that another app relies on. There are ways around it. Luckily, there’s actually not a ton of reliance on syslog events in the current UCS app.

What I recommend in the short term is to go into the Splunk_KB_CiscoUCS app. Make a local/ folder if you haven’t already. Open default/eventtypes.conf and copy the first three stanzas out to local/eventtypes.conf:

[ucs-syslog]
search = index=cisco_ucs sourcetype="ciscoucs:syslog"
description = Cisco UCS events received via syslog

[ucs-syslog-event]
search = index=cisco_ucs sourcetype="ciscoucs:syslog" mnemonic="EVENT"
description = Cisco UCS events received via syslog

[ucs-syslog-audit]
search = index=cisco_ucs sourcetype="ciscoucs:syslog" mnemonic="AUDIT"
description = Cisco UCS events received via syslog

And in each search line above, alter it to match what you need it to be to account for the IOS app’s overriding. I suggest perhaps a combination of sourcetype and host fields. Don’t forget that you can use macros here as well. So you could add a new macro, say call it cisco_ucs_manager_manager_hosts. Then your search lines might look like:

search= sourcetype=cisco:ios ` cisco_ucs_manager_manager_hosts`

I ship some macros in the main dashboard app, so you may want to put your macros there--but they can be stored anywhere that's in context for the users of the app. But for example, in SplunkAppForCiscoUCS/local/macros.conf:

[cisco_ucs_manager_manager_hosts]
definition = host=ucsmanager*

Or,

definition = host=host1 OR host=host2 OR host=host3

Obviously, you’ll need to edit the lines to suit your environment.

Register for .conf21 Now! Go Vegas or Go Virtual!

How will you .conf21? You decide! Go in-person in Las Vegas, 10/18-10/21, or go online with .conf21 Virtual, 10/19-10/20.