Security

Cisco CSA Indexing Issue

cvajs
Contributor

v4.3.1 on sles 11.1

i have my cisco csa v5.2.0.278 alerts going to splunk via udp 162 (snmp traps)

the raw data (via splunk view) looks like this
0\x82\u0002\xC3\u0002\u0001\u0001\u0004\u0006public\xA7\x82\u0002\xB4\u0002\u0003\u001b3\xFA\u0002\u0001

and the Client Security App doesnt display anything. i set the source type to cisco_csa, so why this App not working?

the raw data from tcpdump for a snmp trap is:
16:56:05.188160 IP (tos 0x0, ttl 128, id 24807, offset 0, flags [none], proto UDP (17), length 762) venom.prod.org.cplscrambler-lg > SPLUNK.PROD.ORG.snmptrap: { SNMPv2c { V2Trap(715) R=1783164 system.sysUpTime.0=116503972 S:1.1.4.1.0=E:8590.3.1 E:8590.2.1=10317498 E:8590.2.2=1374 E:8590.2.3="HOSTB.prod.org" E:8590.2.4="2012-04-04 16:56:00.000" E:8590.2.5=2 E:8590.2.6=179 E:8590.2.7= E:8590.2.8= E:8590.2.9= E:8590.2.10= E:8590.2.11="The 'Service Control Manager' service logged event code 7036 into the system event log: The Ati HotKey Poller service entered the stopped state. " E:8590.2.12=1658 E:8590.2.13="10.132.194.174" E:8590.2.14="W" E:8590.2.15= E:8590.2.16= E:8590.2.17="NT Event log" E:8590.2.18="" E:8590.2.19=280 E:8590.2.20="CSA Service Monitoring" E:8590.2.21= E:8590.2.22= E:8590.2.23=0 } }

Tags (4)
0 Karma

cvajs
Contributor

Splunk shows just this for the source data of an event (why does it truncate the data?)

2012-04-05 15:38:29 syslog.prod.org [UDP: [10.219.0.134]:1086->[10.222.1.253]]:

but the raw log file shows this for same event

2012-04-05 15:38:29 syslog.prod.org [UDP: [10.219.0.134]:1086->[10.222.1.253]]:
DISMAN-EVENT-MIB::sysUpTimeInstance = Timeticks: (124678725) 14 days, 10:19:47.25       SNMPv2-MIB::snmpTrapOID.0 = OID: CSAMC-SNMPv2-MIB::csaTraps.1        CSAMC-SNMPv2-MIB::eventID = INTEGER: 10329635   CSAMC-SNMPv2-MIB::ruleID = Wrong Type (should be INTEGER): NULL      CSAMC-SNMPv2-MIB::hostName = STRING: "hostB.prod.org"   CSAMC-SNMPv2-MIB::eventTime = STRING: "2012-04-05 15:38:29.617"      CSAMC-SNMPv2-MIB::severityCode = INTEGER: 2     CSAMC-SNMPv2-MIB::eventCode = INTEGER: 164   CSAMC-SNMPv2-MIB::processName = STRING: "<remote application>"  CSAMC-SNMPv2-MIB::fileName = Wrong Type (should be OCTET STRING): NULL       CSAMC-SNMPv2-MIB::sourceIPAddress = Wrong Type (should be IpAddress): NULL      CSAMC-SNMPv2-MIB::destinationIPAddress = Wrong Type (should be IpAddress): NULL      CSAMC-SNMPv2-MIB::eventText = STRING: "The process '<remote application>' has triggered too many log records in the last few minutes. Further messages will be logged at a decreased rate for 10 minutes."        CSAMC-SNMPv2-MIB::hostID = INTEGER: 209 CSAMC-SNMPv2-MIB::currentHostIPAddress = Wrong Type (should be IpAddress): STRING: "10.132.194.158"  CSAMC-SNMPv2-MIB::hostOSType = STRING: "W"      CSAMC-SNMPv2-MIB::sourcePort = Wrong Type (should be INTEGER): NULL  CSAMC-SNMPv2-MIB::destinationPort = Wrong Type (should be INTEGER): NULL        CSAMC-SNMPv2-MIB::eventType = STRING: "Administrative"       CSAMC-SNMPv2-MIB::ruleDescription = Wrong Type (should be OCTET STRING): NULLCSAMC-SNMPv2-MIB::ruleModuleID = Wrong Type (should be INTEGER): NULL   CSAMC-SNMPv2-MIB::ruleModuleName = Wrong Type (should be OCTET STRING): NULL CSAMC-SNMPv2-MIB::buttonCode = Wrong Type (should be INTEGER): NULL     CSAMC-SNMPv2-MIB::userName = STRING: "myDOMAIN\\WSecGat_Px"   CSAMC-SNMPv2-MIB::flags = INTEGER: 0
DISMAN-EVENT-MIB::sysUpTimeInstance = Timeticks: (124679186) 14 days, 10:19:51.86       SNMPv2-MIB::snmpTrapOID.0 = OID: CSAMC-SNMPv2-MIB::csaTraps.1        CSAMC-SNMPv2-MIB::eventID = INTEGER: 10329637   CSAMC-SNMPv2-MIB::ruleID = INTEGER: 1374    CSAMC-SNMPv2-MIB::hostName = STRING: "hostA.prod.org"    CSAMC-SNMPv2-MIB::eventTime = STRING: "2012-04-05 15:38:29.999"      CSAMC-SNMPv2-MIB::severityCode = INTEGER: 2     CSAMC-SNMPv2-MIB::eventCode = INTEGER: 179      CSAMC-SNMPv2-MIB::processName = Wrong Type (should be OCTET STRING): NULL    CSAMC-SNMPv2-MIB::fileName = Wrong Type (should be OCTET STRING): NULL       CSAMC-SNMPv2-MIB::sourceIPAddress = Wrong Type (should be IpAddress): NULL      CSAMC-SNMPv2-MIB::destinationIPAddress = Wrong Type (should be IpAddress): NULL      CSAMC-SNMPv2-MIB::eventText = STRING: "The 'Service Control Manager' service logged event code 7036 into the system event log: The LiveUpdate service entered the running state. "   CSAMC-SNMPv2-MIB::hostID = INTEGER: 2206     CSAMC-SNMPv2-MIB::currentHostIPAddress = Wrong Type (should be IpAddress): STRING: "10.10.10.10"   CSAMC-SNMPv2-MIB::hostOSType = STRING: "W"      CSAMC-SNMPv2-MIB::sourcePort = Wrong Type (should be INTEGER): NULL CSAMC-SNMPv2-MIB::destinationPort = Wrong Type (should be INTEGER): NULL CSAMC-SNMPv2-MIB::eventType = STRING: "NT Event log"CSAMC-SNMPv2-MIB::ruleDescription = ""   CSAMC-SNMPv2-MIB::ruleModuleID = INTEGER: 280   CSAMC-SNMPv2-MIB::ruleModuleName = STRING: "CSA Service Monitoring"  CSAMC-SNMPv2-MIB::buttonCode = Wrong Type (should be INTEGER): NULL     CSAMC-SNMPv2-MIB::userName = Wrong Type (should be OCTET STRING): NULL       CSAMC-SNMPv2-MIB::flags = INTEGER: 
0 Karma

cvajs
Contributor

i think the field extractions of the app are wrong for my version of CSA, and, possibly loading the CSA MIB is mucking up the event data. i cant find any info about using the app MIB with snmptrapd, etc. i would think using the MIB would be preferred.

as example, the field "hostName" is pulling pulling out 'STRING: "hostname"' when i think it should just be 'hostname' etc

0 Karma

cvajs
Contributor

ok, i copied props.conf from default to local in the CSA app.

here's my props now in local dir
[cisco_csa]
TRANSFORMS = csa_hostoverride
REPORT-extract = csafields
SHOULD_LINEMERGE = true
TRUNCATE = 0
BREAK_ONLY_BEFORE = ^(\d{4}-\d{2}-\d{2}\s\d{2}:\d{2}:\d{2})

i cleaned the index to clear the junk, and now get events in full, but CSA app returns no data for any of its reports.

whats weird is that splunk shows two eventtype fields:

host=myHost |sourcetype=cisco_csa |eventtype=cisco_csa |eventtype=cisco_syslog_types cisco firewall network

0 Karma

Ayn
Legend

Yes, you need to configure line merging in props.conf.

0 Karma

cvajs
Contributor

so i need to tell the preview that my event is "DATE \n EVENT_DATA \n"
(not that way exactly, but you get my point). oddly, if i write snmptrapd out to syslog then Splunk will read those events from syslog file fine, but CSA app still doesnt work with that data.

0 Karma

Ayn
Legend

I don't know in detail how the preview data functionality works so I don't know, sorry. But, the default behaviour for event breaking in Splunk is that the first time a timestamp is encountered on a new line, Splunk breaks and creates a new event.

0 Karma

cvajs
Contributor

i dont quite understand. if i manually add the source and do a "Preview" of the log data it shows the events correctly, so why would it mess up event breaking after that?

0 Karma

Ayn
Legend

I think you need to configure event breaking. Splunk is probably breaking events because it finds what it views as timestamps after the first line you pasted, and default behaviour is for Splunk to create a new event when it finds a valid timestamp on a new line.

0 Karma

cvajs
Contributor

anyone have an idea as to why splunk does this?

0 Karma

cvajs
Contributor

TRUNCATE=0 to default props in CSA app, but Splunk still shows:
2012-04-09 11:24:34 myHost.prod.org [UDP: [10.8.0.134]:1086->[10.1.1.53]]:

props.conf
[cisco_csa]
TRANSFORMS = csa_hostoverride
REPORT-extract = csafields
SHOULD_LINEMERGE = false
TRUNCATE=0

transfroms.conf
[csafields]
REGEX = ^[^\|]+\|([^\|]+)\|([^\|]+)\|([^\|]+)\|([^\|]+)\|([^\|]+)\|([^\|]+)
FORMAT = nbtname::$1 ip::$2 ruleid::$3 code::$4 remotetime::$5 alert::$6

[csa_hostoverride]
DEST_KEY = MetaData:Host
REGEX = ^[^\|]+\|([^\|]+)\|([^\|]+)\|([^\|]+)\|([^\|]+)\|([^\|]+)\|([^\|]+)
FORMAT = host::$1

0 Karma

MarioM
Motivator

you could try adding TRUNCATE=0 in your cisco_csa stanza

0 Karma

cvajs
Contributor

which props?

i dont have a system/local props
i dont have a local app props

i have this props
/opt/splunk/etc/apps/Splunk_CiscoClientSecurityAgent/default

[cisco_csa]
TRANSFORMS = csa_hostoverride
REPORT-extract = csafields
SHOULD_LINEMERGE = false

0 Karma

MarioM
Motivator

what your props.conf looks like?

0 Karma

cvajs
Contributor

this looks like a bug. when i create the source from file and preview it, Splunk correctly shows each event, but when i finish setting up the source and look at the data from a search it only shows that 1st line, not all of the event data.

0 Karma

MarioM
Motivator

snmp traps are binary then your have to convert it in ascii : receive and index SNMP traps

0 Karma

cvajs
Contributor

i read the directions, still doesnt work. the raw log file has a bunch of ascii data but Splunk only shows part of it for some reason. when i say "syslog-ng" i mean i use -Ls with snmptrapd instead of -Lf, this allows me to have better control over where the data goes, etc.

snmptrapd -M /usr/share/snmp/mibs -m +ALL -Lf /logs/snmp/hostA --disableAuthorization=yes

or

snmptrapd -M /usr/share/snmp/mibs -m +ALL -Ls 0 --disableAuthorization=yes

the latter cmd logs the same data as the 1st, but Splunk seems to have a problem showing/parsing the 1st.

0 Karma

MarioM
Motivator

i think you didnot read the instructions that exactly same but snmptrapd with splunk instead of syslog-ng...

0 Karma

cvajs
Contributor

i am working on alternate solution, using snmptrapd and my syslog-ng. snmptrapd is not dynamic enough to handle numerous snmp traps from different hosts, etc. thnx.

0 Karma

MarioM
Motivator

and tcpdump actually decode snmp traps,Splunk donot...

0 Karma

MarioM
Motivator

as per my answer to index snmp traps you need to follow the instructions from the link.

0 Karma
Register for .conf21 Now! Go Vegas or Go Virtual!

How will you .conf21? You decide! Go in-person in Las Vegas, 10/18-10/21, or go online with .conf21 Virtual, 10/19-10/20.