I'm running some script to gather logs every 10 mins, one of them is Cisco ASA VPN-sessiondb info, I'd like to use Splunk to generate
1. Number of users timechart
2. bandwidth consumption per user timechart
I'm having challenge with the multi-line log looks like the following
FIREWALLNAME# show clock
10:09:50.431 PDT Mon May 15 2017
FIREWALLNAME# show vpn-sessiondb anyconnect sort p-ipaddress
Session Type: AnyConnect
Username : User1 Index : 159
Assigned IP : x.x.x.x Public IP : y.y.y.y
Protocol : AnyConnect-Parent SSL-Tunnel DTLS-Tunnel
License : AnyConnect Premium
Encryption : AnyConnect-Parent: (1)RC4 SSL-Tunnel: (1)AES128 DTLS-Tunnel: (1)AES128
Hashing : AnyConnect-Parent: (1)SHA1 SSL-Tunnel: (1)SHA1 DTLS-Tunnel: (1)SHA1
Bytes Tx : 31430848 Bytes Rx : 2918291
Group Policy : VPN_Service Tunnel Group : DefaultWEBVPNGroup
Login Time : 08:27:04 PDT Mon May 15 2017
Duration : 1h:42m:48s
Inactivity : 0h:00m:00s
VLAN Mapping : N/A VLAN : none
Audt Sess ID : ac12121c0009f0005919c8c8
Security Grp : none
Username : User2 Index : 157
Assigned IP : z.z.z.z Public IP : v.v.v.v
Protocol : AnyConnect-Parent SSL-Tunnel DTLS-Tunnel
License : AnyConnect Premium
Encryption : AnyConnect-Parent: (1)RC4 SSL-Tunnel: (1)AES128 DTLS-Tunnel: (1)AES128
Hashing : AnyConnect-Parent: (1)SHA1 SSL-Tunnel: (1)SHA1 DTLS-Tunnel: (1)SHA1
Bytes Tx : 52439947 Bytes Rx : 11136783
Group Policy : VPN_Service Tunnel Group : DefaultWEBVPNGroup
Login Time : 08:14:04 PDT Mon May 15 2017
Duration : 1h:55m:48s
Inactivity : 0h:00m:00s
VLAN Mapping : N/A VLAN : none
Audt Sess ID : ac12121c0009d0005919c5bc
Security Grp : none
Omitting more users
So far I was able to extract the usernames ^Username\s+:\s+(?P\w+\s+)
but unable to generate a timechart i believe because it's multi-line
I'm also trying to extract the "Bytes Tx" and "Bytes Rx" per user to calculate the Diff's (The commands run every 10 mins) and use it to calculate the average bandwidth consumption.
First, try to follow adonio's suggestion, because a TA will handle lots of stuff for you.
Second, if that doesn't work, then
1) extract the fields manually with rex
into individual MV fields
2) mvzip
the multivalue fields together so that all the related fields are attached to each other
3) use mvexpand
to break each one into a separate record
4) use another rex
or a makemv
(depending on how you mvzipped them) to break them into separate fields again.
Once you've done all that, then you'll probably need to
5) sort
the records into order (remember to use sort 0
to retain all results).
6) use delta
(or more likely streamstats
) to calculate the difference for the period
7) use some kind of smoothing to deal with each user's first record and/or last record.
this might work out for you , but doesn't include the deltas between collections..you could look at streamstats for that.
sourcetype="vpn-stats" "show clock" | rex field=_raw max_match=0 "Username\s+\:\s+(?<Username>\S+)" | rex field=_raw max_match=0 "Bytes\s+Tx\s+\:\s+(?<Bytes_tx>\d+)" | rex field=_raw max_match=0 "Bytes\s+Rx\s+\:\s+(?<Bytes_rx>\d+)" | eval zipped = mvzip(Username, Bytes_tx, ",") | eval zipped = mvzip(zipped, Bytes_rx, ",") | mvexpand zipped | makemv delim="," zipped | eval Username = mvindex(zipped, 0) | eval Bytes_tx = mvindex(zipped, 1) | eval Bytes_rx = mvindex(zipped, 2) | timechart max(Bytes_rx) AS bytes_rx max(Bytes_tx) AS bytes_tx by Username
First, try to follow adonio's suggestion, because a TA will handle lots of stuff for you.
Second, if that doesn't work, then
1) extract the fields manually with rex
into individual MV fields
2) mvzip
the multivalue fields together so that all the related fields are attached to each other
3) use mvexpand
to break each one into a separate record
4) use another rex
or a makemv
(depending on how you mvzipped them) to break them into separate fields again.
Once you've done all that, then you'll probably need to
5) sort
the records into order (remember to use sort 0
to retain all results).
6) use delta
(or more likely streamstats
) to calculate the difference for the period
7) use some kind of smoothing to deal with each user's first record and/or last record.
Thanks for your input DalJeanis.
I've tried to follow the steps but somehow it didn't seperate the fields into their own lines and i can't draw a graph with it.
Here's the search string I used
sourcetype="vpn-stats" "show clock" | rex field=_raw "Username : (?<VPNUser>\w+)" max_match=0 | rex field=_raw "Bytes Tx : (?<VPNTX>\w+)" max_match=0 | eval fields = mvzip(VPNUser,VPNTX) | mvexpand fields | eval fields = mvzip(VPNUser,VPNTX) | table _time fields
The result is like following
_time fields
5/15/17 2:57 PM User1,641674831
User2,92081181
User3,52384414
…
5/15/17 2:57 PM User1,641674800
User2,92081100
User3,52384400
…
5/15/17 2:57 PM …
Maybe my mvexpand was wrong?
You did well up thru step 3, and misread step 4. I've added one more field and a sample rex to your code...
sourcetype="vpn-stats" "show clock"
| rex field=_raw "Username : (?<VPNUser>\w+)" max_match=0
| rex field=_raw "Bytes Tx : (?<VPNTX>\w+)\s+Bytes Rx\s+: (?<VPNRX>\w+)" max_match=0
| eval myfields = mvzip(mvzip(VPNUser,VPNTX),VPNRX)
| mvexpand myfields
| rex field=myfield "(?<VPNUser>[^,]+),(?<VPNTX>[^,]+),(?<VPNRX>.*)"
| table _time, myfield, VPNUser, VPNTX, VPNRX
Thanks DalJeanis for your quick response. Looks like its working
i believe using the Cisco ASA TA will help with all fields extractions