Splunk Search

How to generate a search to find the number of users and their bandwidth consumption using timechart?

barryy
Explorer

I'm running some script to gather logs every 10 mins, one of them is Cisco ASA VPN-sessiondb info, I'd like to use Splunk to generate
1. Number of users timechart
2. bandwidth consumption per user timechart

I'm having challenge with the multi-line log looks like the following

FIREWALLNAME# show clock
10:09:50.431 PDT Mon May 15 2017
FIREWALLNAME# show vpn-sessiondb anyconnect sort p-ipaddress 

Session Type: AnyConnect

Username     : User1                  Index        : 159
Assigned IP  : x.x.x.x           Public IP    : y.y.y.y
Protocol     : AnyConnect-Parent SSL-Tunnel DTLS-Tunnel
License      : AnyConnect Premium
Encryption   : AnyConnect-Parent: (1)RC4  SSL-Tunnel: (1)AES128  DTLS-Tunnel: (1)AES128
Hashing      : AnyConnect-Parent: (1)SHA1  SSL-Tunnel: (1)SHA1  DTLS-Tunnel: (1)SHA1
Bytes Tx     : 31430848               Bytes Rx     : 2918291
Group Policy : VPN_Service        Tunnel Group : DefaultWEBVPNGroup
Login Time   : 08:27:04 PDT Mon May 15 2017
Duration     : 1h:42m:48s
Inactivity   : 0h:00m:00s
VLAN Mapping : N/A                    VLAN         : none
Audt Sess ID : ac12121c0009f0005919c8c8
Security Grp : none

Username     : User2               Index        : 157
Assigned IP  : z.z.z.z           Public IP    : v.v.v.v
Protocol     : AnyConnect-Parent SSL-Tunnel DTLS-Tunnel
License      : AnyConnect Premium
Encryption   : AnyConnect-Parent: (1)RC4  SSL-Tunnel: (1)AES128  DTLS-Tunnel: (1)AES128
Hashing      : AnyConnect-Parent: (1)SHA1  SSL-Tunnel: (1)SHA1  DTLS-Tunnel: (1)SHA1
Bytes Tx     : 52439947               Bytes Rx     : 11136783
Group Policy : VPN_Service        Tunnel Group : DefaultWEBVPNGroup
Login Time   : 08:14:04 PDT Mon May 15 2017
Duration     : 1h:55m:48s
Inactivity   : 0h:00m:00s
VLAN Mapping : N/A                    VLAN         : none
Audt Sess ID : ac12121c0009d0005919c5bc
Security Grp : none

Omitting more users

So far I was able to extract the usernames ^Username\s+:\s+(?P\w+\s+) but unable to generate a timechart i believe because it's multi-line
I'm also trying to extract the "Bytes Tx" and "Bytes Rx" per user to calculate the Diff's (The commands run every 10 mins) and use it to calculate the average bandwidth consumption.

0 Karma
1 Solution

DalJeanis
Legend

First, try to follow adonio's suggestion, because a TA will handle lots of stuff for you.

Second, if that doesn't work, then
1) extract the fields manually with rex into individual MV fields
2) mvzip the multivalue fields together so that all the related fields are attached to each other
3) use mvexpand to break each one into a separate record
4) use another rex or a makemv (depending on how you mvzipped them) to break them into separate fields again.

Once you've done all that, then you'll probably need to
5) sort the records into order (remember to use sort 0 to retain all results).
6) use delta (or more likely streamstats) to calculate the difference for the period
7) use some kind of smoothing to deal with each user's first record and/or last record.

View solution in original post

0 Karma

rphillips_splk
Splunk Employee
Splunk Employee

this might work out for you , but doesn't include the deltas between collections..you could look at streamstats for that.

sourcetype="vpn-stats" "show clock" | rex field=_raw max_match=0 "Username\s+\:\s+(?<Username>\S+)" | rex field=_raw max_match=0 "Bytes\s+Tx\s+\:\s+(?<Bytes_tx>\d+)" | rex field=_raw max_match=0 "Bytes\s+Rx\s+\:\s+(?<Bytes_rx>\d+)"  | eval zipped = mvzip(Username, Bytes_tx, ",") | eval zipped = mvzip(zipped, Bytes_rx, ",") | mvexpand zipped | makemv delim="," zipped | eval Username = mvindex(zipped, 0) | eval Bytes_tx = mvindex(zipped, 1) | eval Bytes_rx = mvindex(zipped, 2)  | timechart max(Bytes_rx) AS bytes_rx max(Bytes_tx) AS bytes_tx by Username
0 Karma

DalJeanis
Legend

First, try to follow adonio's suggestion, because a TA will handle lots of stuff for you.

Second, if that doesn't work, then
1) extract the fields manually with rex into individual MV fields
2) mvzip the multivalue fields together so that all the related fields are attached to each other
3) use mvexpand to break each one into a separate record
4) use another rex or a makemv (depending on how you mvzipped them) to break them into separate fields again.

Once you've done all that, then you'll probably need to
5) sort the records into order (remember to use sort 0 to retain all results).
6) use delta (or more likely streamstats) to calculate the difference for the period
7) use some kind of smoothing to deal with each user's first record and/or last record.

0 Karma

barryy
Explorer

Thanks for your input DalJeanis.

I've tried to follow the steps but somehow it didn't seperate the fields into their own lines and i can't draw a graph with it.

Here's the search string I used

sourcetype="vpn-stats" "show clock" | rex field=_raw "Username : (?<VPNUser>\w+)" max_match=0 | rex field=_raw "Bytes Tx : (?<VPNTX>\w+)" max_match=0 | eval fields = mvzip(VPNUser,VPNTX) | mvexpand fields | eval fields = mvzip(VPNUser,VPNTX) | table _time fields

The result is like following

_time                   fields
5/15/17 2:57 PM     User1,641674831
                        User2,92081181
                        User3,52384414
                        …
5/15/17 2:57 PM     User1,641674800
                        User2,92081100
                        User3,52384400
                        …
5/15/17 2:57 PM     …

Maybe my mvexpand was wrong?

0 Karma

DalJeanis
Legend

You did well up thru step 3, and misread step 4. I've added one more field and a sample rex to your code...

sourcetype="vpn-stats" "show clock" 
| rex field=_raw "Username : (?<VPNUser>\w+)" max_match=0 
| rex field=_raw "Bytes Tx : (?<VPNTX>\w+)\s+Bytes Rx\s+: (?<VPNRX>\w+)" max_match=0 
| eval myfields = mvzip(mvzip(VPNUser,VPNTX),VPNRX) 
| mvexpand myfields 
| rex field=myfield "(?<VPNUser>[^,]+),(?<VPNTX>[^,]+),(?<VPNRX>.*)" 
| table _time, myfield, VPNUser, VPNTX, VPNRX
0 Karma

barryy
Explorer

Thanks DalJeanis for your quick response. Looks like its working

adonio
Ultra Champion

i believe using the Cisco ASA TA will help with all fields extractions

0 Karma
Get Updates on the Splunk Community!

How to Monitor Google Kubernetes Engine (GKE)

We’ve looked at how to integrate Kubernetes environments with Splunk Observability Cloud, but what about ...

Index This | How can you make 45 using only 4?

October 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with this ...

Splunk Education Goes to Washington | Splunk GovSummit 2024

If you’re in the Washington, D.C. area, this is your opportunity to take your career and Splunk skills to the ...