Splunk Search
Highlighted

How to extract the numeric value and IP address from a string in my sample data?

Explorer

hello,

My log contains below entries.

2017-10-06T04:19:25.658+0000 I NETWORK [initandlisten] connection accepted from 12.34.56.789:12345 #192 (10 connections now open)

I am looking for 2 things.

  1. I want to create a timechart for "Totalconnections". This information will come from the string "(10 connections now open)" and I want to timechart the number 10
  2. I want to count the IPaddress to know how many connections there are per IP.
Highlighted

Re: How to extract the numeric value and IP address from a string in my sample data?

SplunkTrust
SplunkTrust

You need to first capture those IP and connection number into field, like this

your base search | rex "connection accepted from (?<IPAddress>\d+\.\d+\.\d+\.\d+):[^\(]+\((?<ConnectionCount>\d+) connections"

Now you can do total connection timechart like this

above search | timechart sum(ConnectionCount) as TotalConnections

For count of connections per IP address

above search | timechart sum(ConnectionCount) as TotalConnections by IPAddress
Highlighted

Re: How to extract the numeric value and IP address from a string in my sample data?

Explorer

Below query is giving me output 500 to 1100 connections but as per my logs connections are in between 10 to 30.
ring=xxxx source=xxxx "NETWORK" earliest=-4h | rex "connection accepted from (?\d+.\d+.\d+.\d+):[^(]+((?\d+) connections"| timechart sum(ConnectionCount) as TotalConnections

lets not worry about Connection per IP for now, I just need connection count i.e 10
from this string (10 connections now open) because these are real connections.

Here is the log sample:

2017-10-06T04:01:24.889+0000 I NETWORK [conn183] end connection xxx (9 connections now open)

0 Karma
Highlighted

Re: How to extract the numeric value and IP address from a string in my sample data?

SplunkTrust
SplunkTrust

In stats, use max or latest instead of sum.

your base search | rex "connection accepted from (?<IPAddress>\d+\.\d+\.\d+\.\d+):[^\(]+\((?<ConnectionCount>\d+) connections" | timechart max(ConnectionCount) as TotalConnections

OR

your base search | rex "connection accepted from (?<IPAddress>\d+\.\d+\.\d+\.\d+):[^\(]+\((?<ConnectionCount>\d+) connections" | timechart latest(ConnectionCount) as TotalConnections
Highlighted

Re: How to extract the numeric value and IP address from a string in my sample data?

Explorer

Thanks Somesoni!

Can't I get exact connected sessions graph instead of Max/Latest/avg?

Example:
From log entries at 04:05:53.268 I have 12 open connections (I just want to see in my graph 12 at that timestamp) and at 4:19:25.658 I have 10 connections open, so when I do plot a graph I want to see exact count so that I will get idea how many sessions were active at particular time.

2017-10-06T04:05:53.268+0000 I NETWORK [initandlisten] connection accepted from IP:PORT #187 (12 connections now open)

2017-10-06T04:19:25.658+0000 I NETWORK [initandlisten] connection accepted from IP:Port (10 connections now open)

2017-10-06T04:23:55.733+0000 I NETWORK [initandlisten] connection accepted from #193 (10 connections now open)

Sorry I am very new to splunk, we just started using this.

0 Karma
Highlighted

Re: How to extract the numeric value and IP address from a string in my sample data?

SplunkTrust
SplunkTrust

You can actually do this

your base search | rex "connection accepted from (?<IPAddress>\d+\.\d+\.\d+\.\d+):[^\(]+\((?<ConnectionCount>\d+) connections" | table _time ConnectionCount

This will display all the points with corresponding connection count. Please note that there is a limit on how many points can be plotted in the chart so it may not show all points based on how much data you select. See this for more details:
https://docs.splunk.com/Documentation/Splunk/7.0.0/Viz/ChartDisplayissues#Time_charting

0 Karma
Highlighted

Re: How to extract the numeric value and IP address from a string in my sample data?

Explorer

Excellent that worked Somesoni!

But I have multiple hosts on that ring, how do I get per host level?

0 Karma
Highlighted

Re: How to extract the numeric value and IP address from a string in my sample data?

SplunkTrust
SplunkTrust

Try this (will create a new field with same name as value of field host, and that new field will contains corresponding connection count value)

your base search | rex "connection accepted from (?<IPAddress>\d+\.\d+\.\d+\.\d+):[^\(]+\((?<ConnectionCount>\d+) connections" | table _time host ConnectionCount | eval {host}=ConnectionCount | fields - host ConnectionCount
0 Karma
Highlighted

Re: How to extract the numeric value and IP address from a string in my sample data?

Explorer

Somesoni, My graph is not showing as timestamp based, it's giving random time results.

Graph showing like this.
example:
First it showing 13:19:31 sessions count, second 13:48:01 sessions count and then 13:39:03 timestamp sessions count, it just shows random order.

0 Karma
Highlighted

Re: How to extract the numeric value and IP address from a string in my sample data?

SplunkTrust
SplunkTrust

Hi Chandukreddi,

can you please try below search??

YOUR SEARCH | rex "(?<IPAddress>\d+\.\d+\.\d+\.\d+):[^\(]+\((?<ConnectionCount>\d+)" | chart sum(ConnectionCount) as ConnectionCount over _time by  IPAddress

View solution in original post