Dashboards & Visualizations

How to count Unique User Agents in the message log?

davidsumner
Engager

I would like to setup a dashboard that tracks the totals for user agents in incoming requests.  I couldn't find a "user agent", "user-agents" or any other field listed.  When I exported the search results to CSV, I saw the following heading: 

"_raw","_time",cloudaccount,host,index,linecount,message,source,sourcetype,"splunk_server"

It appears that the info containing the user agent is contained in the message field, enclosed in double double-quotes.

I assume that in order to count each type of user agent, I first need to isolate these values, then count them.  What's the best way to do that?

Labels (2)
Tags (1)
0 Karma
1 Solution

richgalloway
SplunkTrust
SplunkTrust

This works with the sample event

 

<<your query>>
| rex "https?\S+\s\\\"+(?<UA>[^\\\"]+)"
| stats count by UA

 

---
If this reply helps you, Karma would be appreciated.

View solution in original post

somesoni2
Revered Legend

Give this a try

Find count of events by userAgent

Your base search
| rex "\]\s+(\"+[^\"]+){3}\"+\s+\"+(?<userAgent>[^\"]+)"
| stats count by userAgent

 

 Trend of distinct count of userAgents over time

Your base search
| rex "\]\s+(\"+[^\"]+){3}\"+\s+\"+(?<userAgent>[^\"]+)"
| timechart dc(userAgent) as distinct_userAgents
0 Karma

davidsumner
Engager

When I did a table view I realized that the data I'm looking for is actually part of the _raw field.  I'm trying to figure out how isolate the user-agent portion (in bold) and count the different unique values that get reported.  Here's a sanitized version of a record.

"<133>1 2023-09-21T14:53:43+00:00 host-29490 example.apache-access - - - 208.207.1.214 - - [21/Sep/2023:14:53:43 +0000] ""GET / HTTP/1.1"" 302 46779 ""https://edit.onlineshop.example.com/"" ""Mozilla/5.0 (Windows NT 6.2; WOW64) AppleWebKit/537.4 (KHTML, like Gecko) Chrome/98 Safari/537.4 (StatusCake)"" vhost=example.prod.my-sites.com host=edit.example.com hosting_site=example pid=24164 request_time=106271 forwarded_for=""208.51.62.14, 64.220.85.15, 23.120.51.94"" request_id=""reqid-a88558b0-5a8e-1ee-6e0-ea57887e2d"" location=""/user/login"" ","2023-09-21T10:53:43.000-04:00",778910529448,"52.22.171.60",application,1,,example,"tcp-raw","splunk-indexer-ip-10-128-128-5.ec2.internal"

0 Karma

richgalloway
SplunkTrust
SplunkTrust

This works with the sample event

 

<<your query>>
| rex "https?\S+\s\\\"+(?<UA>[^\\\"]+)"
| stats count by UA

 

---
If this reply helps you, Karma would be appreciated.

richgalloway
SplunkTrust
SplunkTrust

Since rex is my favorite command, I'll say that's the best way to extract the field.  Of course, that depends on the details.  Please share (sanitized) sample events with message fields.

---
If this reply helps you, Karma would be appreciated.
Get Updates on the Splunk Community!

More Control Over Your Monitoring Costs with Archived Metrics!

What if there was a way you could keep all the metrics data you need while saving on storage costs?This is now ...

New in Observability Cloud - Explicit Bucket Histograms

Splunk introduces native support for histograms as a metric data type within Observability Cloud with Explicit ...

Updated Team Landing Page in Splunk Observability

We’re making some changes to the team landing page in Splunk Observability, based on your feedback. The ...