Splunk Search

How to count sequence of strings

lpolo
Motivator

I have the following log:

01/02/2011:00:00:01 q=UP
01/02/2011:00:00:02 q=UP A
01/02/2011:00:00:03 q=UP AL
01/02/2011:00:00:04 q=UP ALF
01/02/2011:00:00:05 q=UP ALL

And I would like to have these result set:

q COunt
UP ALF 1
UP ALL 1

Any ideas?
Thanks

Tags (2)
0 Karma
1 Solution

kristian_kolb
Ultra Champion

You should extract the q field by adding to/creating your stanza for the source/sourcetype in props.conf

[your_source_or_sourcetype_here]
EXTRACT-q_string = \sq=(?<qstring>.*)$

Then you can have a search like

... | stats count AS Count by qstring | addcoltotals Count labelfield=qstring label="Total no. queries" 

Which should give you;

qstring             Count
--------------------------
UP                      6
UP ALL                  4
UP AF                   2
Total no. queries      12

EDIT: typo in the field extraction..

Hope this helps,

Kristian

View solution in original post

0 Karma

lpolo
Motivator

After some work I think that this query does the work:

sourcetype="queries" q NOT NULL |
eval q=upper(q) |
eval QueryFirstLetter=substr(q,1,1) |
transaction maxpause=0s QueryFirstLetter delim="," mvlist=q |
eval Query=mvindex(q, -1) |
top limit=0 Query

0 Karma

lpolo
Motivator

I replace the top command by stats. Now it is working.

sourcetype="queries" q NOT NULL |
eval q=upper(q) |
eval QueryFirstLetter=substr(q,1,1) |
transaction maxpause=0s QueryFirstLetter delim="," mvlist=q |
eval Query=mvindex(q, -1) |
stats count by Query|sort - Query

0 Karma

lpolo
Motivator

This query is working fine but If I select a large time period it fails. I will update this notes once I have more information.

0 Karma

kristian_kolb
Ultra Champion

You should extract the q field by adding to/creating your stanza for the source/sourcetype in props.conf

[your_source_or_sourcetype_here]
EXTRACT-q_string = \sq=(?<qstring>.*)$

Then you can have a search like

... | stats count AS Count by qstring | addcoltotals Count labelfield=qstring label="Total no. queries" 

Which should give you;

qstring             Count
--------------------------
UP                      6
UP ALL                  4
UP AF                   2
Total no. queries      12

EDIT: typo in the field extraction..

Hope this helps,

Kristian

0 Karma

kristian_kolb
Ultra Champion

Well, what results did you get?

/k

0 Karma

lpolo
Motivator

It does not work... The query does not return the result set I presented in the example.....

Thanks,

0 Karma

southeringtonp
Motivator

Assuming you always want everything after the equals sign, it's pretty straightforward:

...
| rex field="_raw" "q=(?<querystring>.*)"
| stats count by querystring

Or, you can set up a more permanent field extraction as below, and then use stats without the rex command:

#In transforms.conf...
[qstring]
REGEX = q=(.*)
FORMAT = querystring::$1

#In props.conf...
[putYourSourcetypeHere]
REPORT-qstring = qstring

(search for ... | stats count by querystring )

0 Karma

lpolo
Motivator

Thanks for your response. The issue is not about how extract the value of q.

0 Karma

lpolo
Motivator

That is the actual log. q is a set of query searches. In this example the user typed q in the sequence I presented as a result the intention of the user was UP ALF. Then another user typed UP ALL.
q can be any string.

Thanks

0 Karma

kristian_kolb
Ultra Champion

Hi,

Is that your actual log, or just a sample of what the log could look like? Do the log messages end just after A, AL or ALF etc?

Not 100% sure of the output you want either. Are you only interested in the count of events for ALF and ALL, but not for A or AL?

Are you familiar with field extractions?

Yes, I have ideas, but some more info would be good.

/kristian

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...