topic Re: Creating a simple chart from extracting specific string in Splunk Search

Creating a simple chart from extracting specific string

philallen1 — Tue, 15 Oct 2013 17:05:11 GMT

This should be nice and easy for you lot.

I have an application producing thousands of logs a day. In some of these logs there will be the phrase: "IncomeData SCV link" and some of the other logs there will be the phrase: "IncomeData started in". The two phrases will never be in the same log.
They are also in different positions within the log, so I think field extractions is out of the question)

I want to simply produce a column chart like below, that counts how many logs have the first phrase in and how many have the second phrase in.

I use the below search to pull out all the logs, but I can't finish it off to produce the chart.

sourcetype="UserLogs" Username=* "IncomeData SCV link" OR "IncomeData started in"

Re: Creating a simple chart from extracting specific string

jtrucks — Tue, 15 Oct 2013 17:34:25 GMT

By "how many logs" do you mean a count of events? so in your UserLogs sourcetype, say you had 1000 entries with "IncomeData SCV link" and 1500 entries with "IncomeData started in" you want the chart like you showed to reflet those numbers?

Re: Creating a simple chart from extracting specific string

_d_ — Tue, 15 Oct 2013 17:35:21 GMT

The basic unit of performing statistics in splunk is called a field. In your case you need to make those phrases (or more precisely, the presence of those phrases in events) into fields before you can build charts/tables. Here is one way of doing it:

sourcetype="UserLogs" Username=* "IncomeData SCV link" OR "IncomeData started in" | rex "(?<phrase>IncomeData SCV link)" | rex "(?<phrase>IncomeData started in)" | stats count by phrase

Re: Creating a simple chart from extracting specific string

somesoni2 — Tue, 15 Oct 2013 17:47:16 GMT

You can try this

sourcetype="userlogs" UserName=* "IncomeData SCV link" | stats count as "IncomeData SCV link" | appendcols [search sourcetype="userlogs" UserName=* "IncomeData started in" | stats count as "IncomeData started in"]

Re: Creating a simple chart from extracting specific string

jtrucks — Tue, 15 Oct 2013 17:47:19 GMT

I would do it like this (with one rex):

sourcetype="UserLogs" Username=* "IncomeData SCV link" OR "IncomeData started in" | rex field=_raw ".*\s(?<phrase>IncomeData (SCV link|started in))\s.*" | stats count by phrase

This is slightly more efficient and on large data volumes will be more efficient (faster).

Re: Creating a simple chart from extracting specific string

_d_ — Tue, 15 Oct 2013 17:55:29 GMT

Actually, the backtracking due to (two!) greedy quantifiers (.*) as well as the alternation (|) within the capture group make that a very inefficient regex. But, it does work.

Re: Creating a simple chart from extracting specific string

philallen1 — Wed, 16 Oct 2013 10:08:40 GMT

This worked perfectly - thank you

Re: Creating a simple chart from extracting specific string

philallen1 — Wed, 16 Oct 2013 10:09:11 GMT

Yes, that's exactly right. It's the count of events.