All Apps and Add-ons

Aggregated reporting on a field with variable value

Engager

Hi, I am trying to create a scheduled report that will count the number of events for specific keywords. The event entry looks somewhat like:

10 Mar 2013 10:10:35,559 MyApplicationProcessor : MyProcess PIDNo [123456] MyProcessGroup [PRG_NM1] FailureRetryCount[x] ExceptionRetryCount [y] Finishing Process

10 Mar 2013 10:10:35,123 MyApplicationProcessor : MyProcess PIDNo [123456] MyProcessGroup [PRG_NM2] FailureRetryCount[x] ExceptionRetryCount [y] Finishing Process

10 Mar 2013 10:10:35,234 MyApplicationProcessor : MyProcess PIDNo [123454] MyProcessGroup [PRG_NM1] FailureRetryCount[x] ExceptionRetryCount [y] Exit

10 Mar 2013 10:10:35,234 MyApplicationProcessor : MyProcess PIDNo [123454] MyProcessGroup [PRG_NM1] FailureRetryCount[x] ExceptionRetryCount [y] Finishing Process

10 Mar 2013 10:10:35,234 MyApplicationProcessor : MyProcess PIDNo [123454] MyProcessGroup [PRG_NM1] FailureRetryCount[x] ExceptionRetryCount [y] Enter

The PIDNo,x,y, PRG_NM etc are variables I am particularly looking to create a report based on PRD_NM where the even ends with Finishing Process. The PRG_NM can be any of the 100+ possible values.

The reports needs to look like:

PRG_NM1 - 35

PRG_NM2 - 43

PRG_NM3 - 21

PRG_NM4 - 42

Any help is greatly appreciated.

0 Karma
1 Solution

SplunkTrust
SplunkTrust

You could do something like this:

your base search | rex "\[(?<prg>PRG.*?)\]" | stats count by prg

If you can control the format of that log it's nicer to have these PRG_NM1 fields printed in key=value format, something like this:

10 Mar 2013 10:10:35,559 processor=MyApplicationProcessor process=MyProcess pid=PIDNo something=123456 processGroup=MyProcessGroup prg=PRG_NM1 failureRetryCount=x exceptionRetryCount=y message="Finishing Process"

Then Splunk will understand all the fields automatically, and humans will still understand the log as well.

View solution in original post

SplunkTrust
SplunkTrust

For charting over time you usually want to use timechart like so:

... | timechart span=1h count

0 Karma

SplunkTrust
SplunkTrust

Use chart command instead of stats..

...| chart count over prg by date_hour

0 Karma

Engager

Awesome. Thank you for the quick response. Much appreciated.

Unfortunately, the events are logged by a legacy program and we have very little control on how and what is being logged

can this be modify to make it look like below (where h=current hour)? I tried stats count by prg, date_hour
but it inserts one row per hour per prg.

PRGName   h-2   h-1   h

PRGNM1    35   23   76

PRGNM2    43   53   12

0 Karma

SplunkTrust
SplunkTrust

You could do something like this:

your base search | rex "\[(?<prg>PRG.*?)\]" | stats count by prg

If you can control the format of that log it's nicer to have these PRG_NM1 fields printed in key=value format, something like this:

10 Mar 2013 10:10:35,559 processor=MyApplicationProcessor process=MyProcess pid=PIDNo something=123456 processGroup=MyProcessGroup prg=PRG_NM1 failureRetryCount=x exceptionRetryCount=y message="Finishing Process"

Then Splunk will understand all the fields automatically, and humans will still understand the log as well.

View solution in original post