Aggregated reporting on a field with variable value

[ilyashussain]

Hi, I am trying to create a scheduled report that will count the number of events for specific keywords. The event entry looks somewhat like:

10 Mar 2013 10:10:35,559 MyApplicationProcessor : MyProcess PIDNo [123456] MyProcessGroup [PRG_NM1] FailureRetryCount[x] ExceptionRetryCount [y] Finishing Process

10 Mar 2013 10:10:35,123 MyApplicationProcessor : MyProcess PIDNo [123456] MyProcessGroup [PRG_NM2] FailureRetryCount[x] ExceptionRetryCount [y] Finishing Process

10 Mar 2013 10:10:35,234 MyApplicationProcessor : MyProcess PIDNo [123454] MyProcessGroup [PRG_NM1] FailureRetryCount[x] ExceptionRetryCount [y] Exit

10 Mar 2013 10:10:35,234 MyApplicationProcessor : MyProcess PIDNo [123454] MyProcessGroup [PRG_NM1] FailureRetryCount[x] ExceptionRetryCount [y] Finishing Process

10 Mar 2013 10:10:35,234 MyApplicationProcessor : MyProcess PIDNo [123454] MyProcessGroup [PRG_NM1] FailureRetryCount[x] ExceptionRetryCount [y] Enter

The PIDNo,x,y, PRG_NM etc are variables I am particularly looking to create a report based on PRD_NM where the even ends with Finishing Process. The PRG_NM can be any of the 100+ possible values.

The reports needs to look like:

PRG_NM1 - 35

PRG_NM2 - 43

PRG_NM3 - 21

PRG_NM4 - 42

Any help is greatly appreciated.

[martin_mueller]

For charting over time you usually want to use timechart like so:

... | timechart span=1h count

[somesoni2]

Use chart command instead of stats..

...| chart count over prg by date_hour

[ilyashussain]

Awesome. Thank you for the quick response. Much appreciated.

Unfortunately, the events are logged by a legacy program and we have very little control on how and what is being logged

can this be modify to make it look like below (where h=current hour)? I tried stats count by prg, date_hour
but it inserts one row per hour per prg.

PRGName   h-2   h-1   h

PRGNM1    35   23   76

PRGNM2    43   53   12

[martin_mueller · Accepted solution]

You could do something like this:

your base search | rex "\[(?<prg>PRG.*?)\]" | stats count by prg

If you can control the format of that log it's nicer to have these PRG_NM1 fields printed in key=value format, something like this:

10 Mar 2013 10:10:35,559 processor=MyApplicationProcessor process=MyProcess pid=PIDNo something=123456 processGroup=MyProcessGroup prg=PRG_NM1 failureRetryCount=x exceptionRetryCount=y message="Finishing Process"

Then Splunk will understand all the fields automatically, and humans will still understand the log as well.