Splunk Search

How to use the head command with group by?

jimmymccauley
Explorer

Hi All

We are building a security toolkit that performs a number of different scans as part of the application build process when any new code is committed.
We're using Splunk for monitoring, alerting and reporting with all events generated by the security tests being indexed. We're all relative noobs.
One reporting dashboard we need to present to the security team requires us to show the security test outcome for each application across the 5 most recent builds; the output should be as per the table below:
alt text

There is an app_name field in the event that can contain any number of application names. I was planning on using the head command to get the test outcomes for the 5 most recent builds for each application.
This works fine when reporting against a single application, but if I want to report against multiple applications (as per the table above) I cannot find any way to retrieve the first five results for each application. I thought it might be able possible to combine head and group by app_name but I can't find any evidence that this can be done, or any workaround.
Really appreciate any help or recommendations on how I might implement this? It's driving me to drink and you will be saving my liver 🙂
The SPL that I am currently using is below - I've specified an application name in the app_name field:

`index="osesat-dev-01" source=fluentd environment="test-b" platform_region="eu-west-1" kubernetes.pod_name="trufflehog*" OR kubernetes.pod_name="sat-ctrlr-*" app_name=sec-ref-app request_uuid="*" 
| fields - _raw 
| fields _time, request_uuid, app_name, artifact_source, artifact_branch, artifact_revision, kubernetes.pod_name
| join request_uuid [search index="osesat-dev-01" source=fluentd environment="test-b" platform_region="eu-west-1" request_uuid="*" kubernetes.pod_name="trufflehog*" OR kubernetes.pod_name="sat-ctrlr-*" result="pass" OR result="error" OR (result="fail" AND ("High Entropy count">0 OR "Keys count">0 )) 
    | fields - _raw 
    | fields _time, request_uuid, artifact_source, artifact_branch, artifact_revision, result, kubernetes.pod_name, report, "High Entropy count", "keys count"] 
| dedup artifact_revision
| head 5
| table _time, request_uuid, app_name, artifact_source, artifact_branch, artifact_revision, kubernetes.pod_name, result,  "High Entropy count", "keys count", report 
| chart values(result)  as "Test Outcome" over app_name by artifact_revision 
| rename app_name AS "Application", artifact_revision AS "Git Revision Hash"`

Thanks in advance,
Jimmy

0 Karma
1 Solution

somesoni2
Revered Legend

Just replace | head 5 with | dedup 5 app_name. Dedup also keeps specified number of recent events for a field (default to 1i.e. most recent entry).

View solution in original post

somesoni2
Revered Legend

Just replace | head 5 with | dedup 5 app_name. Dedup also keeps specified number of recent events for a field (default to 1i.e. most recent entry).

jimmymccauley
Explorer

Somesoni2 - brilliant. Thanks very much! You seem to exist to help. Top man 🙂

0 Karma
Get Updates on the Splunk Community!

Technical Workshop Series: Splunk Data Management and SPL2 | Register here!

Hey, Splunk Community! Ready to take your data management skills to the next level? Join us for a 3-part ...

Spotting Financial Fraud in the Haystack: A Guide to Behavioral Analytics with Splunk

In today's digital financial ecosystem, security teams face an unprecedented challenge. The sheer volume of ...

Solve Problems Faster with New, Smarter AI and Integrations in Splunk Observability

Solve Problems Faster with New, Smarter AI and Integrations in Splunk Observability As businesses scale ...