topic Re: Is it possible to count the number of times a field occurs within a transaction? in Splunk Search

Is it possible to count the number of times a field occurs within a transaction?

popdeluxe — Tue, 23 May 2017 17:32:11 GMT

Is it possible to get the number of times a Field occurs within an event?

I've read posts on how to arrive at unique values of a Field using mvcount. In my case however I have custom logging that includes the same field=value across multiple lines. I'm trying to find a way of counting the number of times this Field occurs within the transaction, so that I can afterwards filter, perhaps with a where clause, based on that that count.

Example logging:

(1)
RequestId=123 RequestType=A
RequestId=123 Consolidate=True
RequestId=123 RequestType=A
RequestId=123 Consolidate=True

(2)
RequestId=456 RequestType=A
RequestId=456 RequestType=A
RequestId=456 Consolidate=True

I'm trying to arrive at a search that can build a transaction with RequestId where the count (number of occurrences of Consolidate) is 2. So the search would return the transaction with RequestId 123 but not 456. Hoping this makes sense.

Re: Is it possible to count the number of times a field occurs within a transaction?

cmerriman — Tue, 23 May 2017 17:47:50 GMT

try:

...|eventstats count(eval(match(Consolidate,"True"))) as ConsolidateTrue|search ConsolidateTrue=2

you could add another count if you want False, also.

Re: Is it possible to count the number of times a field occurs within a transaction?

popdeluxe — Tue, 23 May 2017 18:46:43 GMT

thank you cmerriman.

eventstats is gathering the total count of Consolidate=True across all events. Is it possible to get the count by individual transaction? Using the logging example, this search.....

index=myindex | transaction RequestId | eventstats count(eval(match(Consolidate,"True"))) as ConsolidateTrue| table RequestId ConsolidateTrue

returns

RequestId   ConsolidateTrue
123               3
456               3

I'm trying to find a way of identifying RequestId 123 has 2 Consolidate fields, and 456 only has 1 (so that I can filter this event out)

Any ideas?

Re: Is it possible to count the number of times a field occurs within a transaction?

woodcock — Tue, 23 May 2017 18:54:09 GMT

Add this to the end of your search:

| rex max_match=0 "(?<mvc>RequestId=123 Consolidate=True)" | where mvcount(mvc) > 2

Re: Is it possible to count the number of times a field occurs within a transaction?

cmerriman — Tue, 23 May 2017 18:55:28 GMT

sorry, try adding by RequestId in the eventstats.

index=myindex | transaction RequestId | eventstats count(eval(match(Consolidate,"True"))) as ConsolidateTrue by RequestId| table RequestId ConsolidateTrue

Re: Is it possible to count the number of times a field occurs within a transaction?

popdeluxe — Tue, 23 May 2017 19:39:38 GMT

thank you very much! That was the missing piece, plus I had to move the eventstats prior to the transaction clause. I suppose given the same field=value, when in the context of the transaction it is seen as 1 occurrence of Consolidate in the event.

final query:

 index=myindex | eventstats count(eval(match(Consolidate,"True"))) as ConsolidateTrue by RequestId| transaction RequestId |  table RequestId ConsolidateTrue

results in

 RequestId   ConsolidateTrue
 123               2
 456               1

thank you for the assist cmerriman!!!