All Apps and Add-ons

How to join two sources with summary indexing to improve performance?

bmarshall24
New Member

Hello,

I am quite green at Splunk and have a problem I could use some help with.

My data is coming from a postgres database via the Splunk DB Connect App, where each input (source) into Splunk is a postgres table.

I am trying to join two sources, which I can do in a regular search, but am trying to improve performance since my join search is running quite long, so I am looking at summary indexing.

The two sources are as follows:

action_times

action_time
act_id

actions_table

act_id
operation

Here is the base search that returns the expected results.

source="action_times" | join type=inner act_id [search source="actions_table"] | stats count by operation

I have been able to set up a summary index and schedule a report which runs the search above, but the actions_table really does not update often so most subsequent runs of the scheduled report return no events, despite there being tens of thousands of events from action_times.

Sample Input with Expected Output
Input - action_times

Row 1: action_time = 2020-06-03 11:58:10.123. act_id = 1
Row 2: action_time = 2020-06-03 11:59:18.563. act_id = 2
Row 3: action_time = 2020-06-03 11:55:28.752. act_id = 1

Input - actions_table
Row 1: act_id = 1. operation = "read register"
Row 2: act_id = 2. operation = "write register"

Expected Output
Row 1: "read register" - 2
Row 2: "write register" - 1

What I would like to do...

  • I would like to use summary indexing to pull in the joined data, either with an actual join command, or without.

If there is any other helpful information I can provide, please let me know.

Thank you,

Labels (1)
0 Karma

to4kawa
Ultra Champion
 source="action_times"  OR source="actions_table" 
| stats count by act_id operation
| fields - act_id

that's all

0 Karma

bmarshall24
New Member

@to4kawa Thank you for taking the time to respond. However, I tried to run the search you provided and it returned No results.

I added some information to my original post with "Sample Input and Expected Output" in case that helps clear anything up.

0 Karma

to4kawa
Ultra Champion

I see. check my latest answer.

0 Karma
Get Updates on the Splunk Community!

More Ways To Control Your Costs With Archived Metrics | Register for Tech Talk

Tuesday, May 14, 2024  |  11AM PT / 2PM ET Register to Attend Join us for this Tech Talk and learn how to ...

.conf24 | Personalize your .conf experience with Learning Paths!

Personalize your .conf24 Experience Learning paths allow you to level up your skill sets and dive deeper ...

Threat Hunting Unlocked: How to Uplevel Your Threat Hunting With the PEAK Framework ...

WATCH NOWAs AI starts tackling low level alerts, it's more critical than ever to uplevel your threat hunting ...