All Apps and Add-ons

How to join two sources with summary indexing to improve performance?

bmarshall24
New Member

Hello,

I am quite green at Splunk and have a problem I could use some help with.

My data is coming from a postgres database via the Splunk DB Connect App, where each input (source) into Splunk is a postgres table.

I am trying to join two sources, which I can do in a regular search, but am trying to improve performance since my join search is running quite long, so I am looking at summary indexing.

The two sources are as follows:

action_times

action_time
act_id

actions_table

act_id
operation

Here is the base search that returns the expected results.

source="action_times" | join type=inner act_id [search source="actions_table"] | stats count by operation

I have been able to set up a summary index and schedule a report which runs the search above, but the actions_table really does not update often so most subsequent runs of the scheduled report return no events, despite there being tens of thousands of events from action_times.

Sample Input with Expected Output
Input - action_times

Row 1: action_time = 2020-06-03 11:58:10.123. act_id = 1
Row 2: action_time = 2020-06-03 11:59:18.563. act_id = 2
Row 3: action_time = 2020-06-03 11:55:28.752. act_id = 1

Input - actions_table
Row 1: act_id = 1. operation = "read register"
Row 2: act_id = 2. operation = "write register"

Expected Output
Row 1: "read register" - 2
Row 2: "write register" - 1

What I would like to do...

  • I would like to use summary indexing to pull in the joined data, either with an actual join command, or without.

If there is any other helpful information I can provide, please let me know.

Thank you,

Labels (1)
0 Karma

to4kawa
Ultra Champion
 source="action_times"  OR source="actions_table" 
| stats count by act_id operation
| fields - act_id

that's all

0 Karma

bmarshall24
New Member

@to4kawa Thank you for taking the time to respond. However, I tried to run the search you provided and it returned No results.

I added some information to my original post with "Sample Input and Expected Output" in case that helps clear anything up.

0 Karma

to4kawa
Ultra Champion

I see. check my latest answer.

0 Karma
Get Updates on the Splunk Community!

Introducing the 2024 SplunkTrust!

Hello, Splunk Community! We are beyond thrilled to announce our newest group of SplunkTrust members!  The ...

Introducing the 2024 Splunk MVPs!

We are excited to announce the 2024 cohort of the Splunk MVP program. Splunk MVPs are passionate members of ...

Splunk Custom Visualizations App End of Life

The Splunk Custom Visualizations apps End of Life for SimpleXML will reach end of support on Dec 21, 2024, ...