All Apps and Add-ons

How to join two sources with summary indexing to improve performance?

bmarshall24
New Member

Hello,

I am quite green at Splunk and have a problem I could use some help with.

My data is coming from a postgres database via the Splunk DB Connect App, where each input (source) into Splunk is a postgres table.

I am trying to join two sources, which I can do in a regular search, but am trying to improve performance since my join search is running quite long, so I am looking at summary indexing.

The two sources are as follows:

action_times

action_time
act_id

actions_table

act_id
operation

Here is the base search that returns the expected results.

source="action_times" | join type=inner act_id [search source="actions_table"] | stats count by operation

I have been able to set up a summary index and schedule a report which runs the search above, but the actions_table really does not update often so most subsequent runs of the scheduled report return no events, despite there being tens of thousands of events from action_times.

Sample Input with Expected Output
Input - action_times

Row 1: action_time = 2020-06-03 11:58:10.123. act_id = 1
Row 2: action_time = 2020-06-03 11:59:18.563. act_id = 2
Row 3: action_time = 2020-06-03 11:55:28.752. act_id = 1

Input - actions_table
Row 1: act_id = 1. operation = "read register"
Row 2: act_id = 2. operation = "write register"

Expected Output
Row 1: "read register" - 2
Row 2: "write register" - 1

What I would like to do...

  • I would like to use summary indexing to pull in the joined data, either with an actual join command, or without.

If there is any other helpful information I can provide, please let me know.

Thank you,

Labels (1)
0 Karma

to4kawa
Ultra Champion
 source="action_times"  OR source="actions_table" 
| stats count by act_id operation
| fields - act_id

that's all

0 Karma

bmarshall24
New Member

@to4kawa Thank you for taking the time to respond. However, I tried to run the search you provided and it returned No results.

I added some information to my original post with "Sample Input and Expected Output" in case that helps clear anything up.

0 Karma

to4kawa
Ultra Champion

I see. check my latest answer.

0 Karma
Career Survey
First 500 qualified respondents will receive a $20 gift card! Tell us about your professional Splunk journey.

Can’t make it to .conf25? Join us online!

Get Updates on the Splunk Community!

Can’t Make It to Boston? Stream .conf25 and Learn with Haya Husain

Boston may be buzzing this September with Splunk University and .conf25, but you don’t have to pack a bag to ...

Splunk Lantern’s Guide to The Most Popular .conf25 Sessions

Splunk Lantern is a Splunk customer success center that provides advice from Splunk experts on valuable data ...

Unlock What’s Next: The Splunk Cloud Platform at .conf25

In just a few days, Boston will be buzzing as the Splunk team and thousands of community members come together ...