Splunk Search

Issue with parsing large dataset using Join

kiranpatil1985
New Member

Hello,
I am using the following search to parse 2 indexes since I want to combine the results from both indexes based on common field "email". I am running this search on my local Splunk instance and both indexes are uploaded CSV. I have configured limits.conf file to handle large dataset. I get different output for Clicked_link and delivered_email when I use the OR operator in the Join versus when I just use either of them I get the correct output. Am I missing something here. Why is the OR operator trimming output result. I see 0 output for bunch of tables when normally it is populated with some number.

index=IndexA
| join type=inner email [ search index=IndexB ( event=delivered OR event=click ) | dedup email event | fields email, event ]
| stats count(eval('event'="delivered")) as Email_Delivered
count(eval('event'="click")) as Clicked_links
by Region, Division, Country, Location
| table Region, Division, Country, Location, "Email_Delivered" , Clicked_links

Tags (4)
0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi kiranpatil1985,
there is a limit of 50,000 results in subsearches, for this reason and because join command is very very slow, I suggest to approach this problem in a different way, using stats command.

index=IndexA OR index=IndexB ( event=delivered OR event=click ) 
| dedup email event | fields email, event ]
| stats count(eval('event'="delivered")) as Email_Delivered count(eval('event'="click")) as Clicked_links BY email Region, Division, Country, Location
| table Region, Division, Country, Location, "Email_Delivered" , Clicked_links

Bye.
Giuseppe

0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

Mastering Threat Intelligence in ES 8.5, Splunk AI Assistant v2, and More from Splunk ...

Splunk Lantern is Splunk’s customer success center that provides practical guidance from Splunk experts on key ...

Break the Build: Inside the KubeDoom Lounge at .conf26

    You step up to the machine. The pixelated corridors of a certain 1993 FPS load in front of you, EMP Pulse ...

Splunk Auto Ingestion Parallel Pipeline Scaling

Why this feature matters Many Splunk environments experience ingestion pressure long before the host is fully ...