Splunk Search

How to join two sources based on two search fields?

vmnguyen
New Member

I have two sets of data:
1. sourcetype=app "DEBUG A" function=UpdateCartItemStatus status=Rejected
2. sourcetype=app "DEBUG B" function=UpdateCartItemStatus

Set 1 (DEBUG A) also has the fields unitID1, unitID2, and user1
Set 2 (DEBUG B) also has the fields unitID1, unitID2, and user2

I would like to join data set 1 with data set 2 on unitID1 and unitID2 and get a count of the number of instances this occurs per user2. Ideally this would be as efficient as possible as the data sources are large, searches can span long periods of time, and they are constantly being refreshed. A join is not required, it was just the first thing I thought of.

I am using the dashboard editor for Splunk Enterprise.

0 Karma

sundareshr
Legend

Try this?

search for set 1 data | eval unit=unitd1."::::".unitd2 | append [search for set 2 data | eval  unit=unitd1."::::".unitd2] | chart limit=0 count by unit over user
0 Karma

vmnguyen
New Member

Hm...let me clarify:

Datapoints:

  1. sourcetype=app "DEBUG A" function=UpdateCartItemStatus status=Rejected unitID1=1008908999 unitID2 = front user=john
  2. sourcetype=app "DEBUG A" function=UpdateCartItemStatus status=Rejected unitID1=1008908999 unitID2 = back user=john
  3. sourcetype=app "DEBUG B" function=UpdateCartItemStatus status=Printed unitID1=1008908999 unitID2 = front user=mary
  4. sourcetype=app "DEBUG B" function=UpdateCartItemStatus status=Printed unitID1=1008908999 unitID2 = back user=mary

I want to count how many times Mary updated the status as "Printed" and DEBUG A reported that it was Rejected. The output should have a count of 2 for user=Mary.
Thus, I want to link datapoint 3 to datapoint 1 on and datapoint 4 to datapoint 2 using unitID1 and unitID2.

0 Karma

sundareshr
Legend

will the unitid be the same for all (1008908999). how did you know that 3 should be liked to 1?

0 Karma

sundareshr
Legend

How about this

search to get all data using append | transaction unitID2 unitID1 startswith="debug=A" endswith="debug=B" maxevents=2 keepevicted=f

The transaction command has a few more options you can explore

0 Karma

vmnguyen
New Member

Yes, all four of those datapoints would have the same unitID1, but there are millions of datapoints with different unitIDs. We know 3 should be linked to 1 because they share the same unitID1 and unitID2.

0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

Best Practices: Splunk auto adjust pipeline queue

When you enable autoAdjustQueue in Splunk, maxSize should be understood as the queue size Splunk starts with ...

Laser Bananas and Edge Hubs: Exploring Operational Technology (OT) Data Through a ...

  OT is a different environment to traditional IT and can have interesting challenges when interfacing the ...

Event Series: Mastering AI Tokenomics and Splunk Agent Observability

Beyond the Black Box: Correlating AI Performance and Tokenomics with Splunk Agent Observability   As ...