Getting Data In

How to combine different types of events from different data sources into one event?

akawacz
Path Finder

Hello

I have three different data sources (so 3 different types of events)

DataSource_1: Event_Number Ticket
DataSource_2: Event_Number Create_Event_Date
DataSource_3: Ticket Create_Ticket_Date

I would like to get one event which has: Ticket Create_Ticket_date Create_Event_Date

Is it possible to build a search which shows this as one event, taking also into account possible new completely different data sources (e.g. DataSource_4)?

I was trying transaction (not efficient), lookups (which I created from two different sources and use inputlookup populated to the 3rd one. In the search, there was some specification to those 3 data sources. Doing a lot of reports I would always need to take into account this part.

Thank you

0 Karma
1 Solution

woodcock
Esteemed Legend

Try this:

... | eventstats values(Ticket) AS Ticket BY Event_Number
| eventstats values(Event_Number) AS Event_Number BY Ticket
| stats values(*) AS * by Ticket

View solution in original post

0 Karma

woodcock
Esteemed Legend

Try this:

... | eventstats values(Ticket) AS Ticket BY Event_Number
| eventstats values(Event_Number) AS Event_Number BY Ticket
| stats values(*) AS * by Ticket
0 Karma

Richfez
SplunkTrust
SplunkTrust

You didn't mention how you tried to use transaction. I admit it's not the most efficient thing, but I've used it on fairly large datasets well if you can limit the time and events it's operating on.

... | transaction Event_Number Ticket maxspan=15m maxpause=15m maxevents=3

If speed is still a problem, you could create a Data Model containing that information and accelerate it.

That does assume Event_Number and Ticket aren't equal to one another within a 15 minute period, and obviously assuming they get generated within 15 minutes of each other. Adjust as necessary.

0 Karma

pranav_agile
Explorer

I would like to use the below transaction in Data Model
| transaction maxevents=2 keeporphans=true
what is the best way to do this?

0 Karma

woodcock
Esteemed Legend

Avoid transaction like the plague that it is. It should only be used for transitive key mapping (e.g. some events have EmployeeID, others have Address, others have loginID and each of this is fully unique to a single individual. Then the best way to link is to use | transaction EmployeeID Address loginID. Otherwise do yourself a favor and do not ever use it; it does not scale.

0 Karma

pranav_agile
Explorer

Sure, thanks Woodcock.

0 Karma

MuS
SplunkTrust
SplunkTrust

just a funny thought; who about this:

 base search here | eval corr_field= coalesce(Event_Number, Ticket) | stats values(*) AS * by corr_field
0 Karma

woodcock
Esteemed Legend

What correlation fields do you have to link events between sources?

0 Karma

akawacz
Path Finder

Correlation is by the same column name

So DataSouce_1 Event_Number = DataSource_2 Event_Number
DataSource_1 Ticket= DataSource_3 Ticket

0 Karma
Get Updates on the Splunk Community!

Last Chance to Submit Your Paper For BSides Splunk - Deadline is August 12th!

Hello everyone! Don't wait to submit - The deadline is August 12th! We have truly missed the community so ...

Ready, Set, SOAR: How Utility Apps Can Up Level Your Playbooks!

 WATCH NOW Powering your capabilities has never been so easy with ready-made Splunk® SOAR Utility Apps. Parse ...

DevSecOps: Why You Should Care and How To Get Started

 WATCH NOW In this Tech Talk we will talk about what people mean by DevSecOps and deep dive into the different ...