I have three different data sources (so 3 different types of events)
DataSource1: EventNumber Ticket
DataSource2: EventNumber CreateEventDate
DataSource3: Ticket CreateTicket_Date
I would like to get one event which has: Ticket CreateTicketdate CreateEventDate
Is it possible to build a search which shows this as one event, taking also into account possible new completely different data sources (e.g. DataSource_4)?
I was trying transaction (not efficient), lookups (which I created from two different sources and use inputlookup populated to the 3rd one. In the search, there was some specification to those 3 data sources. Doing a lot of reports I would always need to take into account this part.
What correlation fields do you have to link events between sources?
Correlation is by the same column name
So DataSouce1 EventNumber = DataSource2 EventNumber
DataSource1 Ticket= DataSource3 Ticket
You didn't mention how you tried to use transaction. I admit it's not the most efficient thing, but I've used it on fairly large datasets well if you can limit the time and events it's operating on.
... | transaction Event_Number Ticket maxspan=15m maxpause=15m maxevents=3
If speed is still a problem, you could create a Data Model containing that information and accelerate it.
That does assume Event_Number and Ticket aren't equal to one another within a 15 minute period, and obviously assuming they get generated within 15 minutes of each other. Adjust as necessary.
just a funny thought; who about this:
base search here | eval corr_field= coalesce(Event_Number, Ticket) | stats values(*) AS * by corr_field
I would like to use the below transaction in Data Model
| transaction maxevents=2 keeporphans=true
what is the best way to do this?
transaction like the plague that it is. It should only be used for transitive key mapping (e.g. some events have
EmployeeID, others have
Address, others have
loginID and each of this is fully unique to a single individual. Then the best way to link is to use
| transaction EmployeeID Address loginID. Otherwise do yourself a favor and do not ever use it; it does not scale.
... | eventstats values(Ticket) AS Ticket BY Event_Number | eventstats values(Event_Number) AS Event_Number BY Ticket | stats values(*) AS * by Ticket