topic how to find the Unique field value which is present in two different source logs in Splunk Search

how to find the Unique field value which is present in two different source logs

RashmiGowda — Fri, 07 Mar 2014 04:51:31 GMT

My question is how to find the uniqueId which is present in two different source logs..?

I have 2 source logs say, abc.log and xyz.log. abc.log has fields called "RequestID" and "RequestStartTime". xyz.log has fields called "TransactionID" and "TransEndTime". Now i have to find and display a UniqueID which is present in both "RequestID" and "TransactionID" with RequestStartTime and TransEndTime..??

Can anyone please suggest how m able to do this in splunk..??

Thanks in advance 🙂

Re: how to find the Unique field value which is present in two different source logs

gauldridge — Fri, 07 Mar 2014 05:06:25 GMT

You could try:

source="abc.log" OR source="xyz.log" | eval UniqueID=coalesce(RequestID,TransactionID)

and then depending on how you want the output formatted, you could do something like:

| stats c by UniqueID,RequestStartTime,TransEndTime | fields - c

| chart values(RequestStartTime) AS Start values(TransEndTime) AS End by UniqueID

Re: how to find the Unique field value which is present in two different source logs

RashmiGowda — Fri, 07 Mar 2014 05:31:48 GMT

Thank you. But how to eliminate the duplicate TransactionID. because in xyz.log there are Transactions which are appering more than once. Could you please suggest..??

Re: how to find the Unique field value which is present in two different source logs

RashmiGowda — Fri, 07 Mar 2014 05:57:24 GMT

above query is giving duplicate results also. its also displaying id's whcih are not present in RequestID and TransactionID.. 😞

Re: how to find the Unique field value which is present in two different source logs

gauldridge — Fri, 07 Mar 2014 05:58:02 GMT

Are there multiple "copies" of the same transaction in the xyz.log source or is the TransactionID being recycled/reused by multiple transactions? Also, is the beginning of a transaction always in abc.log and the end always in xyz.log?

Re: how to find the Unique field value which is present in two different source logs

RashmiGowda — Fri, 07 Mar 2014 06:33:53 GMT

Actually TransactionID is being reused by multiple transactions. Its not that beginning of transaction in abc.log and end in xyz.log.

What i need to do is, i need to corelate the events from abc.log and xyz.log based on the UniqueiD which is present in both the logs. so i created "RequestID" for abc.log and TransactionID for xyz.log.

now i need to pick up the uniqueid which is present in both RequestID and TransactionID. RequestID contains uniqueID of abc.log and TransactionID contains uniqueiD of xyz.log

@gauldridge

Re: how to find the Unique field value which is present in two different source logs

gauldridge — Fri, 07 Mar 2014 06:45:57 GMT

Does that mean that RequestID is also used by multiple transactions?

Re: how to find the Unique field value which is present in two different source logs

RashmiGowda — Fri, 07 Mar 2014 06:54:21 GMT

No.. RequestID is Unique. Its not used by Multiple Transactions
@gauldridge

Re: how to find the Unique field value which is present in two different source logs

gauldridge — Sun, 09 Mar 2014 13:55:32 GMT

So, is it correct to say that the RequestID is unique per transaction but the TransactionID is not?

Is the UniqueID you mention something that actually exists in the events or something you need to create on-the-fly?

Is there something in either one or both of the log types that always indicates the beginning and end of the transaction?

Is it possible for you to share a snippet of each of the log sources? Even if it is scrubbed, it might be easier to troubleshoot with an example of the data at this point.