Splunk Search

Reading same field from multiple log files

New Member

I have 2 log files from different sources. Both log files have statements either indicating a "Transaction-Start" or "Transaction-End" . "EPOCH" is a field common in both log files indicating the timestamp of either start or end of a transaction.

Now I want to write a query that fetches EPOCH of "Transaction-Start" from log file 1, call it as, say start and EPOCH of "Transaction-End" from log file 2, call it as, say end. Following this, I want to find the difference between end and start and display only those logs with a difference higher than a threshold, say 10000

What I have tried writing is below :

index=someIndex ENVIRONMENT="someEnv" (source="/log/source1.log" "Transaction-Start" "EPOCH" as start) OR (source="/log/source2.log" "Transaction-End" "EPOCH" as end) 
    | eval difference=end-start 
    | where difference>10000

But this is not working. Looking for help in composing this search in the right way.

0 Karma
1 Solution

SplunkTrust
SplunkTrust

Try this. It assumes EPOCH is an integer. If it isn't then you'll need to use strptime to convert it into an integer.

index=someIndex ENVIRONMENT="someEnv" EPOCH=* (source="/log/source1.log" "Transaction-Start") OR (source="/log/source2.log" "Transaction-End") 
| eval start = if(source="/log/source1.log", EPOCH, null()), end = if (source="/log/source2.log", EPOCH, null())
| stats values(*) as * by TXID
| eval difference=end-start 
| where difference>10000
---
If this reply helps you, an upvote would be appreciated.

View solution in original post

SplunkTrust
SplunkTrust

Try this. It assumes EPOCH is an integer. If it isn't then you'll need to use strptime to convert it into an integer.

index=someIndex ENVIRONMENT="someEnv" EPOCH=* (source="/log/source1.log" "Transaction-Start") OR (source="/log/source2.log" "Transaction-End") 
| eval start = if(source="/log/source1.log", EPOCH, null()), end = if (source="/log/source2.log", EPOCH, null())
| stats values(*) as * by TXID
| eval difference=end-start 
| where difference>10000
---
If this reply helps you, an upvote would be appreciated.

View solution in original post

SplunkTrust
SplunkTrust

One cannot rename fields in a base search (the part before the first |). That's done using eval or rename.
Also, since the start and end times are in separate events, you will not find both 'start' and 'end' together. The starting and ending events must be combined via some common field. Is there a transaction ID or other field that can be used to join the two events?

---
If this reply helps you, an upvote would be appreciated.
0 Karma

New Member

@richgalloway, yes, there is a "TXID" field that is common field in both log files.

0 Karma
State of Splunk Careers

Access the Splunk Careers Report to see real data that shows how Splunk mastery increases your value and job satisfaction.

Find out what your skills are worth!