topic Re: how to join 2 log files using splunk in Splunk Search

how to join 2 log files using splunk

infyravi — Thu, 03 Jan 2013 17:43:25 GMT

Hi,
I am having 2 log files like this
1) abc.log
2) master.log

In the master.log I am having master data like

URI=/ABC/HOME | Name=LandingPage | SLA=10
URI=/ABC/SOMETRANSACTION| Name=XYZ | SLA=10

In ABC.LOG file I am having entries like this
URI=/ABC/home|TT=10|CLS=ABC
URI=/ABC/home|TT=20|CLS=ABC
URI=/ABC/SOMETRANSACTION|TT=20|CLS=CDER

Now I want to create a report like this

Business Txn ----> Volume -----> Average Response time---> SLA

Landing page -----> 2 ----------> 15 ------> RED

XYZ ------> 1 -----> 20 -----> RED

It means the URI should be shown as Name value from master.log file and it should also do the
count of number of occurences along with Average and SLA calculation.

Landing page should come as hyperlink so that when clicked it should show all landing page related transactions in another table.

Can anyone let me know how we can achieve this.

Re: how to join 2 log files using splunk

kristian_kolb — Thu, 03 Jan 2013 22:36:26 GMT

is the master.log really a log file? It seems a little bit like a table of acceptable SLA's per URL. In that case it may prove beneficial to use the master log as a lookup table. See the docs on lookup tables.

Re: how to join 2 log files using splunk

sideview — Thu, 03 Jan 2013 22:45:44 GMT

The approach I would try first is:

source="abc.log" OR source="master.log" | stats last(SLA) as SLA last(Name) as Name last max(TT) as maxTT avg(TT) as avgTT by URI

You can use the eval or rangemap commands to get you your Red/Yellow/Green column, and you may want/need to use eval to do more massaging with eval etc. but that will give you the basic idea.

Beware the join command. Users new to Splunk often overuse the join and append commands because the metaphor matches what they are familiar with from SQL. However it's almost always better to use lookups, eval+stats or transaction. Think of the join and append commands as last resorts - square SQL pegs in a round Splunk world.

Here's a flow chart I made a long time ago that attempts to help you through all this. http://sideviewapps.com/misc/grouping_flow_chart.png

Re: how to join 2 log files using splunk

infyravi — Fri, 04 Jan 2013 05:42:31 GMT

I will try this approach and update accordingly