Getting Data In
Highlighted

How to join separate logs together for further information about host.

Hello,

This may not be possible but is worth questioning anyway.

I'm currently pulling several log files which relate to the same device. We recently rolled out an update to these devices and I'm pulling logs back that show if a devices version has been updated, they look like;

2015-07-05 16:55:19,121 [30] INFO log4net [(null)] - protocol: 77268 02:02:00

Is there anyway I can get this version number (02:02:00) statically linked to hosts so that I can use it as a filter etc for my other searches. So for example I have host B0990T99 - I can perform searches and then also find out what version this host is running...

Hopefully that makes sense.

Thanks.

Tags (2)
0 Karma
Highlighted

Re: How to join separate logs together for further information about host.

SplunkTrust
SplunkTrust

What is in the cited event that tells you it belongs to host B0990T99? Without the ability to associate events in one log to events in another using some common field, you will not be able to accomplish your goal.

---
If this reply helps you, an upvote would be appreciated.
0 Karma
Highlighted

Re: How to join separate logs together for further information about host.

SplunkTrust
SplunkTrust

Here you would want to use a lookup file. Lookup files are generated using search commands. I'll do a simple one, you will want to research and make it "production ready".

Start with your search. I'm assuming you have already performed the extractions to get out the protocol and version number.

 index=myindex sourcetype=log4net | stats latest(version_number) as vn by host protocol

Now, add this:

 |outputlookup protocol_versions_lookup.csv

So all together now!

index=myindex sourcetype=log4net earliest=-1h@h  | stats latest(version_number) as vn by host protocol | inputlookup protocol_versions_lookup.csv append=t|dedup host protocol | outputlookup protocol_versions_lookup.csv

Save this and schedule it to run every hour. This will pull the data and put it in a lookup file on the Search head. It will also take what is there, append it, and then de duplicate the data.

You then can use it like this:

 host=myhost | lookup protocol_versions_lookup.csv host | ....

A static file, with dynamically generated data. Again, I wrote this from the top of my head, so it might not be 100% working for your environment. But gives you the idea of how to move forward.

Start here: http://docs.splunk.com/Documentation/Splunk/6.2.3/Knowledge/Aboutlookupsandfieldactions, and enjoy.

Highlighted

Re: How to join separate logs together for further information about host.

Thanks for this.

I've managed to export the .csv (I believe) using;

index = spire protocol | stats latest(Version) as vn by host| inputlookup spireversions.csv append=t|dedup host|outputlookup spireversions.csv

How would I now link this to my other searches, for example;

index = spire Login Failed| top limit=20 host

This is a seperate search I would like to incorporate the version number into, so for example I can see how many Login Failed we are getting with the different versions.

Thanks a lot for the help.

0 Karma
Highlighted

Re: How to join separate logs together for further information about host.

Esteemed Legend

If I understand you correctly, you do it like this:

Every day or so you do this:

... | rex "protocol:\s+\d+(?<version>.*)$" | table host version | inputlookup append=true host2version | dedup host | outputlookup host2version 

Then you can add this to any search:

... | lookup host2version host OUTPUT version
0 Karma