Getting Data In

How to join separate logs together for further information about host.

rickyholland87
Engager

Hello,

This may not be possible but is worth questioning anyway.

I'm currently pulling several log files which relate to the same device. We recently rolled out an update to these devices and I'm pulling logs back that show if a devices version has been updated, they look like;

2015-07-05 16:55:19,121 [30] INFO log4net [(null)] - protocol: 77268 02:02:00

Is there anyway I can get this version number (02:02:00) statically linked to hosts so that I can use it as a filter etc for my other searches. So for example I have host B0990T99 - I can perform searches and then also find out what version this host is running...

Hopefully that makes sense.

Thanks.

Tags (2)
0 Karma

woodcock
Esteemed Legend

If I understand you correctly, you do it like this:

Every day or so you do this:

... | rex "protocol:\s+\d+(?<version>.*)$" | table host version | inputlookup append=true host2version | dedup host | outputlookup host2version 

Then you can add this to any search:

... | lookup host2version host OUTPUT version
0 Karma

alacercogitatus
SplunkTrust
SplunkTrust

Here you would want to use a lookup file. Lookup files are generated using search commands. I'll do a simple one, you will want to research and make it "production ready".

Start with your search. I'm assuming you have already performed the extractions to get out the protocol and version number.

 index=myindex sourcetype=log4net | stats latest(version_number) as vn by host protocol

Now, add this:

 |outputlookup protocol_versions_lookup.csv

So all together now!

index=myindex sourcetype=log4net earliest=-1h@h  | stats latest(version_number) as vn by host protocol | inputlookup protocol_versions_lookup.csv append=t|dedup host protocol | outputlookup protocol_versions_lookup.csv

Save this and schedule it to run every hour. This will pull the data and put it in a lookup file on the Search head. It will also take what is there, append it, and then de duplicate the data.

You then can use it like this:

 host=myhost | lookup protocol_versions_lookup.csv host | ....

A static file, with dynamically generated data. Again, I wrote this from the top of my head, so it might not be 100% working for your environment. But gives you the idea of how to move forward.

Start here: http://docs.splunk.com/Documentation/Splunk/6.2.3/Knowledge/Aboutlookupsandfieldactions, and enjoy.

rickyholland87
Engager

Thanks for this.

I've managed to export the .csv (I believe) using;

index = spire protocol | stats latest(Version) as vn by host| inputlookup spire_versions.csv append=t|dedup host|outputlookup spire_versions.csv

How would I now link this to my other searches, for example;

index = spire Login Failed| top limit=20 host

This is a seperate search I would like to incorporate the version number into, so for example I can see how many Login Failed we are getting with the different versions.

Thanks a lot for the help.

0 Karma

richgalloway
SplunkTrust
SplunkTrust

What is in the cited event that tells you it belongs to host B0990T99? Without the ability to associate events in one log to events in another using some common field, you will not be able to accomplish your goal.

---
If this reply helps you, Karma would be appreciated.
0 Karma
Get Updates on the Splunk Community!

Detecting Remote Code Executions With the Splunk Threat Research Team

WATCH NOWRemote code execution (RCE) vulnerabilities pose a significant risk to organizations. If exploited, ...

Enter the Splunk Community Dashboard Challenge for Your Chance to Win!

The Splunk Community Dashboard Challenge is underway! This is your chance to showcase your skills in creating ...

.conf24 | Session Scheduler is Live!!

.conf24 is happening June 11 - 14 in Las Vegas, and we are thrilled to announce that the conference catalog ...