How to parse and extract fields from Syslog data for particular hosts?

[snehalk]

Hello All,

We have the cluster environment and the syslog data is coming in from UDP port. We need to parse some of the syslog data from particular hosts.

Usually I will go with deploying transforms.conf and props.conf files for particular a sourcetype to search heads, but am not getting how we can achieve in this condition because we need to parse by host.

Could you please help me on this?

[snehalk]

Hello All,

I done the syslog parsing using the field extraction in Splunk Web UI, and its working as expected.

[damode]

Hi @snehalk,

I am looking for ways to parse data solely based on the host IP. I had created a regex to extract host IP, but I realised not all of the logs contain the host IP. Hence, I was looking for more ways to parse data.

Can you please advise what exact steps did you to take to achieve this solution ?

[snehalk]

Hello Damode,

i have used Splunk Web UI where we have direct option "Extract Field".

[splunkboob]

i did this and it worked on my syslog files after importing them. but it doesnt work on live data (real-time). do you have a fix for that?

[renems]

Hi snehalk!

I'm not 100% sure if I understand your question correct. This is what I suspect your question is: how to modify/select data, coming in from a syslog stream, per host?

Best way that I could think of, is to have your (r)syslog configuration make a difference on the incoming streams in (r)syslog.conf. So that it sets those hosts apart from the rest (for example, based on IP?), writing its output to a different filepath.
Once you achieved so, you then can configure splunk in the inputs.conf to monitor the seperate hosts in a dedicated sourcetype. This can very well be a splunk heavy forwarder, sending its data to the indexer cluster.

Is this of any help?

[Jeremiah]

Do you want to apply different field extractions or props settings by host?

http://docs.splunk.com/Documentation/Splunk/6.3.3/Admin/propsconf

So if your sourcetype is syslog, you could do

[syslog]
REPORT-bar = bar

You can also do

[host::foo]
REPORT-bar = bar

You can apply wildcard patterns (see the link above) if you have many hosts you need to match.

[snehalk]

Hello Jeremiah,
Thanks for reply, so i need to create props.conf file as below

 Props.conf

[host::myhost1,myhost2]
REPORT-bar = syslog_extract

Transforms.conf

[syslog_extract]
DELIMS =" "
FIELDS =field1,field2...

And deployed it to Search heads ?

Please let me know on this.

Thanks

[snehalk]

Hello Renems,
Thanks for the reply, The Syslog data is already in Splunk ( cluster) now i want to extract say eg, date,time,IP from the particular host.
so could you please guide here? how i can approach this?