topic Re: How to parse and extract fields from Syslog data for particular hosts? in Getting Data In

How to parse and extract fields from Syslog data for particular hosts?

snehalk — Tue, 09 Feb 2016 09:59:48 GMT

Hello All,

We have the cluster environment and the syslog data is coming in from UDP port. We need to parse some of the syslog data from particular hosts.

Usually I will go with deploying transforms.conf and props.conf files for particular a sourcetype to search heads, but am not getting how we can achieve in this condition because we need to parse by host.

Could you please help me on this?

Re: How to parse and extract fields from Syslog data for particular hosts?

renems — Tue, 09 Feb 2016 10:30:21 GMT

Hi snehalk!

I'm not 100% sure if I understand your question correct. This is what I suspect your question is: how to modify/select data, coming in from a syslog stream, per host?

Best way that I could think of, is to have your (r)syslog configuration make a difference on the incoming streams in (r)syslog.conf. So that it sets those hosts apart from the rest (for example, based on IP?), writing its output to a different filepath.
Once you achieved so, you then can configure splunk in the inputs.conf to monitor the seperate hosts in a dedicated sourcetype. This can very well be a splunk heavy forwarder, sending its data to the indexer cluster.

Is this of any help?

Re: How to parse and extract fields from Syslog data for particular hosts?

snehalk — Tue, 09 Feb 2016 11:39:11 GMT

Hello Renems,
Thanks for the reply, The Syslog data is already in Splunk ( cluster) now i want to extract say eg, date,time,IP from the particular host.
so could you please guide here? how i can approach this?

Re: How to parse and extract fields from Syslog data for particular hosts?

Jeremiah — Tue, 09 Feb 2016 19:40:35 GMT

Do you want to apply different field extractions or props settings by host?

http://docs.splunk.com/Documentation/Splunk/6.3.3/Admin/propsconf

So if your sourcetype is syslog, you could do

[syslog]
REPORT-bar = bar

You can also do

[host::foo]
REPORT-bar = bar

You can apply wildcard patterns (see the link above) if you have many hosts you need to match.

Re: How to parse and extract fields from Syslog data for particular hosts?

snehalk — Fri, 12 Feb 2016 12:49:17 GMT

Hello Jeremiah,
Thanks for reply, so i need to create props.conf file as below

 Props.conf

[host::myhost1,myhost2]
REPORT-bar = syslog_extract

Transforms.conf

[syslog_extract]
DELIMS =" "
FIELDS =field1,field2...

And deployed it to Search heads ?

Please let me know on this.

Thanks

Re: How to parse and extract fields from Syslog data for particular hosts?

snehalk — Wed, 17 Feb 2016 14:27:15 GMT

Hello All,

I done the syslog parsing using the field extraction in Splunk Web UI, and its working as expected.

Re: How to parse and extract fields from Syslog data for particular hosts?

damode — Fri, 20 Oct 2017 02:58:04 GMT

Hi @snehalk,

I am looking for ways to parse data solely based on the host IP. I had created a regex to extract host IP, but I realised not all of the logs contain the host IP. Hence, I was looking for more ways to parse data.

Can you please advise what exact steps did you to take to achieve this solution ?

Re: How to parse and extract fields from Syslog data for particular hosts?

snehalk — Mon, 23 Oct 2017 09:51:32 GMT

Hello Damode,

i have used Splunk Web UI where we have direct option "Extract Field".

Re: How to parse and extract fields from Syslog data for particular hosts?

splunkboob — Tue, 22 Mar 2022 08:54:54 GMT

i did this and it worked on my syslog files after importing them. but it doesnt work on live data (real-time). do you have a fix for that?