Getting Data In

How to parse and extract fields from Syslog data for particular hosts?

snehalk
Communicator

Hello All,

We have the cluster environment and the syslog data is coming in from UDP port. We need to parse some of the syslog data from particular hosts.

Usually I will go with deploying transforms.conf and props.conf files for particular a sourcetype to search heads, but am not getting how we can achieve in this condition because we need to parse by host.

Could you please help me on this?

0 Karma

snehalk
Communicator

Hello All,

I done the syslog parsing using the field extraction in Splunk Web UI, and its working as expected.

damode
Motivator

Hi @snehalk,

I am looking for ways to parse data solely based on the host IP. I had created a regex to extract host IP, but I realised not all of the logs contain the host IP. Hence, I was looking for more ways to parse data.

Can you please advise what exact steps did you to take to achieve this solution ?

0 Karma

snehalk
Communicator

Hello Damode,

i have used Splunk Web UI where we have direct option "Extract Field".

0 Karma

splunkboob
Explorer

i did this and it worked on my syslog files after importing them. but it doesnt work on live data (real-time). do you have a fix for that?

0 Karma

renems
Communicator

Hi snehalk!

I'm not 100% sure if I understand your question correct. This is what I suspect your question is: how to modify/select data, coming in from a syslog stream, per host?

Best way that I could think of, is to have your (r)syslog configuration make a difference on the incoming streams in (r)syslog.conf. So that it sets those hosts apart from the rest (for example, based on IP?), writing its output to a different filepath.
Once you achieved so, you then can configure splunk in the inputs.conf to monitor the seperate hosts in a dedicated sourcetype. This can very well be a splunk heavy forwarder, sending its data to the indexer cluster.

Is this of any help?

0 Karma

Jeremiah
Motivator

Do you want to apply different field extractions or props settings by host?

http://docs.splunk.com/Documentation/Splunk/6.3.3/Admin/propsconf

So if your sourcetype is syslog, you could do

[syslog]
REPORT-bar = bar

You can also do

[host::foo]
REPORT-bar = bar

You can apply wildcard patterns (see the link above) if you have many hosts you need to match.

0 Karma

snehalk
Communicator

Hello Jeremiah,
Thanks for reply, so i need to create props.conf file as below

 Props.conf

[host::myhost1,myhost2]
REPORT-bar = syslog_extract

Transforms.conf

[syslog_extract]
DELIMS =" "
FIELDS =field1,field2...

And deployed it to Search heads ?

Please let me know on this.

Thanks

0 Karma

snehalk
Communicator

Hello Renems,
Thanks for the reply, The Syslog data is already in Splunk ( cluster) now i want to extract say eg, date,time,IP from the particular host.
so could you please guide here? how i can approach this?

0 Karma
Get Updates on the Splunk Community!

Splunk Answers Content Calendar, June Edition

Get ready for this week’s post dedicated to Splunk Dashboards! We're celebrating the power of community by ...

What You Read The Most: Splunk Lantern’s Most Popular Articles!

Splunk Lantern is a Splunk customer success center that provides advice from Splunk experts on valuable data ...

See your relevant APM services, dashboards, and alerts in one place with the updated ...

As a Splunk Observability user, you have a lot of data you have to manage, prioritize, and troubleshoot on a ...