Getting Data In

How to parse and extract fields from Syslog data for particular hosts?

snehalk
Communicator

Hello All,

We have the cluster environment and the syslog data is coming in from UDP port. We need to parse some of the syslog data from particular hosts.

Usually I will go with deploying transforms.conf and props.conf files for particular a sourcetype to search heads, but am not getting how we can achieve in this condition because we need to parse by host.

Could you please help me on this?

0 Karma

snehalk
Communicator

Hello All,

I done the syslog parsing using the field extraction in Splunk Web UI, and its working as expected.

damode
Motivator

Hi @snehalk,

I am looking for ways to parse data solely based on the host IP. I had created a regex to extract host IP, but I realised not all of the logs contain the host IP. Hence, I was looking for more ways to parse data.

Can you please advise what exact steps did you to take to achieve this solution ?

0 Karma

snehalk
Communicator

Hello Damode,

i have used Splunk Web UI where we have direct option "Extract Field".

0 Karma

splunkboob
Explorer

i did this and it worked on my syslog files after importing them. but it doesnt work on live data (real-time). do you have a fix for that?

0 Karma

renems
Communicator

Hi snehalk!

I'm not 100% sure if I understand your question correct. This is what I suspect your question is: how to modify/select data, coming in from a syslog stream, per host?

Best way that I could think of, is to have your (r)syslog configuration make a difference on the incoming streams in (r)syslog.conf. So that it sets those hosts apart from the rest (for example, based on IP?), writing its output to a different filepath.
Once you achieved so, you then can configure splunk in the inputs.conf to monitor the seperate hosts in a dedicated sourcetype. This can very well be a splunk heavy forwarder, sending its data to the indexer cluster.

Is this of any help?

0 Karma

Jeremiah
Motivator

Do you want to apply different field extractions or props settings by host?

http://docs.splunk.com/Documentation/Splunk/6.3.3/Admin/propsconf

So if your sourcetype is syslog, you could do

[syslog]
REPORT-bar = bar

You can also do

[host::foo]
REPORT-bar = bar

You can apply wildcard patterns (see the link above) if you have many hosts you need to match.

0 Karma

snehalk
Communicator

Hello Jeremiah,
Thanks for reply, so i need to create props.conf file as below

 Props.conf

[host::myhost1,myhost2]
REPORT-bar = syslog_extract

Transforms.conf

[syslog_extract]
DELIMS =" "
FIELDS =field1,field2...

And deployed it to Search heads ?

Please let me know on this.

Thanks

0 Karma

snehalk
Communicator

Hello Renems,
Thanks for the reply, The Syslog data is already in Splunk ( cluster) now i want to extract say eg, date,time,IP from the particular host.
so could you please guide here? how i can approach this?

0 Karma
Get Updates on the Splunk Community!

Splunk Observability Cloud | Unified Identity - Now Available for Existing Splunk ...

Raise your hand if you’ve already forgotten your username or password when logging into an account. (We can’t ...

Index This | How many sides does a circle have?

February 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

Registration for Splunk University is Now Open!

Are you ready for an adventure in learning?   Brace yourselves because Splunk University is back, and it's ...