Getting Data In

How to parse and extract fields from Syslog data for particular hosts?

snehalk
Communicator

Hello All,

We have the cluster environment and the syslog data is coming in from UDP port. We need to parse some of the syslog data from particular hosts.

Usually I will go with deploying transforms.conf and props.conf files for particular a sourcetype to search heads, but am not getting how we can achieve in this condition because we need to parse by host.

Could you please help me on this?

0 Karma

snehalk
Communicator

Hello All,

I done the syslog parsing using the field extraction in Splunk Web UI, and its working as expected.

damode
Motivator

Hi @snehalk,

I am looking for ways to parse data solely based on the host IP. I had created a regex to extract host IP, but I realised not all of the logs contain the host IP. Hence, I was looking for more ways to parse data.

Can you please advise what exact steps did you to take to achieve this solution ?

0 Karma

snehalk
Communicator

Hello Damode,

i have used Splunk Web UI where we have direct option "Extract Field".

0 Karma

splunkboob
Explorer

i did this and it worked on my syslog files after importing them. but it doesnt work on live data (real-time). do you have a fix for that?

0 Karma

renems
Communicator

Hi snehalk!

I'm not 100% sure if I understand your question correct. This is what I suspect your question is: how to modify/select data, coming in from a syslog stream, per host?

Best way that I could think of, is to have your (r)syslog configuration make a difference on the incoming streams in (r)syslog.conf. So that it sets those hosts apart from the rest (for example, based on IP?), writing its output to a different filepath.
Once you achieved so, you then can configure splunk in the inputs.conf to monitor the seperate hosts in a dedicated sourcetype. This can very well be a splunk heavy forwarder, sending its data to the indexer cluster.

Is this of any help?

0 Karma

Jeremiah
Motivator

Do you want to apply different field extractions or props settings by host?

http://docs.splunk.com/Documentation/Splunk/6.3.3/Admin/propsconf

So if your sourcetype is syslog, you could do

[syslog]
REPORT-bar = bar

You can also do

[host::foo]
REPORT-bar = bar

You can apply wildcard patterns (see the link above) if you have many hosts you need to match.

0 Karma

snehalk
Communicator

Hello Jeremiah,
Thanks for reply, so i need to create props.conf file as below

 Props.conf

[host::myhost1,myhost2]
REPORT-bar = syslog_extract

Transforms.conf

[syslog_extract]
DELIMS =" "
FIELDS =field1,field2...

And deployed it to Search heads ?

Please let me know on this.

Thanks

0 Karma

snehalk
Communicator

Hello Renems,
Thanks for the reply, The Syslog data is already in Splunk ( cluster) now i want to extract say eg, date,time,IP from the particular host.
so could you please guide here? how i can approach this?

0 Karma
Get Updates on the Splunk Community!

Tech Talk | Elevating Digital Service Excellence: The Synergy of Splunk RUM & APM

Elevating Digital Service Excellence: The Synergy of Real User Monitoring and Application Performance ...

Adoption of RUM and APM at Splunk

    Unleash the power of Splunk Observability   Watch Now In this can't miss Tech Talk! The Splunk Growth ...

March Community Office Hours Security Series Uncovered!

Hello Splunk Community! In March, Splunk Community Office Hours spotlighted our fabulous Splunk Threat ...