topic Best pratice for adding knowledge from syslog NG file with forwarder in Getting Data In

Best pratice for adding knowledge from syslog NG file with forwarder

Starlette — Sun, 26 Sep 2010 20:25:26 GMT

Hello,

I have a syslogNG with a forwader to splunkindexer, the syslogNG contains 1000+ hosts Default this will be a single sourcetype (syslog) and shows a nice hostoveride in splunk Question is now, how to do the enrichment for the field extracts? Is it possible to do the sourcetype overides on the forwarder? ( in the inputsfile?) And is it common to do fi 40 or more sourcetypes overrides on this forwarder?

Re: Best pratice for adding knowledge from syslog NG file with forwarder

highiqboy — Sun, 03 Oct 2010 04:32:05 GMT

I believe you can do what you are wanting by overriding Metadata:Sourcetype in your transforms.conf file based on the regular expression (i.e. REGEX=foo ) pattern match and then mapping it to syslog sourcetype in your props.conf file.

However, I believe a better practice, depending on your particular reason for wanting to override the sourcetype, would be to leave it as syslog sourcetype and then creating an eventtype for each of your 40+ "situations". That way you have flexibility to add, delete, and/or change the eventtype definitions as you need to, without having to re-index the syslog events.

(See this page regarding eventtypes and how to setup:)
http://www.splunk.com/base/Documentation/latest/Knowledge/Configureeventtypes

You can also create host tags as well, to group your hosts together for easier logical searching across common host groups, which when combined with eventtype, makes for a very powerful combination to leverage at search time, rather than at forwarding/indexing time.

(see this page regarding tagging your hosts:)
http://www.splunk.com/base/Documentation/latest/Knowledge/Tagthehostfield

Re: Best pratice for adding knowledge from syslog NG file with forwarder

Starlette — Mon, 04 Oct 2010 16:04:18 GMT

Hai

Thanks for the info,,,bottemline here is the field extractions..so the differentation here is source/sourctype/host,,,and with biljions of events I dont want to do that on a single sourcetype right?