Getting Data In
Highlighted

Which properties are available for a Universal Forwarder in Props/Transforms ?

Contributor

Hi,

I can't find any reference in the docs (i.e. : http://docs.splunk.com/Documentation/Splunk/6.5.2/Admin/Propsconf) of Props or Transforms about which attributes are available/working on an Universal Forwarder.

Is there any exhaustive documentation about it?

Thank you

Highlighted

Re: Which properties are available for a Universal Forwarder in Props/Transforms ?

SplunkTrust
SplunkTrust
Highlighted

Re: Which properties are available for a Universal Forwarder in Props/Transforms ?

SplunkTrust
SplunkTrust

This should give you information on how data moves from source to Splunk and what all activities are performed and by which node (forwarder/heavy forwarder/indexer)
http://docs.splunk.com/Documentation/Splunk/6.5.2/Deploy/Componentsofadistributedenvironment
http://docs.splunk.com/Documentation/Splunk/6.5.2/Deploy/Datapipeline

0 Karma
Highlighted

Re: Which properties are available for a Universal Forwarder in Props/Transforms ?

Esteemed Legend

Your title says "for" a UF and your question says "on" a UF. I am going to assume that you literally mean "on a UF". Because the UF does not index the data (with the exception of INDEXED EXTRACTIONS), very little in those files makes any sense to deploy "to" and use "on" the UF. Some that DO include:

props.conf:
TZ, sourcetype, NOBINARYCHECK, CHECKMETHOD, priority, and of course INDEXEDEXTRACTIONS (and its associates).

I cannot think of anything in transforms.conf that takes effect on the UF.

0 Karma
Highlighted

Re: Which properties are available for a Universal Forwarder in Props/Transforms ?

SplunkTrust
SplunkTrust

The whole props/transforms shebang does take effect for data that is cooked on the UF through INDEXED_EXTRACTIONS.

0 Karma
Highlighted

Re: Which properties are available for a Universal Forwarder in Props/Transforms ?

Esteemed Legend

Yes, that is why I called it out as an exception.

0 Karma
Highlighted

Re: Which properties are available for a Universal Forwarder in Props/Transforms ?

Contributor

Thanks, I was hoping for an exhaustive list, it is strange that Splunk does not provide one.

What about the following :

CHARSET

DATETIMECONFIG
TIME
FORMAT
TIMEPREFIX
MAX
TIMESTAMP_LOOKAHEAD

LINEBREAKER
BREAK
ONLYBEFORE
MUST
BREAK_AFTER

TRUNCATE
SHOULD_LINEMERGE

?

0 Karma
Highlighted

Re: Which properties are available for a Universal Forwarder in Props/Transforms ?

Esteemed Legend

The problem is that some of this is "it depends", especially when you use INDEXED_EXTRACTIONS.

0 Karma
Highlighted

Re: Which properties are available for a Universal Forwarder in Props/Transforms ?

SplunkTrust
SplunkTrust

CHARSET applies at input, see http://docs.splunk.com/Documentation/Splunk/6.5.2/Admin/propsconf (search for "input time" for a fairly exhaustive list)

Linebreaking happens at parsing ( LINE_BREAKER, TRUNCATE).
Line Merging happens at merging ( BREAK_ONLY_BEFORE, MUST_BREAK_AFTER, SHOULD_LINEMERGE).
Timestamping happens at typing ( DATETIME_CONFIG, TIME_FORMAT, TIME_PREFIX, MAX_TIMESTAMP_LOOKAHEAD)
See http://wiki.splunk.com/Community:HowIndexingWorks

Where those four happen depends on what path the data takes through Splunk. Input usually happens on a UF, while the other three happen where the data is cooked. By default that's the indexer, if you have a heavy forwarder along the route it's usually the first heavy forwarder, and for indexed extractions it's usually the inputting forwarder - even a UF.

View solution in original post

Highlighted

Re: Which properties are available for a Universal Forwarder in Props/Transforms ?

Contributor

Thanks for the tip about searching "input time", it is really helping!

So If I need to apply all the settings you're mentionning, except "CHARSET", I need to do it on the Indexer side?

On the other hand, according to woodcock's answer, TZ attribute is working on the UF, but in the docs (http://docs.splunk.com/Documentation/Splunk/6.5.2/Admin/propsconf) there is no "input time" mention.

0 Karma