topic Re: Problem of indexing data in indexer in Getting Data In

Problem of indexing data in indexer

sieutruc — Fri, 09 Nov 2012 16:24:16 GMT

Hello ,

I get a problem of indexing data in indexer.
I have a file test.csv. When i monitored it locally in indexer with sourcetype substate ( configured in props.conf in indexer), indexer can parse data basing those fields inside props(ex:SHOULD_LINEMERGE=false)

But if i use Universal Forwarder (UF) to monitor that csv file,its entire data content is sent to indexer without using SHOULD_LINEMERGE=false above.

i know that at parsing time, we can configure in indexer, heavy forwarder, but impossible in UF.

So how can i configure to let Splunk indexer be able to index it correctly ?

Re: Problem of indexing data in indexer

ClementG — Fri, 09 Nov 2012 16:48:53 GMT

Hi,

You can add a props.conf on the indexer with a stanza on the specified sourcetype, source or host.

E.G.

[source::my.csv]
SHOULD_LINEMERGE = false

http://splunk-base.splunk.com/answers/48280/forwarder-configuration

Regards

Clement

Re: Problem of indexing data in indexer

sowings — Fri, 09 Nov 2012 16:49:10 GMT

Parse-time rules would still apply where the parsing is done. In environments where the forwarders are Universal Forwarders (UF), the parsing occurs on the indexer. Note that the inputs.conf on the UF may set a sourcetype--it is the rules for this sourcetype that would be applied at parse time.

Re: Problem of indexing data in indexer

sieutruc — Fri, 09 Nov 2012 20:46:50 GMT

yes, i did, but it's not totally working

Re: Problem of indexing data in indexer

sieutruc — Fri, 09 Nov 2012 20:49:58 GMT

In fact, my Splunk deployment is a bit more complicated. There is a intermediate forwarder(heavy forwarder) that will forward data from UF to indexer. I see 2 event from UF with the same sourcetype substate, but one seemed not to be applied sourcetype into.

So do i need to configure the same props in intermediate forwarder ?

Re: Problem of indexing data in indexer

ClementG — Fri, 09 Nov 2012 22:06:04 GMT

You have to do it on the Heavy Forwarder, you can set your stanza on either the sourcetype or the source of the file.