Getting Data In

Logs with parameter INDEXED_EXTRACTIONS on UF could no be forwarded from indexer via syslog

ludoz13
Path Finder

Hello splunker,

I have some trouble to forward data to third-party systems via syslog.
All logs are forwarded via syslog except one where parameter INDEXED_EXTRACTIONS is set on a uf.
I share my configuration to explaint you my problem :

UF :
inputs :
[monitor://c:\tmp\logs.csv]
sourcetype = fileshared

props:
[fileshared]

KV_MODE=none
INDEXED_EXTRACTIONS=csv

INDEXER :

props:
[fileshared]
TRANSFORMS-syslog = send_to_syslog

transfoms:
[send_to_syslog]
REGEX = .
DEST_KEY = _SYSLOG_ROUTING
FORMAT = my_syslog_group

outputs:
[syslog:my_syslog_group]
server = loghost.example.com:514

Logs with sourcetype fileshared are indexed as expected (fields and logs indexed) but there are not forwarded via syslog
All others logs from this uf without INDEXED_EXTRACTIONS parameter are indexed and forwarded via syslog

On Splunk documentation, I see that :

When you forward structured data to an indexer, it is not parsed when it arrives at the indexer, even if you have configured props.conf on that indexer with INDEXED_EXTRACTIONS. Forwarded data skips the following pipelines on the indexer, which precludes any parsing of that data on the indexer:

I understand that Is not possible to set parsing and transformation on indexer for sourcetype with INDEXED_EXTRACTION paramater . Do you know if there is a way to forward this kind of data without another method ?

Thanks for your help,

Regards,

vince2010091
Path Finder

Hi Ludoz13,

maybe REGEX = .*

Bye

0 Karma

ludoz13
Path Finder

Hi vince,

I reuse the same regex mentioned in the documentation
it not the problem but thanks for your help

Bye

0 Karma

sbbadri
Motivator

@ludoz13

try this,

UF :
inputs :
[monitor://c:\tmp\logs.csv]
sourcetype = fileshared
props:
[fileshared]
KV_MODE=none
INDEXED_EXTRACTIONS=csv

INDEXER :

props:
[fileshared]
TRANSFORMS-routing=syslogRouting

transfoms:
[syslogRouting]
REGEX=.
DEST_KEY=_TCP_ROUTING
FORMAT=syslogGroup

outputs:
[tcpout:syslogGroup]
server = loghost.example.com:514

For further details, please check the below link,
http://docs.splunk.com/Documentation/SplunkCloud/6.6.3/Forwarding/Routeandfilterdatad

0 Karma

ludoz13
Path Finder

Hi ssbadri,

Thanks for your help,
Unfortunenelty, this does not work.

In my understanding, the setting "[fileshared] TRANSFORMS-routing=syslogRouting" is not read on the indexer because of INDEXED_EXTRACTIONS paramater.

I think that the only way is either removing INDEXED_EXTRACTIONS parameter or either using forwardedindex.0.whitelist option to select only indexes I want to forward data

I am taking all others ideas 🙂

Thanks,

0 Karma
Get Updates on the Splunk Community!

Optimize Cloud Monitoring

  TECH TALKS Optimize Cloud Monitoring Tuesday, August 13, 2024  |  11:00AM–12:00PM PST   Register to ...

What's New in Splunk Cloud Platform 9.2.2403?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.2.2403! Analysts can ...

Stay Connected: Your Guide to July and August Tech Talks, Office Hours, and Webinars!

Dive into our sizzling summer lineup for July and August Community Office Hours and Tech Talks. Scroll down to ...