topic Why won't data parsing while sending data through UF? in Getting Data In

Why won't data parsing while sending data through UF?

SanjayReddy — Fri, 02 Dec 2022 14:41:14 GMT

Hi All,

Need help on sending data through UF.

Background

We have single PROD Splunk instance acting as all in one server and all the configs are present in this server(ex. props , transforms. etc). Currently we are ingesting data using add data from Splunk UI.

we are uploading data for couple of sources and using props.conf for data parsing.

props.conf is defined on basis on sourcetype ex:sourcetypeA, and this config is present in app called appA.

and when we upload data using upload data option data is parsing correctly , this way of ingesting happing for more than year and everything working fine .

Current Issue

recently we installed UF on one of the system and configured UF to send the data to Splunk instance(which is single component)

UF---->SH

as a part of testing we sent file A from UF for sourcetype sourcetypeA to Splunk instance ,
props.conf settings are not applied on search head. later we used the same file A, ingested using data upload option in UI, mentioned souretype as sourcetypeA, parsing working fine(which is excepted behavior).

but its not working while sending data from UF

Checked internal logs of both UF and SH no errors found for this source type.

what causing issue to not apply props ? can you anyone suggest.

inputs.conf on UF

[monitor://fileA]
index = index1
_TCP_ROUTING = uf_default
crcSalt = <SOURCE>
sourcetype = sourcetypeA

props.conf on SH

[sourcetypeA]
CHARSET = MS-ANSI
FIELD_DELIMITER = ;
INDEXED_EXTRACTIONS = csv
KV_MODE = none
NO_BINARY_CHECK = true
SHOULD_LINEMERGE = false
TIMESTAMP_FIELDS = Time
TIME_FORMAT = %d.%m.%Y %H:%M
TZ = IST
category = Structured
disabled = false
pulldown_type = true
TRUNCATE = 50000
FIELD_QUOTE = "
BREAK_ONLY_BEFORE_DATE =
DATETIME_CONFIG =
LINE_BREAKER = ([\r\n]+)
EVAL-name = <condition>
LOOKUP-name = <condition>
FIELDALIAS-name = <condition>

Re: Data Parsing Not Happing While Sending Data Through UF

gcusello — Thu, 01 Dec 2022 07:36:04 GMT

Hi @SanjayReddy,

persing is usuali done on Indexers or (if present) on Heavy Forwarders, with the only exception of INDEXED_EXTRACTIONS that is partially done on Universal Forwarders, so you have to put your props.conf also on UF.

Ciao.

Giuseppe

Re: Data Parsing Not Happing While Sending Data Through UF

SanjayReddy — Thu, 01 Dec 2022 08:57:05 GMT

Hi @gcusello

Many Thanks for your reply,

do I need to add only one INDEXED_EXTRACTIONS in props.conf of UF or
entire props.conf on UF ?

Re: Data Parsing Not Happing While Sending Data Through UF

gcusello — Thu, 01 Dec 2022 11:01:24 GMT

Hi @SanjayReddy,

the best option is to use tha same Add-On (containing props.conf), eventually with a different inputs.conf in local folder also on the UF.

Ciao.

Giuseppe

Re: Data Parsing Not Happing While Sending Data Through UF

PickleRick — Thu, 01 Dec 2022 13:27:42 GMT

Typically you use the same set of props/transforms on all components working with the same set of data (sourcetype, source, host) - settings not applicable in given context (like search-time extractions for UF) will simply not be applied.

Re: Data Parsing Not Happing While Sending Data Through UF

SanjayReddy — Fri, 02 Dec 2022 04:53:41 GMT

Hi @gcusello

after your sugesstion I placed the props.conf on UF and it started working:) , Thanks for your help.

Hi @PickleRick
yes I have sime set of settings, and afrter placing them on UF are working fine.