topic uf agent sending data to forwarder in Getting Data In

uf agent sending data to forwarder

jiaqya — Tue, 24 Mar 2020 13:17:46 GMT

i have a csv file which is comma delimited
i am creating an inputs.conf file and sending this file to HF
but when i search, all the data is on single line, the comma is not honored.

how to set it up so that comma is honored and i can see individual columns as data after ingest..

Re: uf agent sending data to forwarder

PavelP — Tue, 24 Mar 2020 13:42:23 GMT

you can use "Add Data" UI Option to create working props.conf and only then when you sure that it 100% works copy the config to UF/HF. Be aware that UI wizard creates props.conf with INDEXED_EXTRACTIONS which leads to CSV files are being parsed on UF.

Re: uf agent sending data to forwarder

jiaqya — Tue, 24 Mar 2020 13:55:22 GMT

i tried using the Add Data UI on test box , but it did not create the props.conf in the app, although the delimit worked, but i need a working copy of props.conf to put on the uf agent.. any ideas ?

So i dont need the props.conf if i use this method ?
i see its creating a sourcetype...

Re: uf agent sending data to forwarder

PavelP — Tue, 24 Mar 2020 15:02:42 GMT

you can work with the wizard until it works and then in the second step (set sourcetype) open "Advanced" section and use "copy to clipboard" link - it opens a new popup window with a text area where you can copy the working props.conf configuration. Then cancel the wizard and use the copied date to create props.conf either on UF or on HF.

Be aware that you can parse CSV on UF directly and send it to Indexer directly, skipping HF. Alternatively you can just send raw data to indexer and parse & index there. In both cases no HF necessary.

Re: uf agent sending data to forwarder

jiaqya — Wed, 30 Sep 2020 04:49:40 GMT

i did exactly what you said, but i still get data with columns, its just not honoring the delimiter.
however , on the UI, i can see them splitting correctly by comma.
but from the UF agent its not doing the splitting by columns based on comma.

below is props.conf

[ MSSAlertsCher ]
SHOULD_LINEMERGE=false
LINE_BREAKER=([\r\n]+)
NO_BINARY_CHECK=true
CHARSET=UTF-8
INDEXED_EXTRACTIONS=csv
KV_MODE=none
category=Structured
description=Comma-separated value format. Set header and other settings in "Delimited Settings"
disabled=false
pulldown_type=true