topic Re: Impossible to define fields in transforms.conf. in Getting Data In

Impossible to define fields in transforms.conf.

spisiakmi — Tue, 24 Sep 2019 13:01:05 GMT

Hi,

I have simple tab delimited text file.

1 05:45:12 first message 97
1 05:52:15 second message 110
1 05:52:46 third message 97
1 05:53:09 fourth message 110

I want to index it with header definined in transforms.conf
Here are my config files:

**inputs.conf**

[monitor://c:\temp\seho\err\]
disabled = false
index = seho_err_tmp
sourcetype = tsv_WINDOWS-1252
crcSalt=

**props.conf**

[tsv_WINDOWS-1252]
BREAK_ONLY_BEFORE_DATE = 
CHARSET = WINDOWS-1252
INDEXED_EXTRACTIONS = tsv
KV_MODE = none
LINE_BREAKER = ([\r\n]+)
NO_BINARY_CHECK = true
SHOULD_LINEMERGE = false
category = Structured
description = Tab-separated value format. Set header and other settings in "Delimited Settings"
disabled = false
pulldown_type = 1
REPORT-getfields=seho_err_fields

transforms.conf

[seho_err_fields]
DELIMS=":\t"
FIELDS=Fehler,Zeit,Fehlermeldungtext,Fehlernummer

I tried also \t, "\t".

The defined fields never appear in Splunk and the first row from the file is defined as a header by default. Can anybody help me, please?

Re: Impossible to define fields in transforms.conf.

adonio — Wed, 25 Sep 2019 03:52:50 GMT

nothing better then a question with "Impossible" at the headline
here are the steps to accomplish:
your data created in a file tsv_no_header.txt

1 05:45:12 first message 97
1 05:52:15 second message 110
1 05:52:46 third message 97
1 05:53:09 fourth message 110

in props.conf

[tsv_no_header]
SHOULD_LINEMERGE = false
REPORT-no_header = no_header
LINE_BREAKER = ([\r\n]+)

in transforms.conf

[no_header]
DELIMS = " ","\t"
FIELDS = a,b,c,d,e

note: "\t" supposed to be enough, i used both delimiters as i copied to a text file

screenshot:

dont forget to restart splunk on the first full instance that "touches" the data, HF or Indexer/s

hope it helps

Re: Impossible to define fields in transforms.conf.

spisiakmi — Wed, 25 Sep 2019 11:31:07 GMT

Hi Adonio,

thank you for the reaction. The props.conf and the transforms.conf should be defined on the FW or on the Splunk Server site?

Re: Impossible to define fields in transforms.conf.

spisiakmi — Wed, 25 Sep 2019 11:36:44 GMT

And if on the Splunk Server (Indexer) site, the Splunk Server should be restarted?

Re: Impossible to define fields in transforms.conf.

spisiakmi — Wed, 25 Sep 2019 12:34:07 GMT

Hi Adonio,

i made all the steps, you mentioned, also with the restart of the fw. And unfortunatelly only the first row from the file has been indexed and without the field a and the last value from the first row 97. b=1, c=05:45:12, d=first, e=message.
See the screenshots
https://ibb.co/F4MRJKn
https://ibb.co/5RZjZsH

Re: Impossible to define fields in transforms.conf.

adonio — Wed, 25 Sep 2019 14:33:27 GMT

@spisiakmi please read my answer all the way
the configurations should be on the first FULL SPLUNK INSTANCE e.g. Heavy Forwarder OR Indexer/s - not a Universal Forwarder
you need to restart that instance after applying configarions

Re: Impossible to define fields in transforms.conf.

spisiakmi — Wed, 25 Sep 2019 14:50:10 GMT

Thank you. But I have no possibility to restart the Indexer.

Re: Impossible to define fields in transforms.conf.

spisiakmi — Wed, 25 Sep 2019 14:52:17 GMT

I found a solution, which works. Because I have no possibility to restart the Indexer, I created props.conf on UniFW site like this:

props.conf

[tsv_seho_err]
CHARSET = WINDOWS-1252
DATETIME_CONFIG = 
FIELD_DELIMITER = tab
FIELD_NAMES = Fehler, Zeit, Fehlermeldungtext, Fehlernummer
INDEXED_EXTRACTIONS = tsv
LINE_BREAKER = ([\r\n]+)
NO_BINARY_CHECK = true
category = Benutzerdefiniert
description = Tab getrennte Werte ohne Header
pulldown_type = 1

and it works.

Re: Impossible to define fields in transforms.conf.

spisiakmi — Wed, 25 Sep 2019 15:20:05 GMT

And if I want to skip indexing the third column, I can use this syntax
FIELD_NAMES = Fehler, Zeit, , Fehlernummer

Re: Impossible to define fields in transforms.conf.

adonio — Wed, 25 Sep 2019 15:25:34 GMT

good, as long as it is not "Impossible"