Getting Data In

Impossible to define fields in transforms.conf.

spisiakmi
Communicator

Hi,

I have simple tab delimited text file.

1 05:45:12 first message 97
1 05:52:15 second message 110
1 05:52:46 third message 97
1 05:53:09 fourth message 110

I want to index it with header definined in transforms.conf
Here are my config files:

**inputs.conf**

[monitor://c:\temp\seho\err\]
disabled = false
index = seho_err_tmp
sourcetype = tsv_WINDOWS-1252
crcSalt=

**props.conf**

[tsv_WINDOWS-1252]
BREAK_ONLY_BEFORE_DATE = 
CHARSET = WINDOWS-1252
INDEXED_EXTRACTIONS = tsv
KV_MODE = none
LINE_BREAKER = ([\r\n]+)
NO_BINARY_CHECK = true
SHOULD_LINEMERGE = false
category = Structured
description = Tab-separated value format. Set header and other settings in "Delimited Settings"
disabled = false
pulldown_type = 1
REPORT-getfields=seho_err_fields

transforms.conf

[seho_err_fields]
DELIMS=":\t"
FIELDS=Fehler,Zeit,Fehlermeldungtext,Fehlernummer

I tried also \t, "\t".

The defined fields never appear in Splunk and the first row from the file is defined as a header by default. Can anybody help me, please?

0 Karma
1 Solution

spisiakmi
Communicator

I found a solution, which works. Because I have no possibility to restart the Indexer, I created props.conf on UniFW site like this:

props.conf

[tsv_seho_err]
CHARSET = WINDOWS-1252
DATETIME_CONFIG = 
FIELD_DELIMITER = tab
FIELD_NAMES = Fehler, Zeit, Fehlermeldungtext, Fehlernummer
INDEXED_EXTRACTIONS = tsv
LINE_BREAKER = ([\r\n]+)
NO_BINARY_CHECK = true
category = Benutzerdefiniert
description = Tab getrennte Werte ohne Header
pulldown_type = 1

and it works.

View solution in original post

0 Karma

spisiakmi
Communicator

I found a solution, which works. Because I have no possibility to restart the Indexer, I created props.conf on UniFW site like this:

props.conf

[tsv_seho_err]
CHARSET = WINDOWS-1252
DATETIME_CONFIG = 
FIELD_DELIMITER = tab
FIELD_NAMES = Fehler, Zeit, Fehlermeldungtext, Fehlernummer
INDEXED_EXTRACTIONS = tsv
LINE_BREAKER = ([\r\n]+)
NO_BINARY_CHECK = true
category = Benutzerdefiniert
description = Tab getrennte Werte ohne Header
pulldown_type = 1

and it works.

0 Karma

spisiakmi
Communicator

And if I want to skip indexing the third column, I can use this syntax
FIELD_NAMES = Fehler, Zeit, , Fehlernummer

0 Karma

adonio
Ultra Champion

good, as long as it is not "Impossible"

0 Karma

adonio
Ultra Champion

nothing better then a question with "Impossible" at the headline
here are the steps to accomplish:
your data created in a file tsv_no_header.txt

1 05:45:12 first message 97
1 05:52:15 second message 110
1 05:52:46 third message 97
1 05:53:09 fourth message 110

in props.conf

[tsv_no_header]
SHOULD_LINEMERGE = false
REPORT-no_header = no_header
LINE_BREAKER = ([\r\n]+)

in transforms.conf

[no_header]
DELIMS = " ","\t"
FIELDS = a,b,c,d,e

note: "\t" supposed to be enough, i used both delimiters as i copied to a text file

screenshot:

alt text

dont forget to restart splunk on the first full instance that "touches" the data, HF or Indexer/s

hope it helps

0 Karma

spisiakmi
Communicator

Hi Adonio,

i made all the steps, you mentioned, also with the restart of the fw. And unfortunatelly only the first row from the file has been indexed and without the field a and the last value from the first row 97. b=1, c=05:45:12, d=first, e=message.
See the screenshots
https://ibb.co/F4MRJKn
https://ibb.co/5RZjZsH

0 Karma

adonio
Ultra Champion

@spisiakmi please read my answer all the way
the configurations should be on the first FULL SPLUNK INSTANCE e.g. Heavy Forwarder OR Indexer/s - not a Universal Forwarder
you need to restart that instance after applying configarions

0 Karma

spisiakmi
Communicator

Thank you. But I have no possibility to restart the Indexer.

0 Karma

spisiakmi
Communicator

Hi Adonio,

thank you for the reaction. The props.conf and the transforms.conf should be defined on the FW or on the Splunk Server site?

0 Karma

spisiakmi
Communicator

And if on the Splunk Server (Indexer) site, the Splunk Server should be restarted?

0 Karma
Get Updates on the Splunk Community!

Last Chance to Submit Your Paper For BSides Splunk - Deadline is August 12th!

Hello everyone! Don't wait to submit - The deadline is August 12th! We have truly missed the community so ...

Ready, Set, SOAR: How Utility Apps Can Up Level Your Playbooks!

 WATCH NOW Powering your capabilities has never been so easy with ready-made Splunk® SOAR Utility Apps. Parse ...

DevSecOps: Why You Should Care and How To Get Started

 WATCH NOW In this Tech Talk we will talk about what people mean by DevSecOps and deep dive into the different ...