Hi all,
I'm in enviroment so configured:
1 uf, 1 hf, 4 indexers, 1 search head, 1 master cluster.
I've to index a large CSV, read from the universal forwarder, which forwards data to the HF which pass the data to the indexer.
The CSV has 150 fields and I want to index only 10 of these. So I've configured these things:
on universal forwarder:
------------------
inputs.conf
------------------
[monitor:///myfolder/Interface*]
disabled = 0
index = interface_metrics
sourcetype = if_csv
on heavy forwarder
------------------
inputs.conf
------------------
[splunktcp://9996]
index=interface_metrics
sourcetype = if_csv
------------------
props.conf
------------------
[if_csv]
INDEXED_EXTRACTIONS = CSV
HEADER_FIELD_LINE_NUMBER=1
HEADER_FIELD_DELIMITER =,
FIELD_DELIMITER=,
HEADER_FIELD_LINE_NUMBER = 0
TRANSFORMS-set=setnull, setparsing, nullhead
------------------
transforms.conf
------------------
[setnull]
REGEX = .
DEST_KEY = queue
FORMAT = nullQueue
[nullhead]
REGEX = ifInDiscardsDelta
DEST_KEY = queue
FORMAT = nullQueue
[setparsing]
REGEX = ^([^,]),([^,]),(?:[^,]+,\s*)([^,]),([^,])(?:[^,]+,\s*){5}([^,]),([^,])(?:[^,]+,\s*){3}([^,]),([^,]),(?:[^,]+,\s*){2}([^,]),([^,])(?:[^,]+,\s*){7}([^,])(?:(?:[^,]+,?\s)|(?:[,,])){123}([^,]),([^,])
DEST_KEY = queue
FORMAT = indexQueue
example CSV row :
0ef1fa5f-586c-48a4-a902-827aef967f47,1569309580446,300.0,100,9,0,0,0,0,6.6107712E7,5.0463189E7,151356.0,150857.0,0.176,0.135,0.0,0.0,0.0,0.0,0,0,0,0,0,0,4b16e13e-c391-4626-b364-2890fe5a009a,0,0,0,0,,,151351,149267,0,0,451,5,1139,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,039550ed-1d39-487f-9b12-276ad9472771,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,3.0,3.0,,,0.056,3.4E-4,0.2,3.0,3.0,,,0.056,3.4E-4,0.2,1569309300000,300
I want to keep the fields:
1,2,4,5,10,11,14,15,26,149,150
I don't succeed in indexing only the fields that I choose, but the whole row.
What I'm wrong ?
Thanks
Fabrizio
