@blbr123 Here is the tested settings, they need to be split among Uf/HF if you have intermediate forwarder (HF). Copy these settings to both UF and HF, if no intermediate forwarder copy it to Indexer. There are some settings don't need to be on HF however no harm it just ignores! I would recommend to test the CSV with 'Add Data' option in dev and load to see with props settings changing to get the desired output. Field names might be different in your case, ## props.conf
[set-csv-test]
INDEXED_EXTRACTIONS = CSV
TIMESTAMP_FIELDS = Datetime
FIELD_NAMES = Units,Variable_code,Variable_name,Variable_category, Datetime
HEADER_FIELD_LINE_NUMBER = 1
TIME_FORMAT = %d/%m/%Y %H:%M:%S
TRANSFORMS-remove-header= Testing-header-removal-csv
# transforms.conf
[Testing-header-removal-csv]
REGEX = ^Units\,
DEST_KEY = queue
FORMAT = nullQueue CSV sample tested: Units,Variable_code,Variable_name,Variable_category,Datetime
Dollars (millions),H01,Total income,Financial performance, 05/04/2022 11:00:00
Dollars (millions),H04,Sales, government funding grants and subsidies Financial performance, 05/04/2022 11:00:01
Dollars (millions),H05,Interest, dividends and donations Financial performance, 05/04/2022 11:00:02
Dollars (millions),H07,Non-operating income,Financial performance, 05/04/2022 11:00:03
Dollars (millions),H08,Total expenditure,Financial performance, 05/04/2022 11:00:04
Dollars (millions),H09,Interest and donations,Financial performance, 05/04/2022 11:00:05
Dollars (millions),H10,Indirect taxes,Financial performance, 05/04/2022 11:00:06
Dollars (millions),H11,Depreciation,Financial performance, 05/04/2022 11:00:07
Dollars (millions),H12,Salaries and wages paid,Financial performance, 05/04/2022 11:00:08
Dollars (millions),H13,Redundancy and severance,Financial performance, 05/04/2022 11:00:09
Dollars (millions),H14,Salaries and wages to self employed commission agents,Financial performance, 05/04/2022 11:00:10
Dollars (millions),H19,Purchases and other operating expenses,Financial performance, 05/04/2022 11:00:11
Dollars (millions),H20,Non-operating expenses,Financial performance, 05/04/2022 11:00:12 Tested final look, no header and timestamp parsed and header being extracted as fields. Hope this helps!
... View more