Getting Data In

Why is my tsv data out of order

Communicator

Hello

We are parsing data from a TSV source

The data file has a header that is very long, about 281 columns.
What is happening is that we are getting data in the wrong fields.
For example:

Field:
data_poc_technical_name

Values:  
todd.waller@toddwaller.com

this should be the value in the field: datapoctechnicalemail.
I also notice that when exporting the data its out of order as well, maybe the issue lies in parsing and configs?
Originally when I tested I got field names in the fields values, I think when data was NULL so I removed FIELD
NAMES from the props and it seemed to have parsed correctly but now doesnt look that way.

These are the props on the UF

[fp:tsv]
FIELD_DELIMITER = \t
HEADER_FIELD_DELIMITER = \t
INDEXED_EXTRACTIONS = TSV
TIMESTAMP_FIELDS = md_createdAt

and the props from the indexer

[fp:tsv]
FIELD_DELIMITER = \t
HEADER_FIELD_DELIMITER = \t
TIMESTAMP_FIELDS = md_createdAt
KV_MODE = none

Any thoughts?
Thanks for the help!
Todd

0 Karma
1 Solution

Communicator

I figured this out
It seems the issue was timestamping. Once I fixed the timestamp field recognition and reindexed the data it seems to be correct now.

My edited props on the UF were

[fp:tsv]
TIME_FORMAT = %m/%d/%Y %H:%M:%S
FIELD_DELIMITER = \t
HEADER_FIELD_DELIMITER = \t
INDEXED_EXTRACTIONS = TSV
TIMESTAMP_FIELDS = md.createdAt

In case it helps anyone else.

View solution in original post

0 Karma

Communicator

I figured this out
It seems the issue was timestamping. Once I fixed the timestamp field recognition and reindexed the data it seems to be correct now.

My edited props on the UF were

[fp:tsv]
TIME_FORMAT = %m/%d/%Y %H:%M:%S
FIELD_DELIMITER = \t
HEADER_FIELD_DELIMITER = \t
INDEXED_EXTRACTIONS = TSV
TIMESTAMP_FIELDS = md.createdAt

In case it helps anyone else.

View solution in original post

0 Karma