Getting Data In

Why is my tsv data out of order

tkwaller_2
Communicator

Hello

We are parsing data from a TSV source

The data file has a header that is very long, about 281 columns.
What is happening is that we are getting data in the wrong fields.
For example:

Field:
data_poc_technical_name

Values:  
todd.waller@toddwaller.com

this should be the value in the field: data_poc_technical_email.
I also notice that when exporting the data its out of order as well, maybe the issue lies in parsing and configs?
Originally when I tested I got field names in the fields values, I think when data was NULL so I removed FIELD_NAMES from the props and it seemed to have parsed correctly but now doesnt look that way.

These are the props on the UF

[fp:tsv]
FIELD_DELIMITER = \t
HEADER_FIELD_DELIMITER = \t
INDEXED_EXTRACTIONS = TSV
TIMESTAMP_FIELDS = md_createdAt

and the props from the indexer

[fp:tsv]
FIELD_DELIMITER = \t
HEADER_FIELD_DELIMITER = \t
TIMESTAMP_FIELDS = md_createdAt
KV_MODE = none

Any thoughts?
Thanks for the help!
Todd

0 Karma
1 Solution

tkwaller_2
Communicator

I figured this out
It seems the issue was timestamping. Once I fixed the timestamp field recognition and reindexed the data it seems to be correct now.

My edited props on the UF were

[fp:tsv]
TIME_FORMAT = %m/%d/%Y %H:%M:%S
FIELD_DELIMITER = \t
HEADER_FIELD_DELIMITER = \t
INDEXED_EXTRACTIONS = TSV
TIMESTAMP_FIELDS = md.createdAt

In case it helps anyone else.

View solution in original post

0 Karma

tkwaller_2
Communicator

I figured this out
It seems the issue was timestamping. Once I fixed the timestamp field recognition and reindexed the data it seems to be correct now.

My edited props on the UF were

[fp:tsv]
TIME_FORMAT = %m/%d/%Y %H:%M:%S
FIELD_DELIMITER = \t
HEADER_FIELD_DELIMITER = \t
INDEXED_EXTRACTIONS = TSV
TIMESTAMP_FIELDS = md.createdAt

In case it helps anyone else.

0 Karma
Get Updates on the Splunk Community!

Extending Observability Content to Splunk Cloud

Watch Now!   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to leverage ...

More Control Over Your Monitoring Costs with Archived Metrics!

What if there was a way you could keep all the metrics data you need while saving on storage costs?This is now ...

New in Observability Cloud - Explicit Bucket Histograms

Splunk introduces native support for histograms as a metric data type within Observability Cloud with Explicit ...