Getting Data In

Need Help with Time Prefix and "|" character

wbfoxii
Communicator

I've got data that looks like this:


YCTC3|YCTC3|A277537|20131013|225102|316739|E|001|TP0|THPNBAV05|10.124.130.71|||||||PAR|A|0000119501|00|||

Date is the fourth column, and time is the fifth. Got any ideas about how to get TIME_PREFIX, TIME_FORMAT and MAX_TIME_LOOKAHEAD to get this right?

My latest try is:


TIME_PREFIX = ^[^|]|[^|]|[^|]*|
TIME_FORMAT = %Y%m%d|%H%M%S
MAX_TIMESTAMP_LOOKAHEAD = 20

Tags (2)

emiller42
Motivator

I think the only issue is your TIME_PREFIX. The regex you have only matches a single non-pipe character between each pipe. For what you have, you want:

TIME_PREFIX = ^[^\|]+\|[^\|]+\|[^\|]+\|

Then the rest should work as intended.

(Replace + with * if any of the preceeding fields might be empty. |||20131013|...)

EDIT: Also need to escape the pipes, as sowings mentioned.

sowings
Splunk Employee
Splunk Employee

Good call, I missed the "only one char" thing.

sowings
Splunk Employee
Splunk Employee

| has special meaning in a regex, you'll have to escape it with a \.

TIME_PREFIX= ^[^\|]\|{3}

There are three groups of "non-pipe characters followed by a pipe".

emiller42
Motivator

I think the {3} only applies to the previous token, so you'd have to group before using it for it to apply to the whole pattern.

^(?:[^\|]+\|){3}
0 Karma

wbfoxii
Communicator

This is the one that eventually worked. I didn't test the others too hard. This one looked elegant.

0 Karma
Get Updates on the Splunk Community!

Observe and Secure All Apps with Splunk

  Join Us for Our Next Tech Talk: Observe and Secure All Apps with SplunkAs organizations continue to innovate ...

Splunk Decoded: Business Transactions vs Business IQ

It’s the morning of Black Friday, and your e-commerce site is handling 10x normal traffic. Orders are flowing, ...

Fastest way to demo Observability

I’ve been having a lot of fun learning about Kubernetes and Observability. I set myself an interesting ...