Date Format and Time Format

[SplunkDash]

Hello,

What would be my TIME_FORMAT for prop configuration file for this events

2021-06-08T13:26:53.665000-04:00|PGM|mtb1120ppcdwap6|vggtb|26462|

2021-06-08T13:26:54.478000-04:00|PGM|mtb1120ppcdwap6|vggtb|26462|

I wrote this not covering entire range

TIME_PREFIX=^

TIME_FORMAT=%Y-%m-%dT%H:%M:%S.%f%z

Any help will be highly appreciated. Thank you so much.

[gcusello · Accepted solution]

Hi @SplunkDash,

just a little update:

TIME_PREFIX=^
TIME_FORMAT=%Y-%m-%dT%H:%M:%S.%6N%:z

because you have 6 milliseconds digits and in your timezone you have the format -5:00

For more infos see at https://docs.splunk.com/Documentation/Splunk/8.2.1/SearchReference/Commontimeformatvariables

Ciao.

Giuseppe

[SplunkDash]

Hello, since event has the pipe "|" ...I wanted to use following props conf ...but not working.., any help will be highly appreciated!

SHOULD_LINEMERGE = false

LINE_BREAKER = ([\r\n]+)

INDEXED_EXTRACTIONS = psv

TIME_FORMAT = %Y%m%d %H:%M:%S:%Q

TIMESTAMP_FIELDS = TIMESTAMP

[gcusello]

Hi @SplunkDash,

to use indexed extractions, you have to define:

• the kind of indexed extraction, in your case psv,
• the separator, in your case pipe "|",
• the field list.

About timestamp, if it's raining the above extraction, I'd use it

Anyway, please try something like this:

SHOULD_LINEMERGE = false
LINE_BREAKER = ([\r\n]+)
INDEXED_EXTRACTIONS = psv
TIME_PREFIX=^
TIME_FORMAT=%Y-%m-%dT%H:%M:%S.%6N%:z
TIMESTAMP_FIELDS = TIMESTAMP
PREAMBLE_REGEX = ^ost:
FIELD_DELIMITER = |
FIELD_NAMES = TimeStamp, field2, field3, field4, field5

Ciao.

Giuseppe

P.S.: Karma Points are appreciated 😉

[SplunkDash]

Perfect ...working as expected, thank you so much ...appreciated.....just one more issue... my source is text file....how would I make my PROPS Conf file not to read first line ....as first line is not an event..

ost: 'XXXpcdwa', OS: 'LIN X64', Release: '35.0.0-X1127.19.1.ex7.x86_128',  Version: '

2021-06-08T13:26:53.665000-04:00|PGM|mtb1120ppcdwap6|vggtb|26462|

2021-06-08T13:26:54.478000-04:00|PGM|mtb1120ppcdwap6|vggtb|26462|

[gcusello]

Hi @SplunkDash,

good for you, please accept my answer for the other people of Community.

About log filtering, if you can find a regex (e.g. in your case "^ost:"), you can filter your data flow excluding events that match the regex, following the configuration at https://docs.splunk.com/Documentation/SplunkCloud/latest/Forwarding/Routeandfilterdatad#Discard_spec...

In your case:

props.conf

[your_sourcetype]
TRANSFORMS-null= setnull

transforms.conf

[setnull]
REGEX = ^ost:
DEST_KEY = queue
FORMAT = nullQueue

Ciao.

Giuseppe

P.S.: Karma Points are appreciated 😉