topic Re: Date Format and Time Format in Splunk Search

Date Format and Time Format

SplunkDash — Thu, 12 Aug 2021 15:54:20 GMT

Hello,

What would be my TIME_FORMAT for prop configuration file for this events

2021-06-08T13:26:53.665000-04:00|PGM|mtb1120ppcdwap6|vggtb|26462|

2021-06-08T13:26:54.478000-04:00|PGM|mtb1120ppcdwap6|vggtb|26462|

I wrote this not covering entire range

TIME_PREFIX=^

TIME_FORMAT=%Y-%m-%dT%H:%M:%S.%f%z

Any help will be highly appreciated. Thank you so much.

Re: Date Format and Time Format

gcusello — Thu, 12 Aug 2021 16:09:40 GMT

Hi @SplunkDash,

just a little update:

TIME_PREFIX=^ TIME_FORMAT=%Y-%m-%dT%H:%M:%S.%6N%:z

because you have 6 milliseconds digits and in your timezone you have the format -5:00

For more infos see at https://docs.splunk.com/Documentation/Splunk/8.2.1/SearchReference/Commontimeformatvariables

Ciao.

Giuseppe

Re: Date Format and Time Format

SplunkDash — Thu, 12 Aug 2021 16:26:18 GMT

Perfect ...working as expected, thank you so much ...appreciated.....just one more issue... my source is text file....how would I make my PROPS Conf file not to read first line ....as first line is not an event..

ost: 'XXXpcdwa', OS: 'LIN X64', Release: '35.0.0-X1127.19.1.ex7.x86_128', Version: '

2021-06-08T13:26:53.665000-04:00|PGM|mtb1120ppcdwap6|vggtb|26462|

2021-06-08T13:26:54.478000-04:00|PGM|mtb1120ppcdwap6|vggtb|26462|

Re: Date Format and Time Format

gcusello — Thu, 12 Aug 2021 16:33:23 GMT

Hi @SplunkDash,

good for you, please accept my answer for the other people of Community.

About log filtering, if you can find a regex (e.g. in your case "^ost:"), you can filter your data flow excluding events that match the regex, following the configuration at https://docs.splunk.com/Documentation/SplunkCloud/latest/Forwarding/Routeandfilterdatad#Discard_specific_events_and_keep_the_rest

In your case:

props.conf

[your_sourcetype] TRANSFORMS-null= setnull

transforms.conf

[setnull] REGEX = ^ost: DEST_KEY = queue FORMAT = nullQueue

Ciao.

Giuseppe

P.S.: Karma Points are appreciated 😉

Re: Date Format and Time Format

SplunkDash — Thu, 12 Aug 2021 17:08:38 GMT

Hello, since event has the pipe "|" ...I wanted to use following props conf ...but not working.., any help will be highly appreciated!

SHOULD_LINEMERGE = false

LINE_BREAKER = ([\r\n]+)

INDEXED_EXTRACTIONS = psv

TIME_FORMAT = %Y%m%d %H:%M:%S:%Q

TIMESTAMP_FIELDS = TIMESTAMP

Re: Date Format and Time Format

gcusello — Fri, 13 Aug 2021 07:03:05 GMT

Hi @SplunkDash,

to use indexed extractions, you have to define:

the kind of indexed extraction, in your case psv,
the separator, in your case pipe "|",
the field list.

About timestamp, if it's raining the above extraction, I'd use it

Anyway, please try something like this:

SHOULD_LINEMERGE = false LINE_BREAKER = ([\r\n]+) INDEXED_EXTRACTIONS = psv TIME_PREFIX=^ TIME_FORMAT=%Y-%m-%dT%H:%M:%S.%6N%:z TIMESTAMP_FIELDS = TIMESTAMP PREAMBLE_REGEX = ^ost: FIELD_DELIMITER = | FIELD_NAMES = TimeStamp, field2, field3, field4, field5

Ciao.

Giuseppe

P.S.: Karma Points are appreciated 😉