Splunk Search

Date Format and Time Format

malekmo
Contributor

Hello,

What would be my TIME_FORMAT for prop configuration file for this events

2021-06-08T13:26:53.665000-04:00|PGM|mtb1120ppcdwap6|vggtb|26462|

2021-06-08T13:26:54.478000-04:00|PGM|mtb1120ppcdwap6|vggtb|26462|

I wrote this not covering entire range

TIME_PREFIX=^

TIME_FORMAT=%Y-%m-%dT%H:%M:%S.%f%z

 

Any help will be highly appreciated. Thank you so much.

 

 

Labels (1)
Tags (1)
0 Karma
1 Solution

gcusello
SplunkTrust
SplunkTrust

Hi @malekmo,

just a little update:

TIME_PREFIX=^
TIME_FORMAT=%Y-%m-%dT%H:%M:%S.%6N%:z

because you have 6 milliseconds digits and in your timezone you have the format -5:00

For more infos see at https://docs.splunk.com/Documentation/Splunk/8.2.1/SearchReference/Commontimeformatvariables

Ciao.

Giuseppe

View solution in original post

0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @malekmo,

just a little update:

TIME_PREFIX=^
TIME_FORMAT=%Y-%m-%dT%H:%M:%S.%6N%:z

because you have 6 milliseconds digits and in your timezone you have the format -5:00

For more infos see at https://docs.splunk.com/Documentation/Splunk/8.2.1/SearchReference/Commontimeformatvariables

Ciao.

Giuseppe

View solution in original post

0 Karma

malekmo
Contributor

Hello, since event has the pipe "|" ...I wanted to use following props conf ...but not working.., any help will be highly appreciated!

SHOULD_LINEMERGE = false

LINE_BREAKER = ([\r\n]+)

INDEXED_EXTRACTIONS = psv

TIME_FORMAT = %Y%m%d %H:%M:%S:%Q

TIMESTAMP_FIELDS = TIMESTAMP

0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @malekmo,

to use indexed extractions, you have to define:

  • the kind of indexed extraction, in your case psv,
  • the separator, in your case pipe "|",
  • the field list.

About timestamp, if it's raining the above extraction, I'd use it

Anyway, please try something like this:

 

SHOULD_LINEMERGE = false
LINE_BREAKER = ([\r\n]+)
INDEXED_EXTRACTIONS = psv
TIME_PREFIX=^
TIME_FORMAT=%Y-%m-%dT%H:%M:%S.%6N%:z
TIMESTAMP_FIELDS = TIMESTAMP
PREAMBLE_REGEX = ^ost:
FIELD_DELIMITER = |
FIELD_NAMES = TimeStamp, field2, field3, field4, field5

 

Ciao.

Giuseppe

P.S.: Karma Points are appreciated 😉

malekmo
Contributor

Perfect ...working as expected, thank you so much ...appreciated.....just one more issue... my source is text file....how would I make my PROPS Conf file not to read first line ....as first line is not an event..

ost: 'XXXpcdwa', OS: 'LIN X64', Release: '35.0.0-X1127.19.1.ex7.x86_128',  Version: '

2021-06-08T13:26:53.665000-04:00|PGM|mtb1120ppcdwap6|vggtb|26462|

2021-06-08T13:26:54.478000-04:00|PGM|mtb1120ppcdwap6|vggtb|26462|

 

 

 

 

0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @malekmo,

good for you, please accept my answer for the other people of Community.

About log filtering, if you can find a regex (e.g. in your case "^ost:"), you can filter your data flow excluding events that match the regex, following the configuration at https://docs.splunk.com/Documentation/SplunkCloud/latest/Forwarding/Routeandfilterdatad#Discard_spec...

In your case:

props.conf

[your_sourcetype]
TRANSFORMS-null= setnull

transforms.conf

[setnull]
REGEX = ^ost:
DEST_KEY = queue
FORMAT = nullQueue

Ciao.

Giuseppe

P.S.: Karma Points are appreciated 😉

0 Karma