Splunk Search

Date Format and Time Format

SplunkDash
Motivator

Hello,

What would be my TIME_FORMAT for prop configuration file for this events

2021-06-08T13:26:53.665000-04:00|PGM|mtb1120ppcdwap6|vggtb|26462|

2021-06-08T13:26:54.478000-04:00|PGM|mtb1120ppcdwap6|vggtb|26462|

I wrote this not covering entire range

TIME_PREFIX=^

TIME_FORMAT=%Y-%m-%dT%H:%M:%S.%f%z

 

Any help will be highly appreciated. Thank you so much.

 

 

Labels (1)
Tags (1)
0 Karma
1 Solution

gcusello
SplunkTrust
SplunkTrust

Hi @SplunkDash,

just a little update:

TIME_PREFIX=^
TIME_FORMAT=%Y-%m-%dT%H:%M:%S.%6N%:z

because you have 6 milliseconds digits and in your timezone you have the format -5:00

For more infos see at https://docs.splunk.com/Documentation/Splunk/8.2.1/SearchReference/Commontimeformatvariables

Ciao.

Giuseppe

View solution in original post

0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @SplunkDash,

just a little update:

TIME_PREFIX=^
TIME_FORMAT=%Y-%m-%dT%H:%M:%S.%6N%:z

because you have 6 milliseconds digits and in your timezone you have the format -5:00

For more infos see at https://docs.splunk.com/Documentation/Splunk/8.2.1/SearchReference/Commontimeformatvariables

Ciao.

Giuseppe

0 Karma

SplunkDash
Motivator

Hello, since event has the pipe "|" ...I wanted to use following props conf ...but not working.., any help will be highly appreciated!

SHOULD_LINEMERGE = false

LINE_BREAKER = ([\r\n]+)

INDEXED_EXTRACTIONS = psv

TIME_FORMAT = %Y%m%d %H:%M:%S:%Q

TIMESTAMP_FIELDS = TIMESTAMP

0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @SplunkDash,

to use indexed extractions, you have to define:

  • the kind of indexed extraction, in your case psv,
  • the separator, in your case pipe "|",
  • the field list.

About timestamp, if it's raining the above extraction, I'd use it

Anyway, please try something like this:

 

SHOULD_LINEMERGE = false
LINE_BREAKER = ([\r\n]+)
INDEXED_EXTRACTIONS = psv
TIME_PREFIX=^
TIME_FORMAT=%Y-%m-%dT%H:%M:%S.%6N%:z
TIMESTAMP_FIELDS = TIMESTAMP
PREAMBLE_REGEX = ^ost:
FIELD_DELIMITER = |
FIELD_NAMES = TimeStamp, field2, field3, field4, field5

 

Ciao.

Giuseppe

P.S.: Karma Points are appreciated 😉

SplunkDash
Motivator

Perfect ...working as expected, thank you so much ...appreciated.....just one more issue... my source is text file....how would I make my PROPS Conf file not to read first line ....as first line is not an event..

ost: 'XXXpcdwa', OS: 'LIN X64', Release: '35.0.0-X1127.19.1.ex7.x86_128',  Version: '

2021-06-08T13:26:53.665000-04:00|PGM|mtb1120ppcdwap6|vggtb|26462|

2021-06-08T13:26:54.478000-04:00|PGM|mtb1120ppcdwap6|vggtb|26462|

 

 

 

 

0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @SplunkDash,

good for you, please accept my answer for the other people of Community.

About log filtering, if you can find a regex (e.g. in your case "^ost:"), you can filter your data flow excluding events that match the regex, following the configuration at https://docs.splunk.com/Documentation/SplunkCloud/latest/Forwarding/Routeandfilterdatad#Discard_spec...

In your case:

props.conf

[your_sourcetype]
TRANSFORMS-null= setnull

transforms.conf

[setnull]
REGEX = ^ost:
DEST_KEY = queue
FORMAT = nullQueue

Ciao.

Giuseppe

P.S.: Karma Points are appreciated 😉

0 Karma
Get Updates on the Splunk Community!

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...