topic Re: How to get props and transforms to extract time from source? in Getting Data In

How to get props and transforms to extract time from source?

power12 — Mon, 01 May 2023 11:20:54 GMT

Hello Splunkers ,

I have the following source file which has the date/time in it .. How do I write the props and transforms to use the source date/time as the _time

Below is the sample file

/project/admin/sv/re/sniff/pre/logs/2022-12-16T11-57-36/status

i want the _time or indexed time to be 2022-12-16 11-57-36.

Thanks in Advance

Re: props and transforms to extract time from source

gcusello — Sat, 29 Apr 2023 06:16:03 GMT

Hi @power12,

you have to configure datetime_config.xml and point to it in props.conf:

props.conf:

[your_sourcetype] DATETIME_CONFIG = /opt/splunk/etc/apps/your_app/local/datetime.xml

in datetime_config.xml:

As described at https://docs.splunk.com/Documentation/Splunk/9.0.4/Data/Configuredatetimexml#Examples_of_custom_datetime.xml_configuration

Ciao.

Giuseppe

Re: props and transforms to extract time from source

power12 — Mon, 01 May 2023 18:23:57 GMT

@gcusello Thank you for your message...do you think there is a ) missing before the T

(\d{4})-(\d{2})-(\d{2}T(\d{2})-(\d{2})-(\d{2})

(\d{4})-(\d{2})-(\d{2})T(\d{2})-(\d{2})-(\d{2})

Re: props and transforms to extract time from source

gcusello — Tue, 02 May 2023 06:20:41 GMT

Hi @power12,

yes, correct, sorry!

Ciao.

Giuseppe

Re: props and transforms to extract time from source

gcusello — Thu, 25 May 2023 04:02:26 GMT

hi @power12 ,

good for you, see next time!

Please accept one answer for the other people of Community

Ciao and happy splunking

Giuseppe

P.S.: Karma Points are appreciated 😉

Re: How to get props and transforms to extract time from source?

isoutamo — Thu, 25 May 2023 08:05:09 GMT

Another way is use to INGEST_EVAL. I cannot say which one is more efficient.

Add props.conf

[<source or sourcetype name>] TRANSFORMS-set_time = set_time_from_file_path

transforms.conf

[set_time_from_file_path] INGEST_EVAL = _time = strptime(replace(source, ".*(\d{4}-\d\d-\d\dT\d\d-\d\d-\d\d)/status","\1"),"%Y-%m-%dT%H-%M-%S")

You also need to check TZ information should you add it here or not? Definitely needs if you have source systems on different timezones and/or source systems and indexers / HFs are on different TZ.

r. Ismo