topic Use source path time when ingesting data in Getting Data In

Use source path time when ingesting data

power12 — Tue, 27 Feb 2024 21:15:03 GMT

I have the following source .I want to extract time from source when data is ingesting

source="/logs/gs/ute-2024-02-05a/2024-02-05_16-17-54/abc.log"

in props

TRANSFORMS-set_time =source_path_time

In transforms

[set_time_from_file_path] INGEST_EVAL = | eval _time = strptime(replace(source, ".*/ute-(\\d{4}-\\d{2}-\\d{2}[a-z])/([^/]+/[^/]+).*","\1"),"%y-%m-%d_%H-%M-%S")

I tried testing it but I am unable to get the _time

| makeresults | eval source="/logs/gs/ute-2024-02-05a/2024-02-05_16-17-54/abc.log" | fields - _time ``` above set test data ``` | eval _time = strptime(replace(source, ".*/compute-(\\d{4}-\\d{2}-\\d{2}[a-z])/([^/]+/[^/]+).*","\1"),"%y-%m-%d_%H-%M-%S")

Thanks in Advance

Re: Use source path time when ingesting data

victor_menezes — Wed, 28 Feb 2024 02:52:19 GMT

Your props is not matching the stanza name of transforms. Not sure if that was a typo...
About a typo, you don't need that first pipe in the ingest_eval. Try this instead (I changed the regex a bit)

Props.conf:
[your_sourcetype]
TRANSFORMS-set_time = set_time_from_file_path

Transforms.conf
[set_time_from_file_path]
INGEST_EVAL = eval _time = strptime(replace(source, ".*/ute-(\\d{4}-\\d{2}-\\d{2}[a-z]+)/([^/]+/[^/]+).*","\\1"), "%Y-%m-%d_%H-%M-%S")

Re: Use source path time when ingesting data

power12 — Wed, 28 Feb 2024 14:59:48 GMT

Thank You @victor_menezes .I tried below and it worked .

| eval _time = strptime(replace(source, ".*(\d\d\d\d-\d\d-\d\d\_\d\d-\d\d-\d\d).*","\1"),"%Y-%m-%d_%H-%M-%S")

Re: Use source path time when ingesting data

victor_menezes — Wed, 28 Feb 2024 15:30:43 GMT

Awesome! Glad to know that.

Please remember to mark this as resolved so others can know about it.

Happy splunking!