topic Re: Need to write a regex for time extraction in Getting Data In

Need to write a regex for time extraction

Praz_123 — Wed, 28 May 2025 04:53:27 GMT

Need to write a regex for same as time and same as event given below in image

Re: Need to write a regex for time extraction

kiran_panchavat — Wed, 28 May 2025 05:33:52 GMT

@Praz_123

\b\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}(?:\.000)?\b

Re: Need to write a regex for time extraction

Praz_123 — Wed, 28 May 2025 05:39:36 GMT

@kiran_panchavat
am getting the error as I need the same date and time extraction while using the time format and time prefix am getting the below error

Below is my props.conf

[ _json ]
SHOULD_LINEMERGE=false
LINE_BREAKER=([\S\s\n]+"predictions":\s\[\s*)|}(\s*\,\s*){|([\s\n\r]*\][\s\n\r]*}[\s\n\r]*)
NO_BINARY_CHECK=true
TIME_FORMAT=%Y-%m-%dT%H:%M:%S
TIME_PREFIX=\[|ds\"\:\s\".

Re: Need to write a regex for time extraction

kiran_panchavat — Wed, 28 May 2025 05:40:59 GMT

@Praz_123

Let me try in my lab and get back to you shortly.

Re: Need to write a regex for time extraction

Praz_123 — Wed, 28 May 2025 05:43:28 GMT

@kiran_panchavat

Same data would be like :-

{
"version": "200",
"predictions": [
{
"ds": "2023-01-01T01:00:00",
"y": 25727,
"yhat_lower": 23595.643771045987,
"yhat_upper": 26531.786203915904,
"marginal_upper": 26838.980030149163,
"marginal_lower": 23183.715141246714,
"anomaly": false
},
{
"ds": "2023-01-01T02:00:00",
"y": 24710,
"yhat_lower": 21984.478022195697,
"yhat_upper": 24966.416390280523,
"marginal_upper": 25457.020250925423,
"marginal_lower": 21744.743048120385,
"anomaly": false
},
{
"ds": "2023-01-01T03:00:00",
"y": 23908,
"yhat_lower": 21181.498740796877,
"yhat_upper": 24172.09825724038,
"marginal_upper": 24449.705257711226,
"marginal_lower": 20726.645610860345,
"anomaly": false
},

Re: Need to write a regex for time extraction

kiran_panchavat — Wed, 28 May 2025 05:54:38 GMT

@Praz_123

Check this

Re: Need to write a regex for time extraction

Praz_123 — Wed, 28 May 2025 06:17:48 GMT

@kiran_panchavat

Not working for me are you using different event breaks and timestamp

I used the below props.conf

[ <SOURCETYPE NAME> ]
CHARSET=AUTO
SHOULD_LINEMERGE=true
LINE_BREAKER=([\r\n]+)
TIME_FORMAT=%Y-%m-%dT%H:%M:%S
TIME_PREFIX= "ds":\s*"

Re: Need to write a regex for time extraction

livehybrid — Wed, 28 May 2025 06:23:43 GMT

Hi @Praz_123

Did the time extraction I provided in the previous thread not work for you for some reason?

TIME_PREFIX="ds":\s" TIME_FORMAT=%Y-%m-%dT%H:%M:%S MAX_TIMESTAMP_LOOKAHEAD=20

The reason for your error is the extra "." you have in your TIME_PREFIX which is causing it to skip the first character of the year. Also you need to specify the MAX_TIMESTAMP_LOOKAHEAD.

Below is my previous response incase you missed it.

@livehybrid wrote:

[yourSourceType] SHOULD_LINEMERGE=false LINE_BREAKER=([\S\s\n]+"predictions":\s\[\s*)|}(\s*\,\s*){|([\s\n\r]*\][\s\n\r]*}[\s\n\r]*) NO_BINARY_CHECK=true TIME_PREFIX="ds":\s" TIME_FORMAT=%Y-%m-%dT%H:%M:%S MAX_TIMESTAMP_LOOKAHEAD=20

🌟 Did this answer help you? If so, please consider:

Adding karma to show it was useful
Marking it as the solution if it resolved your issue
Commenting if you need any clarification

Your feedback encourages the volunteers in this community to continue contributing

Re: Need to write a regex for time extraction

kiran_panchavat — Wed, 28 May 2025 06:27:05 GMT

@Praz_123

Check the data which you uploaded it should be .json format not .txt format.

[ jsontest ]
CHARSET=UTF-8
LINE_BREAKER=([\S\s\n]+"predictions":\s\[\s*)|}(\s*\,\s*){|([\s\n\r]*\][\s\n\r]*}[\s\n\r]*)
NO_BINARY_CHECK=true
SHOULD_LINEMERGE=true
category=Custom
pulldown_type=true

Re: Need to write a regex for time extraction

Praz_123 — Wed, 28 May 2025 06:39:30 GMT

@kiran_panchavat

Same issue again if I take in json or in txt both are not giving me results

Re: Need to write a regex for time extraction

kiran_panchavat — Wed, 28 May 2025 06:44:35 GMT

@Praz_123

Select source type something like, I can see it's showing default

********* Ignore this *****

Re: Need to write a regex for time extraction

Praz_123 — Wed, 28 May 2025 06:47:33 GMT

@kiran_panchavat

I got the answer thanks as I used the below props.conf , @livehybrid thanks for your help

[ <SOURCETYPE NAME> ]
LINE_BREAKER=([\S\s\n]+"predictions":\s\[\s*)|}(\s*\,\s*){|([\s\n\r]*\][\s\n\r]*}[\s\n\r]*)
NO_BINARY_CHECK=true
TIME_FORMAT=%Y-%m-%dT%H:%M:%S
TIME_PREFIX="ds":\s"
MAX_TIMESTAMP_LOOKAHEAD=20
MAX_DAYS_AGO=5000

Re: Need to write a regex for time extraction

livehybrid — Wed, 28 May 2025 06:48:05 GMT

Hi @Praz_123 @kiran_panchavat

You need to set MAX_DAYS_AGO=5000 and MAX_TIMESTAMP_LOOKAHEAD=20

🌟 Did this answer help you? If so, please consider:

Adding karma to show it was useful
Marking it as the solution if it resolved your issue
Commenting if you need any clarification

Your feedback encourages the volunteers in this community to continue contributing