Need Proper props configuration for extracting date-time information and breaking down events.

[Praz_123]

Need to know while am adding the data in splunk am getting the below error 

Same data would be like :-

{
"version": "200",
"predictions": [
{
"ds": "2023-01-01T01:00:00",
"y": 25727,
"yhat_lower": 23595.643771045987,
"yhat_upper": 26531.786203915904,
"marginal_upper": 26838.980030149163,
"marginal_lower": 23183.715141246714,
"anomaly": false
},
{
"ds": "2023-01-01T02:00:00",
"y": 24710,
"yhat_lower": 21984.478022195697,
"yhat_upper": 24966.416390280523,
"marginal_upper": 25457.020250925423,
"marginal_lower": 21744.743048120385,
"anomaly": false
},
{
"ds": "2023-01-01T03:00:00",
"y": 23908,
"yhat_lower": 21181.498740796877,
"yhat_upper": 24172.09825724038,
"marginal_upper": 24449.705257711226,
"marginal_lower": 20726.645610860345,
"anomaly": false
},

[livehybrid]

Okay @Praz_123 

Lets try again!

[yourSourceType]
SHOULD_LINEMERGE=false
LINE_BREAKER=([\S\s\n]+"predictions":\s\[\s*)|}(\s*\,\s*){|([\s\n\r]*\][\s\n\r]*}[\s\n\r]*)
NO_BINARY_CHECK=true
TIME_PREFIX="ds":\s"
TIME_FORMAT=%Y-%m-%dT%H:%M:%S
MAX_TIMESTAMP_LOOKAHEAD=20

🌟 Did this answer help you? If so, please consider:

• Adding karma to show it was useful
• Marking it as the solution if it resolved your issue
• Commenting if you need any clarification

Your feedback encourages the volunteers in this community to continue contributing

[Praz_123]

@livehybrid 

not sure like how its working for you as still am unable to get the results.

[livehybrid · Accepted solution]

Hi @Praz_123 

I think the issue here could be that in the original data I had for the my example, the date is in 2023, however in this example the data is in 2012.

In props.conf there is a MAX_DAYS_AGO setting which defaults to 2000 - which is some time in 2019 - If the date you want to extract is prior to this date then you need to increase MAX_DAYS_AGO!

Try setting MAX_DAYS_AGO=5000

[yourSourceType]
SHOULD_LINEMERGE=false
LINE_BREAKER=([\S\s\n]+"predictions":\s\[\s*)|}(\s*\,\s*){|([\s\n\r]*\][\s\n\r]*}[\s\n\r]*)
NO_BINARY_CHECK=true
TIME_PREFIX="ds":\s"
TIME_FORMAT=%Y-%m-%dT%H:%M:%S
MAX_TIMESTAMP_LOOKAHEAD=20
MAX_DAYS_AGO=5000

If this doesnt work then please show the error by hovering over the error icon.

🌟 Did this answer help you? If so, please consider:

• Adding karma to show it was useful
• Marking it as the solution if it resolved your issue
• Commenting if you need any clarification

Your feedback encourages the volunteers in this community to continue contributing

[Praz_123]

@livehybrid 

THANKS for your help it worked finally ,

[livehybrid]

Hi @Praz_123 

Under Advanced try setting a LINE_BREAKER to

"predictions"\s*:\s*\[|}\s*,\s*{|}\s*\]?

🌟 Did this answer help you? If so, please consider:

• Adding karma to show it was useful
• Marking it as the solution if it resolved your issue
• Commenting if you need any clarification

Your feedback encourages the volunteers in this community to continue contributing

[Praz_123]

@livehybrid 

Now it came like in 1 event 

[Praz_123]

@livehybrid 

Able to break down the events , but still can't extract the date-time information ,getting error 

[livehybrid]

Ah sorry about that! Leave it with me, just working on it locally to check.

[livehybrid]

[yourSourceType]
SHOULD_LINEMERGE=false
LINE_BREAKER=([\S\s\n]+"predictions":\s\[\s*)|}(\s*\,\s*){|([\s\n\r]*\][\s\n\r]*}[\s\n\r]*)
NO_BINARY_CHECK=true
TIME_PREFIX="ds":\s"
TIME_FORMAT=%Y-%m-%dT%H:%M:%S
MAX_TIMESTAMP_LOOKAHEAD=20

🌟 Did this answer help you? If so, please consider:

• Adding karma to show it was useful
• Marking it as the solution if it resolved your issue
• Commenting if you need any clarification

Your feedback encourages the volunteers in this community to continue contributing