Summary index bug, timestamps set "2020-12-09" as "2020-09-12"

[LucLu]

We meet on question is , when the u_worked_date is “2020-12-09”, some of them timestamps is “2020-12-09”, others is “2020-09-12”,I do not know what cause this, Other dates in December can be set as timestamps normally. So I think it maybe a bug of summary index.

Bellow is my SPL

index=idx_snow_task_time sourcetype=snow_task_time

| dedup sys_id

| table sys_id u_worked_date time_worked rate_type sys_updated_by task u_task_category u_actual_time user

| eval _time=strptime(u_worked_date,"%Y-%m-%d")

| collect index=idx_summary_snow_task_time_by_worked_date source="Snow Task Time by Worked Date"

[gcusello]

Hi @LucLu,

I see that you define _time in your scheduled search but you don't use it in the table command, please, try in this way:

index=idx_snow_task_time sourcetype=snow_task_time
| dedup sys_id
| eval _time=strptime(u_worked_date,"%Y-%m-%d")
| table _time sys_id u_worked_date time_worked rate_type sys_updated_by task u_task_category u_actual_time user
| collect index=idx_summary_snow_task_time_by_worked_date source="Snow Task Time by Worked Date"

In this way, you're sure that in the summary index you have the correct _time.

Ciao.

Giuseppe

[gcusello]

Hi @LucLu,

can you share some sample of your data?

Ciao.

Giuseppe

[LucLu]

this is the SPL data:

and this is data from summary index:

you can see that only head 10 _time = u_worked_date

[gcusello]

Hi @LucLu,

could you share a sample of the row data? at least one or two samples of the wrong data and one or two samples of the correct data.

Ciao.

Giuseppe

[LucLu]

wrong raw data:

correct raw data:

[gcusello]

Hi @LucLu,

to avoid that Splunk makes an error in timestamp reading, configure

[your_sourcetype]
TIME_FORMAT = %m/%d/%Y %H:%M:%S %z

in yous props.conf for that sourcetype.

Ciao.

Giuseppe

[LucLu]

oh ,thanks @gcusello 

this is an summary index , do not have sourcetype,  do you mean set the default sourcetype stash's time_format? if I change this , will it affect all of the summary index?

[gcusello]

Hi @LucLu,

The problem, probably isn't in the scheduled search to populate the summary index, check if in the indexed data (not in summary) the timestamp in those events is correctly read.

if not, the only way is to use the TIME_FORMAT option in props (I usually set everytime this parameter!). 

Ciao.

Giuseppe

[LucLu]

@gcusello 

when I run the scheduled search the date is correct, just in summary index have some problem.

I set the TIME_FORMART , but the problem have not resolved. 😭