Custom timestamp detection for a sourcetype with some event w/o timestamp

[D2SI]

Hello there,

For a particular sourcetype there are events with a timestamp and events without timestamp.

As Splunk cannot detect a timestamp in the events without timestamp, it generates plenty of errors saying 'Defaulting to timestamp of previous event'.

I am trying to clean this up.

I see that I can configure a local datetime.xml to take into account various timestamp formats.

What I would like to do is :

for event with a timestamp -> assign the correct timestamp

for events without timestamp -> assign the indexing time as the timestamp and don't throw errors

Is there a way in datetime.xml to say : for this format, apply indextime timestamp?

Thanks in advance for any hint!

[jarizeloyola]

Another way to solve this is to use INGEST_EVAL in transforms.conf.

I tried this config and works for me.

props.conf
[test]
LINE_BREAKER = ([\r\n]+)(\d+|\w+)
SHOULD_LINEMERGE = false
TIME_PREFIX = ^
TIME_FORMAT = %Y-%m-%d %H:%M:%S.%3N
MAX_TIMESTAMP_LOOKAHEAD = 30
TRANSFORMS = modtimestamp

---

transforms.conf

[modtimestamp]
INGEST_EVAL = _time[float]:=if(match(_raw,"^(?!\d+-\d+-\d+).*"),time(),_time)

it filters the events that doesnt start with the year and apply the condition

time() is more expensive and acts on every event separately. now() appears to only be valid in search and is the time of the search itself.

Note: This configuration might be very impactful on the indexer's performance.

[adonio]

imho the workaround i suggested in the comment directly under the question performs better.
and only being used if needed (search time)

[jarizeloyola]

Depends on his requirements , either at the index or search time. whatever that helps him. 🙂

[D2SI]

Thank both of you for your suggested solutions!

@adonio Good idea, thank you, I will test this with eval in the props.conf!

@jarizeloyola Thanks! I am using ingest_eval for another source which timestamp can only be found in the filename, and it works great! However, I see splunkd timestamp errors for that source, so I have to check if inges_eval works in a way there will still have timestamp issues generated even if OK, or if it is something else!

[mloyola_splunk]

I would suggest to use the big 6/8 of props.conf
Your props should contain these "TIME_PREFIX, TIME_FORMAT, MAX_TIMESTAMP_LOOKAHEAD" and
add "DATETIME_CONFIG = current".

DATETIME_CONFIG is when Splunk can't find something to match TIME_* .

[D2SI]

Thanks for your answer.

Your are saying that with TIME_* parameters configured and DATETIME_CONFIG set to current, events with a timestamp will get timestamped according to TIME_* parameters while events from the same sourcetype without timestamp will have the indextime as the timestamp.

To my understanding setting DATETIME_CONFIG to current overrides all the TIME_* parameters and set the indextime as timestamp but I will test this.

[adonio]

would be very interesting, to know the result of your test @D2SI as i had the same understanding you had.
couple links for reference:
https://docs.splunk.com/Documentation/Splunk/8.0.1/Admin/Propsconf#Timestamp_extraction_configuratio...
https://wiki.splunk.com/Community:HowIndexingWorks

[D2SI]

Alright I have tested it and it does not work, I mean if DATETIME_CONFIG is set to CURRENT, it overrides any other TIME_* parameter and set timestamp to indextime.

Thanks anyhow for the suggestion.

[adonio]

probably doesnt answer your question directly, but here is a workaround.
index all events with current timestamp
in search extract the timestamp for the field that contains time, lets say you will call it event_time
now in search (if need to use time other then current), run something like this:
... | eval _time = if(isnull(event_time),_time,event_time) ... or something along those lines
hope it helps a little bit