How to parse timestamp which is in epoch and assign it to the same field timestamp

[sabollam]

Hello All,

I have log file which has the following content in json format, I would like to parse the timestamp and convert it to "%m-%d-%Y %H:%M:%S.%3N" and assign it to the same field timestamp.

Can someone assist me on this on what should be props.conf and transforms.conf.

i tried to use _json sourcetype but it producing none for the timestamp field.

Note: I'm trying to test this locally.

```

{"level":"warn","service":"resource-sweeper","timestamp":1744302465965,"message":"1 nodes are not allocated"}

{"level":"warn","service":"resource-sweeper","timestamp":1744302475969,"message":"1 nodes are not allocated"}

{"level":"warn","service":"resource-sweeper","timestamp":1744302858869,"message":"1 nodes are not allocated"}

{"level":"warn","service":"resource-sweeper","timestamp":1744304731808,"message":"1 nodes are not allocated"}

{"level":"warn","service":"resource-sweeper","timestamp":1744304774636,"message":"1 nodes are not allocated"}

```

[PickleRick]

Ok, there have been many ideas here but  oone asked the main question. Why do you want to do it?

[livehybrid]

Hi @sabollam 

You can use the following to update this within the _raw event at searchtime:

| eval _raw=json_set(_raw, "timestamp",strftime(json_extract(_raw,"timestamp")/1000,"%m-%d-%Y %H:%M:%S.%3N"))

However if you want to do this at index time then you need to do the following:

== props.conf == 
[yourSourcetype]
TRANSFORM-overrideTimeStamp = overrideTimeStamp

== transforms.conf ==
[overrideTimeStamp]
INGEST_EVAL = _raw=json_set(_raw, "timestamp",strftime(json_extract(_raw,"timestamp")/1000,"%m-%d-%Y %H:%M:%S.%3N"))

🌟 Did this answer help you? If so, please consider:

• Adding karma to show it was useful
• Marking it as the solution if it resolved your issue
• Commenting if you need any clarification

Your feedback encourages the volunteers in this community to continue contributing

[sabollam]

Hi @livehybrid, I tried to apply props and transforms like you mentioned earlier but i don't see events are breaking,  the value of the timestamp is still showing the epoch value not the time format I needed. it's also showing none value in the results which is not expected, how to eliminate the none in the results.

[sabollam]

I have this in props and transforms.

[resource_timestamp]

SHOULD_LINEMERGE = false

INDEXED_EXTRACTIONS = json

KV_MODE = none

TIME_PREFIX = "timestamp":

TIME_FORMAT = %s%3N

DATETIME_CONFIG = NONE

TRANSFORMS-overrideTimeStamp = overrideTimeStamp

[overrideTimeStamp]

INGEST_EVAL = _raw=json_set(_raw, "timestamp",strftime(json_extract(_raw,"timestamp")/1000,"%m-%d-%Y %H:%M:%S.%3N"))

#INGEST_EVAL = _raw=strftime(json_extract(_raw, "timestamp")/1000, "%m-%d-%Y %H:%M:%S.%3N")

I can now see the intended timeformat is being updated in the timestamp field but i also see the value of timestamp twice with none and epoch format, how do i eliminate none value.

[sabollam]

Hi @livehybrid , 

I wanted this while indexing data. I don't see the value of the timestamp is overriden with the actual value it has(epoch), Addition to it, i see the value none returning in the timestamp values.

I wanted the event to be shown something like this in the splunk results.

raw_event: before indexing.

{"level":"warn","service":"resource-sweeper","timestamp":1744382735963,"message":"1 nodes are not allocated"}

{"level":"warn","service":"resource-sweeper","timestamp":1744390525975,"message":"1 nodes are not allocated"}

{"level":"warn","service":"resource-sweeper","timestamp":1744390538019,"message":"2 nodes are not allocated"}

{"level":"warn","service":"resource-sweeper","timestamp":1744390555970,"message":"1 nodes are not allocated"}

I wanted the events to be shown in splunk this way:

{"level":"warn","service":"resource-sweeper","timestamp":1744382735963,"message":"1 nodes are not allocated"}

{"level":"warn","service":"resource-sweeper","timestamp":1744390525975,"message":"1 nodes are not allocated"}

{"level":"warn","service":"resource-sweeper","timestamp":1744390538019,"message":"2 nodes are not allocated"}

{"level":"warn","service":"resource-sweeper","timestamp":1744390555970,"message":"1 nodes are not allocated"}

{"level":"warn","service":"resource-sweeper","timestamp”:04/16/2025 16:55:23.650,”message":"1 nodes are not allocated"}

{"level":"warn","service":"resource-sweeper","timestamp":04/16/2025 16:55:25.975,"message":"1 nodes are not allocated"}

{"level":"warn","service":"resource-sweeper","timestamp":04/16/2025 16:55:38.019,"message":"2 nodes are not allocated"}

{"level":"warn","service":"resource-sweeper","timestamp":04/16/2025 16:55:55.970,”message":"1 nodes are not allocated"}

The values of the timestamp should be the above one's.

[livehybrid]

Hi @sabollam 

I think you first need to address the issue of the multiple JSON events displaying in a single event as per your screenshot. I suspect that the reason you are getting the "none" value is because its failing to do the json_extract to get the timestamp value because the JSON is not valid/there are multiple events.

If you are able to get the event breaking properly then I think the INGEST_EVAL should work.

As others have said, its worth making sure you are consciously doing this based on valid decision - there may be other ways to achieve this.

🌟 Did this answer help you? If so, please consider:

• Adding karma to show it was useful
• Marking it as the solution if it resolved your issue
• Commenting if you need any clarification

Your feedback encourages the volunteers in this community to continue contributing

[isoutamo]

Are you try to put it into _time field instead of timestamp? Then just modify @livehybrid INGEST_EVAL example to put that value into _time field instead of timestamp. Remember to remove json_set part also.

[ITWhisperer]

Why are you trying to do this at index time? timestamps can be better manipulated/compared when they are epochs, they only "need" to be converted to strings when being displayed in reports and dashboards.

[sabollam]

The reason is, Our dev team requires the timestamp which is in epoch needs to be formatted to "%d-%m-%d %H:%M:%S.%3N", Have already created a calculated field to convert this to the format we require. But still they need this to be done at indexing stage.

props.conf

[resource_timestamp]

SHOULD_LINEMERGE = false

INDEXED_EXTRACTIONS = json

KV_MODE = none

TIME_PREFIX = \"timestamp\"\:

TIME_FORMAT = %s%3N

MAX_TIMESTAMP_LOOKAHEAD = 13

TRANSFORMS-updateTimestamp = updateTimestamp

TRANSFORMS-overrideTimeStamp = overrideTimeStamp

transforms.conf

[overrideTimeStamp]

INGEST_EVAL = _raw=json_set(_raw, "timestamp",strftime(json_extract(_raw,"timestamp")/1000,"%m-%d-%Y %H:%M:%S.%3N"))

[updateTimestamp]

#INGEST_EVAL = timestamp=json_extract(_raw, "timestamp"

INGEST_EVAL = timestamp=strftime(json_extract(_raw, "timestamp") / 1000, "%m-%d-%Y %H:%M:%S.%3N")

I was able to format the timestamp in _raw but the timestamp field in the interesting field is still showing up as epoch, How can I transform the value of the timestamp similar to _raw.

[PickleRick]

1. Ekhm, your "dev team" cannot handle epoch timestamp? That is... surprising to say the least.

2. Who produces those logs? Another app written by another "dev team"?

[kiran_panchavat]

@sabollam