How to Change Timezone of Cloudflare logs

[ibraheem]

Hello,

I have successfully integrated Cloudflare with Splunk Enterprise using the pull method. This integration was set up on a Heavy Forwarder, so the logs are first received by the HF before being forwarded to the Indexers. While the integration itself is working correctly, I encountered an issue with the time zone in the logs.

The API we are using requires the timestamps to be in UTC. As a result, when the API fetches the logs, the events are recorded in the UTC timezone. However, I need to convert these timestamps from UTC to UTC+5 (Pakistan Standard Time, PKT).

Here is a sample log event from Cloudflare:

"

---

EdgeEndTimestamp: 2024-08-26T09:07:43Z
EdgeResponseBytes: 72322
EdgeResponseStatus: 206
EdgeStartTimestamp: 2024-08-26T09:07:43Z

---

"

We are extracting the EdgeStartTimestamp and using it for the _time field, but this timestamp is in UTC format.

In my props.conf file on the Heavy Forwarder, I have the following configuration:

[cloudflare:json]
disabled = false
TIME_PREFIX = \"EdgeStartTimestamp\":\"
TIME_FORMAT = %Y-%m-%dT%H:%M:%S
MAX_TIMESTAMP_LOOKAHEAD = 19

I also tried adding the TZ setting to props.conf:

[cloudflare:json]
TZ = Asia/Karachi

However, this didn't work because the events themselves contain timezone information (UTC), so the TZ setting doesn't have any effect.

I then tried using TZ_ALIAS in props.conf:

[cloudflare:json]
TZ_ALIAS = Z=UTC+5

This didn't work either. Finally, I tried the following in props.conf, but it still didn't resolve the issue:

[cloudflare:json]
EVAL-_time = _time + 5*3600

Any help would be appreciated.

[ibraheem · Accepted solution]

Issue is resolved. I just have to tell Splunk that the time you receive is in UTC and it worked. In props.conf, I just add this:

TZ=UTC

@richgalloway  and @PickleRick  appreciate the effort 🙂

[ibraheem]

Default System Timezone is selected in my preferences. I don't think is the problem because my other searches are working fine.

[PickleRick]

Additionally to what @richgalloway already said - you don't need to "convert" timestamps to another timezone. The timestamps are reported by source in some timezone (the timezone info might be included in the timestamp or not; if it is you can use it, if it is not you have to set it explicitly). But the timestamp as parsed out into the _time field will be stored as an "absolute" timestamp and will be shown in the UI using your user's defined timezone. So the same event will be shown at 14:39 if your user uses UTC or 16:39 if he uses CEST and so on. But the event's contents will remain the same.

[richgalloway]

The time zone is included in the timestamp.  Tell Splunk about it and it will automatically convert the timestamp.

[cloudflare:json]
disabled = false
TIME_PREFIX = "EdgeStartTimestamp":"
TIME_FORMAT = %Y-%m-%dT%H:%M:%S%Z
MAX_TIMESTAMP_LOOKAHEAD = 20

[ibraheem]

I did the same, but nothing changed

TIME_FORMAT = %Y-%m-%dT%H:%M:%S%Z

I ran the above search at 5:35 PM and the latest event time is: 12:30, so approx. 5 hours behind

[PickleRick]

If this is indeed your actual event and those are your actual props settings, Splunk will never find the timestamp because you have a very low lookahead set. There is no timestamp within first 20 characters of the event.

Additionally - are you using indexed extractions?

[richgalloway]

The settings should be fine.  MAX_TIMESTAMP_LOOKAHEAD starts after TIME_PREFIX ends.

[PickleRick]

Ahh, right you are. Still there might be an issue with indexed extractions. Anyway the timestamp doesn't seem to be "evenly" offset so it's more like a current timestamp, not the one from the event.

There might also be an issue with the prefix itself. We don't see the raw data so there might be any number of whitespace characters there. (And escaping quotes is not needed; but shouldn't be harmful in this case)

[richgalloway]

Splunk appears to be interpreting the event timestamp correctly.  The displayed time is based on your selected time zone.  What do you have select in your preferences?

[ibraheem]

Default System Timezone is selected in my preferences. I don't think is the problem because my other searches are working fine.