topic Re: Timestamp is extracted as none in Splunk Search

Timestamp is extracted as none

alvinsullivan01 — Thu, 31 Jul 2025 09:04:33 GMT

I have issue to transform data and extracting the fields value. Here is my sample data.

2025-07-20T10:15:30+08:00 h1 test[123456]: {"data": {"a": 1, "b": 2, "c": 3}}

The data has timestamp and other information in the beginning, and the data dictionary at the end. I want my data to go into Splunk in JSON format. Other than data, what I need is the timestamp. So I create a transform to pick only the data dictionary and move the timestamp into that dictionary. Here is my transforms.

[test_add_timestamp] DEST_KEY = _raw REGEX = ([^\s]+)[^:]+:\s*(.+)} FORMAT = $2, "timestamp": "$1"} LOOKAHEAD = 32768

Here is my props to use the transforms.

[test_log] SHOULD_LINEMERGE = false TRUNCATE = 0 KV_MODE = json LINE_BREAKER = ([\r\n]+) NO_BINARY_CHECK = true disabled = false pulldown_type = true TIME_PREFIX = timestamp MAX_TIMESTAMP_LOOKAHEAD = 100 TIME_FORMAT = %Y-%m-%dT%H:%M:%S TZ = UTC TRANSFORMS-timestamp = test_add_timestamp

After transform, the data will look like this.

{"data": {"a": 1, "b": 2, "c": 3}, "timestamp": "2025-07-20T10:15:30+08:00"}

But when I search the data in Splunk, why do I see "none" as value in timestamp as well?

Another thing I noticed is in my Splunk index that has many data, I can see few data has this timestamp extracted, and most of them have no timestamp, which is fine. But when I click "timestamp" under interesting fields, why is it showing only "none"? I also noticed some of the JSON keys are not available under "interesting fields". What is the logic behind this?

Re: Timestamp is extracted as none

gcusello — Thu, 31 Jul 2025 10:07:53 GMT

Hi @alvinsullivan01 ,

this seems to be a json format, the raw data in a json format are different than the visualized ones, so to check you regex, open the raw data visualization and not the json visualization: probablu you have to add some backslash to your regex.

Ciao.

Giuseppe

Re: Timestamp is extracted as none

livehybrid — Thu, 31 Jul 2025 12:27:06 GMT

Hi @alvinsullivan01

If this is a JSON event then you should be able to use

TIMESTAMP_FIELDS = timestamp #Also adjust TIME_FORMAT to include the timezone. TIME_FORMAT = %Y-%m-%dT%H:%M:%S%z

🌟 Did this answer help you? If so, please consider:

Adding karma to show it was useful
Marking it as the solution if it resolved your issue
Commenting if you need any clarification

Your feedback encourages the volunteers in this community to continue contributing

Re: Timestamp is extracted as none

PickleRick — Thu, 31 Jul 2025 13:32:53 GMT

If you're ok with the timestamp just being assigned to an event (no need to have it explicitly written in the event itself), just parse out the timestamp, cut the whole header and just leave the json part on its own.

Timestamp recognition takes place very early in the ingestion pipeline so you can do this way and not have to have the "timestamp" field in your json. You'll just have the _time field.

Re: Timestamp is extracted as none

alvinsullivan01 — Fri, 01 Aug 2025 02:22:16 GMT

Thank you for your reply @gcusello

Here is how it looks like in raw visualization.

It looks like a valid JSON format. Do you have any idea why do I have both "2025-07-20" and "none" value under timestamp?

Re: Timestamp is extracted as none

alvinsullivan01 — Fri, 01 Aug 2025 02:24:25 GMT

Thank you for your reply @livehybrid

I am currently using TIME_FORMAT as you suggested, but correct me if I'm wrong, is that for _time, not "timestamp" field?

Re: Timestamp is extracted as none

alvinsullivan01 — Fri, 01 Aug 2025 02:30:43 GMT

Thank you for your reply @PickleRick

Yes you are right, that is another possible solution. But if possible, the requirement I have now is to have the "timestamp" field included in the event JSON. Do you have any idea why my "timestamp" field has both value "2025-07-20" and "none"?

Re: Timestamp is extracted as none

PrewinThomas — Fri, 01 Aug 2025 04:23:16 GMT

@alvinsullivan01

If the field does not exist at the moment Splunk attempts to extract time, it might report it as "none".
Can you try with below config,

props.conf

[test_log] SHOULD_LINEMERGE = false KV_MODE = json LINE_BREAKER = ([\r\n]+) NO_BINARY_CHECK = true disabled = false pulldown_type = true TRUNCATE = 0 TRANSFORMS-addtimestamp = test_add_timestamp TIME_PREFIX = "timestamp":" TIME_FORMAT = %Y-%m-%dT%H:%M:%S%z MAX_TIMESTAMP_LOOKAHEAD = 50 TZ = UTC

transforms.conf

[test_add_timestamp] DEST_KEY = _raw REGEX = ^([^\s]+).*?({.*})$ FORMAT = {"timestamp":"$1", "data":$2}

Regards,
Prewin
Splunk Enthusiast | Always happy to help! If this answer helped you, please consider marking it as the solution or giving a Karma. Thanks!

Re: Timestamp is extracted as none

alvinsullivan01 — Fri, 01 Aug 2025 04:39:30 GMT

Thank you for your reply @PrewinThomas

Unfortunately I'm still getting the same both "2025-07-20" and "none" value in 1 event.

Re: Timestamp is extracted as none

PickleRick — Fri, 01 Aug 2025 06:03:42 GMT

It's hard to say without knowing your full config.

Anyway, answering your earlier question, the "interesting fields" section shows only fields for which there is a value in at least some part of your all events (don't remember the threshold - 10%? 15%?). So if you have a field which appears in just 3% of your search results it will not be listed under interesting fields.

One more thing, if you have a value of "none" for your timestamp field it means that it has been explicitly assigned this value. If there is no value there's just no value. "none" in this case is a string saying "none".

Re: Timestamp is extracted as none

alvinsullivan01 — Fri, 01 Aug 2025 06:11:29 GMT

@PickleRick Thank you for the info! That may explain why some of the fields are not showing up in interesting fields. Though I can see as in my screenshot that there is only 1 "timestamp" field in my JSON and it has value, so I'm still not sure where is the "none" coming from.

Re: Timestamp is extracted as none

gcusello — Fri, 01 Aug 2025 06:34:42 GMT

Hi @alvinsullivan01 ,

you used a wrong TIME_PREFIX: timestap il calculated as first operation, so please try:

TIME_PREFIX = ^

Ciao.

Giuseppe

Re: Timestamp is extracted as none

PickleRick — Fri, 01 Aug 2025 06:37:15 GMT

That's why I'm saying that without knowing your full config it's impossible to tell you where it comes from. The "none" value seems like some kind of a safeguard calculated value so that you have _some_ value assigned even if no/wrong value was extracted in the first place (it's a typical operation for some fields in CIM datamodels for example). But normally I'd expect it to overwrite the original field. So for no we can only guess.

Re: Timestamp is extracted as none

PickleRick — Fri, 01 Aug 2025 06:39:33 GMT

Good catch. Timestamp recognition takes place right after line breaking. Definitely way before all this fields shuffling. So the timestamp at this point is still in the header of the event.

Re: Timestamp is extracted as none

alvinsullivan01 — Fri, 01 Aug 2025 06:50:21 GMT

Thank you, that works!

Out of curiosity, do you know why Splunk extracted both "2025-07-20" and "none" when TIME_PREFIX was wrong?

Re: Timestamp is extracted as none

PickleRick — Fri, 01 Aug 2025 07:32:52 GMT

Ahhh... I doublechecked and it seems Splunk also populates the timestamp by default (to be honest, I didn't know that; I thought it was only the _time field and those "partial" fields like date_hour, date_day and so on).

So when you had your TIME_PREFIX wrong Splunk - since as I said timestamp recognition happend very early in the ingestion pipeline - would attempt to parse out the timestamp from the message. Since the prefix was wrong Splunk wouldn't properly extract the time so it would set the _time using default logic (either to the timestamp of the previous event or the current time of the receiving host; the details of the time assignment are pretty well docummented, for example here - https://help.splunk.com/en/data-management/get-data-in/get-data-into-splunk-cloud-platform/10.0.2503/configure-timestamps/how-timestamp-assignment-works ) and it would set the timestamp field to "none".

And then in search time your already transformed json would get parsed into fields so the value from the event would be added to the already present value of "none" making the field a multivalued field.

Re: Timestamp is extracted as none

alvinsullivan01 — Fri, 01 Aug 2025 08:10:04 GMT

I see, that's good to know. Thanks for the explanation!