Hello,
I recently enabled a SolarWinds alert in the inputs.conf on the heavy forwarder. The data is now ingesting into Splunk, but the timestamps are appearing in UTC instead of local time. The interval is set to 899. All other SolarWinds alerts ingesting into Splunk are showing the correct local time, so this issue seems isolated to the newly enabled alert. Any guidance on what might cause this specific alert to default to UTC—whether related to the alert configuration, timestamp parsing, or a missing props/transforms setting—would be appreciated.
for current and future learners reference, i would like to post the Splunk's "Magic 8"
https://kinneygroup.com/blog/splunk-magic-8-props-conf/
Apart from all the things already mentioned by @richgalloway , check the contents of the event itself and compare to the other - "working" - events. Do the timestamps in those alert contain timestamps in the same timezone? Are they formatted the same way? Are they even extracted and used properly by Splunk or is the ingestion time used instead?
Check the props.conf settings for the sourcetype ingested by the inputs.conf entry. Chances are the time parsing settings are incorrect. Verify the TIME_FORMAT and TIME_PREFIX settings in particular. You may need to add a TZ setting.