Getting Data In

How to set up timestamps in this situation?

gauravnj1
Engager

Below is a sample of the log that is generated at the source. This timestamp is in UTC:

2018-02-24T21:21:43.176112 src="yy.yy.yy.yy", direction="inbound", protocol="ip", ids_type="network", vendor_product="Amun", type="amun.events", app="amun", dest="xx.xx.xx.xx", dest_port="80", signature="Connection to Honeypot", src_port="40244", sensor="xyz", transport="tcp", severity="high"

On the forwarder at this source, this is how the inputs.conf looks like:

[monitor:///var/log/mhn/mhn-splunk.log]
sourcetype = mhn
disabled = false

On the indexer, this is what I have in my props.conf:

[sourcetype=mhn]
TZ = UTC

When I search for logs in the last 24 hours, I don't get anything. When I change the timeframe to All time, that's when I see all the logs. How do I correctly set-up timestamps to make sure that I get results?

0 Karma

martin_mueller
SplunkTrust
SplunkTrust

props.conf stanzas are assumed to be sourcetypes unless prefixed with source:: or host::, so that stanza applies to a sourcetype literally called sourcetype=mhn - make sure you use just [mhn] to configure the mhn sourcetype.

The timestamp itself should be auto-recognized by Splunk, but to be sure everything works the way you want it to, I'd use these settings for timestamp recognition:

TIME_PREFIX = ^
MAX_TIMESTAMP_LOOKAHEAD = 30
TIME_FORMAT = %Y-%m-%dT%H:%M:%S.%6N
TZ = UTC

That way, there is no risk for something else in the event that might look like a timestamp to be recognized as one incorrectly. As an added bonus, helping Splunk here increases throughput when indexing.

For even more accuracy and speed, help Splunk break up events:

LINE_BREAKER = ([\r\n]+)\d\d\d\d-\d\d-\d\d
SHOULD_LINEMERGE = false
0 Karma
Get Updates on the Splunk Community!

.conf25 technical session recap of Observability for Gen AI: Monitoring LLM ...

If you’re unfamiliar, .conf is Splunk’s premier event where the Splunk community, customers, partners, and ...

A Season of Skills: New Splunk Courses to Light Up Your Learning Journey

There’s something special about this time of year—maybe it’s the glow of the holidays, maybe it’s the ...

Announcing the Migration of the Splunk Add-on for Microsoft Azure Inputs to ...

Announcing the Migration of the Splunk Add-on for Microsoft Azure Inputs to Officially Supported Splunk ...