topic detected_host, detected_timestamp, etc. with JSON file source in Getting Data In

detected_host, detected_timestamp, etc. with JSON file source

yuanliu — Mon, 14 Jun 2021 16:41:52 GMT

If I upload a file containing JSON records or monitor such a file/scripted input, a field named host becomes "detected_host", timestamp becomes "detected_timestamp", etc. Is there some way to persuade indexer to accept these fields as host, _time, etc.?

I am looking at a number of such sources all with varying field names for these. So, I hope by renaming/setting corresponding commonly used fields I could just use the default _json sourcetype without resorting to search time tricks.

Re: detected_host, detected_timestamp, etc. with JSON file source

venkatasri — Tue, 15 Jun 2021 00:55:18 GMT

Hi @yuanliu

Splunk doesn't prefix the fieldnames as detected* this could be some pre-existing settings at search/index time modifying host, timestamp fields. Let's continue...to solution

To answers your query, for timestamp field use props.conf - TIME_PREFIX, TIME_FORMAT variables to consider as _time by Splunk. Should be deployed to HF/indexer.

host field in your json can be assigned to host default field using both props.conf, transforms.conf.

##This is just example of host override, should be deployed to HF/indexer. #works for any input type UF/scripted input #props.conf [your_json_sourcetype/host::<hostname>/source::<your_source>] TRANSFORMS-host = hostoverride #Transforms.conf Override host: , REGEX should match your host in json payload [hostoverride] DEST_KEY = MetaData:Host REGEX = \s(\w*)$ FORMAT = host::$1

---

An upvote would be appreciated if it helps!

Re: detected_host, detected_timestamp, etc. with JSON file source

yuanliu — Thu, 17 Jun 2021 01:40:24 GMT

Splunk doesn't prefix the fieldnames as detected* this could be some pre-existing settings at search/index time modifying host, timestamp fields. Let's continue...to solution

Well, this is the behavior on a pretty clean install.

As you explained, if I want to set host/time/eventtype, etc., at index time for a JSON file, there is no escape from doing custom sourcetype, even if I change field names to host, _time, etc. Thanks, @venkatasri!