topic Re: Why can't I set a new timestamp via props.conf? in Getting Data In

Why can't I set a new timestamp via props.conf?

Clovisa — Wed, 04 Apr 2018 15:53:39 GMT

Hi!

I have the following JSON and I would like to set the field "Date" as timestamp. Splunk is currently setting the date and time corresponding to when I index the data.

JSON

{"Date":"2018-02-26","Id commande":"L4512XXX","Type":"A","Quantité vendue":"1000","Support de vente":"Livre","Code pays":"FR","Référence":"REFXXX"}

In order to set the field Date as timestamp, here is my configuration file :

props.conf

[json_sourcetype]
KV_MODE = json
NO_BINARY_CHECK = 1
SHOULD_LINEMERGE = true
TRUNCATE = 0
TIME_PREFIX = Date":"
MAX_TIMESTAMP_LOOKAHEAD = 200 
TIME_FORMAT = %Y-%m-%d

But it is not working at all, it stills takes the indexation date as if this config was not taken into account. Do you know where it could come from?

Thanks!

Re: Why can't I set a new timestamp via props.conf?

richgalloway — Tue, 29 Sep 2020 18:52:33 GMT

KV_MODE only applies at search time. Use INDEXED_EXTRACTIONS at index-time. Try these props.conf settings.

[ json_sourcetype]
SHOULD_LINEMERGE=true
NO_BINARY_CHECK=true
INDEXED_EXTRACTIONS=json
KV_MODE=none
TIME_PREFIX=Date\":\"
TIME_FORMAT=%Y-%m-%d

Re: Why can't I set a new timestamp via props.conf?

Clovisa — Thu, 05 Apr 2018 07:35:07 GMT

Hi @richgalloway, thanks for your answer. I tried your configuration but it is still not working as you can see in the screenshot.

Could it come from somewhere else, in an other config file ?

Re: Why can't I set a new timestamp via props.conf?

Azeemering — Tue, 29 Sep 2020 18:56:06 GMT

Can you try:

MAX_TIMESTAMP_LOOKAHEAD=10
NO_BINARY_CHECK=true
SHOULD_LINEMERGE=true
TIME_FORMAT=%Y-%m-%d
TIME_PREFIX={\"\Date\":\"
INDEXED_EXTRACTIONS=json
KV_MODE=none

Re: Why can't I set a new timestamp via props.conf?

Clovisa — Thu, 05 Apr 2018 12:15:47 GMT

It is still giving me as timestamp the indexation time ... I'll put below all the file parts that could have an impact on this, I'm necessarily doing something wrong somewhere ! Thanks again 🙂

Request

curl -k  http://splunk:8088/services/collector -H "Authorization: Splunk <my_token>" -d '{"sourcetype": "json_sourcetype", "event": {<my_json>}}'

inputs.conf (/opt/splunk/etc/apps/splunk_httpinput/local/inputs.conf , because it is a HTTP data input)

[http]
disabled=0
port=8088
enableSSL=0

[http://Vente]
disabled = 0
index = sales
token = <my_token>
sourcetype = json_sourcetype

indexes.conf

[sales]
homePath = $SPLUNK_DB/sales/db
maxTotalDataSizeMB = 512000
enableDataIntegrityControl = 0
thawedPath = $SPLUNK_DB/sales/thaweddb
enableTsidxReduction = 0
coldPath = $SPLUNK_DB/sales/colddb

props.conf

[json_sourcetype]
MAX_TIMESTAMP_LOOKAHEAD=10
NO_BINARY_CHECK=true
SHOULD_LINEMERGE=true
TIME_FORMAT=%Y-%m-%d
TIME_PREFIX={\"\Date\"\:\"
INDEXED_EXTRACTIONS=json
KV_MODE=none

Re: Why can't I set a new timestamp via props.conf?

FrankVl — Thu, 05 Apr 2018 12:21:36 GMT

Where did you deploy this props.conf? Index time configuration like setting the timestamp should be done on the indexers, or if any Heavy Forwarder is used before it reaches the indexer(s), it should go on the first Heavy Forwarder that processes the data.

Re: Why can't I set a new timestamp via props.conf?

richgalloway — Thu, 05 Apr 2018 12:54:11 GMT

Don't escape the D. There's no need for the leading {\", either.

Re: Why can't I set a new timestamp via props.conf?

Clovisa — Thu, 05 Apr 2018 13:04:19 GMT

Sadly even without it, the result is the same !