Getting Data In

Why can't I set a new timestamp via props.conf?

Path Finder

Hi!

I have the following JSON and I would like to set the field "Date" as timestamp. Splunk is currently setting the date and time corresponding to when I index the data.

JSON

{"Date":"2018-02-26","Id commande":"L4512XXX","Type":"A","Quantité vendue":"1000","Support de vente":"Livre","Code pays":"FR","Référence":"REFXXX"}

In order to set the field Date as timestamp, here is my configuration file :

props.conf

[json_sourcetype]
KV_MODE = json
NO_BINARY_CHECK = 1
SHOULD_LINEMERGE = true
TRUNCATE = 0
TIME_PREFIX = Date":"
MAX_TIMESTAMP_LOOKAHEAD = 200 
TIME_FORMAT = %Y-%m-%d

But it is not working at all, it stills takes the indexation date as if this config was not taken into account. Do you know where it could come from?

Thanks!

0 Karma

Ultra Champion

Where did you deploy this props.conf? Index time configuration like setting the timestamp should be done on the indexers, or if any Heavy Forwarder is used before it reaches the indexer(s), it should go on the first Heavy Forwarder that processes the data.

0 Karma

SplunkTrust
SplunkTrust

KV_MODE only applies at search time. Use INDEXED_EXTRACTIONS at index-time. Try these props.conf settings.

[ json_sourcetype]
SHOULD_LINEMERGE=true
NO_BINARY_CHECK=true
INDEXED_EXTRACTIONS=json
KV_MODE=none
TIME_PREFIX=Date\":\"
TIME_FORMAT=%Y-%m-%d
---
If this reply helps you, an upvote would be appreciated.
0 Karma

Path Finder

Hi @richgalloway, thanks for your answer. I tried your configuration but it is still not working as you can see in the screenshot.

Date not taken into account

Could it come from somewhere else, in an other config file ?

0 Karma

Builder

Can you try:

MAX_TIMESTAMP_LOOKAHEAD=10
NO_BINARY_CHECK=true
SHOULD_LINEMERGE=true
TIME_FORMAT=%Y-%m-%d
TIME_PREFIX={\"\Date\":\"
INDEXED_EXTRACTIONS=json
KV_MODE=none

0 Karma

Path Finder

It is still giving me as timestamp the indexation time ... I'll put below all the file parts that could have an impact on this, I'm necessarily doing something wrong somewhere ! Thanks again 🙂

Request

curl -k  http://splunk:8088/services/collector -H "Authorization: Splunk <my_token>" -d '{"sourcetype": "json_sourcetype", "event": {<my_json>}}'

inputs.conf (/opt/splunk/etc/apps/splunk_httpinput/local/inputs.conf , because it is a HTTP data input)

[http]
disabled=0
port=8088
enableSSL=0

[http://Vente]
disabled = 0
index = sales
token = <my_token>
sourcetype = json_sourcetype

indexes.conf

[sales]
homePath = $SPLUNK_DB/sales/db
maxTotalDataSizeMB = 512000
enableDataIntegrityControl = 0
thawedPath = $SPLUNK_DB/sales/thaweddb
enableTsidxReduction = 0
coldPath = $SPLUNK_DB/sales/colddb

props.conf

[json_sourcetype]
MAX_TIMESTAMP_LOOKAHEAD=10
NO_BINARY_CHECK=true
SHOULD_LINEMERGE=true
TIME_FORMAT=%Y-%m-%d
TIME_PREFIX={\"\Date\"\:\"
INDEXED_EXTRACTIONS=json
KV_MODE=none
0 Karma

SplunkTrust
SplunkTrust

Don't escape the D. There's no need for the leading {\", either.

---
If this reply helps you, an upvote would be appreciated.
0 Karma

Path Finder

Sadly even without it, the result is the same !

0 Karma
Don’t Miss Global Splunk
User Groups Week!

Free LIVE events worldwide 2/8-2/12
Connect, learn, and collect rad prizes
and swag!