Splunk Search

Json timestamp miliseconds

Jakub
Explorer

Hi, hello,

Splunk is not showing up miliseconds for JSON logs. I have find some Questions and Answers here in splunk community, but without success.

Description:

I have HFs, indexer cluster and search head cluster.

HF props.conf

[k8s:dev]
#temporary removed to fix 123123
#INDEXED_EXTRACTIONS = JSON
TIME_PREFIX = {\\"@timestamp\\":\\"
TIME_FORMAT = %Y-%m-%dT%H:%M:%S.%6N
TRUNCATE = 200000
TRANSFORMS-discard_events = setnull_whitespace_indented,setnull_debug_logging
SEDCMD-RemoveLogProp = s/("log":)(.*)(?="stream":)//

 

HF transforms.conf

[setnull_java_stacktrace_starttab]
SOURCE_KEY = field:log
REGEX = ^\tat\s.*
DEST_KEY = queue
FORMAT = nullQueue

[setnull_whitespace_indented]
SOURCE_KEY = field:log
REGEX = ^\s+.*
DEST_KEY = queue
FORMAT = nullQueue

[setnull_debug_logging]
SOURCE_KEY = field:log
REGEX = .*?\sDEBUG\s
DEST_KEY = queue
FORMAT = nullQueue

 

Search props.conf

 

#workaround, see 123123


[k8s:dev]
KV_MODE = json

 

Everything looks fine in web ADD DATA in HF and SEARCH too.

Jakub_0-1627990412004.png

But not when I search it.

Jakub_1-1627990543783.png

 

I can insert only part of the JSON.

{"log":"{\"@timestamp\":\"2021-08-03T09:00:57.539+02:00\",\"@version\":\"1\",\"message\":

 

Also when I am in HF ADD DATA and I remove TIME_PREFIX and TIME_FORMAT the miliseconds still appear, but when I a little bit "destroy" TIME_PREFIX there is error and file timestamp is used(I think its file timestamp).

Question is:

1.what am I doing wrong? 

2. Is it possible to configure TIME_PREFIX and TIME_FORMAT for KV_MODE on search? Because as I know they are used in HF during parsing.

3. Is it possible to configure KV_MODE?

 

Thank you very much for your suggestions.

 

 

Labels (1)
0 Karma

Jakub
Explorer

Actually nothing works. What else I can try?

timestamp.PNGtimstampp.PNG

0 Karma

Jakub
Explorer

Also SEDCMD should not affect that as  that is done after timestamp parsing.

 

Thank you!

 

0 Karma
Get Updates on the Splunk Community!

Splunk App for Anomaly Detection End of Life Announcment

Q: What is happening to the Splunk App for Anomaly Detection?A: Splunk is officially announcing the ...

Aligning Observability Costs with Business Value: Practical Strategies

 Join us for an engaging Tech Talk on Aligning Observability Costs with Business Value: Practical ...

Mastering Data Pipelines: Unlocking Value with Splunk

 In today's AI-driven world, organizations must balance the challenges of managing the explosion of data with ...