Getting Data In

Why are the JSON event lines missing?

splunkLPN
Path Finder

the line format is :

{"tim":"2018-07-12 15:23:16","pre":"ayisha.udam","fir":"Ayisha","las":"UDAM","pe1":false}

Some lines present in the source file if I look at it with a text editor don't appear in a search or in a raw export.
in a json export :

{"preview":false,"result":{"_raw":"{\"tim\":\"2018-07-12 15:23:46\",\"pre\":\"ayisha.adam\",\"fir\":\"Ayisha\",\"las\":\"UDAM\" ...}

The sourcetype (I don't think it's the problem) is like that :

INDEXED_EXTRACTIONS:json
KV_MODE:json
NO_BINARY_CHECK:true
SHOULD_LINEMERGE:false
category:Structured
description:JavaScript Object Notation format. For more information, visit http://json.org/
disabled:false
pulldown_type:true

I've checked for differences in the source : line breaks, quote, I can't see any differences.

What else can I check?

thank's

0 Karma
1 Solution

akocak
Contributor

I believe this should be updated in your sourcetype:
KV_MODE:none

also if you can't guarantee single line events:
SHOULD_LINEMERGE:true

Default Splunk Sourcetype for _json with
./splunk cmd btool props list

[_json]
ADD_EXTRA_TIME_FIELDS = True
ANNOTATE_PUNCT = True
AUTO_KV_JSON = true
BREAK_ONLY_BEFORE = 
BREAK_ONLY_BEFORE_DATE = True
CHARSET = AUTO
DATETIME_CONFIG = \etc\datetime.xml
DEPTH_LIMIT = 1000
HEADER_MODE = 
INDEXED_EXTRACTIONS = json
KV_MODE = none
LEARN_MODEL = true
LEARN_SOURCETYPE = true
LINE_BREAKER_LOOKBEHIND = 100
MATCH_LIMIT = 100000
MAX_DAYS_AGO = 2000
MAX_DAYS_HENCE = 2
MAX_DIFF_SECS_AGO = 3600
MAX_DIFF_SECS_HENCE = 604800
MAX_EVENTS = 256
MAX_TIMESTAMP_LOOKAHEAD = 128
MUST_BREAK_AFTER = 
MUST_NOT_BREAK_AFTER = 
MUST_NOT_BREAK_BEFORE = 
SEGMENTATION = indexing
SEGMENTATION-all = full
SEGMENTATION-inner = inner
SEGMENTATION-outer = outer
SEGMENTATION-raw = none
SEGMENTATION-standard = standard
SHOULD_LINEMERGE = True
TRANSFORMS = 
TRUNCATE = 10000
category = Structured
description = JavaScript Object Notation format. For more information, visit http://json.org/
detect_trailing_nulls = auto
maxDist = 100
priority = 
pulldown_type = true
sourcetype = 

View solution in original post

0 Karma

akocak
Contributor

I believe this should be updated in your sourcetype:
KV_MODE:none

also if you can't guarantee single line events:
SHOULD_LINEMERGE:true

Default Splunk Sourcetype for _json with
./splunk cmd btool props list

[_json]
ADD_EXTRA_TIME_FIELDS = True
ANNOTATE_PUNCT = True
AUTO_KV_JSON = true
BREAK_ONLY_BEFORE = 
BREAK_ONLY_BEFORE_DATE = True
CHARSET = AUTO
DATETIME_CONFIG = \etc\datetime.xml
DEPTH_LIMIT = 1000
HEADER_MODE = 
INDEXED_EXTRACTIONS = json
KV_MODE = none
LEARN_MODEL = true
LEARN_SOURCETYPE = true
LINE_BREAKER_LOOKBEHIND = 100
MATCH_LIMIT = 100000
MAX_DAYS_AGO = 2000
MAX_DAYS_HENCE = 2
MAX_DIFF_SECS_AGO = 3600
MAX_DIFF_SECS_HENCE = 604800
MAX_EVENTS = 256
MAX_TIMESTAMP_LOOKAHEAD = 128
MUST_BREAK_AFTER = 
MUST_NOT_BREAK_AFTER = 
MUST_NOT_BREAK_BEFORE = 
SEGMENTATION = indexing
SEGMENTATION-all = full
SEGMENTATION-inner = inner
SEGMENTATION-outer = outer
SEGMENTATION-raw = none
SEGMENTATION-standard = standard
SHOULD_LINEMERGE = True
TRANSFORMS = 
TRUNCATE = 10000
category = Structured
description = JavaScript Object Notation format. For more information, visit http://json.org/
detect_trailing_nulls = auto
maxDist = 100
priority = 
pulldown_type = true
sourcetype = 
0 Karma

splunkLPN
Path Finder

Thank you for made me discover btool. I must now investigate. Config Quest app will help me.
Your suggestion solved my problems !

0 Karma

akocak
Contributor

Can you pick my solution as answer then 😄 ? No problem, I learned a lot here from other people

0 Karma

splunkLPN
Path Finder

That was my intention ! I don't see how change the "accepted answer"

Get Updates on the Splunk Community!

Splunk Certification Support Alert | Pearson VUE Outage

Splunk Certification holders and candidates!  Please be advised of an upcoming system maintenance period for ...

Enterprise Security Content Update (ESCU) | New Releases

In September, the Splunk Threat Research Team had two releases of new security content via the Enterprise ...

New in Observability - Improvements to Custom Metrics SLOs, Log Observer Connect & ...

The latest enhancements to the Splunk observability portfolio deliver improved SLO management accuracy, better ...