Getting Data In

Why are the JSON event lines missing?

splunkLPN
Path Finder

the line format is :

{"tim":"2018-07-12 15:23:16","pre":"ayisha.udam","fir":"Ayisha","las":"UDAM","pe1":false}

Some lines present in the source file if I look at it with a text editor don't appear in a search or in a raw export.
in a json export :

{"preview":false,"result":{"_raw":"{\"tim\":\"2018-07-12 15:23:46\",\"pre\":\"ayisha.adam\",\"fir\":\"Ayisha\",\"las\":\"UDAM\" ...}

The sourcetype (I don't think it's the problem) is like that :

INDEXED_EXTRACTIONS:json
KV_MODE:json
NO_BINARY_CHECK:true
SHOULD_LINEMERGE:false
category:Structured
description:JavaScript Object Notation format. For more information, visit http://json.org/
disabled:false
pulldown_type:true

I've checked for differences in the source : line breaks, quote, I can't see any differences.

What else can I check?

thank's

0 Karma
1 Solution

akocak
Contributor

I believe this should be updated in your sourcetype:
KV_MODE:none

also if you can't guarantee single line events:
SHOULD_LINEMERGE:true

Default Splunk Sourcetype for _json with
./splunk cmd btool props list

[_json]
ADD_EXTRA_TIME_FIELDS = True
ANNOTATE_PUNCT = True
AUTO_KV_JSON = true
BREAK_ONLY_BEFORE = 
BREAK_ONLY_BEFORE_DATE = True
CHARSET = AUTO
DATETIME_CONFIG = \etc\datetime.xml
DEPTH_LIMIT = 1000
HEADER_MODE = 
INDEXED_EXTRACTIONS = json
KV_MODE = none
LEARN_MODEL = true
LEARN_SOURCETYPE = true
LINE_BREAKER_LOOKBEHIND = 100
MATCH_LIMIT = 100000
MAX_DAYS_AGO = 2000
MAX_DAYS_HENCE = 2
MAX_DIFF_SECS_AGO = 3600
MAX_DIFF_SECS_HENCE = 604800
MAX_EVENTS = 256
MAX_TIMESTAMP_LOOKAHEAD = 128
MUST_BREAK_AFTER = 
MUST_NOT_BREAK_AFTER = 
MUST_NOT_BREAK_BEFORE = 
SEGMENTATION = indexing
SEGMENTATION-all = full
SEGMENTATION-inner = inner
SEGMENTATION-outer = outer
SEGMENTATION-raw = none
SEGMENTATION-standard = standard
SHOULD_LINEMERGE = True
TRANSFORMS = 
TRUNCATE = 10000
category = Structured
description = JavaScript Object Notation format. For more information, visit http://json.org/
detect_trailing_nulls = auto
maxDist = 100
priority = 
pulldown_type = true
sourcetype = 

View solution in original post

0 Karma

akocak
Contributor

I believe this should be updated in your sourcetype:
KV_MODE:none

also if you can't guarantee single line events:
SHOULD_LINEMERGE:true

Default Splunk Sourcetype for _json with
./splunk cmd btool props list

[_json]
ADD_EXTRA_TIME_FIELDS = True
ANNOTATE_PUNCT = True
AUTO_KV_JSON = true
BREAK_ONLY_BEFORE = 
BREAK_ONLY_BEFORE_DATE = True
CHARSET = AUTO
DATETIME_CONFIG = \etc\datetime.xml
DEPTH_LIMIT = 1000
HEADER_MODE = 
INDEXED_EXTRACTIONS = json
KV_MODE = none
LEARN_MODEL = true
LEARN_SOURCETYPE = true
LINE_BREAKER_LOOKBEHIND = 100
MATCH_LIMIT = 100000
MAX_DAYS_AGO = 2000
MAX_DAYS_HENCE = 2
MAX_DIFF_SECS_AGO = 3600
MAX_DIFF_SECS_HENCE = 604800
MAX_EVENTS = 256
MAX_TIMESTAMP_LOOKAHEAD = 128
MUST_BREAK_AFTER = 
MUST_NOT_BREAK_AFTER = 
MUST_NOT_BREAK_BEFORE = 
SEGMENTATION = indexing
SEGMENTATION-all = full
SEGMENTATION-inner = inner
SEGMENTATION-outer = outer
SEGMENTATION-raw = none
SEGMENTATION-standard = standard
SHOULD_LINEMERGE = True
TRANSFORMS = 
TRUNCATE = 10000
category = Structured
description = JavaScript Object Notation format. For more information, visit http://json.org/
detect_trailing_nulls = auto
maxDist = 100
priority = 
pulldown_type = true
sourcetype = 
0 Karma

splunkLPN
Path Finder

Thank you for made me discover btool. I must now investigate. Config Quest app will help me.
Your suggestion solved my problems !

0 Karma

akocak
Contributor

Can you pick my solution as answer then 😄 ? No problem, I learned a lot here from other people

0 Karma

splunkLPN
Path Finder

That was my intention ! I don't see how change the "accepted answer"

Get Updates on the Splunk Community!

New in Observability - Improvements to Custom Metrics SLOs, Log Observer Connect & ...

The latest enhancements to the Splunk observability portfolio deliver improved SLO management accuracy, better ...

Improve Data Pipelines Using Splunk Data Management

  Register Now   This Tech Talk will explore the pipeline management offerings Edge Processor and Ingest ...

3-2-1 Go! How Fast Can You Debug Microservices with Observability Cloud?

Register Join this Tech Talk to learn how unique features like Service Centric Views, Tag Spotlight, and ...