Getting Data In

Data is not getting parsed on HEC

rahulg
Explorer

I have props.conf

[source::tcp:7660]
TRUNCATE=10000000
LINE_BREAKER = {\"time
NO_BINARY_CHECK = true
SHOULD_LINEMERGE = false
category = Custom
pulldown_type = true
KV_MODE = json
#TZ = America/Chicago
TZ=UTC
=====================================

 

I see some of events are not parsed in json format

 

Labels (2)
Tags (3)
0 Karma

PickleRick
Ultra Champion

Which endpoint are you send your events to?

0 Karma

somesoni2
Revered Legend

Is the HEC configured on Heavy forwarder/indexer? Check if events which are not parsed as json is in pure JSON format. Did you setup KV_MODE=JSON on your search head(s)?

0 Karma

rahulg
Explorer

HEC configured on Heavy forwarder and i dont have KV_MODE=JSON on search head(s) will that help?

0 Karma

PickleRick
Ultra Champion

Firstly, I think you don't need line breaking settings since you supply whole single events to the /event endpoint.

Secondly - KV_MODE is a search-time setting so yes, you need it on search-heads, not on indexers/HF's

0 Karma

rahulg
Explorer

Here is sample events which is working fine

 

{ [-]
command: C:\Windows\System32\sdfhlsdhjfjsnsdf
company_1: Microsoft Corporation
company_2: Microsoft Corporation
connection_count: 0
created: Mon Nov 1 07:52:10 2021
created_1: Sun Jun 6 1
created_2: Sun Jun 6 14:52:03.721 2021
desc_1: Runtime Broker
desc_2: Host Process for Windows Services
exists_1: yes
exists_2: yes
file_1: C:\Windows\System32\kjksnkfhskf
file_2: C:\Windows\System32\svchost.exe
firstbytes_1: jhsfkszhkfhnkkllks.ndklfsf
firstbytes_2: hkdhfkgkdhfgknzdlfgnl.sdflgndlkfgnld
hostname: nkdnf.ks
imphash_1: .nsdlkfnlszknflsNLfnzslkdnfksnkfnskfn
imphash_2: nsdnfknfaksnfdksnflnfknskdfnksnafdks
legal_copyright_1: © Microsoft Corporation. All rights reserved.
legal_copyright_2: © Microsoft Corporation. All rights reserved.
level: Info
listen_ports:
md5: nbzkdfnkzshdkfjskJnfkznfksnk
md5_1: ksndlfn.ksndfknsakf
md5_2: nKSndkfksdfnksandfknsak
message: Process info
module: ProcessCheck
name: RuntimeBroker.exe
owner: NM\JOIN4029
owner_1: NT SERVICE\TrustedInstaller
owner_2: NT SERVICE\TrustedInstaller
parent: C:\Windows\System32\svchost.exe
path: C:\Windows\System32\RuntimeBroker.exe
pid: 24080
ppid: 1264
scanid: S-bszkdbfksnbdfkjs
sha1_1: kndfnkzdnfkdnakgnkfgnxkdzn
sha1_2: ndxnfvkznfnkmzfxbvkzdbfvkbzkbxdv
sha256_1: oiajsosfu094ursjofjlsjdflk
sha256_2: knsldkflzsdjflkslkf
size_1: 8679890
size_2: 567890
time: 2021-11-01T14:18:26Z
type_1: EXE
type_2: EXE
}

 

if it has file_1 and file_2 works fine and if addition file_3 or similar sha256_3  or any _etc field ect gets added it doesnt  shows ja\son format

0 Karma

PickleRick
Ultra Champion

Check if the fields are present in raw event. Then you'll know if it's a parsing problem or ingestion one.

0 Karma
Get Updates on the Splunk Community!

Improve Your Security Posture

Watch NowImprove Your Security PostureCustomers are at the center of everything we do at Splunk and security ...

Maximize the Value from Microsoft Defender with Splunk

 Watch NowJoin Splunk and Sens Consulting for this Security Edition Tech TalkWho should attend:  Security ...

This Week's Community Digest - Splunk Community Happenings [6.27.22]

Get the latest news and updates from the Splunk Community here! News From Splunk Answers ✍️ Splunk Answers is ...