Splunk Enterprise

Do I need to modify props to capture 2 format of logs?

mahesh27
Communicator

Sample data:
i have 2 types of data and below props given, i am seeing internal logs like

ERROR JsonLineBreaker - JSON StramID:13457545565443322455 had parsing error: Unexpected character: 'a' - data_source........

Do i need to modify props to capture 2 format of logs??

props:
[sourcetype]
INDEXED_EXTRACTIONS=json
KV_MODE=none
SHOULD_LINEMERGE=true
TIMESTAMP_FIELDS=timestamp
LINE_BREAKER=([\r\n]+)

 

{[-]
UserID: Null
host: apl-45678
level: medium
message: cliendid: null, secondaryClientid: null, userid: unknown, respinsetime:1.34455
timestamp: 2022-01-22T21:23:44.897Z
}

 

 

 

{"timestamp": "2022-01-22T21:23:44.897Z", "level":"applevel", "host":"apl-12345", "userid": "NA", "message": apl-12345-20144 - unknown - GET - / - REQ-NAMES - {"accept": "text/plain, application/json:*************************************************************************, "host:""apl-12345", "connection":"unknown"}"}

 

 

 

{[-]
UserID: Null
host: apl-45678
level: medium
message: cliendid: null, secondaryClientid: null, userid: unknown, respinsetime:1.34455
timestamp: 2022-01-22T21:23:44.897Z
}

 

 

 

{"timestamp": "2022-01-22T21:23:44.897Z", "level":"applevel", "host":"apl-12345", "userid": "NA",
 "message": apl-12345-20144 - unknown - GET - / - REQ-NAMES - {"accept": "text/plain, application/json:*************************************************************************, "host:""apl-12345", "connection":"unknown"}"}

 


 

Labels (1)
0 Karma

richgalloway
SplunkTrust
SplunkTrust

You say you have two types of data, but the example look very similar to me.  In general, yes, two types of data call for 2 set of props, but I believe that is not the case here.

In this case, I believe the problem is the data is not well-formed JSON so Splunk cannot parse it.  Paste the events into jsonlint.com to see what I mean.

---
If this reply helps you, Karma would be appreciated.
0 Karma

mahesh27
Communicator

hi @richgalloway, actually i have only json logs before, but now logs with timestamp added.
so i need props to fetch this other logs as well to avoid json parsing issues.

0 Karma

yeahnah
Motivator

If the pasted JSON is correct then this is badly formatted JSON

"host:""apl-12345"

 

0 Karma

mahesh27
Communicator

ok will try to change it, but can you please confirm the props i am using is correct??

0 Karma

yeahnah
Motivator

If it is valid JSON and you want to use INDEXED_EXTRACTIONS then this is all that is needed.

[sourcetype]
INDEXED_EXTRACTIONS=json

Note the implications of using this setting though.

https://docs.splunk.com/Documentation/Splunk/9.0.4/Admin/Propsconf#Structured_Data_Header_Extraction...

 

 

0 Karma
Get Updates on the Splunk Community!

Announcing Scheduled Export GA for Dashboard Studio

We're excited to announce the general availability of Scheduled Export for Dashboard Studio. Starting in ...

Extending Observability Content to Splunk Cloud

Watch Now!   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to leverage ...

More Control Over Your Monitoring Costs with Archived Metrics GA in US-AWS!

What if there was a way you could keep all the metrics data you need while saving on storage costs?This is now ...