topic Re: import json file in Getting Data In

How to import json file?

vernikose — Thu, 16 Feb 2023 15:55:44 GMT

Hello,

I am trying to import a json file to SPLUNK. It seems that the file is imported into one event but not all of it, it looks like that the file is imported by 10% (or less).

Could it be because of a configuration that I have to change?

the file is of this format

{"resultsPerPage":344,"startIndex":0,"totalResults":344,"format":"NVD_CVE","version":"2.0","timestamp":"2023-02-15T09:42:40.560","vulnerabilities":[{"cve":{"id":"CVE-2013-10012","sourceIdentifier":"cna@vuldb.com","published":"2023-01-16T11:15:10.037","lastModified":"2023-01-24T15:14:10.117","vulnStatus":"Analyzed","descriptions":[{"lang":"en","value":"A vulnerability, which was classified as critical, was found in antonbolling clan7ups. Affected is an unknown function of the component Login\/Session. The manipulation leads to sql injection. The name of the patch is 25afad571c488291033958d845830ba0a1710764. It is recommended to apply a patch to fix this issue. The identifier of this vulnerability is VDB-218388."}],"metrics":{"cvssMetricV31":[{"source":"nvd@nist.gov","type":"Primary","cvssData":{"version":"3.1","vectorString":"CVSS:3.1\/AV:N\/AC:L\/PR:N\/UI:N\/S:U\/C:H\/I:H\/A:H","attackVector":"NETWORK","attackComplexity":"LOW","privilegesRequired":"NONE","userInteraction":"NONE","scope":"UNCHANGED","confidentialityImpact":"HIGH","integrityImpact":"HIGH","availabilityImpact":"HIGH","baseScore":9.8,"baseSeverity":"CRITICAL"},"exploitabilityScore":3.9,"impactScore":5.9}],"cvssMetricV30":[{"source":"cna@vuldb.com","type":"Secondary","cvssData":{"version":"3.0","vectorString":"CVSS:3.0\/AV:A\/AC:L\/PR:L\/UI:N\/S:U\/C:L\/I:L\/A:L","attackVector":"ADJACENT_NETWORK","attackComplexity":"LOW","privilegesRequired":"LOW","userInteraction":"NONE","scope":"UNCHANGED","confidentialityImpact":"LOW","integrityImpact":"LOW","availabilityImpact":"LOW","baseScore":5.5,"baseSeverity":"MEDIUM"},"exploitabilityScore":2.1,"impactScore":3.4}],"cvssMetricV2":[{"source":"cna@vuldb.com","type":"Secondary","cvssData":{"version":"2.0","vectorString":"AV:A\/AC:L\/Au:S\/C:P\/I:P\/A:P","accessVector":"ADJACENT_NETWORK","accessComplexity":"LOW","authentication":"SINGLE","confidentialityImpact":"PARTIAL","integrityImpact":"PARTIAL","availabilityImpact":"PARTIAL","baseScore":5.2},"baseSeverity":"MEDIUM","exploitabilityScore":5.1,"impactScore":6.4,"acInsufInfo":false,"obtainAllPrivilege":false,"obtainUserPrivilege":false,"obtainOtherPrivilege":false,"userInteractionRequired":false}]},"weaknesses":[{"source":"cna@vuldb.com","type":"Primary","description":[{"lang":"en","value":"CWE-89"}]}],"configurations":[{"nodes":[{"operator":"OR","negate":false,"cpeMatch":[{"vulnerable":true,"criteria":"cpe:2.3:a:clan7ups_project:clan7ups:*:*:*:*:*:*:*:*","versionEndExcluding":"2013-02-12","matchCriteriaId":"12D82AEE-3A68-4121-811C-C3462BCEAF25"}]}]}],"references":[{"url":"https:\/\/github.com\/antonbolling\/clan7ups\/commit\/25afad571c488291033958d845830ba0a1710764","source":"cna@vuldb.com","tags":["Patch","Third Party Advisory"]}

I would appreciate any help

Thank you

Re: import json file

scelikok — Thu, 16 Feb 2023 12:03:04 GMT

Hi @vernikose,

If the file is bigger than 10000 characters and Splunk tires to import as one event your should be hitting TRUNCATE=10000 default limit. You can change this parameter on your sourcetype and try again.

In order to split the file into 344 events you should set LINE_BREAKER settings accordingly.

Re: import json file

vernikose — Thu, 16 Feb 2023 12:41:15 GMT

Hi @scelikok,

thanks for your feedback. I don't mind to have it in one event.

where about do I change the TRUNCATE=10000?

Re: import json file

vernikose — Thu, 16 Feb 2023 12:52:43 GMT

I have added TRUNCATE = 0 at /opt/splunk/etc/system/props.conf and the file didn't upload it at all (

I cannot see anywhere else to have it

/opt/splunk/etc/system/local# grep -i -r "TRUNCATE" .
./props.conf:TRUNCATE = 0
./limits.conf:truncate_report = 0

Re: import json file

scelikok — Thu, 16 Feb 2023 12:55:45 GMT

If you are using Add Data GUI method, you can add new parameter as like TRUNCATE=100000.

Or you should add your props.conf like below and restart Splunk.

props.conf [your_sourcetype] TRUNCATE = 100000

Re: import json file

vernikose — Thu, 16 Feb 2023 13:11:28 GMT

When I add the TRUNCATE = 100000 the file is not uploaded. I have no results. even with TRUNCATE = 0

Re: import json file

scelikok — Thu, 16 Feb 2023 13:39:24 GMT

Can you please share your full config about TRUNCATE setting? Did you enter it into the right stanza?

Re: import json file

vernikose — Thu, 16 Feb 2023 13:42:56 GMT

/opt/splunk/etc/system/local# cat props.conf
[test]
SHOULD_LINEMERGE = true
TRUNCATE = 100000

Re: How to import json file?

scelikok — Fri, 17 Feb 2023 05:47:38 GMT

This seems fine and should not cause problem with uploading.

I can not think any reason for the problem.