Getting Data In

How to import json file?

vernikose
Explorer

Hello,

I am trying to import a json file to SPLUNK. It seems that the file is imported into one event but not all of it, it looks like that the file is imported by 10% (or less).

Could it be because of a configuration that I have to change?

the file is of this format

 

 

{"resultsPerPage":344,"startIndex":0,"totalResults":344,"format":"NVD_CVE","version":"2.0","timestamp":"2023-02-15T09:42:40.560","vulnerabilities":[{"cve":{"id":"CVE-2013-10012","sourceIdentifier":"cna@vuldb.com","published":"2023-01-16T11:15:10.037","lastModified":"2023-01-24T15:14:10.117","vulnStatus":"Analyzed","descriptions":[{"lang":"en","value":"A vulnerability, which was classified as critical, was found in antonbolling clan7ups. Affected is an unknown function of the component Login\/Session. The manipulation leads to sql injection. The name of the patch is 25afad571c488291033958d845830ba0a1710764. It is recommended to apply a patch to fix this issue. The identifier of this vulnerability is VDB-218388."}],"metrics":{"cvssMetricV31":[{"source":"nvd@nist.gov","type":"Primary","cvssData":{"version":"3.1","vectorString":"CVSS:3.1\/AV:N\/AC:L\/PR:N\/UI:N\/S:U\/C:H\/I:H\/A:H","attackVector":"NETWORK","attackComplexity":"LOW","privilegesRequired":"NONE","userInteraction":"NONE","scope":"UNCHANGED","confidentialityImpact":"HIGH","integrityImpact":"HIGH","availabilityImpact":"HIGH","baseScore":9.8,"baseSeverity":"CRITICAL"},"exploitabilityScore":3.9,"impactScore":5.9}],"cvssMetricV30":[{"source":"cna@vuldb.com","type":"Secondary","cvssData":{"version":"3.0","vectorString":"CVSS:3.0\/AV:A\/AC:L\/PR:L\/UI:N\/S:U\/C:L\/I:L\/A:L","attackVector":"ADJACENT_NETWORK","attackComplexity":"LOW","privilegesRequired":"LOW","userInteraction":"NONE","scope":"UNCHANGED","confidentialityImpact":"LOW","integrityImpact":"LOW","availabilityImpact":"LOW","baseScore":5.5,"baseSeverity":"MEDIUM"},"exploitabilityScore":2.1,"impactScore":3.4}],"cvssMetricV2":[{"source":"cna@vuldb.com","type":"Secondary","cvssData":{"version":"2.0","vectorString":"AV:A\/AC:L\/Au:S\/C:P\/I:P\/A:P","accessVector":"ADJACENT_NETWORK","accessComplexity":"LOW","authentication":"SINGLE","confidentialityImpact":"PARTIAL","integrityImpact":"PARTIAL","availabilityImpact":"PARTIAL","baseScore":5.2},"baseSeverity":"MEDIUM","exploitabilityScore":5.1,"impactScore":6.4,"acInsufInfo":false,"obtainAllPrivilege":false,"obtainUserPrivilege":false,"obtainOtherPrivilege":false,"userInteractionRequired":false}]},"weaknesses":[{"source":"cna@vuldb.com","type":"Primary","description":[{"lang":"en","value":"CWE-89"}]}],"configurations":[{"nodes":[{"operator":"OR","negate":false,"cpeMatch":[{"vulnerable":true,"criteria":"cpe:2.3:a:clan7ups_project:clan7ups:*:*:*:*:*:*:*:*","versionEndExcluding":"2013-02-12","matchCriteriaId":"12D82AEE-3A68-4121-811C-C3462BCEAF25"}]}]}],"references":[{"url":"https:\/\/github.com\/antonbolling\/clan7ups\/commit\/25afad571c488291033958d845830ba0a1710764","source":"cna@vuldb.com","tags":["Patch","Third Party Advisory"]}

 

 

 

I would appreciate any help 

Thank you

Labels (1)
0 Karma
1 Solution

scelikok
SplunkTrust
SplunkTrust

If you are using Add Data GUI method, you can add new parameter as like TRUNCATE=100000.

Or you should add your props.conf like below and restart Splunk.

props.conf

[your_sourcetype]
TRUNCATE = 100000

 

If this reply helps you an upvote and "Accept as Solution" is appreciated.

View solution in original post

scelikok
SplunkTrust
SplunkTrust

This seems fine and should not cause problem with uploading. 

I can not think any reason for the problem.

If this reply helps you an upvote and "Accept as Solution" is appreciated.
0 Karma

scelikok
SplunkTrust
SplunkTrust

Can you please share your full config about TRUNCATE setting? Did you enter it into the right stanza? 

If this reply helps you an upvote and "Accept as Solution" is appreciated.
0 Karma

vernikose
Explorer

/opt/splunk/etc/system/local# cat props.conf
[test]
SHOULD_LINEMERGE = true
TRUNCATE = 100000

0 Karma

scelikok
SplunkTrust
SplunkTrust

If you are using Add Data GUI method, you can add new parameter as like TRUNCATE=100000.

Or you should add your props.conf like below and restart Splunk.

props.conf

[your_sourcetype]
TRUNCATE = 100000

 

If this reply helps you an upvote and "Accept as Solution" is appreciated.

vernikose
Explorer

When I add the TRUNCATE = 100000 the file is not uploaded. I have no results. even with TRUNCATE = 0

0 Karma

scelikok
SplunkTrust
SplunkTrust

Hi @vernikose,

If the file is bigger than 10000 characters and Splunk tires to import as one event your should be hitting TRUNCATE=10000 default limit. You can change this parameter on your sourcetype and try again.

In order to split the file into 344 events you should set LINE_BREAKER settings accordingly.

 

If this reply helps you an upvote and "Accept as Solution" is appreciated.
0 Karma

vernikose
Explorer

Hi @scelikok,

thanks for your feedback. I don't mind to have it in one event.

 

where about do I change the TRUNCATE=10000?

0 Karma

vernikose
Explorer

I have added TRUNCATE = 0 at /opt/splunk/etc/system/props.conf and the file didn't upload it at all (

I cannot see anywhere else to have it

 /opt/splunk/etc/system/local# grep -i -r "TRUNCATE" .
./props.conf:TRUNCATE = 0
./limits.conf:truncate_report = 0

0 Karma
Get Updates on the Splunk Community!

AppDynamics Summer Webinars

This summer, our mighty AppDynamics team is cooking up some delicious content on YouTube Live to satiate your ...

SOCin’ it to you at Splunk University

Splunk University is expanding its instructor-led learning portfolio with dedicated Security tracks at .conf25 ...

Credit Card Data Protection & PCI Compliance with Splunk Edge Processor

Organizations handling credit card transactions know that PCI DSS compliance is both critical and complex. The ...