Getting Data In

Need help with a custom source type

pir8radio
Path Finder

Hi, starting fresh. maybe I can explain a bit better here..   I found another similar issue to mine here:   https://community.splunk.com/t5/Getting-Data-In/How-to-split-a-json-array-into-multiple-events-with-...               I need it to break out the 20+ items in the string. For some reason setting up my source type like in this post, just gives me the first user worth of info. It doesnt break them all out..   Here is a dump of one of the raw json requests.  its truncated at then end, looking into that.  But i basically need to break out each user in this list with their stats just like that previous post above talks about.   any help would be appreciated.  

Spoiler
 
Labels (1)
0 Karma
1 Solution

kamlesh_vaghela
SplunkTrust
SplunkTrust

@pir8radio 

Can you please try this?

[YOUR_SOURCETYPE]
DATETIME_CONFIG = CURRENT
SHOULD_LINEMERGE=true
LINE_BREAKER=}(\,){\"address\"
CHARSET=UTF-8
SEDCMD-a=s/{"miners":\[//g
SEDCMD-b=s/(\].*)//g

 

Screenshot 2021-06-28 at 11.35.11 AM.png

 

Thanks
KV
▄︻̷̿┻̿═━一

If any of my reply helps you to solve the problem Or gain knowledge, an upvote would be appreciated. 

View solution in original post

kamlesh_vaghela
SplunkTrust
SplunkTrust

@pir8radio 

Can you please try this?

[YOUR_SOURCETYPE]
DATETIME_CONFIG = CURRENT
SHOULD_LINEMERGE=true
LINE_BREAKER=}(\,){\"address\"
CHARSET=UTF-8
SEDCMD-a=s/{"miners":\[//g
SEDCMD-b=s/(\].*)//g

 

Screenshot 2021-06-28 at 11.35.11 AM.png

 

Thanks
KV
▄︻̷̿┻̿═━一

If any of my reply helps you to solve the problem Or gain knowledge, an upvote would be appreciated. 

pir8radio
Path Finder

looks like it is working with the below config.. thank you!  😄 

pir8radio_0-1624897235897.png

 

0 Karma

pir8radio
Path Finder

it will not let me set "should_linemerge=true" with line_breaker set to 

}(\,){\"address\"
Tags (1)
0 Karma

kamlesh_vaghela
SplunkTrust
SplunkTrust

@pir8radio 

Can you please share your configuration?

KV

pir8radio
Path Finder

This is how i have it configured today, but it only grabs the first user "Pir8radio" ignores the rest.

pir8radio_0-1624806183555.png

 

And this is result, but result is only the first array item. not the rest of them.  

pir8radio_0-1624806639990.png

 

 

0 Karma

kamlesh_vaghela
SplunkTrust
SplunkTrust

@pir8radio 

Your shared JSON looks invalid. Can you please share valid _raw event?

 

Screenshot 2021-06-27 at 9.19.23 PM.png

KV

0 Karma

pir8radio
Path Finder

Hope i made sense. 🙂

0 Karma

pir8radio
Path Finder

I am using splunk REST plugin to grab json data.   Plugin may strip raw info i am not sure.  But here is the real API json data:  https://signapool.notallmine.net/api/getMiners

@kamlesh_vaghela  If you can make input work with raw json i can just write a script that grabs json and puts it into a text file every 2 mins and not use REST plugin.

0 Karma

pir8radio
Path Finder

I guess the community isnt as active any more..  😞

 

0 Karma

pir8radio
Path Finder

must be harder than i thought.......  🙂

0 Karma
Get Updates on the Splunk Community!

Stay Connected: Your Guide to November Tech Talks, Office Hours, and Webinars!

🍂 Fall into November with a fresh lineup of Community Office Hours, Tech Talks, and Webinars we’ve ...

Transform your security operations with Splunk Enterprise Security

Hi Splunk Community, Splunk Platform has set a great foundation for your security operations. With the ...

Splunk Admins and App Developers | Earn a $35 gift card!

Splunk, in collaboration with ESG (Enterprise Strategy Group) by TechTarget, is excited to announce a ...