topic Re: Need help with a custom source type in Getting Data In

Need help with a custom source type

pir8radio — Wed, 30 Jun 2021 22:18:27 GMT

Hi, starting fresh. maybe I can explain a bit better here.. I found another similar issue to mine here: https://community.splunk.com/t5/Getting-Data-In/How-to-split-a-json-array-into-multiple-events-with-... I need it to break out the 20+ items in the string. For some reason setting up my source type like in this post, just gives me the first user worth of info. It doesnt break them all out.. Here is a dump of one of the raw json requests. its truncated at then end, looking into that. But i basically need to break out each user in this list with their stats just like that previous post above talks about. any help would be appreciated.

Re: Need help with a custom source type

pir8radio — Sun, 27 Jun 2021 03:48:56 GMT

must be harder than i thought....... 🙂

Re: Need help with a custom source type

pir8radio — Sun, 27 Jun 2021 14:17:53 GMT

I guess the community isnt as active any more.. 😞

Re: Need help with a custom source type

kamlesh_vaghela — Sun, 27 Jun 2021 14:56:24 GMT

@pir8radio

Can you please share your configuration?

Re: Need help with a custom source type

pir8radio — Sun, 27 Jun 2021 15:10:46 GMT

This is how i have it configured today, but it only grabs the first user "Pir8radio" ignores the rest.

And this is result, but result is only the first array item. not the rest of them.

Re: Need help with a custom source type

kamlesh_vaghela — Sun, 27 Jun 2021 15:50:11 GMT

@pir8radio

Your shared JSON looks invalid. Can you please share valid _raw event?

Re: Need help with a custom source type

pir8radio — Sun, 27 Jun 2021 16:16:25 GMT

I am using splunk REST plugin to grab json data. Plugin may strip raw info i am not sure. But here is the real API json data: https://signapool.notallmine.net/api/getMiners

@kamlesh_vaghela If you can make input work with raw json i can just write a script that grabs json and puts it into a text file every 2 mins and not use REST plugin.

Re: Need help with a custom source type

pir8radio — Sun, 27 Jun 2021 20:37:14 GMT

Hope i made sense. 🙂

Re: Need help with a custom source type

kamlesh_vaghela — Mon, 28 Jun 2021 06:05:25 GMT

@pir8radio

Can you please try this?

[YOUR_SOURCETYPE] DATETIME_CONFIG = CURRENT SHOULD_LINEMERGE=true LINE_BREAKER=}(\,){\"address\" CHARSET=UTF-8 SEDCMD-a=s/{"miners":\[//g SEDCMD-b=s/(\].*)//g

Thanks
KV
▄︻̷̿┻̿═━一

If any of my reply helps you to solve the problem Or gain knowledge, an upvote would be appreciated.

Re: Need help with a custom source type

pir8radio — Mon, 28 Jun 2021 16:10:12 GMT

it will not let me set "should_linemerge=true" with line_breaker set to

}(\,){\"address\"

Re: Need help with a custom source type

pir8radio — Mon, 28 Jun 2021 16:20:41 GMT

looks like it is working with the below config.. thank you! 😄