Help with props.conf to break events

vrmandadi — Sat, 30 Apr 2022 19:46:00 GMT

I am trying to work on props.conf to parse and break correctly.I am pushing data using CURL commands but it is sending 50 logs in one event.It worked through UI but failing when sent from CURL commands.I want to break it into individual events .Only the first event start with "{"sourcetype": "json","event": {"

AND ends with "last_updated" (EXAMPLE:"last_updated": "2022-03-24T02:35:41.148727Z"
},) .Rest of the events START WITH ID and end with last_updated....There are lot of nested ID in the event which I did not post but the syntax should be something that will break after last_updated

I want the events to BREAK AFTER THE "last_updated" followed by closed flower brackets and the new event should start from

NOTE:ONLY THE first event start is different ..rest all events start with id and end with last_updated.

I tried BREAK_ONLY_BEFORE=\"\w*\"\:\s\"\d*\-\d*\-\d*\w\d*\:\d*\:\d*\.\d*\w\" ... but its not breaking correctly

{
"id":

Following are the sample events that I want to break

Event1:

{"sourcetype": "json","event": { . . . . . }, "created": "2022-02-07", "last_updated": "2022-03-24T02:35:41.083145Z"

Event 2:

{ "id": 150749, "name": "no hostname 1660322000234", . . . . . "created": "2022-02-07", "last_updated": "2022-03-24T02:35:41.148727Z" }

I used the below props...it worked uploading sample file via GUI but when I used this sourcetype in CURL through HEC it is not breaking.

[ Netbox ]
CHARSET=UTF-8
DATETIME_CONFIG=CURRENT
LINE_BREAKER=([\r\n]+)\s+{
MUST_BREAK_BEFORE=\"\w*\"\:\s\"\d*\-\d*\-\d*\w\d*\:\d*\:\d*\.\d*\w\"
NO_BINARY_CHECK=true
SHOULD_LINEMERGE=false
category=Custom
disabled=false
pulldown_type=true

CURL:

curl -k http://10.xx.xx.xx:8088/services/collector/event -H 'Authorization: Splunk <TOKEN>' -d '{"sourcetype": "Netbox","event": '"$SITEINFO"'}'

Re: Help with props.conf to break events

PickleRick — Sat, 30 Apr 2022 06:40:53 GMT

It doesn't work that way. The HEC /collector/event endpoint is meant for already split events. On ingestion from this endpoint Splunk skips the event breaking altogether because you already said where the event starts and where it ends. You should split it before and send as multiple events. You can however send multiple events in one HTTP transaction. Just make sure they are properly formed. Like

{ "sourcetype":"whatever","index":"test1","event":"a b c d"}
{"event":"another one"}
{"index":"test2","event":{"field1":"value1","field2":"value2"}}

I'm not sure if you need commas between the events.

topic Help with props.conf to break events in Getting Data In

Help with props.conf to break events

Re: Help with props.conf to break events