I am trying to work on props.conf to parse and break correctly.I am pushing data using CURL commands but it is sending 50 logs in one event.It worked through UI but failing when sent from CURL commands.I want to break it into individual events .Only the first event start with "{"sourcetype": "json","event": {"
AND ends with "last_updated" (EXAMPLE:"last_updated": "2022-03-24T02:35:41.148727Z"
},) .Rest of the events START WITH ID and end with last_updated....There are lot of nested ID in the event which I did not post but the syntax should be something that will break after last_updated
I want the events to BREAK AFTER THE "last_updated" followed by closed flower brackets and the new event should start from
NOTE:ONLY THE first event start is different ..rest all events start with id and end with last_updated.
I tried BREAK_ONLY_BEFORE=\"\w*\"\:\s\"\d*\-\d*\-\d*\w\d*\:\d*\:\d*\.\d*\w\" ... but its not breaking correctly
{
"id":
Following are the sample events that I want to break
Event1:
{"sourcetype": "json","event": {
.
.
.
.
.
},
"created": "2022-02-07",
"last_updated": "2022-03-24T02:35:41.083145Z"
Event 2:
{
"id": 150749,
"name": "no hostname 1660322000234",
.
.
.
.
.
"created": "2022-02-07",
"last_updated": "2022-03-24T02:35:41.148727Z"
}
I used the below props...it worked uploading sample file via GUI but when I used this sourcetype in CURL through HEC it is not breaking.
[ Netbox ]
CHARSET=UTF-8
DATETIME_CONFIG=CURRENT
LINE_BREAKER=([\r\n]+)\s+{
MUST_BREAK_BEFORE=\"\w*\"\:\s\"\d*\-\d*\-\d*\w\d*\:\d*\:\d*\.\d*\w\"
NO_BINARY_CHECK=true
SHOULD_LINEMERGE=false
category=Custom
disabled=false
pulldown_type=true
CURL:
curl -k http://10.xx.xx.xx:8088/services/collector/event -H 'Authorization: Splunk <TOKEN>' -d '{"sourcetype": "Netbox","event": '"$SITEINFO"'}'
It doesn't work that way. The HEC /collector/event endpoint is meant for already split events. On ingestion from this endpoint Splunk skips the event breaking altogether because you already said where the event starts and where it ends. You should split it before and send as multiple events. You can however send multiple events in one HTTP transaction. Just make sure they are properly formed. Like
{ "sourcetype":"whatever","index":"test1","event":"a b c d"}
{"event":"another one"}
{"index":"test2","event":{"field1":"value1","field2":"value2"}}
I'm not sure if you need commas between the events.