topic Re: Creating new events via REST receivers endpoint in Getting Data In

Creating new events via REST receivers endpoint

misteryuku — Mon, 19 Mar 2012 08:39:22 GMT

Whenever i want to create new events via REST receivers endpoint, can i create new fields and set their values for the fields??

Re: Creating new events via REST receivers endpoint

dart — Mon, 19 Mar 2012 14:14:19 GMT

You can add fields in the data in key=value pairs and they will be extracted automatically.

See also the docs on the receivers endpoint

Re: Creating new events via REST receivers endpoint

misteryuku — Mon, 19 Mar 2012 15:10:54 GMT

Are you sure? It didn't work.
I appended the key value pairs in the REST API endpoint url. I appended the source and sourcetype and they appear during the search, but not the additional fields that i created.

Re: Creating new events via REST receivers endpoint

Damien_Dallimor — Mon, 19 Mar 2012 23:11:20 GMT

If you format the content of your log message using key=value pairs, then Splunk will automatically extract these at search time. This log message gets sent in the body of the REST HTTP Request.

The url argument key=value pairs are for defining Splunk meta data fields(index, source, sourcetype, host, host_regex)

The Splunk Java Logging Framework provides a useful interface to make it easier to create best practice log messages and integrate with your preferred logging framework ie: there are log4j, logback appenders that will seamlessly handle logging to the SPLUNK RestEndpoint. Download it and look at the examples.

Re: Creating new events via REST receivers endpoint

misteryuku — Thu, 22 Mar 2012 06:22:04 GMT

Is it necessary to format the log message using the Splunk logging framework?

Re: Creating new events via REST receivers endpoint

Damien_Dallimor — Thu, 22 Mar 2012 06:30:58 GMT

No, it is simply a framework to make it easier for you.

Re: Creating new events via REST receivers endpoint

misteryuku — Thu, 22 Mar 2012 06:51:13 GMT

What are the other ways to format the log message?

Re: Creating new events via REST receivers endpoint

Ayn — Thu, 22 Mar 2012 06:58:16 GMT

You could format it any way you want. Splunk only extracts keys and values automatically if they follow the key=value standard, but if you format it differently it's just a matter of creating field extractions for your specific log format instead.

Re: Creating new events via REST receivers endpoint

misteryuku — Thu, 22 Mar 2012 07:21:45 GMT

I have been looking through logback and i would like to ask for the log message formatted with key=value pairs, they are sent to the Splunk endpoint by socket appenders. Is that right?

Re: Creating new events via REST receivers endpoint

Ayn — Thu, 22 Mar 2012 07:40:32 GMT

I don't know what socket appenders you are talking about.

I still think the best idea for you would be to show as a complete case what you're trying to achieve, with example data and an actual use-case, rather than asking about small details one at a time. But, that's just me.

Re: Creating new events via REST receivers endpoint

misteryuku — Thu, 22 Mar 2012 08:29:16 GMT

I wanted to format the log message into key value pairs using logback framework and append the log message to the Splunk rest receivers endpoint. I'm doing these all in java I wanted to append the formatted log message to an outputstream appender and get an outputstream object to be sent to the splunk's rest recievers stream endpoint.

I'm wanted to format the log message with sample key value pairs like this.
logger.debug("wrap = true, setValue = false,")

Re: Creating new events via REST receivers endpoint

Ayn — Thu, 22 Mar 2012 08:38:13 GMT

Ok, can't help you with how the logback framework works. Sorry.

Re: Creating new events via REST receivers endpoint

misteryuku — Thu, 22 Mar 2012 08:42:07 GMT

Never mind. Thanks.

Re: Creating new events via REST receivers endpoint

dart — Thu, 22 Mar 2012 14:41:31 GMT

I mean in the same place the data of the event is sent, not as extra parameters

Re: Creating new events via REST receivers endpoint

misteryuku — Fri, 23 Mar 2012 04:04:27 GMT

I set the key=value pairs into the body of the REST HTTP request directly using Java REST SDK API. Example :
RequestMessage reqMsg = new RequestMessage();
reqMsg.setMethod("post");
reqMsg.getHeader().put("x-splunk-input-mode", "streaming");
reqMsg.setContent("hater = yes, nothater = no");
Then i send the message to the simple reciever rest endpoint.
String path = "/services/receivers/simple?host=localhost&index=main&source=addfields&sourcetype=addedfields";
ResponseMessage resMsg = authService.send(path,reqMsg);

Re: Creating new events via REST receivers endpoint

misteryuku — Fri, 23 Mar 2012 04:05:27 GMT

Then, When i opened the search app to see the added data, i saw both the new fields and the raw data which is the key=value pairs that i set directly added.

Re: Creating new events via REST receivers endpoint

misteryuku — Fri, 23 Mar 2012 04:13:45 GMT

I only want to see the added key=value pairs below the raw data, not together with the raw data.

When i tried adding the raw data and the key=value pairs to the content body of rest http request like this using java rest sdk api,
reqMsg.setContent("rawdata1 - hater = yes, nothater = no");

i see this added on the search app.
rawdata1 - hater = yes, nothater = no
(for the added raw data value)
the new fields hater and nohater are added below the raw field.

I just want the rawdata1 as the raw data value. Has it to be done using Java logging framework if i'm using java.