I need some help understanding how to send data from an api to splunk enterprise so that I can create a dashboard about the information. The api is open source and located at https://ghibliapi.herokuapp.com/#. I understand that I can get the information using the curl command, but how do I input this information directly into my splunk instance? I don't have the option to use REST API as an data source for 'Add Data'. So far I've tried to print the output to a txt file and monitor that file using the universal forwarder, but I can't split the data into events properly, as the data is ingested line by line and backwards, regardless of the settings to props.conf. These are the current settings in my props file:
[apiver2]
EVENT_BREAKER_ENABLE = true
EVENT_BREAKER = "(/{)"
SHOULD_LINEMERGE = true
BREAK_ONLY_BEFORE_DATE = false
BREAK_ONLY_BEFORE = ^/{
MUST_BREAK_AFTER = ^/},
MUST_NOT_BREAK_AFTER = ^"id.+
MUST_NOT_BREAK_BEFORE = ^"url.+
And it always displays like this: So I'm thinking that maybe this is because of the data format of the request. What are my options for ingesting this data? A lot of this is new to me, so would HTTP Event Collector work, or is there something else I should do?
Thanks in advance!
You have far more props than is necessary. Please share some sample events so we can help you set the breaking attributes. Be sure to indicate where one event ends and the next begins.
Note that the slashes in the props.conf settings need to be backslashes to properly escape the braces.
Why can you not use the REST API?
I can't use REST API because the option isn't available in my instance. I don't see it under 'Data Inputs'.
I gave an example from my earlier picture. The full event should look something like this:
{
"id": "d868e6ec-c44a-405b-8fa6-f7f0f8cfb500",
"title": "The Red Turtle",
"original_title": "レッドタートル ã‚る島ã®ç‰©èªž",
"original_title_romanised": "ReddotÄtoru aru shima no monogatari",
"description": "A man set adrift by a storm wakes up on a beach. He discovers that he is on a deserted island with plenty of fresh water, fruit and a dense bamboo forest. He builds a raft from bamboo and attempts to sail away, but his raft is destroyed by an unseen monster in the sea, forcing him back to the island. He tries again with another, larger raft, but is again foiled by the creature. A third attempt again ends with the raft destroyed, but this time he is confronted by a giant red turtle, which stares at him, and forces him back to the island.",
"director": "Michaël Dudok de Wit",
"producer": "Toshio Suzuki, Isao Takahata, Vincent Maraval, Pascal Caucheteux, Grégoire Sorlat",
"release_date": "2016",
"running_time": "80",
"rt_score": "93",
"people": [
"https://ghibliapi.herokuapp.com/people/"
],
"species": [
"https://ghibliapi.herokuapp.com/species/"
],
"locations": [
"https://ghibliapi.herokuapp.com/locations/"
],
"vehicles": [
"https://ghibliapi.herokuapp.com/vehicles/"
],
"url": "https://ghibliapi.herokuapp.com/films/d868e6ec-c44a-405b-8fa6-f7f0f8cfb500"
}
But you can see in this picture here that the info comes in reverse, and every line is turned into its own event.
I also tried the backslash and restarting splunk, but that didn't change anything.