Getting Data In

Help with sending data from api to splunk enterprise

bsheppard8
Loves-to-Learn Lots

I need some help understanding how to send data from an api to splunk enterprise so that I can create a dashboard about the information. The api is open source and located at https://ghibliapi.herokuapp.com/#. I understand that I can get the information using the curl command, but how do I input this information directly into my splunk instance? I don't have the option to use REST API as an data source for 'Add Data'. So far I've tried to print the output to a txt file and monitor that file using the universal forwarder, but I can't split the data into events properly, as the data is ingested line by line and backwards, regardless of the settings to props.conf. These are the current settings in my props file:

[apiver2]
EVENT_BREAKER_ENABLE = true
EVENT_BREAKER = "(/{)"
SHOULD_LINEMERGE = true
BREAK_ONLY_BEFORE_DATE = false
BREAK_ONLY_BEFORE = ^/{
MUST_BREAK_AFTER = ^/},
MUST_NOT_BREAK_AFTER = ^"id.+
MUST_NOT_BREAK_BEFORE = ^"url.+

And it always displays like this: splunk api ingest.jpgSo I'm thinking that maybe this is because of the data format of the request. What are my options for ingesting this data? A lot of this is new to me, so would HTTP Event Collector work, or is there something else I should do?

Thanks in advance!

Tags (2)
0 Karma

richgalloway
SplunkTrust
SplunkTrust

You have far more props than is necessary.  Please share some sample events so we can help you set the breaking attributes.  Be sure to indicate where one event ends and the next begins.

Note that the slashes in the props.conf settings need to be backslashes to properly escape the braces. 

Why can you not use the REST API?

---
If this reply helps you, Karma would be appreciated.
0 Karma

bsheppard8
Loves-to-Learn Lots

I can't use REST API because the option isn't available in my instance. I don't see it under 'Data Inputs'.

I gave an example from my earlier picture. The full event should look something like this:

{
"id": "d868e6ec-c44a-405b-8fa6-f7f0f8cfb500",
"title": "The Red Turtle",
"original_title": "レッドタートル ある島の物語",
"original_title_romanised": "Reddotātoru aru shima no monogatari",
"description": "A man set adrift by a storm wakes up on a beach. He discovers that he is on a deserted island with plenty of fresh water, fruit and a dense bamboo forest. He builds a raft from bamboo and attempts to sail away, but his raft is destroyed by an unseen monster in the sea, forcing him back to the island. He tries again with another, larger raft, but is again foiled by the creature. A third attempt again ends with the raft destroyed, but this time he is confronted by a giant red turtle, which stares at him, and forces him back to the island.",
"director": "Michaël Dudok de Wit",
"producer": "Toshio Suzuki, Isao Takahata, Vincent Maraval, Pascal Caucheteux, Grégoire Sorlat",
"release_date": "2016",
"running_time": "80",
"rt_score": "93",
"people": [
"https://ghibliapi.herokuapp.com/people/"
],
"species": [
"https://ghibliapi.herokuapp.com/species/"
],
"locations": [
"https://ghibliapi.herokuapp.com/locations/"
],
"vehicles": [
"https://ghibliapi.herokuapp.com/vehicles/"
],
"url": "https://ghibliapi.herokuapp.com/films/d868e6ec-c44a-405b-8fa6-f7f0f8cfb500"
}

But you can see in this picture here that the info comes in reverse, and every line is turned into its own event.

splunk screenshot.png

 

I also tried the backslash and restarting splunk, but that didn't change anything.

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...