Splunk real-time data input from html via REST

[tamduong16]

I've been trying to look for a way for Splunk to input real-time data and I come across Rest API thinking it could be a solution to my problem. But after I set it up a Rest api base on the instruction from Splunk, no data is being added to Splunk. Could anyone let me know what I did wrong here? For testing purposes, I use a Wikipedia site as an endpoint url and I did not set up any kind of handler. I just want to know if this REST api could get me any type of information from the site.
Someone recommended me to define a custom sourcetype but I don't know what should I define in this custom sourcetype. If this is the way that I could fix it, can anyone please be specific in what I should put in this custom sourcetype? I'm very new to Splunk REST api!

[Sukisen1981 · Accepted solution]

HI,
You can not directly import the wiki url into the REST API.

Try this as yor end point url - https://en.wikipedia.org/w/api.php?action=query&titles=Alan%20Turing&prop=revisions&rvprop=content&f...

I am able to get a JSON stream once set this as my end point in the REST API url.
There are of course many other things you have to do for field extractions, but this gets the data into your REST API.
You can replace Alan Turing with any other search text to retrieve data.
Considerations:
How frequently do you need to poll the data? for a test wiki site like this maybe once in 2-3 days is enough.
You can import data as text, json and xml through the REST API , choose what is relevant or you
You can also use delimiters in the RES API initial set up I choose not to give any delimiter because I just wanted to get the data in.
Your actual api site will surely have splitters, delimiters or is already in JSON format. It will work , main thing you can import a API text, JSON,, XML version of a website which allows this, you can not import an web page as it is.

[tamduong16]

thank you so much. It works!!!

[Sukisen1981]

could you please pass the test wiki url you are using to GET data into the REST API?

[tamduong16]

Here is the site: https://en.wikipedia.org/wiki/Alan_Turing
Thank you!