All Apps and Add-ons

REST API Modular Input: Is there a way to have a REST API call retrieve only new events, not all data?

theouhuios
Motivator

Hello

I am new to writing Rest APi calls so please pardon my ignorance if its really simple. I wrote a rest api call which talks to cloudera and gets the events. This part works fine. The issue is it polls every 60sec( default) as I am using the Rest API Modular Input and it pulls all data instead of getting just the new events which happened.

Is there a way to tell a REST api call to get only new events? I know that I can have the time in the URL argument which will then keep data limited to only that time, but how to I dynamically change that date time value so that it automatically pulls newer data?

Tags (1)
1 Solution

martin_mueller
SplunkTrust
SplunkTrust

If you're on version 1.3.5 then the date token used in the URL should update itself for every call. If you need more tokens you can add them yourself.

View solution in original post

martin_mueller
SplunkTrust
SplunkTrust

If you're on version 1.3.5 then the date token used in the URL should update itself for every call. If you need more tokens you can add them yourself.

martin_mueller
SplunkTrust
SplunkTrust

Does that answer your original question?

0 Karma

theouhuios
Motivator

Yup. Thanks Martin

0 Karma

martin_mueller
SplunkTrust
SplunkTrust

You could set up a transforms.conf rule that routes empty events to the nullQueue.

theouhuios
Motivator

Yeah, did the same. I was thinking if I could do it in the input itself, but since that didn't work used the transforms

0 Karma

theouhuios
Motivator

When I place a datetime as a token it gets empty events which is actually right. But is it possible to not index the json data if there is no value in the items ?

0 Karma

martin_mueller
SplunkTrust
SplunkTrust

The example token only gives you the date, but you can add any tokens to the app you need.

0 Karma

theouhuios
Motivator

But it only gets the date but not the time right? Each time it runs its getting everything for today instead of getting only the changes which happened.

0 Karma
Get Updates on the Splunk Community!

Enterprise Security Content Update (ESCU) | New Releases

In January, the Splunk Threat Research Team had one release of new security content via the Splunk ES Content ...

Expert Tips from Splunk Professional Services, Ensuring Compliance, and More New ...

Splunk Lantern is a Splunk customer success center that provides advice from Splunk experts on valuable data ...

Observability Release Update: AI Assistant, AppD + Observability Cloud Integrations & ...

This month’s releases across the Splunk Observability portfolio deliver earlier detection and faster ...