Getting Data In

Why is Syndication input repeating events?

jovelfer
Engager

Hi all,

I'm using the syndication component (latest version), to fetch data from multiple feeds:

https://www.cloudflarestatus.com/history.atom
https://cloud.ibm.com/status/api/notifications/feed.rss
https://status.aws.amazon.com/rss/all.rss
https://status.cloud.google.com/feed.atom
https://ocistatus.oraclecloud.com/history.rss

By adding the entries, the events have started to repeat every time each feed is processed, which is 5 minutes, that is, it is re-indexing the entire set of events every 5 minutes for each feed. The check is activated so that it only takes into account new events.

When I set one feed, for example google feed with 3 events:

jovelfer_0-1645695438462.png

After 5 min:

jovelfer_1-1645695510537.png

If I make:

index=gcc_extension_1 source = syndication://google_gcc_ext | stats count values(host) values(source) values(sourcetype) values(index) by _raw | WHERE count>0

There are 6 results, note that it is not the entire _raw that is repeated, since the _indextime is different each time the array is processed.


I've been researching and doing all kinds of tests for a long time, but I don't know what the problem could be. If anyone could help me out a bit with this I'd really appreciate it.

Here, the detail of feed conf:

jovelfer_2-1645695596944.png

Aside from screenshots, I can provide configuration as needed.

Thank you very much in advance.

Labels (3)
0 Karma

jovelfer
Engager

Any update on this?

0 Karma
Get Updates on the Splunk Community!

Splunk Observability Cloud | Unified Identity - Now Available for Existing Splunk ...

Raise your hand if you’ve already forgotten your username or password when logging into an account. (We can’t ...

Index This | How many sides does a circle have?

February 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

Registration for Splunk University is Now Open!

Are you ready for an adventure in learning?   Brace yourselves because Splunk University is back, and it's ...