Hi all,
I'm using the syndication component (latest version), to fetch data from multiple feeds:
https://www.cloudflarestatus.com/history.atom
https://cloud.ibm.com/status/api/notifications/feed.rss
https://status.aws.amazon.com/rss/all.rss
https://status.cloud.google.com/feed.atom
https://ocistatus.oraclecloud.com/history.rss
By adding the entries, the events have started to repeat every time each feed is processed, which is 5 minutes, that is, it is re-indexing the entire set of events every 5 minutes for each feed. The check is activated so that it only takes into account new events.
When I set one feed, for example google feed with 3 events:
After 5 min:
If I make:
index=gcc_extension_1 source = syndication://google_gcc_ext | stats count values(host) values(source) values(sourcetype) values(index) by _raw | WHERE count>0
There are 6 results, note that it is not the entire _raw that is repeated, since the _indextime is different each time the array is processed.
I've been researching and doing all kinds of tests for a long time, but I don't know what the problem could be. If anyone could help me out a bit with this I'd really appreciate it.
Here, the detail of feed conf:
Aside from screenshots, I can provide configuration as needed.
Thank you very much in advance.
