Getting Data In

Forwarder Logs retention

gladieu1
Explorer

Dear Community,

I'd like to know what retention logs is possible at Forwarder level ?
We have intention to duplicate the logs from the forwarder to two different Indexer, but what are the mechanism and principle of the logs retention ?
I understood they are stored if both links to Indexers are down, but if only one is down ?

Thanks for your replies,

Regards

0 Karma

koshyk
Super Champion

Your question is vague, without understanding your Splunk Design/tiering

Assumption i'm going to make is
1. You have got Forwarder tier which collect syslog and send to Indexer?
2. You UF on the agents which sends to indexer

In Both Cases, Splunk forwarder does NOT cache or store data. But just reads from the point in which it left off.
for (1), the retention is dependent on the logrotate or syslog rotation functionality. So if you have log retention of 7 days, your data will be there for 7 days

for (2), the logs are kept as per the settings on the client system. For example, Windows may hold eventlogs for 24hrs or when it fills up 2GB etc. So it is purely dependent on the client system

gladieu1
Explorer

Yes it is case 1), to have a Forwarder sending Syslogs to two Indexers (load balancing).

Kind regards

0 Karma

gladieu1
Explorer

Dear,
thanks for your reply.
In fact, in case we have a Forwarder and two Indexers, if one Indexer goes down, do the Logs are stored in the Forwarder as long as the 2nd Indexer goes up again ? If so, how long (I guess it will depend on disk size, but in theory is there any limitation in time) ?
What is the mechanism involved in fact ?

Thanks

kind regards

0 Karma

koshyk
Super Champion

as mentioned, Splunk Software in Forwarder DO NOT store logs. So it is upto your log clearing application to do this job. So if it retains for 1 year, it will be there for 1 year and Splunk CAN start indexing from all remaining logs .

the best way to tackle is, to send data to both your indexers and enable indexer replication. This way when ONE indexer is down, you can still send to other indexer as well as you will have a copy of data from other indexer. Link for overview

Please upvote/accept if the replies helped you.

gladieu1
Explorer

Nobody please ?

0 Karma

adonio
Ultra Champion

can you elaborate on the problem you are trying to solve?
i am confused as to how you use the word retention, if you are monitoring a file, the file sits on the disk and the retention of the file is as long as you (or disk) allows.

0 Karma
Get Updates on the Splunk Community!

Thanks for the Memories! Splunk University, .conf24, and Community Connections

Thank you to everyone in the Splunk Community who joined us for .conf24 – starting with Splunk University and ...

.conf24 | Day 0

Hello Splunk Community! My name is Chris, and I'm based in Canberra, Australia's capital, and I travelled for ...

Enhance Security Visibility with Splunk Enterprise Security 7.1 through Threat ...

 (view in My Videos)Struggling with alert fatigue, lack of context, and prioritization around security ...