Index doesn't show event anymore

[MattiaP]

Hi, I have an index that doesn't show events anymore. 

Could you help me please?

On November I had a problem with Mongo DB and I tried this solutions:

- https://community.splunk.com/t5/Knowledge-Management/Why-are-we-getting-these-errors-KV-Store-Proces...  --> doing this I noticed that permissions of files inside this folder have changed. May this be the cause of the problem? This solutiion didn't work

- I solved the problem doing this

Could you help me please?

Thank you

[PickleRick]

This is such a vague question and there is such little information...

1. Do you see events in other indexes but not in this one or you cannot find any events anywhere?

2. Were there any changes done lately to the environment?

3. Are you ingesting any data at all? Or was it just a "static" environment. In such case the data might have simply rolled over to frozen (got deleted) due to exceeding retention period.

As @gcusello mentioned, KV-store problems don't have much with having the events or not. They can cause other issues but they are not responsible for data suddenly disappearing from indexes.

[MattiaP]

Hi @PickleRick,

1 - I can see other events in other indexes

2 - One month ago I restarted KV store, I didn't make other changes.

3 -I'm ingesting data, there aren't frozen data.

What should I expect regarding the index from inputs.conf file?

Thank you in advance.

Mattia

[PickleRick]

OK. _If_ you are seeing current events in other indexes, it should mean that your "main" part of the environment is working relatively ok.

We don't have much info about your setup so we don't know whether this index you mention should contain events from multiple sources or just one source. If it's just one source, it may be that something caused that source to stop sending the events. Maybe due to turning off the forwarder or due to network problems. If it's an index gathering data from multiple sources... are you sure someone didn't delete it from your setup?

Do you see any events in this index and just don't see recent evens or do you not see any events at all, even the old ones? What are your index parameters? (size limits, retention settings).

[MattiaP]

Hi @PickleRick,

There are multiple sources.

I see event until November, from December zero events.

Thank you,

Mattia

[PickleRick]

Depending on how those event should be ingested I'd try to investigate if they are being properly sent to Splunk. As there are many ways of getting the data into Splunk you need to verify the particular way used in your case. Be it verifying UF connectivity, be it checking syslog traffic or whatever else.

There are no miracles. If your config didn't change and there are no events, they must have stopped "flowing".

[rafamss]

Hi @MattiaP,

Did you validate if your license is active? If no logs are being shown, it could be related to your license.

Kind regards,
Rafael Santos

[MattiaP]

Hi @rafamss,

yes, licensi is active.

Mattia

[gcusello]

Hi @MattiaP,

sorry, what's the relation between an index and MongoDB?

if you haven't events only in one index, you should check the inputs.conf that ingest data stored in that index.

The only excepton is if you have an index overriding, have you this?

Ciao.

Giuseppe

[MattiaP]

Hi @gcusello , 

thank you for answering. Index stopped working when I had problems with MongoDB so I tought it was correlated.

What should I expect to find from inputs.conf? Sorry, I'm a beginner.

ciao,

Mattia