Deployment Architecture

splunk remove data after indexing

TISKAR
Builder

Hello

I have a big problem with the addition of data.
initially given 9 million are added. but after I find Splunk removes one million data.

Can you help please.

Tags (1)
0 Karma

TISKAR
Builder

Hello every one,

Thank you for your participation,
I found the problem , but I did not understand why the problem is that I have more time , I added from the fields of these dates in TIMESTAMP_FIELDS after I left That a single TIMESTAMP_FIELDS field , I'll find out why . and what is the criteria on TIMESTAMP_FIELDS ??

0 Karma

woodcock
Esteemed Legend

What do you see with this search:

 | tstats count valuse(sourcetype) where index=* OR index=_*
0 Karma

TISKAR
Builder

this command gives 0 event and null values

0 Karma

haley_swarnapat
Path Finder

It's a typo, he was mentioning this:
| tstats count values(sourcetype) where index=* OR index=_*

Just FYI, if your free space falls below 5GB, Splunk will stop indexing by default.
If this is the case, you might need to delete some temp files from your OS.
Or you can adjust this limit by going to : Settings -> System Settings -> General Settings -> Pause indexing if free disk space (in MB) falls below *

0 Karma

TISKAR
Builder

after searching , I think the problem comes from three files , I install Splunk entreprise in other computer problems remains with me these three files , I replaced these files by three other files of the same format and size but different data, me the data are added without problem.

Thank you all

0 Karma

woodcock
Esteemed Legend

You have tagged this splunk-enterprise but then you mention "splunk light" What are you using?

0 Karma

TISKAR
Builder

Now I use splubk Entrprise . (before I used splunk light I has not found The Problem)

0 Karma

woodcock
Esteemed Legend

Splunk is a FIFO system so if your index is set at a size of 80G and 10M events is roughly 1G, then the first (earliest) 10G will be frozen (purged) to make room for the last (latest) 10G.

0 Karma

TISKAR
Builder

Thank you for your reaction,
I create author index and , and I created a folder only contains three files, the Splunk began indexing but at some point it removes all that has indexed(COUNT EVENT=0) , knowing that I have not found this problem in Splunk light.

index characteristic:
range=ALL Time
The Max Size=500GB

index.conf:
[indexTest]
coldPath = $SPLUNK_DB/ffjj/colddb
enableDataIntegrityControl = 0
enableTsidxReduction = 0
homePath = $SPLUNK_DB/indexTest/db
maxTotalDataSizeMB = 512000
thawedPath = $SPLUNK_DB/indexTest/thaweddb
disabled = 0

input.conf:
[monitor:///data/splunk/test]
disabled = false
index = indexTest
sourcetype = LICENCIE

0 Karma

haley_swarnapat
Path Finder

Have you set your search time range to "All Time" ?

If it doesn't work, then go to Settings -> Indexes -> click on your index
1. Check your index How big is "Current Size" and "Max Size"?

2. How many "Event Count" are shown?
You might need to add more space if your index is running out of space

0 Karma

TISKAR
Builder

Thank you for you reaction,
range=ALL Time
The Max Size=500GB
The current Size=1MB (I find it also removes all events)
Event Count=0

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...