topic Re: Why is the new index not searchable? in Deployment Architecture

Why is the new index not searchable?

AlteUnke — Mon, 13 Aug 2018 12:08:23 GMT

Hi everyone,

I'm new to Splunk and this is the first Index I created, so hopefully this Question ain't to nooby 😉

This is my inputs.conf:

[monitor:///var/log/app/retry.log]
disabled=false
sourcetype=log4j
index=retry
multiline_event_extra_waittime = true

indexes.conf:

[retry]
homePath=$SPLUNK_DB/retry/db
coldPath=$SPLUNK_DB/retry/colddb
thawedPath=$SPLUNK_DB/retry/thaweddb
repFactor=autor        
maxDataSize=auto

Cluster Bundle Status:

master                        
         cluster_status=None                                                                         
         active_bundle             
                checksum=2924BEA962D9C72179B8CF4D03846EAB 
                timestamp=1533281547 (in localtime=Fri Aug  3 09:32:27 2018) 
         latest_bundle                                      
                checksum=2924BEA962D9C72179B8CF4D03846EAB                                                          
                timestamp=1533281547 (in localtime=Fri Aug  3 09:32:27 2018)                                      
         last_validated_bundle                                                                       
                checksum=2924BEA962D9C72179B8CF4D03846EAB                                                         
                last_validation_succeeded=1                                       
                timestamp=1533281547 (in localtime=Fri Aug  3 09:32:27 2018)                    
         last_check_restart_bundle   
                last_check_restart_result=restart not required  
                checksum=    
                timestamp=0 (in localtime=Thu Jan  1 01:00:00 1970)

 splunkidx2    3F5EEC11-8718-4C0D-AEF7-0F54DABB1D01    default                   
         active_bundle=2924BEA962D9C72179B8CF4D03846EAB                                               
         latest_bundle=2924BEA962D9C72179B8CF4D03846EAB                 
         last_validated_bundle=2924BEA962D9C72179B8CF4D03846EAB                                          
         last_bundle_validation_status=success                                     
         restart_required_apply_bundle=0        
         status=Up     

 splunkidx3    79FD9BAC-9F72-46CB-A043-EDCA31DE8EB7    default                           
         active_bundle=2924BEA962D9C72179B8CF4D03846EAB        
         latest_bundle=2924BEA962D9C72179B8CF4D03846EAB 
         last_validated_bundle=2924BEA962D9C72179B8CF4D03846EAB                   
         last_bundle_validation_status=success    
         restart_required_apply_bundle=0  
         status=Up

 splunkidx1    D2077BB4-988A-46F2-BB00-E261EBF94BC9    default               
         active_bundle=2924BEA962D9C72179B8CF4D03846EAB    
         latest_bundle=2924BEA962D9C72179B8CF4D03846EAB
         last_validated_bundle=2924BEA962D9C72179B8CF4D03846EAB                                               
         last_bundle_validation_status=success                     
         restart_required_apply_bundle=0                                                         
         status=Up

I can see the new "retry" Index in Splunk and add it to roles. But I can't search for it, or find events when search for "index=retry".
But I can see the rawdata/db on the Indexers, so Data is here.
Any Idea what I could have missed?

Thanks in advance!

Re: Why is the new index not searchable?

sudosplunk — Mon, 13 Aug 2018 13:58:31 GMT

Hello, did you try expanding the time range for the results. Try searching for "All Time" (in time range picker) and see if any data shows up.

Re: Why is the new index not searchable?

renjith_nair — Mon, 13 Aug 2018 16:19:28 GMT

Check the roles assigned to the new index if the current role you are using is allowed to see the data

Re: Why is the new index not searchable?

Steve_G_ — Mon, 13 Aug 2018 17:28:21 GMT

Also, in your index definition, it should be "repFactor=auto", not "repFactor=autor"

Re: Why is the new index not searchable?

AlteUnke — Tue, 14 Aug 2018 06:50:57 GMT

Uh, thanks for that! Didn't see that!

Re: Why is the new index not searchable?

AlteUnke — Tue, 14 Aug 2018 06:54:06 GMT

Wow. I was about to answer something like "If it was that easy...". But that's it. Can you tell me what the difference between "All-time (real time)" and "All time" is? I can see the logs only in "All time", which might be the reason why I couldn't see the obvious...

But thanks for the tip!

Re: Why is the new index not searchable?

AlteUnke — Tue, 14 Aug 2018 07:05:41 GMT

I believe my misunderstanding was, that I used logs which all had the date "03.08.2018". I just copy/pasted them, thinking splunk would log them under "last 7 days" depending on the time the log was added and not the date which is written in the log-entry.

Re: Why is the new index not searchable?

bpadmanbhachari — Tue, 14 Aug 2018 07:25:37 GMT

One thing i observed in indexes.conf is " repFactor = autor " which is invalid.

Can you change to "repFactor = auto" and try to push bundles again to peers.

http://docs.splunk.com/Documentation/Splunk/7.1.2/Indexer/Configurethepeerindexes#1._Edit_indexes.conf

Re: Why is the new index not searchable?

sudosplunk — Tue, 14 Aug 2018 13:28:43 GMT

Real-time searches scan events as the events arrive for indexing. When you kick off a real-time search, Splunk software scans the incoming events. The scan looks for events that contain index-time fields that indicate the event could be a match for your search.

"All-time (real-time)" search will continue running until you or another user stops the search or deletes the search job whereas "All-time" search stops once it has returned all the events matching the search criteria.

Re: Why is the new index not searchable?

sudosplunk — Tue, 14 Aug 2018 13:35:42 GMT

Splunk's default timestamp recognition process extracts timestamps found in the events unless instructed otherwise.