Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
×
Are you a member of the Splunk Community?
Sign in or Register with your Splunk account to get your questions answered, access valuable resources and connect with experts!
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
- About athorat
athorat
Communicator
Member since:
04-29-2015
06-24-2024
Community Statistics
| Posts | 180 |
| Solutions | 1 |
| Karma Given | 14 |
| Karma Received | 6 |
| Member Since | 04-29-2015 |
Activity Feed
- Posted SmartStore Retention not deleting data on Getting Data In. 10-03-2023 12:30 AM
- Posted Re: Does it make sense to use SmartStore for all data except hot and warm? on Getting Data In. 10-03-2023 12:25 AM
- Posted How to track scheduled search which are using "All Time" time window? on Knowledge Management. 06-29-2023 06:36 PM
- Posted Re: Splunk Indexer services admin bundle errors on Deployment Architecture. 08-13-2021 08:29 PM
- Got Karma for NOT Like function. 05-28-2021 12:00 AM
- Karma Re: how to find Number of files failed to ingest for a specific Index for richgalloway. 06-05-2020 12:50 AM
- Karma Re: Search head's authentication credentials rejected by peer. I've re-added peers 4 times and this keeps happening! for gozulin. 06-05-2020 12:48 AM
- Karma Re: How to find the oldest log indexed in the indexer instances ? for somesoni2. 06-05-2020 12:48 AM
- Karma Re: How to find the oldest log indexed in the indexer instances ? for woodcock. 06-05-2020 12:48 AM
- Got Karma for Re: Search head's authentication credentials rejected by peer. I've re-added peers 4 times and this keeps happening!. 06-05-2020 12:48 AM
- Karma Re: Applying Quarantine .... Removing quarantine for mmensch. 06-05-2020 12:47 AM
- Karma Re: Why is Splunk not starting on Linux with permission denied errors after upgrading to 6.2.0? for fabioportes. 06-05-2020 12:47 AM
- Karma Re: How to filter data in a search using the AND condition? for Richfez. 06-05-2020 12:47 AM
- Karma Re: How to calculate the difference between the counts from multiple searches? for somesoni2. 06-05-2020 12:47 AM
- Got Karma for How to change the ownership of dashboards to a specific user in the system?. 06-05-2020 12:47 AM
- Got Karma for How to change the ownership of dashboards to a specific user in the system?. 06-05-2020 12:47 AM
- Got Karma for Why is my inputlookup search not pulling a field from a CSV file needed to populate a timechart?. 06-05-2020 12:47 AM
- Got Karma for NOT Like function. 06-05-2020 12:47 AM
- Karma Re: indexing congestion consistenly happening for yannK. 06-05-2020 12:46 AM
- Karma Re: Merging two chart together for MuS. 06-05-2020 12:46 AM
Topics I've Started
| Subject | Karma | Author | Latest Post |
|---|---|---|---|
| 0 | |||
| 0 | |||
| 0 | |||
| 0 | |||
| 0 | |||
| 0 | |||
| 0 | |||
| 0 | |||
| 0 | |||
| 0 |
10-03-2023
12:30 AM
We recently move to S2 and our initial retention was set to 6 months. A month after the migration we decided to reduced the retention to 3 months but did not see any reduction in storage in s3. Support found out that versioning was enabled in AWS by the PS engineer during the migration and that caused this issue. Updated this in indexes.conf remote.s3.supports_versioning =false" Now the new data which is rolled over is deleted but we still have old cruft remaining in s3 which is costing us heavily. Support wants us to delete the data manually by running commands from CLI. Isnt there a better way in doing this? Does AWS lifecycle rules work only for old data which is still lying there? What are the ways to get rid of this old data apart from removing it manually.
... View more
10-03-2023
12:25 AM
@woodcock We recently move to S2 and our initital retention was set to 6 months. A month after the migration we decided to reduced the retention to 3 months but did not see any reduction in storage in s3. Support found out that versioning was enabled in AWS by the PS engineer during the migration and that caused this issue. Now the new data which is rolled over is deleted but we still have old cruft remaining in s3 which is costing us heavily. Support wants us to delete the data manually by running commands from CLI. Isnt there a better way in doing this? Does AWS lifecycle rules work only for old data which is still lying there? What are the ways to get rid of this old data apart from removing it manually.
... View more
06-29-2023
06:36 PM
We have a few users scheduling searches using "all time", time frame.
How can I track those knowledge objets and delete / Pause them?
... View more
Labels
- Labels:
-
smartstore
08-13-2021
08:29 PM
How was this resolved? We are seeing this on the CM a lot and the search performance and also the indexer performance is affected. A support ticket since a month yet no resolutiom Problem replicating config (bundle) to search peer ' <Peer-Name>:8089 ', Upload bundle="/opt/splunk/var/run/Cluster-Master.XXX.XXX-1628153272.bundle" to peer name=<Peer-name>: uri=https://<Peer-name>:8089 failed; http_status=400 http_description="BaseException".
... View more
@woodcock Thanks a lot this helped to troubleshoot one of internal issue.
... View more
03-30-2019
12:15 PM
Want to grant delete permissions for a non admin account for specific prod-test-* indexes
Created a new role and mapped it to an AD group but does not seem to like it. Users still get , Error in 'delete' command: You have insufficient privileges to delete events.
[role_test_access]
srchIndexesAllowed = _audit
deleteIndexesAllowed = prod-test*
delete_by_keyword= enabled
Any suggestions highly appreciated.
... View more
- Tags:
- permissions
- roles
01-03-2019
04:34 PM
Thanks for replying on this @richgalloway,aplogize for the delayed reply. yes we are trying to find the files which never reached splunk.
1 by permissions issue
2 OR by files not matching the whitelist pattern or Unknown reasons.
We have have got the count of files per index which are being read/indexed by Splunk UF
| tstats dc(source) WHERE host=10a*pd-* OR host=ew1a-* OR host=dub01pd-* OR host=uw2*-* OR host=ue1-* index=prod-online* by index
Failed attempt Below:
Now we want to list the number of files which are errored out / Not read by the Splunk UF. So for this the same hosts are being used to filter but how do we get that by Index name and have a bar chart comparing the above query?
index=_internal sourcetype=splunkd splunk_server=usw* host=10a*pd-* OR host=ew1a-* OR host=dub01pd-* OR host=uw2*-* OR host=ue1-* log_level=ERROR
| rex field=message "((?.*))"| stats dc(message) by host | sort – message
... View more
12-31-2018
01:07 PM
How do I find the Number of files failed to ingest for a specific Index.
Trying to compare files ingested vs files failed to ingest for a specific Index in Splunk.
... View more
09-20-2017
11:37 PM
When I use the Data rebalance option from the Cluster master, it shows that it is complete within 10-20 mins
Do not see any data moved from the old indexers to the new indexers which are added in the indexer cluster
Also tried to use :
splunk rebalance cluster-data -action start
and has the same result.
in the UI >> Under the Data Rebalance option>> when I click on "All Indexes" >> it shows only _(indexes)
How do I add all the indexes, pretty sure even the command line is replicating only _internal indexes.
... View more
08-14-2017
04:35 PM
Thank You so much for your inputs. i will try to write the transforms in the mean time and let you know if that works.
... View more
08-14-2017
04:33 PM
index=abc sourcetype=Vserver k=set-collation v.collation = "charset=0x0 root (LROOT)"
Event
{ [-]
k: set-collation
pid: 5076
req: WZ******
sess: 7*********-1:1
sev: info
site: Default
tid: ***
ts: 2017-08-14T16:10:09.317
user: ***.ban\jac***
v: { [-]
collation: charset=0x0 root (LROOT)
column: [sqlserver].[none]
}
}
@somesoni2 @DalJeanis @mdsnmss this how the event looks like.
... View more
08-14-2017
11:59 AM
Hi,
I have a query which filters data in the Splunk search, I want to send the data returned from this query to null queue
I understand that props and transforms needs to be configured, but how about using this filter criteria to be used in the transforms?
index=abc sourcetype=Vserver k=set-collation v.collation = "charset=0x0 root (LROOT)"
All the events for this query needs to be routed to null queue
... View more
08-11-2017
12:25 PM
@somesoni2 nice explanation
1>whats happens if we have set retention period to 1 years in the case (1) maxHotSpanSecs is reached (default 90 days), 2) maxDataSize is reached 2) Splunk got restarted.).
2>after 90 days the hot bucket rolls to warm and stays there ?
3>what if the indexer is restarted more than 300 times? will it still hold the data for 1 year
... View more
07-31-2017
10:01 AM
Events are no breaking by the below used settings in props.conf, all the events are indexed as one event.
Even if I uncomment BREAK_ONLY_BEFORE with a couple of variations , no go
Props.conf
SHOULD_LINEMERGE = false
BREAK_ONLY_BEFORE=[\d+:\d+:\d+]
MAX_TIMESTAMP_LOOKAHEAD=22
LINE_BREAKER = ([\r\n]+)
CHARSET=UTF-8
EVENT_BREAKER_ENABLE = true
EVENT_BREAKER = ([\r\n]+)
disabled=false
[20:53:01] : [BBGen] essbase.yml -> Warning: no bookmarks generated for stanza 1 in configs/essbase.yml.
[20:53:01] : [BBGen] essbase.cfg
[20:53:01] : [BBGen] exasolution.yml -> exasolution.cfg
[20:53:01] : [BBGen] exceldirect.yml -> exceldirect.cfg
[20:53:01] : [BBGen] genericodbc-derived.yml -> genericodbc-derived.cfg
[20:53:01] : [BBGen] genericodbc.yml -> genericodbc.cfg
[20:53:01] : [BBGen] googlecloudsql.yml -> googlecloudsql.cfg
[20:53:01] : [BBGen] greenplum.yml -> greenplum.cfg
[20:53:01] : [BBGen] hadoophive.yml -> hadoophive.cfg
[20:53:01] : [BBGen] hadoophive_AWS.yml -> hadoophive_AWS.cfg
[20:53:01] : [BBGen] hadoophive_hortonworks.yml -> hadoophive_hortonworks.cfg
[20:53:01] : [BBGen] hadoophive_mapr.yml -> hadoophive_mapr.cfg
[20:53:01] : [BBGen] hana.yml -> hana.cfg
[20:53:01] : [BBGen] impala.yml -> impala.cfg
[20:53:01] : [BBGen] jet.yml -> Warning: no bookmarks generated for stanza 1 in configs/jet.yml.
[20:53:01] : [BBGen] jet.cfg
[20:53:01] : [BBGen] kognitio.yml -> kognitio.cfg
[20:53:01] : [BBGen] maps.yml -> maps.cfg
[20:53:01] : [BBGen] marklogic.yml -> Warning: no bookmarks generated for stanza 1 in configs/marklogic.yml.
[20:53:01] : [BBGen] marklogic.cfg
[20:53:01] : [BBGen] mdx.yml -> Warning: no bookmarks generated for stanza 1 in configs/mdx.yml.
[20:53:01] : [BBGen] mdx.cfg
[20:53:01] : [BBGen] memsql.yml -> memsql.cfg
[20:53:01] : [BBGen] msolap.yml -> Warning: no bookmarks generated for stanza 1 in configs/msolap.yml.
[20:53:01] : [BBGen] msolap.cfg
[20:53:01] : [BBGen] mysql.yml -> mysql.cfg
... View more
10-21-2016
03:14 PM
@somesoni2
Thanks for the reply, some how its still now working on this one
21/14:39:43.571 (1ea8/1ecc/1a54) {"BlobManServer"} Deletion of files for blob c37a18fb-8e6c-4994-8cbd-e21c43b9af93 deferred due to 1 additional references
21/14:39:43.571 (1ea8/1ecc/1a54) {"BlobManServer"} Process 19:REL_BLOB_MESSAGE(MsgId=136222) 1/43 released
21/14:39:43.571 (1ea8/1ecc/1a54) {"BlobManServer"} Process 19:ADD_BLOB_REF_MSG(c37a18fb-8e6c-4994-8cbd-e21c43b9af93, size=18544, MsgId=136224)
21/14:39:43.571 (1ea8/1ecc/1a54) {"BlobManServer"} Deletion of files for blob c37a18fb-8e6c-4994-8cbd-e21c43b9af93 deferred due to 1 additional references
21/14:39:43.571 (1ea8/1ecc/1a54) {"BlobManServer"} Process 19:REL_BLOB_MESSAGE(MsgId=136223) 1/43 released
21/14:39:43.571 (180c/1bf0/1a54) {"XmlParser" 0x13178f8} BlobControl: Move ownership started - from me(MsgID=136224) to recieving-msg (forwd=0)
21/14:39:43.571 (1ea8/05f4/1a54) {"XmlParser" 0x2e87748} Got Blob Control Block(MsgId=136224, CompId=19)
21/14:39:43.571 (1ea8/05f4/1a54) Dump-Rsp: Message 'ViewMessageResponse' (1548 byte) from 19(tsl) ---> 15(http) [s167809-t992055]
21/14:39:43.571 (1ea8/05f4/1a54) {"XmlParser" 0x2e87748} BlobControl: Move ownership started - from me(MsgID=0) to recieving-msg (forwd=1)
21/14:39:43.571 (1ea8/05f4/1a54) Connection has been disconnected by target 19(tsl). (State=0/4)
21/14:39:43.571 (1bb0/0704/1a54) {"XmlParser" 0x1c6727c} Got Blob Control Block(MsgId=136224, CompId=19)
21/14:39:43.571 (1ea8/1ecc/1a54) {"BlobManServer"} Blob file KofaxMerlinBlobFile_c37a18fb-8e6c-4994-8cbd-e21c43b9af93.TIF deleted
21/14:39:43.571 (1ea8/1ecc/1a54) {"BlobManServer"} Deleted 1 files for blob c37a18fb-8e6c-4994-
... View more
10-21-2016
12:46 PM
Hi
somehow the date is not being picked up properly by splunk.
the props.conf has %d/%H:%M:%S.3N but its not working..
Any thoughts on this?
[ SOURCETYPE]
SHOULD_LINEMERGE=false
NO_BINARY_CHECK=true
TIME_FORMAT=%D/%H:%M:%S.3N
TIME_PREFIX=^
MAX_TIMESTAMP_LOOKAHEAD=25
28/07:50:42.064 (0a98/0f38/8bea) Dump-Req: Message 'poll' (678 byte) from 1(albin) ---> 6(email) [s593411-t661514]
28/07:50:42.064 (07f4/067c/8bea) Email/MbParametersGet Parameters >IMAP< >imap.tent.trt.csaa.pri:143< Tls >NO<
28/07:50:42.064 (07f4/067c/8bea) Email/MbParametersGet called, MaxRequested >1< ConfirmTimeout >600<
28/07:50:42.064 (07f4/067c/8bea) Email/MbParametersGet bKeepOnServer >1< Folders: >KofaxProcessed< >Rejected<
28/07:50:42.064 (07f4/067c/8bea) Email/MbParametersGet ImapMode >MultiInstance<, TestMode >0<
28/07:50:42.189 (07f4/14c0/8be9) Email/fnMbPoll returns
28/07:50:42.189 (0a98/1f9c/8be9) Dump-Rsp: Message 'poll-res' (291 byte) from 6(email) ---> 1(albin) [s593410-t661513]
28/07:50:42.189 (0a98/1f9c/8be9) Connection has been disconnected by target 6(email). (State=0/4)
28/07:50:42.220 (0a98/19f0/8beb) Dump-Req: Message 'poll' (634 byte) from 1(albin) ---> 6(email) [s593412-t661515]
28/07:50:42.220 (0a98/1328/8bec) Dump-Req: Message 'poll' (637 byte) from 1(albin) ---> 6(email) [s593413-t661516]
28/07:50:42.220 (07f4/154c/8beb) Email/MbParametersGet Parameters >IMAP< >imap.tent.trt.csaa.pri:143< Tls >NO<
28/07:50:42.220 (07f4/154c/8beb) Email/MbParametersGet called, MaxRequested >1< ConfirmTimeout >600<
28/07:50:42.220 (07f4/154c/8beb) Email/MbParametersGet bKeepOnServer >0< Folders: >Processed< >Rejected<
28/07:50:42.220 (07f4/154c/8beb) Email/MbParametersGet ImapMode >MultiInstance<, TestMode >0<
28/07:50:42.220 (07f4/18b4/8bec) Email/MbParametersGet Parameters >IMAP< >imap.tent.trt.csaa.pri:143< Tls >NO<
28/07:50:42.220 (07f4/18b4/8bec) Email/MbParametersGet called, MaxRequested >1< ConfirmTimeout >600<
28/07:50:42.220 (07f4/18b4/8bec) Email/MbParametersGet bKeepOnServer >0< Folders: >Processed< >Rejected<
28/07:50:42.220 (07f4/18b4/8bec) Email/MbParametersGet ImapMode >MultiInstance<, TestMode >0<
28/07:50:42.298 (07f4/067c/8bea) Email/fnMbPoll returns
28/07:50:42.298 (0a98/0f38/8bea) Dump-Rsp: Message 'poll-res' (291 byte) from 6(email) ---> 1(albin) [s593411-t661514]
28/07:50:42.298 (0a98/0f38/8bea) Connection has been disconnected by target 6(email). (State=0/4)
28/07:50:42.439 (07f4/154c/8beb) Email/fnMbPoll returns
28/07:50:42.439 (0a98/19f0/8beb) Dump-Rsp: Message 'poll-res' (291 byte) from 6(email) ---> 1(albin) [s593412-t661515]
28/07:50:42.439 (0a98/19f0/8beb) Connection has been disconnected by target 6(email). (State=0/4)
28/07:50:42.470 (07f4/18b4/8bec) Email/fnMbPoll returns
28/07:50:42.470 (0a98/1328/8bec) Dump-Rsp: Message 'poll-res' (291 byte) from 6(email) ---> 1(albin) [s593413-t661516]
28/07:50:42.470 (0a98/1328/8bec) Connection has been disconnected by target 6(email). (State=0/4)
28/07:50:42.704 (0a98/101c/8bed) Dump-Req: Message 'poll' (679 byte) from 1(albin) ---> 6(email) [s593414-t661517]
28/07:50:42.704 (07f4/08b4/8bed) Email/MbParametersGet Parameters >IMAP< >imap.tent.trt.csaa.pri:143< Tls >NO<
28/07:50:42.704 (07f4/08b4/8bed) Email/MbParametersGet called, MaxRequested >1< ConfirmTimeout >600<
28/07:50:42.704 (07f4/08b4/8bed) Email/MbParametersGet bKeepOnServer >1< Folders: >KofaxProcessed< >Rejected<
28/07:50:42.704 (07f4/08b4/8bed) Email/MbParametersGet ImapMode >MultiInstance<, TestMode >0<
28/07:50:42.735 (0a98/186c/8bee) Dump-Req: Message 'poll' (636 byte) from 1(albin) ---> 6(email) [s593415-t661518]
28/07:50:42.735 (07f4/07c0/8bee) Email/MbParametersGet Parameters >IMAP< >imap.tent.trt.csaa.pri:143< Tls >NO<
... View more
09-06-2016
11:13 AM
Lookuptable = C360_USERS.csv
Fields: USERID,EMPLOYEEID,AVAYAID,FIRSTNAME,LASTNAME,LOCATIONNAME,JOBCODE,JOBTITLE
I want to get all the results from the lookuptable matching userGID from the query. How do I join these two fields userGID and USERID and get all the information in a table format?
index=MRM | eval array=split(_raw,"*"), userGID=mvindex(array,9)|lookup C360_USER ......
... View more
08-22-2016
06:48 PM
@inventsekar
its from an app called DMC installed on the cluster master.
ours is a disributed environment.
... View more
08-20-2016
11:51 PM
We started seeing .dat files for all the indexes since 4th of august at the following path: /opt/splunk/var/lib/splunk
This is where the cold to frozen is pointed to.
There are no changes or upgrades on the server. Is anything wrong with the configurations?
... View more
08-20-2016
11:38 PM
@Lucas K @mmodestino_splunk added a new image
... View more
08-20-2016
11:37 PM
@Anonymous K @mmodestino_splunk
Thanks for the reply. so as I understand
1> so even if the data in the buckets is older than 365 days if the data in that same bucket is new it wont rioll the bucket to frozen, is this correct?
2> like np_aap we have AAP_PROD index for prod index and it has very heavy volumes of data(license usage report says avg of 150gb, but this above images shows it only adding 20gb per index, we have 4 indexers)
this shows data since 2002 / 365 (see image)
for AAP_PROD index I see data from march 9th till today. March 9th was the day when we built this new clustered infrastructure.
Freeze Async Message in Internal Log looks like this, dont know what to search for ...
08-20-2016 23:32:32.228 -0700 INFO StreamedSearch - Streamed search connection terminated: search_id=remote_p01apl.ent.com_1471761143.19590_12B16A82-0C9F-415D-A8A8-AEEB96B9AA2B, server=p01apl.ent.com, active_searches=6, elapsedTime=8.753, search='litsearch index=_internal idex=aap_prod "Freeze Async" | fields keepcolorder=t "*" "_bkt" "_cd" "_si" "host" "index" "linecount" "source" "sourcetype" "splunk_server" | remotetl nb=300 et=2147483647.000000 lt=0.000000 remove=true max_count=1000 max_prefetch=100', savedsearch_name=""
Thanks again for looking into this.
... View more
08-19-2016
07:38 PM
We have this config set up in indexes.conf, but still the data seems to be present even after 365 days.. anything which can be checked?
[nmon]
homePath = volume:hot/nmon/db
coldPath = volume:cold/nmon/colddb
thawedPath = $SPLUNK_DB/nmon/thaweddb
coldToFrozenDir = $SPLUNK_DB/nmon/frozendb
repFactor = auto
frozenTimePeriodInSecs = 31536000
rotatePeriodInSecs = 60
Thanks.
... View more
08-15-2016
01:54 PM
Thanks @sundareshr
it seems it assigned the proper values but the searchType shows only values for "search"
if I Filter data by SearchType(phoneNumber), SearchString field disappears.
Thanks again for looking into this.
... View more
