All Posts

Find Answers
Ask questions. Get answers. Find technical product solutions from passionate members of the Splunk community.

All Posts

I agreee with you on that, if your CPU, IOPS, and searches all seem steady. Some network appliances have default TCP session timeout, If forwarder/indexer sessions idle or ACKs delay just enough, th... See more...
I agreee with you on that, if your CPU, IOPS, and searches all seem steady. Some network appliances have default TCP session timeout, If forwarder/indexer sessions idle or ACKs delay just enough, the connection may be dropped, forcing re-establishment and buffering. Also network switches/routers might prune idle TCP flows, this affects forwarders that don’t constantly send. Regards, Prewin Splunk Enthusiast | Always happy to help! If this answer helped you, please consider marking it as the solution or giving a Karma. Thanks!
The strange thing is that the resource usage is quite equal all time from 09 to 17, with some "normal" CPU peak (i have to add some Indexers asap), also same number of searches, and quality of search... See more...
The strange thing is that the resource usage is quite equal all time from 09 to 17, with some "normal" CPU peak (i have to add some Indexers asap), also same number of searches, and quality of searches (none of them seems to create some loop or resources bottleneck!!!). I was also wondering if some Network Device makes some "refresh" (every 1 hour), maybe breaking the Indexers ACK responses 🤷‍ 🤷‍ 🤷‍ quite strange...
@verbal_666  Splunk doesn’t offer a built-in scheduler for bucket management tasks like rolling or retention. I would say focus on resource monitoring, and possibly scaling your indexer infrastruct... See more...
@verbal_666  Splunk doesn’t offer a built-in scheduler for bucket management tasks like rolling or retention. I would say focus on resource monitoring, and possibly scaling your indexer infrastructure, not on manipulating Splunk's internal maintenance timing. But you can consider below possible tuning, but not a recommended approach. -Tune max_peer_rep_load and max_peer_build_load in server.conf reduce these values to throttle replication -Adjust forwarder behavior by editing autoLBFrequency - reduces how often forwarders switch indexers, lowering channel creation rate #https://community.splunk.com/t5/Getting-Data-In/Why-did-ingestion-slow-way-down-after-I-added-thousands-of-new/m-p/465796 Regards, Prewin Splunk Enthusiast | Always happy to help! If this answer helped you, please consider marking it as the solution or giving a Karma. Thanks!
And what is you stats wants a _time group? index=* source=/my.log | bin span=5m _time | stats count by _time[,source] 00:00 5 00:05 (no records, skipped) 00:10 10 00:15 20 The result will be, ... See more...
And what is you stats wants a _time group? index=* source=/my.log | bin span=5m _time | stats count by _time[,source] 00:00 5 00:05 (no records, skipped) 00:10 10 00:15 20 The result will be, 00:00 5 00:10 10 00:15 20 The only way in this case is to use a timechart index=* source=/my.log | timechart span=5m count by source 00:00 5 00:05 0 00:10 10 00:15 20
Hi. During the day, some on my Indexers completely stops sending back the ACK, so many agents keep data in queue until there's the ACK and the flow restarts (in some cases also 15/20 minutes passes!... See more...
Hi. During the day, some on my Indexers completely stops sending back the ACK, so many agents keep data in queue until there's the ACK and the flow restarts (in some cases also 15/20 minutes passes!!!). Meanwhile, obviously, i have many delays of data and ACK errors. This happens at some hours, from 09:00 to 17:00, during very high data ingestion the issue is clear visible, during the other hours is trasparent, no issue (few data flowing and few users interaction). I'm wondering, maybe an Indexer internal task to manage indexes/buckets, to optimize system and manage retentions? If so, is this task "editable" to run "once per day only" (in night hours)? Thanks.
Hi livehybrid, checked our test and production environment. Neither of them have a lang-setting.
This should be accepted as solution. This workaround works, not sure why Splunk hasn't put this workaround in the known issues section in the docs.
did you able to find solution for this.
just use the <fields> element in your <table> to restrict what fields are shown in the table. All other fields are still available for drilldown with $row.x$ https://docs.splunk.com/Documentation/Sp... See more...
just use the <fields> element in your <table> to restrict what fields are shown in the table. All other fields are still available for drilldown with $row.x$ https://docs.splunk.com/Documentation/Splunk/latest/Viz/PanelreferenceforSimplifiedXML#table  
ご教授ください。 Linuxでsplunk(9.4.3)の構築を学習中です。 WebUIのフォワーダー管理でエージェントが表示されない状態です。 解決策をご存知ないでしょうか。 Deployment ServerはLicense Manager、Monitoring Serverと同居しています。(XX.XX.XX.10) ClientはHeavy Forwarderです。(XX.XX... See more...
ご教授ください。 Linuxでsplunk(9.4.3)の構築を学習中です。 WebUIのフォワーダー管理でエージェントが表示されない状態です。 解決策をご存知ないでしょうか。 Deployment ServerはLicense Manager、Monitoring Serverと同居しています。(XX.XX.XX.10) ClientはHeavy Forwarderです。(XX.XX.XX.8) Monitoring ConsoleではDSもClientもStatus=Upの状態です。 ・各confは下記のように設定 -------[Client(XX.XX.XX.8)]------- $ sudo cat /opt/splunk/etc/system/local/deploymentclient.conf [deployment-client] disabled = false clientName = HF1 [target-broker:deploymentServer] targetUri = XX.XX.XX.10:8089 $ sudo /opt/splunk/bin/splunk show deploy-poll WARNING: Server Certificate Hostname Validation is disabled. Please see server.conf/[sslConfig]/cliVerifyServerName for details. Your session is invalid. Please login. Splunk username: siv_admin Password: Deployment Server URI is set to "XX.XX.XX.10:8089". $ ------------- -------[DS(XX.XX.XX.10)]------- $ sudo cat /opt/splunk/etc/system/local/serverclass.conf [serverClass:HFs] whitelist.0 = host=heavy* whitelist.1 = XX.XX.*.* whitelist.2 = host=HF* whitelist.3 = clientName:HF* [serverClass:HFs:app:app_test] stateOnClient = enabled -------------   ・確認した事 -------[Client(XX.XX.XX.8)]------- #Client=>DS 疎通確認で応答がある $ curl -k https://XX.XX.XX.10:8089/services/server/info -u admin:XXXXX <?xml version="1.0" encoding="UTF-8"?> ・・・・・ <s:list> <s:item>indexer</s:item> <s:item>license_master</s:item> <s:item>license_manager</s:item> <s:item>deployment_server</s:item> <s:item>search_head</s:item> <s:item>kv_store</s:item> </s:list> ・・・・・ -------------- -------[DS(XX.XX.XX.10)]------- #DSにClient の設定はしていない #DS=>Client 疎通確認で応答がある $ sudo /opt/splunk/bin/splunk btool deploymentclient list --debug $ $ curl -k https://XX.XX.XX.8:8089/services/server/info -u admin:XXXXX <?xml version="1.0" encoding="UTF-8"?> ・・・・・ <s:key name="server_roles"> <s:list> <s:item>deployment_client</s:item> <s:item>search_peer</s:item> <s:item>kv_store</s:item> </s:list> </s:key> ・・・・・ --------------   その他気になる事があります。 ・キャッシュ(serverclass_cache.json)が生成されません。 ・テストで作成したAppの配布はできているように見える。 -------[Client(XX.XX.XX.8)]------- $ sudo ls -l /opt/splunk/var/run/HFs total 12 -rw-------. 1 splunk splunk 10240 Jul 22 23:23 app_test-1753173390.bundle -------------- ・splunkd.logでも疎通ができていると思われるログが出ている -------[DS(XX.XX.XX.10)]------- 07-23-2025 23:30:28.287 +0000 INFO PubSubSvr [2431 TcpChannelThread] - Subscribed: channel=deploymentServer/phoneHome/default/reply/heavy-1/HF1 connectionId=connection_XX.XX.XX.8_8089_heavy-1.internal.cloudapp.net_heavy-1_HF1 listener=0x7fbd68ccc400 --------------   よろしくお願いします。
Please can you provide some sample events to demonstrate your issue?
Check out this API reference for being able to use SignalFlow through the API: https://dev.splunk.com/observability/reference/api/signalflow/latest#endpoint-create-websocket-connection Also, if... See more...
Check out this API reference for being able to use SignalFlow through the API: https://dev.splunk.com/observability/reference/api/signalflow/latest#endpoint-create-websocket-connection Also, if you want to work within Splunk Cloud/Enterprise, you can use the Observability Cloud Infrastructure Monitoring TA which will allow you to  use the sim command in your spl and you can use SignalFlow there to get that metric. https://splunkbase.splunk.com/app/5247
Interestingly, when using tostring() before the sort it still identifies as a number?!   The only way I can make it sort as a string is using | sort str(fruit) as @richgalloway  mentioned.  ... See more...
Interestingly, when using tostring() before the sort it still identifies as a number?!   The only way I can make it sort as a string is using | sort str(fruit) as @richgalloway  mentioned.  Did this answer help you? If so, please consider: Adding karma to show it was useful Marking it as the solution if it resolved your issue Commenting if you need any clarification Your feedback encourages the volunteers in this community to continue contributing
hi @pedropiin  Is this the sort of thing you're looking for? Its a little complicated to setup due to the way the expression works in the <format> (cant reference other fields) so you need to create... See more...
hi @pedropiin  Is this the sort of thing you're looking for? Its a little complicated to setup due to the way the expression works in the <format> (cant reference other fields) so you need to create an mvfield containing the two values you want to compare, then use CSS to hide the other! Anyway - let me know what you think! Full example code <dashboard version="1.1"> <label>Colour smaller number cells</label> <description>App1 should be green if App1&lt;App2, App2 should be green if App2&lt;App1</description> <row> <panel> <html> <style> #tableCellColour table tbody td div.multivalue-subcell[data-mv-index="1"]{ display: none; } </style> </html> <table id="tableCellColour"> <search> <query>| makeresults count=10 | streamstats count AS row | eval "Hidden Avg Time App1" = (random() % 100) + 1 | eval "Hidden Avg Time App2" = (random() % 100) + 1 | eval "Avg Time App1" = mvappend('Hidden Avg Time App1', 'Hidden Avg Time App2') | eval "Avg Time App2" = mvappend('Hidden Avg Time App2', 'Hidden Avg Time App1') | fields - Hidden* | table _time *</query> <earliest>@d</earliest> <latest>now</latest> <sampleRatio>1</sampleRatio> </search> <option name="count">50</option> <option name="dataOverlayMode">none</option> <option name="drilldown">none</option> <option name="percentagesRow">false</option> <option name="refresh.display">progressbar</option> <option name="rowNumbers">false</option> <option name="totalsRow">false</option> <option name="wrap">true</option> <format type="color" field="Avg Time App1"> <colorPalette type="expression">case(mvindex(value,1) &gt; mvindex(value,0), "#1ce354", 1=1, "#de2121")</colorPalette> </format> <format type="color" field="Avg Time App2"> <colorPalette type="expression">case(mvindex(value,1) &gt; mvindex(value,0), "#1ce354", 1=1, "#de2121")</colorPalette> </format> </table> </panel> </row> </dashboard>  Did this answer help you? If so, please consider: Adding karma to show it was useful Marking it as the solution if it resolved your issue Commenting if you need any clarification Your feedback encourages the volunteers in this community to continue contributing
When I add  | eval type=typeof(fruit) to the query the results say the numbers are indeed numbers rather than strings.  That would explain the sort. When I use  | sort str(fruit) the results are... See more...
When I add  | eval type=typeof(fruit) to the query the results say the numbers are indeed numbers rather than strings.  That would explain the sort. When I use  | sort str(fruit) the results are in the expected lexicographical order. FWIW, the docs do say "Numeric data is sorted as you would expect for numbers and the sort order is specified as ascending or descending."
Further to my last reply, I've tested with the following which I think does what you need <form version="1.1"> <label>Testing</label> <row> <panel> <title>Support cases</title> <tab... See more...
Further to my last reply, I've tested with the following which I think does what you need <form version="1.1"> <label>Testing</label> <row> <panel> <title>Support cases</title> <table id="myTable"> <search> <query>index=_internal | head 3 | eval "Time (UTC)"=_time | eval "Support ID"="Testing" |eval _epoch_time=_time, _epoch_plus60=_time+60 | table "Support ID","Time (UTC)", _time, _epoch_time, _epoch_plus60</query> <earliest>-15m</earliest> <latest>now</latest> </search> <!-- Drilldown Configuration --> <!-- Enable row-level drilldown --> <option name="drilldown">row</option> <option name="refresh.display">progressbar</option> <drilldown> <link target="_blank">/app/search/search?q=search index=sony* sourcetype=sony_logs support_id="$click.value$"&amp;earliest=$row._epoch_time$&amp;latest=$row._epoch_plus60$</link> </drilldown> </table> </panel> </row> </form>  Did this answer help you? If so, please consider: Adding karma to show it was useful Marking it as the solution if it resolved your issue Commenting if you need any clarification Your feedback encourages the volunteers in this community to continue contributing
The Splunk documentation says that the order rule is lexicographic. I am trying to sort the following values: | makeresults | eval fruit="apple" | append [ | makeresults | eval fruit="Banana" ] ... See more...
The Splunk documentation says that the order rule is lexicographic. I am trying to sort the following values: | makeresults | eval fruit="apple" | append [ | makeresults | eval fruit="Banana" ] | append [ | makeresults | eval fruit="zebra" ] | append [ | makeresults | eval fruit="10" ] | append [ | makeresults | eval fruit="2" ] | append [ | makeresults | eval fruit="20" ] | append [ | makeresults | eval fruit="30" ] | append [ | makeresults | eval fruit="3" ] | append [ | makeresults | eval fruit="1" ] | append [ | makeresults | eval fruit="25" ] | append [ | makeresults | eval fruit="38" ] | table fruit | sort fruit The output I am getting is: 1, 2, 3, 10, 20, 25, 30, 38, Banana, apple, zebra I understand that Banana appears before apple because B<a. But what is up with string numerics? Shouldn't the order be: 1, 10, 2, 20, 25, 3, 30, 38, Banana, apple, zebra ?  Even the documentation says that between 10, 9, 70, 100 the sorted output should be 10, 100, 70, 90.  https://help.splunk.com/en/splunk-enterprise/search/spl-search-reference/9.2/search-commands/sort 
Hi @splunklearner  If you prefix those epoch fields with an underscore _ then it wont show up in the Table, but you can still reference it as a token.  Try the following: <table id="myTable"> <sea... See more...
Hi @splunklearner  If you prefix those epoch fields with an underscore _ then it wont show up in the Table, but you can still reference it as a token.  Try the following: <table id="myTable"> <search> <query>index=sony_summary sourcetype=stash |search <passed drop-down tokens> |sort 0 -"Time (UTC)" |eval _epoch_time=_time, _epoch_plus60=_time+60 (added this now) | table "Support ID","Time (UTC)", _time -------- , _epoch_time, _epoch_plus60</query> </search> <earliest>$time_range.earliest$</earliest> <latest>$time_range.latest$</latest> </search> <!-- Drilldown Configuration --> <!-- Enable row-level drilldown --> <option name="drilldown">row</option> <option name="refresh.display">progressbar</option> <drilldown> <link target="_blank">/app/search/search?q=search index=sony* sourcetype=sony_logs support_id="$click.value$"&amp;earliest=$row._epoch_time$&amp;latest=$row._epoch_plus60</link> </drilldown> ... etc...  Did this answer help you? If so, please consider: Adding karma to show it was useful Marking it as the solution if it resolved your issue Commenting if you need any clarification Your feedback encourages the volunteers in this community to continue contributing
This splunk thread was the answer: https://community.splunk.com/t5/Knowledge-Management/Why-is-KV-Store-initialization-failing-on-one-of-our-add-on-to/m-p/435188#M3872
Hi @ITWhisperer , thanks for your reply I didn't know about this multivalue strategy and it seems very useful. But how would I use, e.g., a "foreach *" structure to implement it between two differ... See more...
Hi @ITWhisperer , thanks for your reply I didn't know about this multivalue strategy and it seems very useful. But how would I use, e.g., a "foreach *" structure to implement it between two different fields? Because the only thing I could think of in the lines of using multivalue is something of the sort: for each field   if field should be compared     if field.value > nextField.value         value = mvappend(value, "RED") But I'm still confused regarding how to use this logic to compare two different fields. Thanks in advance,  Pedro