All Posts

Find Answers
Ask questions. Get answers. Find technical product solutions from passionate members of the Splunk community.

All Posts

@verbal_666  Splunk doesn’t offer a built-in scheduler for bucket management tasks like rolling or retention. I would say focus on resource monitoring, and possibly scaling your indexer infrastruct... See more...
@verbal_666  Splunk doesn’t offer a built-in scheduler for bucket management tasks like rolling or retention. I would say focus on resource monitoring, and possibly scaling your indexer infrastructure, not on manipulating Splunk's internal maintenance timing. But you can consider below possible tuning, but not a recommended approach. -Tune max_peer_rep_load and max_peer_build_load in server.conf reduce these values to throttle replication -Adjust forwarder behavior by editing autoLBFrequency - reduces how often forwarders switch indexers, lowering channel creation rate #https://community.splunk.com/t5/Getting-Data-In/Why-did-ingestion-slow-way-down-after-I-added-thousands-of-new/m-p/465796 Regards, Prewin Splunk Enthusiast | Always happy to help! If this answer helped you, please consider marking it as the solution or giving a Karma. Thanks!
And what is you stats wants a _time group? index=* source=/my.log | bin span=5m _time | stats count by _time[,source] 00:00 5 00:05 (no records, skipped) 00:10 10 00:15 20 The result will be, ... See more...
And what is you stats wants a _time group? index=* source=/my.log | bin span=5m _time | stats count by _time[,source] 00:00 5 00:05 (no records, skipped) 00:10 10 00:15 20 The result will be, 00:00 5 00:10 10 00:15 20 The only way in this case is to use a timechart index=* source=/my.log | timechart span=5m count by source 00:00 5 00:05 0 00:10 10 00:15 20
Hi. During the day, some on my Indexers completely stops sending back the ACK, so many agents keep data in queue until there's the ACK and the flow restarts (in some cases also 15/20 minutes passes!... See more...
Hi. During the day, some on my Indexers completely stops sending back the ACK, so many agents keep data in queue until there's the ACK and the flow restarts (in some cases also 15/20 minutes passes!!!). Meanwhile, obviously, i have many delays of data and ACK errors. This happens at some hours, from 09:00 to 17:00, during very high data ingestion the issue is clear visible, during the other hours is trasparent, no issue (few data flowing and few users interaction). I'm wondering, maybe an Indexer internal task to manage indexes/buckets, to optimize system and manage retentions? If so, is this task "editable" to run "once per day only" (in night hours)? Thanks.
Hi livehybrid, checked our test and production environment. Neither of them have a lang-setting.
This should be accepted as solution. This workaround works, not sure why Splunk hasn't put this workaround in the known issues section in the docs.
did you able to find solution for this.
just use the <fields> element in your <table> to restrict what fields are shown in the table. All other fields are still available for drilldown with $row.x$ https://docs.splunk.com/Documentation/Sp... See more...
just use the <fields> element in your <table> to restrict what fields are shown in the table. All other fields are still available for drilldown with $row.x$ https://docs.splunk.com/Documentation/Splunk/latest/Viz/PanelreferenceforSimplifiedXML#table  
ご教授ください。 Linuxでsplunk(9.4.3)の構築を学習中です。 WebUIのフォワーダー管理でエージェントが表示されない状態です。 解決策をご存知ないでしょうか。 Deployment ServerはLicense Manager、Monitoring Serverと同居しています。(XX.XX.XX.10) ClientはHeavy Forwarderです。(XX.XX... See more...
ご教授ください。 Linuxでsplunk(9.4.3)の構築を学習中です。 WebUIのフォワーダー管理でエージェントが表示されない状態です。 解決策をご存知ないでしょうか。 Deployment ServerはLicense Manager、Monitoring Serverと同居しています。(XX.XX.XX.10) ClientはHeavy Forwarderです。(XX.XX.XX.8) Monitoring ConsoleではDSもClientもStatus=Upの状態です。 ・各confは下記のように設定 -------[Client(XX.XX.XX.8)]------- $ sudo cat /opt/splunk/etc/system/local/deploymentclient.conf [deployment-client] disabled = false clientName = HF1 [target-broker:deploymentServer] targetUri = XX.XX.XX.10:8089 $ sudo /opt/splunk/bin/splunk show deploy-poll WARNING: Server Certificate Hostname Validation is disabled. Please see server.conf/[sslConfig]/cliVerifyServerName for details. Your session is invalid. Please login. Splunk username: siv_admin Password: Deployment Server URI is set to "XX.XX.XX.10:8089". $ ------------- -------[DS(XX.XX.XX.10)]------- $ sudo cat /opt/splunk/etc/system/local/serverclass.conf [serverClass:HFs] whitelist.0 = host=heavy* whitelist.1 = XX.XX.*.* whitelist.2 = host=HF* whitelist.3 = clientName:HF* [serverClass:HFs:app:app_test] stateOnClient = enabled -------------   ・確認した事 -------[Client(XX.XX.XX.8)]------- #Client=>DS 疎通確認で応答がある $ curl -k https://XX.XX.XX.10:8089/services/server/info -u admin:XXXXX <?xml version="1.0" encoding="UTF-8"?> ・・・・・ <s:list> <s:item>indexer</s:item> <s:item>license_master</s:item> <s:item>license_manager</s:item> <s:item>deployment_server</s:item> <s:item>search_head</s:item> <s:item>kv_store</s:item> </s:list> ・・・・・ -------------- -------[DS(XX.XX.XX.10)]------- #DSにClient の設定はしていない #DS=>Client 疎通確認で応答がある $ sudo /opt/splunk/bin/splunk btool deploymentclient list --debug $ $ curl -k https://XX.XX.XX.8:8089/services/server/info -u admin:XXXXX <?xml version="1.0" encoding="UTF-8"?> ・・・・・ <s:key name="server_roles"> <s:list> <s:item>deployment_client</s:item> <s:item>search_peer</s:item> <s:item>kv_store</s:item> </s:list> </s:key> ・・・・・ --------------   その他気になる事があります。 ・キャッシュ(serverclass_cache.json)が生成されません。 ・テストで作成したAppの配布はできているように見える。 -------[Client(XX.XX.XX.8)]------- $ sudo ls -l /opt/splunk/var/run/HFs total 12 -rw-------. 1 splunk splunk 10240 Jul 22 23:23 app_test-1753173390.bundle -------------- ・splunkd.logでも疎通ができていると思われるログが出ている -------[DS(XX.XX.XX.10)]------- 07-23-2025 23:30:28.287 +0000 INFO PubSubSvr [2431 TcpChannelThread] - Subscribed: channel=deploymentServer/phoneHome/default/reply/heavy-1/HF1 connectionId=connection_XX.XX.XX.8_8089_heavy-1.internal.cloudapp.net_heavy-1_HF1 listener=0x7fbd68ccc400 --------------   よろしくお願いします。
Please can you provide some sample events to demonstrate your issue?
Check out this API reference for being able to use SignalFlow through the API: https://dev.splunk.com/observability/reference/api/signalflow/latest#endpoint-create-websocket-connection Also, if... See more...
Check out this API reference for being able to use SignalFlow through the API: https://dev.splunk.com/observability/reference/api/signalflow/latest#endpoint-create-websocket-connection Also, if you want to work within Splunk Cloud/Enterprise, you can use the Observability Cloud Infrastructure Monitoring TA which will allow you to  use the sim command in your spl and you can use SignalFlow there to get that metric. https://splunkbase.splunk.com/app/5247
Interestingly, when using tostring() before the sort it still identifies as a number?!   The only way I can make it sort as a string is using | sort str(fruit) as @richgalloway  mentioned.  ... See more...
Interestingly, when using tostring() before the sort it still identifies as a number?!   The only way I can make it sort as a string is using | sort str(fruit) as @richgalloway  mentioned.  Did this answer help you? If so, please consider: Adding karma to show it was useful Marking it as the solution if it resolved your issue Commenting if you need any clarification Your feedback encourages the volunteers in this community to continue contributing
hi @pedropiin  Is this the sort of thing you're looking for? Its a little complicated to setup due to the way the expression works in the <format> (cant reference other fields) so you need to create... See more...
hi @pedropiin  Is this the sort of thing you're looking for? Its a little complicated to setup due to the way the expression works in the <format> (cant reference other fields) so you need to create an mvfield containing the two values you want to compare, then use CSS to hide the other! Anyway - let me know what you think! Full example code <dashboard version="1.1"> <label>Colour smaller number cells</label> <description>App1 should be green if App1&lt;App2, App2 should be green if App2&lt;App1</description> <row> <panel> <html> <style> #tableCellColour table tbody td div.multivalue-subcell[data-mv-index="1"]{ display: none; } </style> </html> <table id="tableCellColour"> <search> <query>| makeresults count=10 | streamstats count AS row | eval "Hidden Avg Time App1" = (random() % 100) + 1 | eval "Hidden Avg Time App2" = (random() % 100) + 1 | eval "Avg Time App1" = mvappend('Hidden Avg Time App1', 'Hidden Avg Time App2') | eval "Avg Time App2" = mvappend('Hidden Avg Time App2', 'Hidden Avg Time App1') | fields - Hidden* | table _time *</query> <earliest>@d</earliest> <latest>now</latest> <sampleRatio>1</sampleRatio> </search> <option name="count">50</option> <option name="dataOverlayMode">none</option> <option name="drilldown">none</option> <option name="percentagesRow">false</option> <option name="refresh.display">progressbar</option> <option name="rowNumbers">false</option> <option name="totalsRow">false</option> <option name="wrap">true</option> <format type="color" field="Avg Time App1"> <colorPalette type="expression">case(mvindex(value,1) &gt; mvindex(value,0), "#1ce354", 1=1, "#de2121")</colorPalette> </format> <format type="color" field="Avg Time App2"> <colorPalette type="expression">case(mvindex(value,1) &gt; mvindex(value,0), "#1ce354", 1=1, "#de2121")</colorPalette> </format> </table> </panel> </row> </dashboard>  Did this answer help you? If so, please consider: Adding karma to show it was useful Marking it as the solution if it resolved your issue Commenting if you need any clarification Your feedback encourages the volunteers in this community to continue contributing
When I add  | eval type=typeof(fruit) to the query the results say the numbers are indeed numbers rather than strings.  That would explain the sort. When I use  | sort str(fruit) the results are... See more...
When I add  | eval type=typeof(fruit) to the query the results say the numbers are indeed numbers rather than strings.  That would explain the sort. When I use  | sort str(fruit) the results are in the expected lexicographical order. FWIW, the docs do say "Numeric data is sorted as you would expect for numbers and the sort order is specified as ascending or descending."
Further to my last reply, I've tested with the following which I think does what you need <form version="1.1"> <label>Testing</label> <row> <panel> <title>Support cases</title> <tab... See more...
Further to my last reply, I've tested with the following which I think does what you need <form version="1.1"> <label>Testing</label> <row> <panel> <title>Support cases</title> <table id="myTable"> <search> <query>index=_internal | head 3 | eval "Time (UTC)"=_time | eval "Support ID"="Testing" |eval _epoch_time=_time, _epoch_plus60=_time+60 | table "Support ID","Time (UTC)", _time, _epoch_time, _epoch_plus60</query> <earliest>-15m</earliest> <latest>now</latest> </search> <!-- Drilldown Configuration --> <!-- Enable row-level drilldown --> <option name="drilldown">row</option> <option name="refresh.display">progressbar</option> <drilldown> <link target="_blank">/app/search/search?q=search index=sony* sourcetype=sony_logs support_id="$click.value$"&amp;earliest=$row._epoch_time$&amp;latest=$row._epoch_plus60$</link> </drilldown> </table> </panel> </row> </form>  Did this answer help you? If so, please consider: Adding karma to show it was useful Marking it as the solution if it resolved your issue Commenting if you need any clarification Your feedback encourages the volunteers in this community to continue contributing
The Splunk documentation says that the order rule is lexicographic. I am trying to sort the following values: | makeresults | eval fruit="apple" | append [ | makeresults | eval fruit="Banana" ] ... See more...
The Splunk documentation says that the order rule is lexicographic. I am trying to sort the following values: | makeresults | eval fruit="apple" | append [ | makeresults | eval fruit="Banana" ] | append [ | makeresults | eval fruit="zebra" ] | append [ | makeresults | eval fruit="10" ] | append [ | makeresults | eval fruit="2" ] | append [ | makeresults | eval fruit="20" ] | append [ | makeresults | eval fruit="30" ] | append [ | makeresults | eval fruit="3" ] | append [ | makeresults | eval fruit="1" ] | append [ | makeresults | eval fruit="25" ] | append [ | makeresults | eval fruit="38" ] | table fruit | sort fruit The output I am getting is: 1, 2, 3, 10, 20, 25, 30, 38, Banana, apple, zebra I understand that Banana appears before apple because B<a. But what is up with string numerics? Shouldn't the order be: 1, 10, 2, 20, 25, 3, 30, 38, Banana, apple, zebra ?  Even the documentation says that between 10, 9, 70, 100 the sorted output should be 10, 100, 70, 90.  https://help.splunk.com/en/splunk-enterprise/search/spl-search-reference/9.2/search-commands/sort 
Hi @splunklearner  If you prefix those epoch fields with an underscore _ then it wont show up in the Table, but you can still reference it as a token.  Try the following: <table id="myTable"> <sea... See more...
Hi @splunklearner  If you prefix those epoch fields with an underscore _ then it wont show up in the Table, but you can still reference it as a token.  Try the following: <table id="myTable"> <search> <query>index=sony_summary sourcetype=stash |search <passed drop-down tokens> |sort 0 -"Time (UTC)" |eval _epoch_time=_time, _epoch_plus60=_time+60 (added this now) | table "Support ID","Time (UTC)", _time -------- , _epoch_time, _epoch_plus60</query> </search> <earliest>$time_range.earliest$</earliest> <latest>$time_range.latest$</latest> </search> <!-- Drilldown Configuration --> <!-- Enable row-level drilldown --> <option name="drilldown">row</option> <option name="refresh.display">progressbar</option> <drilldown> <link target="_blank">/app/search/search?q=search index=sony* sourcetype=sony_logs support_id="$click.value$"&amp;earliest=$row._epoch_time$&amp;latest=$row._epoch_plus60</link> </drilldown> ... etc...  Did this answer help you? If so, please consider: Adding karma to show it was useful Marking it as the solution if it resolved your issue Commenting if you need any clarification Your feedback encourages the volunteers in this community to continue contributing
This splunk thread was the answer: https://community.splunk.com/t5/Knowledge-Management/Why-is-KV-Store-initialization-failing-on-one-of-our-add-on-to/m-p/435188#M3872
Hi @ITWhisperer , thanks for your reply I didn't know about this multivalue strategy and it seems very useful. But how would I use, e.g., a "foreach *" structure to implement it between two differ... See more...
Hi @ITWhisperer , thanks for your reply I didn't know about this multivalue strategy and it seems very useful. But how would I use, e.g., a "foreach *" structure to implement it between two different fields? Because the only thing I could think of in the lines of using multivalue is something of the sort: for each field   if field should be compared     if field.value > nextField.value         value = mvappend(value, "RED") But I'm still confused regarding how to use this logic to compare two different fields. Thanks in advance,  Pedro
I created a summary index to call it in dashboard because it has so much data and need to run for larger time frames. Configured summary index in this way - <my search query> ---- ---- ---- | eval ... See more...
I created a summary index to call it in dashboard because it has so much data and need to run for larger time frames. Configured summary index in this way - <my search query> ---- ---- ---- | eval log_datetime=strftime(_time, "%Y-%m-%d %H:%M:%S") | rename log_datetime AS "Time (UTC)" |table _time, "Time (UTC)", <wanted fields> | collect index=sony_summary Now calling it in one of my dashboard panel in this way -  index=sony_summary sourcetype=stash |search <passed drop-down tokens> |sort 0 -"Time (UTC)" | table "Support ID","Time (UTC)", _time --------  Now my requirement is I don't want users to see this summary index data. So I have created a drilldown and linked to different search as below. Whenever they click on any field value in table, new search will be opened with clicked support_id   <earliest>$time_range.earliest$</earliest> <latest>$time_range.latest$</latest> </search> <!-- Drilldown Configuration --> <!-- Enable row-level drilldown --> <option name="drilldown">row</option> <option name="refresh.display">progressbar</option> <drilldown> <link target="_blank">/app/search/search?q=search index=sony* sourcetype=sony_logs support_id="$click.value$"&amp;earliest=$time_range.earliest$&amp;latest=$time_range.latest</link> </drilldown>   Now when I click on dashboard panel's field, it is opening with expected support_id as expected, but it is opening with token time range. I am expecting that this should return the particular time range at what time event indexed as per Time (UTC) or _time. Example - An event has support ID with time 07:00 am, when I click on it it should open for 7 am, but it is taking token time range. When I checked in chatgpt, it given in following one and modified it in this way. <table id="myTable"> <search> <query>index=sony_summary sourcetype=stash |search <passed drop-down tokens> |sort 0 -"Time (UTC)" |eval epoch_time=_time, epoch_plus60=_time+60 (added this now) | table "Support ID","Time (UTC)", _time -------- , epoch_time, epoch_plus60</query> </search> <earliest>$time_range.earliest$</earliest> <latest>$time_range.latest$</latest> </search> <!-- Drilldown Configuration --> <!-- Enable row-level drilldown --> <option name="drilldown">row</option> <option name="refresh.display">progressbar</option> <drilldown> <link target="_blank">/app/search/search?q=search index=sony* sourcetype=sony_logs support_id="$click.value$"&amp;earliest=$row.epoch_time$&amp;latest=$row.epoch_plus60</link> </drilldown> Now this is working fine and time range is also coming what I clicked on. but here the issue is I don't want these two new fields - epoch_time, epoch_plus60 to be visible in dashboard. These should get hided completely but still drilldown should work as expected. What to do here? Please suggest me. Am I missing anything? Even if I keep those fields in the last in panel, still my manager said hide it but it should work as expected.
Assuming you are using SimpleXML / Classic dashboards, you might be able to implement something along these lines. For each cell that you want coloured, calculate what the colour should be and make t... See more...
Assuming you are using SimpleXML / Classic dashboards, you might be able to implement something along these lines. For each cell that you want coloured, calculate what the colour should be and make the field a multivalue field, placing the colour as the second value. Then you can use CSS to hide the second multivalue in each cell. This technique has been documented in many answer on this site. For example, https://community.splunk.com/t5/Dashboards-Visualizations/Highlight-row-if-unique-values-exist-within-dynamic-pivot-table/m-p/560950/highlight/true#M39026