After a timechart split by a field you cannot use the field name after the timechart as it no longer exists. The field names are the values of your 'series' field. You need to use the foreach method in your initial post The reason why index=aws returns nothing is that the data you are searching for does not exist in that index. What makes you think it did? It is in the _internal index, so you should definitely include that Change your earliest/latest settings to define the time period you want to search for, or use the time picker instead and remove the earliest and latest totally.   ... View more

hi @gitingua  You could maybe try the solution mentioned in the comments here: https://community.splunk.com/t5/Dashboards-Visualizations/How-to-change-the-width-of-panels-in-xml/m-p/289203  ... View more

AWS is not on-prem (unless you own AWS  😀). ... View more

I changed my prost processing search to: | eval urgency = if(urgency=="-", "unknown", 'urgency') | stats count(eval(urgency="1 - High")) as "1 - High" count(eval(urgency="2 - Medium")) as "2 - Medium" count(eval(urgency=="3 - Low")) as "3 - Low" count(eval(urgency="unknown")) as "unknown" by urgency This puts the values in the key, and then the charting.fieldcolors works. ... View more

Hi @SS1  Have you tired making the panel bigger e.g. <option name="height">500</option>  Thanks, Jamie ... View more

Yes. If you have multiple matching stanzas, results from the highest privilege stanza applies. (If we're talking about overlapping settings) ... View more

my raw event in splunk looks like field1="223" field2="333" event="some text text2 text3 something something2" can you add to this regular formula part, which whill parse field event?  I mean event="..."   ... View more

Hello Jamie, Works, thanks a lot! ... View more

My experience is very much the same as this. I find base searches which are too generic and return a large number of non aggregate events perform very poorly. In addition, they also use a large amount of disk space which contributes to the users disk quota (until the search artifacts are removed). This can in turn lead to a lot of queuing for the users search jobs so it's worth noting how much disk space base searches use via the job inspector or the _introspection logs when developing the dashboard. In general I've found the best approach is to use base searches for summary panels at the top of dashboards which use aggregated data (e.g. single value visualizations) and avoid base searches when you want to display more non aggregated events (e.g. in a table).  One other thing to note here is that, if the dashboard is very heavily used and you can have base searches to reduce the number of searches that run when the dashboard loads (or alternatively have panels behind check boxes etc.) then you could reduce your concurrent search load on your Splunk servers. This could be important in large distributed environments with a large number of concurrent users. In this case you might be happy to have the dashboard take a few extra seconds to load if it results in less searches being executed.   ... View more

Yes I was suspecting frozenTimePeriodInSecs as well based on the docs.. Thx ... View more

hi @jamie00171 , what you said is correct about the follow tail=1, when i removed it, i can see the source which was missing before. It really helped me a lot, thank you. ... View more

@jamie00171    Upon further testing, it seems that doing EVENT_BREAKER_ENABLE = true EVENT_BREAKER = "([\r\n]+)" Works, but only if accompanied by also doing force_local_processing = true   Are the event breaker flags ignored if there's no force local processing? ... View more

index="*" sourcetype="pan:threat" 10.196.246.104 url=* [| makeresults | eval earliest=relative_time("2022-07-29 18:33:20","-1h") | eval latest = relative_time("2022-07-29 18:33:20","+1h") | fields earliest latest] | stats values(url) as url by _time,dest_ip,dest_port,app,category,rule,action,user ... View more

Hi @si_infrastructu  This might work for you: https://docs.splunk.com/Documentation/SCS/current/SearchReference/ConditionalFunctions#like.28.26lt.3Bstr.26gt.3B.2C_.26lt.3Bpattern.26gt.3B.29 ? Thanks,  Jamie ... View more

@PickleRick @jamie00171  Thanks for your suggestion. Discussion with the client are on... Will need to open firewall ports and test it out. Will keep you guys posted.. Thanks, Neerav Mathur ... View more

Works just perfect! Thank you very much!!  ... View more

If this didn’t work you could use INGEST_EVAL on transforms.conf. 1st try to pick _time as normal way and if it didn’t succeed then use INGEST_EVAL. You could found some examples from answers like https://community.splunk.com/t5/Getting-Data-In/Index-time-extraction-and-non-indexed-field/m-p/577740 r. Ismo ... View more

@jamie00171  @Roy_9 , thanks for answering, Probably you are right, there's lack of disk space, but the actual issue is that the user gets a "Queued" message, for example, for every dynamic drop down input field in a dashboard and that makes him "crazy" and angry, because it is not true that searches are "Queued". Searches are simply impossible to be executed, even if the user reloads the page. Do you know a way to monitor per user disk utilization? Is it possible to build some kind of dashboard that represents the percentage of disk quota being used by a user in the last 5 minutes? Thanks a lot and best regards. Luca ... View more

Hello @VatsalJagani thank you for the input. Yes, this concern was because a couple of weeks we had some alerts about memory usage in one search head (root cause a user running hard searches), but during the troubleshooting we notice that we have A LOT of unused memory just sitting unallocated on our servers that are hosting the VMs, we want to use them but first we want to be sure that is not going to generate further issues at the environment. Ok regarding the indexers I have noticed some issues with IOWait and I have try to identify the root cause but no luck there I created this post a few months ago: https://community.splunk.com/t5/Splunk-Enterprise/IOWait-Resource-usage/m-p/578077   So for now I can think there are not issues adding memory to the search heads. Kind Regards. ... View more

Hi @jamie00171 , Thank you for your quick response. I am not seeing anything other than scheduler error logs. Thanks, Anji ... View more

Thank you so much! ... View more

Hi @zacksoft_wf  When you use the collect command, Splunk writes the search results to $SPLUNK_HOME/var/spool/splunk directory/ directory on the disk of the host that executed the search. By default Splunk is configured (via $SPLUNK_HOME/etc/system/default/inputs.conf) to monitor this directory for any new files and put the contents through the indexing pipeline similar to a standard file input. The index used in the collect command determines which index the events go to. The source and sourcetype you use in the collect command will determine what props/transforms (field extractions etc.) apply when indexing (e.g. if the sourcetype creates indexed fields etc.) and the same source and sourcetype will have to be used when searching for the events and therefore the search time parsing rules of the associated source and sourcetype are applied which decide the interesting fields on the left hand side.  "Will my SPL overwrite any of the existing fields ?" - if you don't change the props / transforms config then nothing will be overwritten in terms of field extractions, it's more likely the filed extractions won't work for the data written to the summary index from the new search won't work, depending on how similar the new data is compared with the data that is written from the existing searches.  If the data is different you would typically use different source types with config specific to the format of the data to make sure the data gets parsed correctly. Also, typically the only reason you'd put different types of data into the same index would be if it searched together (i.e. index=myindex sourcetype=mysourcetype OR sourcetype=mysecondsourcetype), if you put a lot of different types of data into the same index it can impact search performance. (I can go into why this is the case in detail if you're interested).   Hope this helps,    Jamie ... View more

Hi @djreschke  Do the first 256 bytes of the file happen to be the same on each file in the same input?  If so, take a look at the following from (https://docs.splunk.com/Documentation/Splunk/9.0.0/Admin/inputsconf😞 crcSalt = <string> * Use this setting to force the input to consume files that have matching CRCs, or cyclic redundancy checks. * By default, the input only performs CRC checks against the first 256 bytes of a file. This behavior prevents the input from indexing the same file twice, even though you might have renamed it, as with rolling log files, for example. Because the CRC is based on only the first few lines of the file, it is possible for legitimately different files to have matching CRCs, particularly if they have identical headers. * If set, <string> is added to the CRC. * If set to the literal string "<SOURCE>" (including the angle brackets), the full directory path to the source file is added to the CRC. This ensures that each file being monitored has a unique CRC. When 'crcSalt' is invoked, it is usually set to <SOURCE>. * Be cautious about using this setting with rolling log files; it could lead to the log file being re-indexed after it has rolled. * In many situations, 'initCrcLength' can be used to achieve the same goals. * Default: empty string Thanks,  Jamie ... View more

Are you searching over the same period from the webui? Your rest call seems to be only searching over last minute or so. ... View more