Hi as @_JP said, MC is just for monitoring the whole Splunk environment. You really need it in any distributed environment. There are also some usable apps in splunkbase.splunk.com which you could look e.g. AdminAlerts. One way to avoid running out of disc space on indexers it migrate your environment to use volumes instead of use SPLUNK_DB on all index configurations. That way splunk automatic frozen needed data based on volume size instead of days or index sizes. r. Ismo ... View more

Hi as other already said this is doable. Here is my old post how to do it https://community.splunk.com/t5/Splunk-Enterprise/Migration-of-Splunk-to-different-server-same-platform-Linux-but/m-p/538062 Of course in your case you must take care of connection between those sites. That must be enough fast to get this working. One other notice is that you must have same splunk version on all nodes in cluster. So you must select between update old first or install old version to AWS first. Based on amount of your data and connection speed between sites you could do this or other option is just create a new idx cluster on AWS and after it is in use then create additional (temporary) cluster for old onprem data and add it as a second cluster for your search heads. That way you could utilise e.g. snowball to transfer data from onsite to AWS (if your connection cannot handle it with reasonable time).   r. Ismo ... View more

Hi @_JP  There are two automatic lookups (for the two csv-s) under Splunk Add-on for Sysmon. Both are enabled. The one I am interested in looks like this: ... View more

yes installed in the server but can we get the related to application services  ... View more

Ok. One at a time 😉 1. While the default retention period for an index might be 6 years (I don't remember exactly but that sounds legit) the retention period for _internal index is just 30 days (you can check where it's defined by doing splunk btool list indexes _internal --debug on your indexer) 2. While your environment might not be that long, there might be some events that were for example produced by hosts with badly configured time. Check the actual events to see where they came from. Remember that the timestamps do not have to reflect the time at which the event was ingested. 3. Events are rotated hot->warm->cold->frozen (deleted) as whole buckets. So the bucket gets rolled to frozen if most recent event from that bucket is older than the retention period. 4. While most buckets will contain events from a relatively small time range, there is a special bucket created for each index which catches all "weird" events - older than expected or from the future. Since this bucket may contain events both very old (as is probably your case) as well as from the future it might actualy not get rotated for a long time. 5. Do a dbinspect of your index to see what buckets you have in your index and what time ranges they contain. ... View more

I'll preface this with there are some best practices I'm skipping over for a production dashboard - formally creating a lookup & setting permissions, scheduling a saved search (aka report) to create this lookup, etc.  I'm also assuming you have admin access to your environment since this example uses data you would have in your index=_internal.  The important thing here is the concept of referencing a lookup and not having an in-line search. I have attached an XML so you can see the SimpleXML dashboard I created for this example.  The left input dropdown does an in-line search to populate the dropdown values (and this could be what you're seeing as slow).  This means it is searching over, and the right input dropdown still runs a search, but all that search does is load a lookup csv file for the data - it's really quick!   The search I run for the left input is the following, and it is configured in the XML to look over the past 120d: index=_internal | dedup component | table component | sort component   The search I run for the right input is the following, and the timeframe doesn't matter - all it does is load a csv, but the results in that csv lookup are the same format/data as the search above: | inputlookup internal_component_list.csv Note:  that's not the real search that generates the csv.  It is just loading it.  To generate the csv, I ran the following search.  It's real similar to the one for the left dropdown, but I added the outputlookup command to make that csv: index=_internal earliest=-120d | dedup component | table component | sort component | outputlookup internal_component_list.csv You can take this outputlookup search and schedule to run once a week (or however often is appropriate for your data).  The key is this search can be scheduled to run behind the scenes when no one is waiting on the results. I just scheduled it as a report: And its search configuration looks like this:   And for this example I decided to schedule it weekly (but notice that the search looks back 120 days with the earliest=-120d in the SPL).  I'm essentially building my dropdown data weekly from the past 120 days of events in _internal:   ... View more

No. Unless explicitly configured to do so (which may be the case but it's beyond us to know how your environment is configured) splunk doesn't "generate" data. It ingests (and possibly modifies if it's configured that way) data it's given for ingestion. And clustering doesn't change the data. It merely replicates (if needed) data to other nodes. ... View more

So, I found a "fix" or "workaround" that I'm not too happy about, but it seems to solve the issue. Surrounding the "require" with a 5 second "setTimeout" call, seems to work fine, as follows: setTimeout(function()  {     require(['splunkjs/mvc'], function(mvc) { ... } }, 5000); If anyone can shed any light on this "issue", please advise. ... View more

Thank you for your message, I checked all occurrences of the original app name in all files and I found that in many .py files the original name was used, so i replaced those with the new name. Now the inputs.conf seems to work and I can see logs, the issue I have now is that the UI of the app shows a new error:   "Configuration page failed to load, the server reported internal errors which may indicate you do not have access to this page." I am checking again what other replacements I need to do.   ... View more

Specifically listing them  using  the GET is proving troublesome. When I search the returned results, I don't find all alerts, but I do find all reports. The POST to create and alert is not an issue.  ... View more

Hi @_JP, I am conceptually agree with you, but the customer already has logs on logstash and wants to use Enterprise Security, that uses CIM. For this reason I have to ingest and parse logstash data, trying to persuade customer to pass to Universal Forwarders. I asked to the Community if someone has already addressed this problem, to have some hint or attention point. Anyway, working by myself, I already reconducted some data flows to standard add-ons. Thank you for your answer. Ciao. Giuseppe ... View more

I'm curious about why you thought eval would not work after stats. There's nothing particularly magical about stats.  It's a transforming command so only the fields used in the command are available to later commands.  They are still fields, however, and can be processed as such.  Note that some stats functions produce multi-value fields, which don't work well in all commands so they may require additional processing. ... View more

The real pro-tip is always in the comments. 😂 ... View more

Thank you for your help for this question Can you also help this related question?    Thank you so much https://community.splunk.com/t5/Splunk-Search/How-to-calculate-total-when-aggregating-using-stats-max-field/m-p/660403#M227978 ... View more

@_JP is correct that single quotes have special meaning in SPL.  Have you tested   index=_internal status=* | rename status AS HTTPStatus | stats count(eval(HTTPStatus >= 400 AND HTTPStatus < 500)) AS fourxxErrors, count(eval(HTTPStatus >= 500 AND HTTPStatus < 600)) AS fivexxErrors, count AS TotalRequests Can you share sample output from this?   ... View more

if only it was supported in dashboard studio!  will use classic for the time being - thank you! ... View more

Are you indexing any of your CMDB info in Splunk already?  You can setup automated entity updates for ITSI, and an easy way to do this is with a Splunk SPL query on data already indexed.  Here is more info in Docs.    If you don't have this sort of thing, a CSV-formatted file is a good starting point since that can already be used to manually update via the gui.  Also keep in mind that if your CMDB can generate that csv for you, then you might as well index that file in Splunk (and you're back to the first paragraph)  😁 ... View more

The destination dashboard would just need to be ready/able to handle the token(s) you are passing it from your source dashboard.   Here's more info on setting up the source dashboard for linking.   ... View more

Thank you very much for this suggestion. ... View more

In a pretty fresh 9.1.1 Splunk install (only Dashboard Examples has been added), if I tried to run the search in the Accepted Solution you linked to I wasn't getting any results because the app and view fields weren't getting extracted in the Search app context in Verbose Mode.  Not sure if something has changed in the field extractions on _internal since that was posted in 2019. If I add a rex command to force the field extract, it seems to be picking up all of the hits to splunkweb to track app usage:   index=_internal sourcetype="splunk_web_access" method="GET" status="200" user!=- | rex field=uri_path "\/app\/(?<app>[^\/]+)\/(?<view>[^\/\s]+)" | stats count latest(_time) as ViewTime by user app view | sort -count | eventstats sum(count) as countByApp list(view) as view list(count) as count list(ViewTime) as ViewTime by user app | convert timeformat="%a %m/%d/%Y %I:%M:%S %p" ctime(ViewTime) | dedup app | appendpipe [stats sum(count) as count by user | eval view = "Total Views"] | sort + user -countByApp     Note:  There are other things happening in splunkweb that you don't have to count - images fetches, calls to the manager endpoints, etc. You can see them here:   index=_internal sourcetype="splunk_web_access" method="GET" status="200" user!=- | rex field=uri_path "\/app\/(?<app>[^\/]+)\/(?<view>[^\/\s]+)" | search NOT app=*     ... View more

I'm assuming you know about & are modeling things after the Splunk Rest Examples?   Can you share a bit of your Python on how you are returning your data? What I'm thinking is if your handle method isn't returning your data in the correct JSON format it needs to generate the response. ... View more

Check logs more "backwards" to see earlier errors. Maybe you mistyped file paths, maybe the password was wrong... ... View more

Just find all events with system_id=aa-1* initially (to limit the number of events you're working with) and then use then regex command to limit the values only to aa-1(-.*)? ... View more