Splunk fields are case sensitive but given the opportunity for confusion it's typically not best practice to overlap spellings.  Unless the app specifically overlaps fields with case issues then I would avoid that. However, it appears the SPL is all based from a field specifically referenced as "LastLogon".  I would look to see if that field still exists after the updated application.  It's possible the information still exists but is not being extracted into the previous field name which the SPL is relying on. If so then you have the option to update the field extraction to that field name or update your SPL to reference any new fields that contain the information you require. ... View more

https://docs.splunk.com/Documentation/Splunk/9.3.2/RESTREF/RESTintrospect#server.2Fsysinfo | rest /services/server/sysinfo | table splunk_server transparent_hugepage.effective_state The above will get you the THP status per server, usually best to run from the DMC in a multi server environment - most if not all devices should be reported there.  Running from a SH or SHC may be limited to SH and IDX. https://docs.splunk.com/Documentation/Splunk/9.3.2/RESTREF/RESTintrospect#server.2Fstatus.2Fresource-usage.2Fhostwide | rest /services/server/status/resource-usage/hostwide | table splunk_server *cpu* *mem* ... View more

1) Can you share the exact SPL search 2) Did you upgrade/refresh the Add On when you upgrade from v8 to v9 ... View more

Awesome information - thank you for demo and tips. ... View more

Can you share a screen cap of the options there but not available to click on? Interim shot in the dark - ask for a rolling restart on the ITSI SH or SHC - assuming you've done this before so shouldn't be a permissions/capability issue - perform a log out and clear cache on browsers, I know you've tried more than one already ... View more

The apps menu toggle recognizes individual unique apps.  The fact that it appears you have the same app multiple times is an indication the app is not designed per best practices.  Even if the version number is in the title the app should be replacing itself so that you are only ever seeing the most recent install. Options a) Fix app so that it install over itself rather than creating unique instances b) Start removing older versions that serve no purpose anymore ... View more

Classic XML dashboards wont let you extend multiple rows. ... View more

Start with the DMC (Distributed Monitoring Console) to review the License usage broken down by index.  This will share with you the daily ingest records for the last 30 days broken down by index.  This is only a starting point as depending on how your environment was setup you may have very specific indexes or things may have been aggregated into only a few indexes. From there you can start decided what questions come next. ... View more

https://docs.splunk.com/Documentation/CIM/6.0.0/User/Install There is still one outstanding index defined in the app as of the most recent install version.  You will need the index defined on the indexes, you can do this full app or custom app to the indexers. ... View more

Do you mean? fieldA fieldB fieldC 1:10 1:3 1:2 1:10   1:2 1:10   1:2     1:1 ... View more

I like this idea but I've always inserted an extra step.  Run a query on the data in Splunk for the Source but then used the SourceType value to search the inputs.  Helps to avoid any issues with wildcards or regex in the log path and filename. To each their own and whatever works is always the best solution. ... View more

You have only defined a single source and destination, you need to compare multiples.  The color*/edge*/weight* is meant to independently review the spread across the comparisons values associated with the source/destination groups. Look at the sample images of visualizations on the splunk base app page for prime examples. https://splunkbase.splunk.com/app/4611   ... View more

Start with disk space - most likely Splunk is so busy trying to manage buckets and ingestion with no additional space that many activities are suffering. ... View more

At one point there was a Splunk-on-Splunk template for ITSI which worked wonders in a previous environment I monitored.  I did supplement the existing template with the inbound syslog system monitoring.  However, I didn't do anything to monitor the Router,FW, and LB since the network was quite large and any HA activities would require additional details.  It would have been too large a task for the return on value. Monitoring these items separately would have a lot of value and if port labels are informative then you can make up for the full integration map. IMO ... View more

I had done something like this in a previous life.  Each HF should get an app which has a props definition under the default stanza.  For a small number of HF's you can do this manually, for a large group to manage from like a DS reference the Splunk environment variables. props.conf [default] splunk_forwarder = <HOSTNAME> It has been a while so play around with this.  I seem to remember it was a props.conf mapped to transforms.conf which inserted the hostname so find what works the best for you.  ... View more

Here we go.  So this could be network transmissions so check for firewall blocks and any routing issues first.  Then look into SSL connection issues last. ... View more

As an outsider with no real knowledge of it I would say it's likely coming soon. Since AWS is their testing ground for all cloud items first they are likely aware of the need to support kernel 6.x.  Also reviewing the link you provided the UF already supports that kernel release. Contact your Sales team or an assigned TSE to your account to see if they can get you this information for tentative release date. ... View more

You need a btool debug output for macros.conf on the ES SHC.  The app is reading the proper file but it appears you have some override of that stanza coming from and outside file. ... View more

The rsyslog is a brand/flavour of application which is dedicated to syslog message protocol and handling.  There are alternatives which the most favorite alternative is likely syslog-ng.  So don't get caught up on the term rsyslog. https://www.rsyslog.com/doc/configuration/index.html Configuring rsyslog or any syslog for your environment can be easy but planning to reduce any gotcha moments requires some for thought.  Separating technology and hosts being key things to help make Splunk ingestion much easier.  A sample thought would be to have all inbound messages to the aggregator server written to file structure such as: /logs/<vendor>/<technology>/<host>/<filename.something> ex /logs/cisco/isa/127.0.0.1/authentication.log /logs/cisco/isa/192.168.0.1/metrics.log * completely fabricated examples Have the logs rotate on a schedule (ie 15mins or 60 mins) and remove files older than 'x' amount of time.  How you do this will be based on volume of logs written and available storage.  I've worked on a x3 original file span as a working bias but again your system may dictate that.  I always keep some incase the UF goes offline for a short period of time, you can recover logs you may otherwise miss.   Once you have that in place then you need to follow the normal UF ingestion process which I wont go through here since your question was more on rsyslog than UF and this community board has far more UF answers than syslog specific examples that are easily searched. ... View more

Scheduled "All Time" searches are typically a no-fly zone and you should look to see what your average or max run time is for those searches.  My general rule of thumb is that any search completion time should be less than 50% of the scheduled reoccurrence. ie. Run Time less than 2 mins 30 seconds if schedule reoccurrence is 5 mins If you absolutely must and I highly doubt there is any good reason that you need an "All Time" search try converting to a TSTATs search which processes much faster.  Even then I would stay away from that since "All Time" can be a significant drain and occupy resources for long periods of time. Can you get away with putting daily results into a summary index, searching "All Time" on the summary index is far superior then searching raw data events. ... View more

Not sure this is really an issue, but I would from practice keep the event and lookup field names a little different.  It could prevent any unintended complications in displaying or transforming data. ... View more