This query will give the number of warnings for each indexer. index=_internal source=*license_usage.log type=SlaveWarnSummary ... View more

I contend the documentation is incorrect.  LINE_BREAKER and BREAK_ONLY_BEFORE are contradictory and shouldn't be used together.  At the very least, great care should be used to ensure the two settings work properly. ... View more

@hemant_lnu wrote: We have one index os_linux which has 2 source type and i see props and transform is written . can you help me to understand how its working . linux:audit Linux_os_syslog   props.conf [Linux_os_syslog] TIME_PREFIX = ^ Tells Splunk to look for the event timestamp at the beginning of the event TIME_FORMAT = %b %d %H:%M:%S Tells Splunk what a timestamp looks like MAX_TIMESTAMP_LOOKAHEAD = 15 How far from TIME_PREFIX the timestamp is allowed to be SHOULD_LINEMERGE = false Don't combine lines LINE_BREAKER = ([\r\n]+) Events break after a newline (CR and/or LF) TRUNCATE = 2048 Cut off each event after 2048 characters TZ = US/Eastern Event timestamps are expected to be in this time zone Transforms.conf [linux_audit] DEST_KEY = MetaData:Sourcetype REGEX = type=\S+\s+msg=audit FORMAT = sourcetype::linux:audit Look for "type=", some text followed by white space, then "msg=audit".  If it's found, set the sourcetype field to "linux:audit" [auditd_node] REGEX = \snode=(\S+) FORMAT = host::$1 DEST_KEY = MetaData:Host Look for "node=" in each event and set the 'host' field to the word that follows it. ... View more

There are no defaults for charting.fieldColors. ... View more

Try increasing the maxKBps setting in the forwarder's limits.conf file.  It sounds like the default of 256 is not enough.  Try 512 or 0 (unlimited). ... View more

Have you enabled receiving of data in Splunk?  Go to Settings->"Forwarding and Receiving"  to turn on receiving. Does "localhost url" include the port number (9997 by default)? Do your firewalls allow connections between Mulesoft and Splunk? ... View more

Check out 'charting.fieldColors' in the Dashboards and Visualization manual. ... View more

Example?  Screenshot? ... View more

It's normal to see "splunkd -p 8089 restart" in the process list because that is the command that launched Splunk.  I'm not sure, however, about why it appears so many times.  AIUI, there should be a single "splunkd -p 8089 restart" process and additional "splunkd" processes for running searches.  I could be mistaken, however. ... View more

It sounds like you've settled on what might be a unsuitable solution to the problem.  Tell us more about the problem itself and we may be able to suggest a better solution. Lookup tables are for enriching events with additional fields based one or more fields already in the events.  It's not a conditional-execution mechanism. If this part of a dashboard (or can be made into a dashboard) then you have better options.  You can have inputs the user can select to determine which calculations are made.  That is well-trodden ground so let us know if that path sounds feasible. ... View more

It should be stated up-front that indexes cannot be reduced in size.  You must wait for buckets to be frozen for data to go away.  The best you can do is reduce how much is stored in new buckets. You've already taken a good first step by eliminating duplicate events. Next, look at indexed fields.  Fields are best extracted at search-time rather than at index-time.  Doing so helps indexer performance, saves space in the indexes, and offers more flexibility with fields. Look at the INDEXED_EXTRACTIONS settings in your props.conf files. Each of them will create index-time fields.  JSON data is especially verbose so KV_MODE=json should be used, instead. ... View more

This is an indication of inefficient bucket use, meaning buckets roll `before they fill up.  This can happen when indexers restart often, but in this case I suspect it's just a matter of the main index getting very few events before maxHotSpecSecs is reached and the bucket rolls to warm. The answer for buckets that are known to contain few events is to set maxDataSize to a value that makes the bucket at least 50% full before it rolls.  The default bucket size is 750MB.  The dbinspect command can tell you the current size of buckets to give you an idea of how to set maxDataSize. Best Practice is to not use the main index at all.  All incoming data should go into a custom index, leaving main empty (and not needing to roll). ... View more

There's no need to edit props or transforms for Ingest Actions as they can be configured easily using the UI.  In fact, use of the UI is recommended to avoid errors in the ruleset config.  If you need to edit the config files for Splunk Cloud, do so in a custom app and upload the app to your Splunk Cloud search head. https://docs.splunk.com/Documentation/SplunkCloud/9.1.2312/Data/DataIngest#Deploy_a_ruleset_on_an_indexer_cluster   ... View more

Install the app you cited in the OP on a heavy forwarder and use that to pull data from Azure using API calls.  The HF will forward the data to Splunk. ... View more

@livehybrid's answer is a good one. In general, HEC cannot pull data from any source.  It is merely a receiver for data pushed to Splunk. ... View more

What have you tried so far?  Where did you get stuck? Why check the whole day every hour?  If nothing was found in the 00:00-01:00 period at the 01:00 run then nothing will be found in the same period at the 02:00 run.  Searching the same data repeatedly is a waste of resources. ... View more

Check out the lookup function.  It should do what you want and put the results in a separate JSON array.   ... View more

Don't add to or touch the Python libraries that ship with Splunk as they will be replaced with each upgrade.  Put the required libraries (that Splunk doesn't provide) in your app.  This is the way. ... View more

To find alerts that have not triggered, try this query | rest /servicesNS/-/-/saved/searches splunk_server=local | fields title disabled triggered_alert_count alert.severity alert.track eai:acl.app | rename alert.track as isAlert, eai:acl.app as App | eval TriggerCount=coalesce(triggered_alert_count, 0) | where disabled=0 AND TriggerCount=0 AND isAlert=1 | table title alert.severity App ... View more

We need more information. ... View more

In addition to adding storage, consider increasing the number of indexers.  Unless the indexers are very over-powered, you probably will need more of them to ingest double the amount of data. ... View more

You can query all alerts using this REST command.  Filter the results to find the information you seek. | rest splunk_server=local /servicesNS/-/-/saved/searches | search alert_type!="always" ... View more

Please try my updated query. ... View more

Perhaps this will help. index=*1644* container_name="ls2-sdp-java" $selected_countries$ | rex field=_raw "for \[(?P<country>\w+),\s*(?P<cobDate>\w+),\s*(?P<sdpType>\w+)" | rex field=_raw "records: (?P<Recordcount>\w+)" | rex field=_raw "^(?<dateTime>\S+)" | eval DateTime=strptime(dateTime, "%Y-%m-%dT%H:%M:%S.%3N%Z") | eval CreatedTime=strftime(DateTime, "%H:%M") | eval CreatedDate=strftime(DateTime, "%Y-%m-%d") ... View more

The only sure way to control access to data is by index.  Have a separate index for each set of access rules. IOW, sources "123456" and "456789" should be in separate indexes and only roles that need access to the Source should have access to corresponding index. ... View more