All,  I am looking GitHub Enterprise logs as captured by my Syslog-ng server on prem. The logs being sent are JSON ...mostly, but we have some values in the JSON key-value-pairs that are breaking characters. The app is not escaping these characters.  SEDCMDing all the these events at the indexer were just overwhelming and don't think this is the correct approach.  I am looking the Splunk Add-on for GitHub and I am seeing it wants Splunk for Syslog Connect container deployed. Before I go and deploy that and learn how it works and what not, how can I check that Splunk has already solved this problem? Just don't want to build that sort of lab out and found out there isn't already some sort of work around in this tool for escaping json chars.    thanks -Daniel  ... View more

All,  I have a simple bash script which pulls down some data from git and restarts a service. I'd like to give a button to my SOC on a Splunk dashboard that they can just click. I can put the script anywhere really. I remember seeing that if I should be able to execute script from splunk/bin from dashboards but I having a hard time finding an example.  Someone have an example/recipe they can point me to?    ... View more

All, So my Management Console Health check is flagging my indexers "Local indexing on non-indexer instances". Did I miss something to pass this health check? Indexers don't shouldn't need to output anything per my understanding? The indexer is correctly assigned the "indexer" role the role in the Management Console setup. Something I missed? Perhaps a .conf file somewhere that also needs updating?   Splunk 9.0 on El8.  ... View more

All,  Setting up an index cluster of 3 nodes soon and sizing some disks. Feels like you would always want a replication factor of 2 searchable copies of 2.  But I see that I can in theory set replication factor of 2 searchable copies of 1, which leaves out tsidx/bloom etc.  Docs really seem to gloss over this.  What benefit does this have? What would recovery look like in a RF=2/SC=1 situation with a lost indexer? How would I bring that replicated copy online if I lost an indexer for good and didn't have a second searchable copy?  ... View more

All, I've started seeing the following error message on Splunk 8.2.1 since installing alert_manager app and I'd like to clean it up. - Error from my deployment server, from btool checks - Version Splunk 8.2.1 - CentOS7 - I have /etc/deployment-apps/alert_manager/README/alert_manager.conf.spec there. So I assume that's what it's looking for. # Error 08-10-2021 13:23:41.948 -0700 WARN Application [28063 MainThread] - No spec file for: /opt/splunk/etc/deployment-apps/alert_manager/default/alert_manager.conf\n 08-10-2021 13:23:41.948 -0700 WARN Application [28063 MainThread] - Invalid key in stanza [alert_manager] in /opt/splunk/etc/deployment-apps/alert_manager/default/alert_actions.conf, line 12: param.urgency (value: low).\n # alert_manager.conf, line 12 under [settings] stanza auto_close_info = false # alert_manager.conf.spec, line 40, under [settings] stanza auto_close_info = [true | false] * Configure if informational events are automatically resolved * Defaults to false   Any ideas on how I'd troubleshoot this?  ... View more

All, Just upgraded to 8.2.1 last night and noticed something today with stats. # This search returns 160k+ events index=netfw 162276 # This returns a 0 in Smart mode, this search returned data in 8.1.x how ever no data in 8.2.1 index=netfw | stats count 0 # Same search in Verbose mode however returns the count index=netfw | stats count 162276 Shouldn't Smart mode have returned the count correctly also? It did work that way in 8.1 ... View more

All,  I have this search here and it's pretty slow. Any recommendations to speed it up? Currently 250.249 seconds and that just seems high.   index=osnixsec sourcetype=linux:audit host=*domain.net earliest=-7d@d latest=-2h@h NOT [ search index=osnixsec sourcetype=linux:audit host=*domain.net earliest=-2h@h latest=now | fields host  | dedup host | table host ] | fields host | dedup host | table host ... View more

All, I am reviewing my data ingestion health and I see the following error.  Breaking event because limit of 256 has been exceeded Changing breaking behavior for event stream because MAX_EVENTS (256) was exceeded without a single event break. Will set BREAK_ONLY_BEFORE_DATE to False, and unset any MUST_NOT_BREAK_BEFORE or MUST_NOT_BREAK_AFTER rules. Typically this will amount to treating this data as single-line only. soucetype=package Seems to be impacting about 20 of my hosts. My props.conf on my indexer has the following customizations for this sourcetype otherwise it's thw default Splunk_TA_nix 8.2x configs that came with it. [package] TRUNCATE = 0 MAX_EVENTS = 512 The resulting props.conf via btool on my indexer is [package] ADD_EXTRA_TIME_FIELDS = True ANNOTATE_PUNCT = True AUTO_KV_JSON = true BREAK_ONLY_BEFORE = BREAK_ONLY_BEFORE_DATE = True CHARSET = UTF-8 DATETIME_CONFIG = CURRENT DEPTH_LIMIT = 1000 HEADER_MODE = KV_MODE = multi LEARN_MODEL = true LEARN_SOURCETYPE = true LINE_BREAKER = ^((?!))$ LINE_BREAKER_LOOKBEHIND = 100 MATCH_LIMIT = 100000 MAX_DAYS_AGO = 2000 MAX_DAYS_HENCE = 2 MAX_DIFF_SECS_AGO = 3600 MAX_DIFF_SECS_HENCE = 604800 MAX_EVENTS = 512 MAX_TIMESTAMP_LOOKAHEAD = 128 MUST_BREAK_AFTER = MUST_NOT_BREAK_AFTER = MUST_NOT_BREAK_BEFORE = SEGMENTATION = indexing SEGMENTATION-all = full SEGMENTATION-inner = inner SEGMENTATION-outer = outer SEGMENTATION-raw = none SEGMENTATION-standard = standard SHOULD_LINEMERGE = false TRANSFORMS = TRUNCATE = 0 detect_trailing_nulls = false maxDist = 100 priority = sourcetype = Any idea why I am getting errors concerning MAX_EVENTS and how I can resolve that? ... View more

All,  Is there any guidance like we get with security inputs (osnix, osnixscript, osnixbash) for the Splunk_TA_nix metric inputs? was going to go with osnixmetrics but if there is something professional services prefers I wanted to make sure that is what I used.    thanks,  ... View more

All,  Anyone have an app or search that can help a non-technical user review retention of data by source?  They'd like to see what the index config is, what the current oldest log is and what the project ETA to fill the volume?    thanks in advance,  -Daniel  ... View more