You can certainly try to set indexed extractions to none. That may affect other things you are configuring though, like the Timestamp fields. Because if you don't have those fields when the data is ingested into Splunk, you probably can't reference it like you're doing now as the timestamp field. Indexed Extractions is sort of an easy button for parsing data (getting data into Splunk) The other option would be to set the auto kv json setting to false for this sourcetype in props.conf. This would just turn off the extractions that try to run at search time. [dateTimeJSON] AUTO_KV_JSON = false If you understand parse time and search time yet, i'd just suggest reading this wiki article https://wiki.splunk.com/Where_do_I_configure_my_Splunk_settings%3F ... View more

Yeah, i would suggest setting AUTO_KV_JSON=false for your sourcetype as well. From the docs.... AUTO_KV_JSON = <boolean> * Used for search-time field extractions only. * Specifies whether to try json extraction automatically. * Default: true ... View more

not sure if something is different with 8.x, but this typically means that KV_MODE=json is set for this sourcetype somehow. So you the fields get indexed with indexed_extractions setting but then also extracted at search time with the kv_mode setting. maybe at least rule it out with btool on your search head?? splunk btool props list dateTimeJSON --debug And if not set on the sourcetype, maybe make sure it's not set in props for the source/host either? ... View more

what part of that doesn't work currently? The timepicker in the child dashboard would have the default range configured, right? So if you go there manually, without populating anything in the url, then it should use those defaults. ... View more

hmm...ok. i'm at work currently but can maybe try to work on an example later tonight, if i can get it working ... View more

is it set up as an input on the destination too? If so, just give that input the same processing rules (prefix, suffix, delim, whatever) as in the source dashboard? ... View more

hmm...ok. so you have errors about untarring the search bundle and errors about apps not existing. i wonder if you can look at the search bundle on that indexer and see if the app is in there? $SPLUNK_HOME/var/run/searchpeers I believe is the location. In there are the search bundles from the search heads that contain all of the config the indexers need to run the searches. If they're not making it there, then maybe the index is throwing an error because it doesn't have the context to run the search? ... View more

those logs will get purged after the search has expired. i'd suggest rerunning the search that is causing the problem, get the sid from the job inspector and then go find that specific search log on the indexer. And when you do find it, you may want to make a copy elsewhere since it will roll eventually. ... View more

Seem like you may be over complicating this a bit? Why not just pass the actual timepicker values in the URL from the other dashboard? Something like this? form.selectedTimestamp.earliest=-5d&form.selectedTimestamp.latest=now And just set your search to use the timepicker's values? <earliest>$selectedTimestamp.earliest$</earliest> <latest>$selectedTimestamp.latest$</latest> ... View more

what if you pass the form token instead? form.app=$form.app$ ... View more

were you looking at the search log on the offending indexer? the one in $SPLUNK_HOME/var/run/splunk/dispatch//search.log? ... View more

can you share your search, masking anything sensitive? Some thoughts.... Have you looked in the _internal or _audit index for this saved search to confirm that it did in fact return 0 results before firing the alert? Is this data being ingested in real-time? Meaning, for that 2 hour period, is the data definitely in Splunk at that time? Or is maybe ingested every 3 hours or something like that? Do you have any knowledge objects (field extraction, lookup, etc.) used in your search that may not be available to the search due to permissions or which app they're in, etc.? ... View more

that probably means the search for that panel actually produces a 0 as opposed to not finding any events. not a ton of info to go on here...but i would run that panel's search to see what it's doing. There are also ways with splunk search to make "no results found" show something else, but i'm assuming you would know if something like that was set up in your dashboard. ... View more

what about this regex with $1::$2 ?? https://regex101.com/r/fXTb3W/1 ... View more

have you tried just putting the spaces in the source:: stanza? Not sure if you need regex there or why splunk wouldn't be able to handle spaces...but i've never tried. Also, how is the data getting sent into Splunk...and is there a reason the sourcetype can't just be set there? I'm a little confused on the Rest API and the Windows UF part of the scenario...but likely it could be set at input time? ... View more

A few things... First, if you're using an indexer cluster, plan to just use master-apps on the cluster master (cm) to push config to your indexers. If that same server is also configured as a deployment server, use that to manage other components of infrastructure. But note that it is recommended (as you scale at least) that the cm is just a cm and nothing else. Second, we typically create apps/add-ons for our configuration. For example, if I work for the ACME corporation and we onboard Application ABC, then we may create an add-on called: TA-acme_abc. In that app would we would have a local folder where we put our configuration. And we would have metadata folder where we would create a simple local.meta file. And then we would put that TA in the master-apps folder and push it out. If you also have a search-time settings in there, you could also put it in deployment-apps and push it to your search head. Third, for your actual conf files. I'm not sure if that's a copy/paste or something you just typed up when created the post. But case matters. All of the settings on the left side of the equal sign should be in all caps. Also, I'd suggest adding the description part to your transforms settings just to be sure it's unique. Also, if you specify source:: in the stanza name, it expects the source not the sourcetype. If you want to use sourcetype, then don't specify a prefix...it's the default behavior for a stanza. So maybe like this: props.conf [splunkSourcetype] TRANSFORMS-mask_sensitive_data = mask_string transforms.conf [mask_string] DEST_KEY = _raw REGEX = regex FORMAT = replacement Also if possible, it's helpful to have a test/dev environment....even if it's just a standalone splunk instance. That way, you can create your add-ons there, manually upload sample data and verify that it works the way you want before pushing it your infrastructure. We don't want to keep pushing to production while trying to get something working. And one final side-note, you could also use SEDCMD in props at parse time to mask data....if you wanted. ... View more

Still seems to be here in 7.3.2. Have you opened a support case by chance? ... View more

glad to help. but just to note, anytime you specify earlier or latest in your search, it will override the whatever is the timepicker. So even if the time picker is set to Last 7 Days, this search would only run for the past 15 minutes index=blah sourcetype=meh earliest=-15m ... View more

ah, looks like you're comparing strings not timestamps, so splunk is going to compare them as strings. Either compare the epochs or reformat the date strings to a format that will work with a string compare, e.g. Year-Month-Date. i'd go with epoch first at least, maybe this? .... | where epochdate < bd_earliest_epoch | .... ... View more

are these the only 2 apps you've used for sophos? I see a few out there...just curious if maybe on your search head, somebody renamed the sourcetype? I guess maybe if that's the case, you could try searching or looking for the _sourcetype field? From props.conf rename = <string> * Renames [<sourcetype>] as <string> at search time * With renaming, you can search for the [<sourcetype>] with sourcetype=<string> * To search for the original source type without renaming it, use the field _sourcetype. * Data from a renamed sourcetype only uses the search-time configuration for the target sourcetype. Field extractions (REPORTS/EXTRACT) for this stanza sourcetype are ignored. * Default: empty string ... View more

Ok, i don't think we're getting anywhere here. Your search should run every 5 minutes LOOKING BACK 5 over the last minutes ago, e.g. 1:00-1:05, 1:05-1:10, 1:10-1:15, and so on. Why would a job that finished more than 5 minutes ago get returned by your search? I don't understand how that is. You're not explaining that at all. But if it is returning ALL of your data every 5 minutes for whatever strange reason, did you try filtering by the end time like i showed above? That would exclude any results where the end time is greater than 5 minutes ago. If your job runs once a day, I have no idea how the alert would be firing more than once with that filter in place Without more details, I can't help any further. You just keep saying the same thing w/o adding anymore info. ... View more