Awesome, this will help many people. Thanks! Chris. ... View more

And for assertHaveCommand() the following needs to be updated (line 47 of bin/common.sh from Splunk_TA_nix 6.0.2): # # # /sbin/ is often absent in non-root users' PATH, and we want it for ifconfig(8) PATH=$PATH:/sbin/ Ubuntu needs this, or it wont be able to find the "service" command when Splunk is running as non-root (splunk). # # # Append path to help find commands when running as non-root, as the non-root paths are different PATH=$PATH:/sbin/:/usr/sbin/ It's called from bin/rlog.sh as follows: assertHaveCommand service Cheers, Chris. ... View more

An update on this after a little digging.. In Splunk_TA_nix 6.0.2, It looks like the rlog.sh script is intended to run as root (the implication is Splunk runs as root), per the following check in common.sh: assertInvokerIsSuperuser () { [ `id -u` -eq 0 ] && return echo "Must be superuser to run this script, quitting" > $TEE_DEST exit 1 } If you enable debugging on rlog.sh (looks like it throws away this important output to /dev/null inside $TEE_DEST unless you have debug enabled): sudo su - splunk $SPLUNK_HOME/bin/splunk cmd $SPLUNK_HOME/etc/apps/Splunk_TA_nix/bin/rlog.sh --debug You'll notice a debug file named debug--rlog.sh-- in the cwd which (unsurprisingly) says: Must be superuser to run this script, quitting Per the logic in assertInvokerIsSuperuser() From an ES point-of-view, this is sub-optimal not only from a security standpoint (running Splunk as root), but the TA is of course designed to work from sourcetype=auditd keyed from the rlog.sh input, so it's not adding the value it could. It's designed this way (i'd guess) because /var/log/audit/audit.log by way of ausearch is out-of-the-box only visible by root (without changes), but with proper unix/posix permissions setup, Splunk running as splunk, can ingest the file via ausearch, rlog.sh, etc. It's debatable whether it's a security risk allowing a non-root user to read the audit.log file, but if you can't bring it up into Splunk to keep eyes on it, it's a relatively small risk to accept. Anyway, just wanted to get to the bottom of why that was happening.. 🙂 PS: There's also a bug in assertHaveCommand() (at least on Ubuntu) i had to also work-around after assertInvokerIsSuperuser() to get it to work but i haven't yet found the root-cause for that, just a work-around, but looking into it.. Cheers, Chris. ... View more

The supplied Mandiant IOCs do false-positive a lot. Looks like you got a match on the file-name (threat_match_field=file_name, with a file-name of setup.exe per threat_match_value="Setup.exe"), would be my guess. You can test it by passing different Setup.exe's (maybe text-files? :)) via the Sourcefire's with different hashes to test the theory 🙂 Chris. ... View more

Correlation searches are just searches at the end of the day, running on a schedule. You could do a search like.. index=_internal sourcetype=scheduler savedsearch_name=part_match Look for a successful run and there should be an evcount (event count) field with a value of 0 or more if it returned any result (depends on your search if no events returned is normal or not). Also important is the 'status' field which says if it ran, was delegated, deferred, or skipped by Splunks scheduler. Please forgive any minor inaccuracies of fields, I'm not in front of a PC at the moment 🙂 Cheers. ... View more

Absolutely - try out the Event Generator app. https://splunkbase.splunk.com/app/1924/ It should generate some data to light up ES to learn on it. Also, check out the BOTS (Boss of the SOC) v1 competition dataset, as well. https://github.com/splunk/botsv1 Cheers. ... View more

George is exactly correct. @a212830 per George's answer, if you do go the SHC route (as someone who has setup one or two :)), ES works well in a SHC. Another reason you might consider a SHC, is if HA is absolutely necessary. SHC has more moving parts is the basic fact, and everything that comes with having more moving parts, applies here as well. It's just different that way, but if it's required, it does work, and works pretty well 🙂 Cheers! ... View more

Depending on authentication load, and given this is a DC, this could be normal, it depends. You could try selecting this link from your notable-event (which you might have already). Ensure your assets.csv correctly reflects reality, with both IPs and DNS/Hostnames, should make your ES experience better. Otherwise you could remove "| search src="companydomain.com" from the search and look at everything that is failing authentication to deduce what's happening. Cheers. ... View more

In the lab i used what i had handy, and had a Windows DS push the Splunk Nix Add-on to an OSX machine. After the app deployed, and wondering why i wasn't getting data from inputs that were turned on, i found the scripted-inputs weren't set to +x and couldn't be executed. A chmod +x *.sh in the app's bin directory got things going again, but that wasn't nice.. I suspect it's because Windows doesn't grok UNIX permissions and just deployed them as regular files. ... View more

Im sure you have a specific use-case in mind but have you seen the "Risk Activity" dashboard in ES? Repeat offenders tally up a score so they bubble up to the top when the risk is high enough from the total score. The risk score is also dotted throughout ES making it easy to see. Just in case you havent come across it is all. Cheers. ... View more

Not sure if it's possible (from a quick read around anyway). You could try a different approach and use Splunk Stream, which can also run on your BIND server. https://splunkbase.splunk.com/app/1809/ It will capture request and response codes, and you can specifically pick fields you want. Stream also has 'config templates' for ES to populate all the ES dashboards. Cheers. ... View more

One way to achieve your goal is to maybe have your "next steps" link implemented as a workflow action? Cheers. ... View more

No problem. Think of CIMing as preparing the data for ES to process correctly, then you can use either the built-in, or write your own correlation-searches, to apply the security logic you want. CIMing is done in the app or TA. When you look for apps on Splunkbase (eg for Bro, Bit9, etc) look on the right column, and it'll list if it's CIM-compliant (and usually a CIM version, eg 4.7, etc..) If it's not CIM compliant it wont work out of the box with ES until it's CIMed. Of course, it'll work with vanilla Splunk, but ES expects CIM'ed data. Then to change your security logic, you can create your own correlation-searches to look for different security threats, or adjust the supplied ones. Remember, correlation-searches are just searches at heart. Also you can adjust the workflow also in ES to suit your runbooks. Some useful links.. http://docs.splunk.com/Documentation/CIM/4.7.0/User/UsetheCIMtonormalizedataatsearchtime http://docs.splunk.com/Documentation/ES/4.6.0/Tutorials/CorrelationSearch Cheers. ... View more

Hey Tyrone, Logs that have sourcetype's applied, if their to be used with ES, would adhere to Splunk's CIM (Common Information Model). As part of the CIM, tags will be added to the logs, that direct which part of ES (datamodels really) the logs apply to (eg, Endpoint, Network, etc..) In theory, you can keep adding new sources of data, as long as your using CIM-compliant app's or TA's, ES should be making use of the data (if it's security relevant data that is..) You can alway adjust the configs yourself, or even write your own TA or app, so ES processes the data as you prefer. Or you could simply write your own correlation-searches, to process pre-CIM'ed data the way you want. Hope it helps.. Cheers. ... View more

As long as your forwarding your indexes to the indexers, ensure your indexer hardware specs meet (preferably exceed) the spec defined in the ES docs. The datamodel acceleration happens on the indexers, so check those. Sounds like your SH is ok. http://docs.splunk.com/Documentation/ES/latest/Install/DeploymentPlanning#Indexer_scaling_considerations_for_Splunk_Enterprise_Security Cheers. ... View more

I don't have ES in front of me, but i believe the "Brute Force Access" (words to that effect) correlation-search looks for "try/fail, try/fail, try/succeed" (within 1 day if memory serves, but could be wrong..) no matter if it's Windows or anything else, and it's also built into ES. ES does it via tags and CIM knowledge provided in the apps/TA's. Hope it helps. ... View more

This also occurred from the ES 4.1.3 -> 4.5.0. The same fix resolved that issue as well. Not sure of the root-cause, it would be nice to be able to go through the setup. ... View more

Hi jwelch, Awesome, thanks that got me back in. 🙂 Cheers. ... View more

Hi Vikas, Sure, the Splunk EventGen (Event Generator) app https://splunkbase.splunk.com/app/1924/ can generate logs to light up ES dashboards. Not sure if it'll work with Splunk Cloud (not sure if you have ES on-prem or in Splunk Cloud), but EventGen is the app to generate dummy logs to allow you to explore ES. Hope it helps! ... View more