Still no results I even tested copying and then executing calc.exe from my desktop and a generic search returns those events, but the full search doesn't return any events ... View more

The lookup is set as follows as it's from the Splunk Security Essentials for Ransomware app: lookup table file - system32_executables.csv lookup definition - isWindowsSystemFile_lookup Permissions are set for All apps for both Thx ... View more

Still no events are returned ... View more

Thx - I did do that earlier and two events are returned sourcetype=XmlWinEventLog:Microsoft-Windows-Sysmon/Operational EventCode=1 Image!=*Windows\\System32* Image!=*Windows\\SysWOW64* defrag.exe ... View more

Plugged in the following but still no hit - thx sourcetype=XmlWinEventLog:Microsoft-Windows-Sysmon/Operational EventCode=1 Image!=*Windows\\System32* Image!=*Windows\\SysWOW64* | rex field=process .*\\\(?<filename>\S+)\s?$ | lookup isWindowsSystemFile_lookup filename OUTPUT systemFile | search systemFile=True | table _time dest host user Image ... View more

Derek, One last question - is there any way you could modify the app to correlate against src_ip as well? Thx ... View more

Derek, Just installed the app and CIM add on to create the relevant aliases and it's great. Thx for taking the time to build this. Any chance you could look to add the FireHOL IP lists (http://iplists.firehol.org/)? Thx, Jeff ... View more

Thx - reviewing the CIM documentation to implement ... View more

Thx for the info ... View more

Looks like something is possibly wrong with the way I'm calculating stdev: eventstats mean(EmailsSent) AS mean_sent, stdev(EmailsSent) AS stdev_sent by SenderAddress Both of these are coming up blank ... View more

Thx for the reply The only thing that's not being calculated is the Z score and I'm trying to figure out why. ... View more

Thx - I'll start plugging away at the two searches you created above (greatly appreciated). Fooling around, I combined your second original search with niketnaily's: index=exchange | bin _time span=1d | stats count as EmailsSent by SenderAddress | eventstats mean(EmailsSent) AS mean_sent, stdev(EmailsSent) AS stdev_sent | eval Z_score=round(((EmailsSent-mean_sent)/stdev_sent),2) | table SenderAddress, EmailsSent, mean_sent, Z_score It appears the mean_sent is comparing every user against the overall average instead of against the user's own average. How would I individually calculate the avg of number of emails sent per user and then calculate the z_score against that avg? Looking for users who are outliers based on the amount of email they send, not the total of emails sent by all users. Thx ... View more

Thx for the reply. I want to calculate the stdev of the avg number of emails per sender per day, regardless of recipients while eliminating the senders who legitimately send a high number of emails (mailing lists, etc), which would leave me with senders who get flagged as outliers. As always the problem is defining the time range for the baseline to compare against. 1 day? 7 days? 30 days? And so on. Eventually, I'd like to put the second search through the ML toolkit to see what can be identified. Thx again ... View more

Thx for the search and the links to anomaly detection and the advanced stats, ... View more

Thx for the reply and info. Correct - IIS events are basic w3c formatted strings inside of a JSON wrapper as we're using nxlog on the servers to forward the IIS logs (and as I said, add some addition k/v pairs for additional info). Duly noted on INDEXED_EXTRACTIONS as I'll remove that line from the stanza. Looking forward to your idea. Thx ... View more

Thx for the reply and information. technically, i can rename the sourcetype as we're ingesting these logs into Hunk, so everytime we start to ingest a new log source I have to define the sourcetype in /opt/splunk/etc/apps/search/local/props.conf. Here is the stanza for our IIS logs: [source::/LogCentral/IIS/EWI_PRD/...] sourcetype = _json INDEXED_EXTRACTIONS = JSON Is it possible to rename the sourcetype from _json to something else like ms:iis:default with still keeping the automated field extractions via json? My worry is renaming the sourcetype from _json to ms:iis:default, I'll wipe out the automated field extractions unless the INDEXED_EXTRACTIONS = JSON is what will keep the automated extractions in place even though the sourcetype will be set to something other than _json. Thx ... View more

Kind of. The events are set to the sourcetype of json already, so before I created the regex to extract fields at searchtime from the key/multivalue field "Event", some fields were already being extracted automatically, such as "SourceModuleName", "SourceModuleType", "FileName" ,"SiteId" ,"WebServer", and "Event". What I thought might be possible would be to define the key/multivalue field"Event" ala http://docs.splunk.com/Documentation/AddOns/released/MSIIS/Setupaddon (Perform additional steps for search-time field extraction) by modifying the transforms.conf file, but wasn't sure how to extract the fields/values from the already defined "Event" field. Thx ... View more

I'm trying to extract fields from the 'event' key in an automated way as I created a regex to extract the IIS fields So "Event" contains the following information in my example: "2017-02-21 13:00:00 x.x.x.x POST /autodiscover/autodiscover.xml - 443 - x.x.x.x Microsoft+Office/16.0+(Windows+NT+6.2;+Microsoft+Outlook+16.0.7571;+Pro) - 301 0 0 0"} ` which breakdown as IIS key/value pairs: Date - 2017-02-21 Time - 13:00:00 s_ip - x.x.x.x cs_method - POST cs-uri-stem - /autodiscover/autodiscover.xml cs-uri-query - s-port - 443 cs-username - c-ip - x.x.x.x cs(User-Agent) - Microsoft+Office/16.0+(Windows+NT+6.2;+Microsoft+Outlook+16.0.7571;+Pro) cs(Referer) - sc-status - 301 sc-substatus - 0 sc-win32-status - 0 time-taken - 0 If we weren't ingesting these files as json, I'd simply modify the transforms.conf file as follows: [auto_kv_for_iis_default] DELIMS = " " FIELDS = date time s-sitename s-ip cs-method cs-uri-stem cs-uri-query s-port cs-username c-ip cs(User-Agent) sc-status sc-substatus sc-win32-status Wasn't sure if it possible to do another field extraction from within an event that has already had its sourcetype defined. Perhaps the regex I have extracting field names at search is the way to go. Thx ... View more

When you state, "The arguments for the stats command is returning the count of MessageId as the field recipient_count. This search by itself will only return a single field: recipient_count, which for me, appears to return the resulting count of events with MessageId in them, renamed as the field "recipient_count", what I'm trying to do is get a count of the number of recipients for a user. Being that I don't have a recipient_count field, I thought I could get a count of the number of recipients for a user by substituting the MessageId field (figuring if I send out an email to 10 people the MessageId would be listed 10 times). Thx ... View more

Thx When I check Exchange message tracking log fields (https://technet.microsoft.com/en-us/library/cc539064.aspx), recipient-count is listed as a field. Unfortunately, pulling the logs from the O365 service, this is a field that is not offered. I'm trying to adapt and apply the search shown in the .conf2016 presentation, "Detecting the Adversary Post-Compromise with Threat Models and Behavioral Analytics" (around the 31 min mark) to look for anomalies. Search is as follows: sourcetype="MSExchange"2010:MessageTracking" sender="[email protected]" recipient_count!=NONE | dedup message_id sortby _time | table _time directionality sender recipient message_subject message_id recipient_count total_bytes | timecahrt sum(recipient_count) as daily_total span=1d | eventstats median(daily_total) AS median, p25(daily_total) as p25, p75(daily_total) as p75, mean(daily_total) as mean | eval iqr = p75 - p25 | eval xplier = 2 | eval low_lim = median - (iqr * xplier) | eval high_lim = median + (iqr * xplier) | eval anomaly = if(daily_total<low_lim OR daily_total > high_lim, daily_total,0) | table _time daily_total anomaly Thx again ... View more