This one is pretty straightforward - keep in mind that the GET request will return in it's response a Search ID (sid). You can then, in a few minutes, return the results of that SID by just visiting the link, such as: https://<host>:8000/en-US/app/search/search?sid=1686664178.1739 If you would rather have the API pull the results back, send another GET to /<app>/jobs/<sid>/results like this - you can specify options such as output_mode and others: curl -u admin:changeme \ -k https://localhost:8089/servicesNS/admin/search/search/jobs/1423855196.339/results/ \ --get -d output_mode=json -d count=5  official docs here: https://docs.splunk.com/Documentation/SplunkCloud/latest/Search/ExportdatausingRESTAPI ... View more

Older question - but this still came up as a hit on my search results while trying to help another customer, so this might be useful here. You have to run (or let the splunk agent run and manage) the monitors and scripts responsible for monitoring exchange related logs. To start - I would locally (like to your laptop) download the Add-On, https://splunkbase.splunk.com/app/3225 From there, explore the inputs.conf configurations available in the subdirectories. These define what is to be collected. If you notice, there are a ton of inputs available. These are set to OFF by default. Let's take a look at one of them under \TA-Windows-Exchange-IIS\default\inputs.conf ####Exchange Server Version 2010 - Start#### [monitor://C:\Program Files\Microsoft\Exchange Server\V14\Logging\Ews] whitelist=\.log$|\.LOG$ sourcetype=MSWindows:2010EWS:IIS queue=parsingQueue index=msexchange disabled=true initCrcLength=8192 The monitor stanza is defining the location of the path - so in this case, the files are stored in that file path (...\logging\EWS). The whitelist/allowist statement specifies that all files in that directory ending in .log or .LOG are to be allowed. The 'disabled' statement is currently set to false, as per the default.  To get this working, copy this stanza to a /local/inputs.conf directory, change disabled to '=false' and deploy to the machine that is running exchange. You will need a a UF, or Heavy Forwarder or some other way to getting data to splunk indexers. Use a similar process for the many other inputs you have available.  Do not turn on all inputs unless you really need them and have done capacity planning as it can be a lot of volume.  ... View more

one reason why you might see this is because the file system maintains a lock on a file if it is in use by a current process. so while you may have issued your file delete command, since splunk was actively using it, the file descriptor may not release the file - splunk will continue ingesting it The option to sinkhole/batch process mentioned is a good way though of deleting the file after ingest ... View more

slight typo on the query - this tstats search is faster than the metadata one if correctly typed: | tstats max(_time) as lastReportedOn where index=A sourcetype=A source=/tmp/A.app.log by index sourcetype source | convert ctime(lastReportedOn) ... View more

one idea for detecting a spike or outlier in logons would be to summarize, by user, how often they login.. say # of times a week or day.. you could try for hour but you'll likely get a lot of noise.. start with something like 'index=<security> sourcetype=<logins> |bin span=1d _time | stats count by <user> _time' Then summary index this data over a recurring period to be able to detect outliers. - try the |collect command Now apply the standard deviation method of detecting outliers - essentially check if the sample is some standard devs from the mean. The MLTK toolkit has some fun features for this, some math behind it is: mean logins by user calculate standard deviation test for : (logins < (mean + 2 standard deviations) ) if true - this is technically an outlier. you can adjust your standard deviation to experiment Then schedule this search to run over a time period and create an alert if there are results.. or to send you a report I would check out the MLTK though, lots of great stuff in there.. Security Essentials has a user case for this as well ... View more

agreed this is what it worked as well ... View more

Thanks for this! Helped us out in our very large environment ... View more

It doesn't return all alerts however - alert.track is set to 1 by default but if someone changes it, or is set otherwise by an app, the query above does not return all alerts, alert action or not. This comment thread serves to inform users of the query above to be on the lookout for this scenario - it is not a guarantee that all configured alerts will be returned. ... View more

This isn't necessarily accurate - if for some reason alert.track has not been set, this will not return all results. You can search for this yourself by using the GUI counts vs the results of the searches above. The workaround would be to narrow down the search results in a different way - most configured alerts will have at least one action associate with it so I used something along the lines of |rest/servicesNS/-/search/saved/searches | search actions!=""|<fields go here> ... View more

worked for me - thanks MHibbin ... View more

I can confirm - this is the definitely the issue!! Thanks hayduk - I will escalate this as much as I can -ali from Splunk PS ... View more