Probably something like sourcetype=xxx | dedup action,id | transaction id ... View more

No, configure inputs.conf as normal, and create each destination index ahead of time via indexes.conf or the Manager. The settings above will override the sourcetype and destination index as the file is indexed. Not sure what happens if the index doesn't exist - it will probably throw an indexing error but it might revert back to the default index. ... View more

Sounds like the dedup command is what you're looking for. ... View more

This is a surprisingly messy problem. Unfortunately, I don't think that mvindex will work, because the multivalue field isn't going to contain duplicates -- each value for employee_id would only show up once. Option 1: Regex Search and Backreference Try this: sourcetype=file_withdata | transaction maxspan=1h workstation_ip | eval distinctusers=mvcount(userid) | search distinctusers>1 | regex _raw="(?s).*?LOGON\s+\S+\s+(?<a>\S+)\s.*LOGON\s+\S+\s+(?<b>\1)" This does a few things: Runs your original transaction search to get the users logging on to each machine Removes entries that don't have at least two different user IDs logging on Pares the results down further, to just those whose first and last login lines have the same user ID Unfortunately, it's not generalized to other log types, but it's a start. Instead of the backreference, you could also use rex instead of regex to pull the first and last match into separate fields, then use eval to check that they're the same. Option 2: Write a Custom Search Command This sort of tracking would lend itself pretty well to a custom search script. Essentially, you'd be rolling your own version of the transaction command, but adding the checks to make sure you have at least one different employee id in the middle. Overall, this may end up being the best option. It also has the fringe benefit of being more generic (you can base it on field values rather than raw text) and producing much shorter/simpler search strings. Take a look here for more detail on that option. Option 3: Simplify Your Requirements If all you really care about is whether more than one user logged in to a given machine in a short period of time, you may be better off using stats : sourcetype=file_withdata | stats dc(employee_id) as dc first(employee_id) as a last(employee_id) as b list(employee_id) | eval is_same=if(a==b,1,0) | search is_same=1 dc>0 This should work with smaller samples, but won't scale well as you increase the time span. Option 4: Map command The map command iterates over multiple search results. In theory you ought to be able to use an initial search and dedup to get a list of unique employee_id . Then use map to iterate over that list and look for your transactions, setting startswith=(employee_id=$employee_id$) and the same for endswith . In practice, I've never been able to get this to work, but not sure why. ... View more

I'm reasonably certain that Splunk does not support expansion of dynamic groups directly. If I'm understanding correctly, however, the overlay should be taking care of the expansion for you. With that in mind, the first step would be to verify that the groups are being expanded correctly, using ldapsearch from the command line as described here. If that works, then make sure that the member attribute exists in the results, and that it contains the valid dn for each user. If not, you'll need to either fix the dynamic group or set groupMemberAttribute and groupMappingAttribute . Some LDAP servers also have a requirement that the objectClass that is the "trigger" for the dynamic group be explicitly included in the search filter before expansion will occur. You may want to try setting groupBaseFilter = (objectclass=GroupofURLs) and see if anything changes. If all else fails, try using a sniffer to watch the LDAP traffic and see the exact queries and responses. That's the most definitive way to tell whether the LDAP server is actually returning all results (indicating a Splunk issue) or whether expansion isn't working (an LDAP isue). ... View more

Have you tried running the same search from the GUI using the savedsearch command there? Does that produce any further detail on the error? ... View more

All of the general searching and reporting should work once you have data coming in and properly sourcetyped. You'll want to configure OSSEC to output via syslog, and make sure that Splunk is listening for it (the input is included but disabled by default). The Agent Status and Agent Management dashboards rely on pyOSSEC, so they will not work. Also, the malware check will never find anything -- OSSEC's syslog output does not include file hashes. ... View more

A few more things to try: Double-check your inputs configuration, and see if you are overriding `source` or `sourcetype` -- if you are, modify the searches suggested by Simeon accordingly. Broaden your searches and try to match on raw text instead of source or sourcetype to see if those fields have something different than expected. Pick a string of text that you know appears in your Windows logs, and search on that: `index=* SomeRawText` Run `netstat -p udp -b` to verify that the Splunk daemon is actually the process bound to that port. If you suspect a firewall issue, try stopping Splunk and installing Kiwi Syslog or similar on that port, just long enough to verify that it can receive syslog messages. That will help narrow down the problem to either network/host configuration or Splunk configuration. ... View more

Possible but not trivial. The limitation on Windows is listed in the KNOWN_ISSUES file, but I will add it to the README for the next release. Interestingly enough, Splunkbase asks you to identify whether a file is platform-dependent when you upload the app, but afterwards doesn't appear to show that information. It's more than just fcntl. The pyOSSEC library included with the app uses pexpect to communicate with the OSSEC agent management tools, and pexpect needs a unix-like system. The good thing about this model is that it's fairly easy to do remote management over SSH; the bad thing is that that it doesn't work on Windows, and it can be overkill when Splunk is running directly on the OSSEC server. For now, the goal has been to limit platform-dependent code to the pyOSSEC library and worry about Windows later. The longer-term goal is to move to more of a client-server model, which will remove that dependency. Since the OSSEC server itself doesn't run on Windows, I don't consider this a huge sticking point. You can still use the reporting functions on syslog or alerts file data. That said, if you're willing to share some information on how your Splunk/OSSEC deployment is laid out, I'll take it into consideration for future development. ... View more

I wondered about that, but wasn't sure if that approach would have differing results depending on whether the end-user script was called directly or via ScriptRunner. Though IIRC, the latter would only apply to custom search scripts and not scripted inputs. ... View more

Personally, I'd rather not see this happen. I actually like the additional requirement of having server access, especially given that (one hopes) pulling Splunk diags should be a rare occurrence. Having this added as a feature request could have some security implications, since splunk diag will contain things like raw config files. Things that would not otherwise be exposed through the application would now be accessible. For example, passwords embedded in config files or app scripts. Presumably this would be restricted to Splunk admins, but it's still a new attack point. That doesn't necessarily implementing it it's a terrible idea, but there definitely should be an option to disable the ability (and it should not be possible to re-enable it via the GUI or REST interfaces). ... View more

The environment will typically have $SPLUNK_HOME set to '/opt/splunk' or whatever your local installation path is. import os APPNAME = 'myapp' FILENAME = 'myscript.conf' SPLUNK_HOME = os.environ['SPLUNK_HOME'] filename = os.path.join(SPLUNK_HOME,'etc','apps',APPNAME,'local',CONFIGNAME) Filename should now contain '/opt/splunk/etc/apps/myapp/local/myscript.conf'. ... View more

It's also worth noting that Splunk has an option to sign indexed data at the block level. Arguably you would need to enable signing to achieve compliance. Whether it's truly needed I'll leave to the auditors to haggle over, but you may wish to read this page: http://www.splunk.com/base/Documentation/4.1.4/Admin/ITDataSigning ... View more

Thanks. I've seen the blog entry, though I'd forgotten it. As a clarification, I'm not so much looking for hard-and-fast rules or a "when x happens you need to upgrade". I'm more interested in objective metrics to support an informed decision on when and how to upgrade, as well as to identify when it's just a configuration issue or poorly written search. The bundled views go a little way towards that goal, but was wondering what else people were looking at, or if anyone had compiled a list of metrics or profiling searches. ... View more

You probably want to avoid tampering with default Python libraries. If you're only concerned with scheduled alerts, then I think you can just do the following in alert_actions.conf: [email] mailserver=mail.example.com:587 This doesn't appear to affect emails sent via the search pipeline (i.e., using | sendemail) -- for that you would still have to call your search like this: search xxx | sendemail to=myaddress@example.com server=mail.example.com:587 But the big advantage is you don't have to do anything "unusual". If you really want to change the overall default, you'll most likely have to modify the sendemail.py in the search app, which has a hard-coded default of "localhost". Overriding the sendemail script may break things, especially during version upgrades. For example, supporting PDF emails release in 4.1 required changes to that script. You have been warned.... Copy apps/search/bin/sendemail.py to sendemail-local.py Around line 51, modify the default value of "server" like so: serverURL = argvals.get("server", "mail.example.com:587") Edit apps/search/local/commands.conf to activate your new script: [sendemail] filename = sendemail-local.py ... View more

Does that mean that it is not possible to have a role inherit from [role_extra_stuff] and not have it be able to access new_index? For example, what if there are settings in [role_extra_stuff] that would be also in [role_one_more]? Those settings would have to be copied into [role_one_more] directly rather than having a third level that inherits from [role_extra_stuff]. Is that right? ... View more

Also, just discovered another old post along these lines - http://answers.splunk.com/questions/3123/message-module-filter-values ... View more

Ah, good to know, thanks! ... View more

The cleanest approach would be to try to find out why the host field is being set to the IP address instead of a hostname on input and fix it there. That would only apply to new events going forward though. If you want to change the display, you will need to modify the dashboard.xml in the search app. It's driven by this search: | metadata type=hosts so you would need to modify it to use a lookup table. Using the nslookup command may also be possible, but I believe that command needs raw events to operate on, and would not work with the output of the metadata command. ... View more