It seems that non-admin users are only able to have three searches running simultaneously. Is there a way to increase this limit? Thx. Craig ... View more

I have a fairly complicated set of searches to create reports for Nessus vulnerability data. One thing I notice is that when I output the results to csv using outputcsv, not all of the columns are ending up in quotes. This is what the header row looks like: hostname,domainname,"dest_ip","service_name",protocol,"dest_port","result_type",risk,score,"nessus_plugin_name","cve_id","nessus_id","access_vector","access_complexity",authentication,"availability_impact","confidentiality_impact","integrity_impact","nessus_solution" The problem is that I can't import the data into Excel because some of the fields that aren't in quotes have commas in them. As far as how those fields were generated: hostname and domainname were created from a lookup table that took the output of the Nessus plugin for resolving host names and extracting the hostname and domainname from the FQDN. The protocol and port fields are extracted using rex at search time, but for some reason protocol isn't in quotes and port is. Risk is extracted from the Nessus plugin files using props/transforms.conf. The score field is assigned to the search result using a lookup table. So, we've got several different fields that are created using several different methods, but only some of them are missing quotes. How do I go about troubleshooting this? Thx. Craig ... View more

What is the best practice for making Splunkweb listen only on localhost or 127.0.0.1? Thx. Craig ... View more

I have been experimenting with indexing Nessus plugins. On my laptop where I have a test Splunk instance and scanner, I was able to successfully get Splunk to treat each plugin file as a single event. This seemed to work because every file starts with a # and comment: [nessus_plugins] TRUNCATE = 0 SHOULD_LINEMERGE = true MUST\_NOT\_BREAK\_AFTER = (#) In our production environment, Nessus doesn't run on the Splunk server. I installed a Splunk forwarder on the Nessus box and used the same inputs.conf file from my laptop. I put the transforms.conf and props.conf on the Splunk server. I noticed that a number of the plugins were being split up into multiple events. Some were multiple single line events, some events were chunks of 10-30 lines each. There didn't seem to be any obvious reason why Splunk was choosing to break the lines where it did. I dropped the index, deleted the fishbucket on the Nessus box, and copied the props.conf file to the Nessus server. Now each config file is being treated as a single event. If an input has some kind of line breaking logic to it, does that mean that the props.conf file must exist on the Splunk instance that manages the input or can it just run on the central indexer? Thx. Craig ... View more

Is it possible for Splunk to natively run a search against a remote Splunk REST API from within a search? For example, let's say I have a search that is monitoring apache logs for suspicious behavior. The central index/search server finds three IP addresses that are behaving suspiciously. I would like Splunk to be able to run a search on the remote web server's REST API that would execute a saved search with a python script in it that adds each address to the iptables chain and reloads the rules. Or would I just have to write the python script on the central server and call it that way? Thx. Craig ... View more

I'm getting some weird behavior with the Deployment monitor app. I've got two servers in our DMZ that act receive data from forwarders at another data center. When I look at the "All Forwarders" tab, it shows one of the DMZ hosts twice, but not the other one. When I do a check from the command line from a host in the remote data center, it shows that forwarding is working properly. So, my question is, why is one host showing up twice and not the other? I've confirmed that they have different hostnames defined in inputs.conf and server.conf. The other question is, do I need to run the Deployment Monitor on all forwarders? If not, then why can I only see the two forwarders that are forwarding directly to the central server and not any of the 3rd tier servers? Thx. Craig ... View more

I have events that look like this: inputs.conf: [monitor://D:\Splunk\NVDB*.xml] crcSalt = disabled = false followTail = 0 sourcetype = nvdb props.conf: [nvdb] SHOULD_LINEMERGE = true BREAK_ONLY_BEFORE = (?i)<entry\sid= MUST_BREAK_AFTER = (?i)</entry> MAX_EVENTS = 10000 REPORT-nvdb_vulnerable_products = nvdb_vulnerable_products EXTRACT-cve_id = (?i)<entry\sid=\"CVE-(?P \d+-\d+) EXTRACT-score = (?i)<cvss:score>(?P [^<]+)< EXTRACT-access_vector = (?i)<cvss:access-vector>(?P [\w+]+)< EXTRACT-access_complexity = (?i)<cvss:access-complexity>(?P [\w+]+)< EXTRACT-authentication = (?i)<cvss:authentication>(?P [\w+]+)< EXTRACT-confidentiality_impact = (?i)<cvss:confidentiality-impact>(?P [\w+]+)< EXTRACT-integrity_impact = (?i)<cvss:integrity-impact>(?P [\w+]+)< EXTRACT-availability_impact = (?i)<cvss:availability-impact>(?P [\w+]+)< The data is XML formatted. The files are treated as a single event and are around 250 lines long. The searches hang at like 538 events (out of tens of thousands). What's the best way to go about troubleshooting this? I have other XML inputs that take no time at all to search through. Thx. Craig ... View more

I'm trying to index a bunch of plugin files such that each file is a single event. I've tried almost every combination of the following options without success. Splunk still treats every line as a separate event. I'm running the latest 4.2.3 build. I feel like this was working eight months ago when last I played with it, but it seems to be broken now. [nessus_plugins] TRUNCATE = 0 SHOULD_LINEMERGE = false BREAK_ONLY_BEFORE = ABCDEFGHIJKLMNOP1234567890 LINE_BREAKER = (?!) I'm noticing that events indexed last year are working, but newly indexed events are not being broken properly. The files are XML formatted. I remember reading that there were some changes to how XML inputs are indexed in some of the later versions. Thanks. ... View more

I'm following the instructions for implementing a reverse DNS lookup at search time. I either get an error saying the fields couldn't be found or I don't get the output I'm expecting. I copied the external_lookup.py to /etc/apps/search/bin Transforms.conf: [dnslookup] external_cmd = external_lookup.py host ip fields_list = host, ip Props.conf: [nessus] LOOKUP-rdns = dnslookup ip OUTPUT host AS hostname I also tried OUTPUTNEW per the example in: http://docs.splunk.com/Documentation/Splunk/4.3/Knowledge/Addfieldsfromexternaldatasources#Set_up_a_fields_lookup_based_on_an_external_command_or_script What am I doing wrong here? The hostname field is empty if I run a search such as: sourcetype=nessus NOT timestamps | dedup host | rename dest_ip AS ip | lookup dnslookup ip | table ip, hostname Thx. ... View more

My understanding was that the best practice for creating redundancy for the forwarders is to create two forwarders and use DNS round robin load balancing to have clients load balance between the two or failover to the active one during an outage. What's the best practice for protecting forwarding using SSL in the scenario above? If I create listeners on those intermediate forwarders, should I have each one use the same CN and cert? My understanding is that if I have two hosts defined in the outputs.conf, it sends one copy of each event to both servers, thus doubling my license consumption and requiring a dedup for all searches. But if they do use the same cert and CN, which fields have to match in the config files? The inputs.conf and the server.conf file? Thx. Craig ... View more

I would like to require that our forwarders use SSL. I've seen references to a bug in 4.2.4 that will break when that setting in exists in inputs.conf. Has in been resolved in 4.2.5? I've searched the site for release notes that detail the bugs fixed in the release, but couldn't find anything. Thx. Craig ... View more

My understanding was that when a forwarder loses its connection to the central Splunk server, it will continue accepting data from other forwarders and it will queue those events until the Splunk server comes back online. One of our engineers is under the impression that if the central Splunk server is unavailable, forwarders stop accepting new data or just drop it on the floor. Could somebody clarify this for me? Thx. Craig ... View more