I am trying to create a macro that will take a field from an existing query. But when I try to call it the macro treats its argument as a literal value rather than the search field value. Specifically what I am trying to do is to lookup info about a queried machine in Carbon Black. the macro looks like this name: reqsensorsearch(1) sensorsearch query="$sensor_search$" I tried testing my macro with |makeresults | eval sensor_search="hostname:<myhost>" | `reqsensorsearch(sensor_search)` but if I do Ctrl+Shift+E (to expand and display), I see this | makeresults | eval sensor_search="hostname:<myhost>" | sensorsearch query="sensor_search" What do I need to do to make the macro accept the value of the field sensor_search? So I can get it to run as |sensorsearch query="hostname:<myhost>" ... View more

I am trying to correlate two resultsets. One is a straight search of apache logs. The other is a table that that took a long time to run (several days) I wanted to know if the web logs are likely related the results in the table, which I define by them happening around the same time. So I tried using the transaction command with a dummy common value |searchA | eval dummy="true", from="searchA" | append [|loadjob sid=job2 | eval dummy="true", from="searchB"] | transaction dummy maxspan=3s And this does create a resultset of transactions, but it ignored the appended loadjob. I think this was because that job's results were a table, not events. So I tried turning it into events |loadjob sid=job2 | eval dummy="true", from="searchB" | eval _raw=_time." ".col1." ".col2 And then used that for the transaction |searchA | eval dummy="true", from="searchA" | append [|loadjob sid=job2 | eval dummy="true", from="searchB" | eval _raw=_time." ".col1." ".col2] | transaction dummy maxspan=3s But this creates distinct transactions for each "from" value even though I only defined the transaction field on "dummy". Does anyone have a suggestion on how I can correlate these two result sets on time? My ultimate goal will be to finish the query with |search from="searchA" from="searchB" So I can find values from one result set that might be related to the other ... View more

I am trying to use a wildcard based lookup table as part of a query that will get all non-wildcard based values so that I can use them in a real lookup. My company maintains 100+ firewalls, sometimes more. All the firewalls send traffic logs to Splunk. I would like to be able to break down the traffic logs into locations without having to know every single firewall name, so I created a lookup table that looks something like this firewalls.csv description, location, search_name, monitored testFW1, location1, fw-loc1-*, Y testFW2, location2, fw-loc2-*, Y testFWsub2, location2, fw-branch*-loc3-*, Y If it were not for the fact that I need a wildcard in the middle of the string to find my lookup data, I could just ask my admin to change some .conf file so that my csv could do wildcard lookups. I solved this by searching for values and then aggregating on the wildcard value, like this |inputlookup firewalls.csv | map maxsearches=20 search="|tstats prestats=t values(host) where index=pan_logs host=$dvc$ | stats values(host) as host | eval dvc=$dvc$" | append [|inputlookup firewalls.csv] | append [|inputlookup firwalls_lookup.csv] | stats values(host) as host first(loc) as loc first(monitored) as monitored first(note) as note by dvc |outputlookup firewalls_lookup.csv Now I can use the hosts values in a lookup index=pan_logs | head 10 | lookup firewalls_lookup.csv host | table src_ip dest_ip host location monitored This seems like a lot of work though. Basically, I am using a csv to create a legit lookup table (would probably run this nightly). Is there a better approach? ... View more

I am trying to summarize network traffic logged by our firewall to determine the factors that have made our index usage exceed estimates by about 50%. The queries that I am running take several days, so I am hoping that there are some optimizations that might help speed them up. There are a lot of data (hundreds of millions of log records per day), so I am asking a lot of Splunk to summarize Indexing logs do not go far enough back to let me do analysis on them. I also have a datamodel that does not go far enough back. So I cannot take advantage of the faster access that these models can provide I have already run queries to determine 3 points in time (a week each) establishing an increase in log record volume Here is what I am doing index=pan_logs | eval loc=case(match(host,"fw_loc1*"), "Loc1", match(host,"fw_loc2.*"), "Loc1", match(host,"fw_loc3.*"), "Loc2", match(host,"fw_loc4.*"), "Loc3", match(host,"fw_loc5.*"), "Loc4", match(host,"fw_loc6.*"), "Loc5", match(host,"fw_loc7.*"), "Loc5", match(host,"fw_loc8.*"), "Loc5", match(host,"fw_loc9.*"),"test", match(host,"fw_locA.*"),"external", match(host,"fw_locB.*"),"external", 1=1,host) | eval istraffic=if(match(eventtype,"pan_traffic"), "Y", null), isurl=if(match(eventtype,"pan_url"), "Y", null), isvuln=if(match(log_subtype,"vulnerability"),"Y",null), trafficDir=src_zone."-".dest_zone | stats count dc(host) values(host) count(istraffic) count(isurl) count(isvuln) dc(trafficDir) by loc The idea is to try and determine if a group of sources is adding to the volume and then understand why. The trafficDir is a quick measure of whether the increase might be related to increased firewall complexity. Does anything here look like I could make it more efficient? ... View more

My admin team frequently needs restart our search heads while I have a long running query still running. When this happens, my search page shows the following message Reading error while waiting for peer . This can be caused by the peer unexpectedly closing or resetting the connection. Search results might be incomplete! If the problem persists, confirm network connectivity between this instance and the peer, and review search.log and splunkd.log on the peer to check its activity. The message notes that "results might be incomplete". Is there a way to tell if they actually are incomplete? ... View more

link textI really want to work with Carbon Black response data in Splunk. While the app will let me run direct queries for things that I already know, Splunk could create conditions that allow me to join on processes and network connections in ways that the Response console is incapable of. The only problem that I have is that Response seems to generate too much data. Following instructions from RedCanary (https://www.redcanary.com/blog/carbon-black-response-splunk-integration/), we tried grabbing process starts and network connections. 6 minutes of data was 1GB --thats on track for 240GB/day (maybe a little less since things should be slower at night) this is more than I can handle (by a lot). Does any one have experience restricting data coming from Carbon Black Response? Any tips or tricks? My understanding is that the Carbon Black event forwarders only filter on event types, which I have already restricted to process starts and network connections. ... View more