You're sending events to a non-existant index on ptc-lpslapp701. You need to create the index. ... View more

No, that is beyond the scope of a simple splunkbase answer I think. It's OK to ask specific questions about specific problems but for very generic "step-by-step" requests like this I think the better option is to dive into the problem yourself first of all and come back if you've encountered specific problems. ... View more

The data you're sending on port 9997 from your UF's is in a format that's very specific to Splunk and is only used for forwarding logs from a Splunk instance (like a forwarder) to another Splunk instance. Your Cisco switches, on the other hand, are probably sending syslog which is a completely different format. In order to receive this, you need to set up a port in Splunk for receiving raw TCP or UDP data (depending on if you're sending TCP or UDP from your switches) and then redirect your switches to send to that port instead. Syslog data is commonly sent to port 514. ... View more

DB Connect IS how you would get data from databases with Splunk. Can you tell us more about why you can't use it? ... View more

I don't entirely follow why you need to split this into separate events? You're not limited to one extraction per event so you can just add all those extractions and have them apply to one single event instead. ... View more

A couple of questions: - Are any other inputs from this forwarder making it to the indexer? Otherwise this might be an output issue rather than input issue. - Did you check the status of the file input? See amrit's excellent script here: http://blogs.splunk.com/2011/01/02/did-i-miss-christmas-2/ alternatively look at the endpoint it uses directly: https:// :8089/services/admin/inputstatus/TailingProcessor:FileStatus ... View more

The problem here is that _time is considered to be an internal field, and as such won't get picked up by the format command that is implicitly run by the subsearch, so it will not be included in the subsearch output. However using a little workaround we can get the output we want anyway, by creating a field called query which will contain the _time filtering string. query is a special field whose value is returned from the subsearch as-is rather than Splunk adding "query=" before it. [search yoursubsearch | eval query="_time="._time | fields query] ... View more

No, you're doing something else wrong in that case. streamstats works perfectly fine with what lguinn suggested. streamstats isn't limited to one field name. I'm not saying you're not having problems because you obviously are, but I think you're jumping to conclusions regarding what the cause to these problems are. ... View more

This is pretty much exactly what subsearches are made for. The docs have excellent explanations on how to use them so I'll just link to those: http://docs.splunk.com/Documentation/Splunk/6.0/SearchTutorial/Useasubsearch ... View more

Just a question: who is saying what? No, using dashboards is not "bad practice" because it's a vital part for many of using Splunk. ... View more

That's trickier, because when running a query from the search head your job server will not be involved at all. I can't think of any way you could throw the job server into the mix if it's not part of the search at all. ... View more

Sure. Just add the index names along with a description to a look and show this in your dashboard. http://docs.splunk.com/Documentation/Splunk/6.0.2/SearchTutorial/Usefieldlookups http://docs.splunk.com/Documentation/Splunk/6.0.2/Knowledge/Addfieldsfromexternaldatasources ... View more

Short answer: no, not natively. You can achieve this, kind of, using dynamic lookups. Longer answer: We have this exact problem in our environment, and I've spent a fair amount of time dealing with it. The problem is, like it says in the post @lukejadamec links to, that when you issue a search on a search head, that search head will put all its knowledge objects into a bundle that will be sent out to its search peers. When the search peers process this search, only the knowledge objects in this bundle will be used. So whatever objects exist locally on the search peer, such as lookups, will/can not be used in the search. While I do not know the full reasons behind why only this mode of operation is supported, one good reason is that since Splunk distributed search operates in a map/reduce fashion, sooner or later when a non-streaming ("reduce type") command is encountered in the search, the search peer is going to return its results to the search head which will carry out the rest of the search centrally. Couple this with knowledge objects that exist on different Splunk instances and the results can be pretty confusing unless you understand fully what is going on. In your example, for instance, only the very first part of the search ( "*" ) would be carried out on the search peers while the head 1 command will cause the rest to be run on the search head, because it's only possible to know which 1 event to present by combining the results from all search peers. If you DO know how to deal with this though, there are ways to work around this limitation. Specifically, what I did was to create dynamic lookups that work similar to Splunk's regular static lookups - i.e. read CSV formatted input on stdin, match this against a CSV formatted file on the file system, populate the requested fields in the CSV from stdin and finally send the result to stdout. What will happen when you use this in a distributed search is that instead of having Splunk send out a static CSV lookup to the search peer it will now send the dynamic lookup script instead. This script is then free to read files on the search peer's local filesystem when it's run, thus making it possible for the lookup to return search peer local values even though it was invoked from a search head. I pasted a version of this dynamic lookup script here: http://answers.splunk.com/answers/85324/regular-expression-in-my-lookup-table It's a modified version that can perform regex matches. But again, NOTE that when you do this it becomes VERY important to keep track of in which order you do things. I've spent many frustrating moments debugging my seemingly not working lookups only to find out that the error was that I had ordered my search incorrectly so the lookup was run on the search head instead of the search peers. As another example, this lookup will run on the search peer: * | lookup mydynamiclookup a OUTPUT b | stats count,values(b) by a While this will run on the search head: * | stats count,values(b) by a | lookup mydynamiclookup a OUTPUT b ... View more

Are other forwarders working in your setup? Also your monitor statements seem to be off. It should be " [monitor:///...] ", not " [monitor:/...] ". ... View more

You don't need any particular addon to receive and index the data - Splunk will happily index whatever text you throw at it. You might however want to install relevant addons for making sense of Cisco data so you get proper fields out of events etc. ... View more

There's an entire site and a specific tutorial with info on how to do this. http://dev.splunk.com/view/rest-api-tutorials/SP-CAAADQ6 ... View more

Asking us to do all the work for you is in my eyes pretty rude. Provide us with info on your nullQueue setup with relevant sections from the conf files you were talking about yourself, then we can have a look and see where you might be going wrong. ... View more

Only if there is a way to output that info from the firewall - Splunk will happily ingest the information if so, but as you can't put a forwarder on these devices it's really up to what you can configure the devices to send to Splunk via syslog. ... View more

The problem here is likely that you don't filter out which fields the subsearch should evaluate when returning its results to the main search. By default, subsearch will take all fields in the final output of the search and create AND:ed filter conditions out of them. So for instance if you have one event with fieldA="1" and fieldB="2", and another with fieldA="A" and fieldB="B", the subsearch will create a filter looking something like this out of it: (((fieldA="1") AND (fieldB="1")) OR ((fieldA="A") AND (fieldB="B"))) But in practice, events don't just carry just a few fields like this - they also always have index, sourcetype, source, host, probably date_hour, date_wday and so on. Unless you filter out these before the end of subsearch, the values for ALL these will be included in the type of filter string shown above. You can have a look yourself at exactly what the subsearch will actually emit to the outer search by running the subsearch on its own and add " | format " at the end (this is actually what the subsearch does implicitly). index=daily remark=internet dstip="*" | format The cure is to choose which fields should be evaluated using the fields command, so that only the fields that should be included in the filter string are there. I also threw "top" in there below so you only get the 100 unique top talkers. Adjust as you see fit. index=daily [search index=daily remark=internet dstip="*" | top 100 dstip | fields dstip] ... View more

Use the "manual" option where you choose sourcetype in the web interface and specify "squid" yourself. ... View more