I am testing splunk. When I do what I consider a very simple search I get the wrong results. Let me say this: our files are random in format and we can not expect them to be in any simple format like log files. I tried setting the source_type as "misc_text" and that seems to work sometimes. Is there a place where the source_type is described?
I've queried on a simple string and get some results. In the output I see another string. Ok, let's search for that string: no dice. Why would that be?
This thing sure ain't idiot proof!
Here are some pages that may be helpful to you:
Keep following the links, and you'll start figuring it all out. Also check out the online videos, tutorials, and other more official training options are available too.
Many specific questions have already been asked here, and if you can find the answer your looking for ask a new question. (The more detailed the more likely you are to get the right answer quickly.)
Please give more detail about what it is that you are trying to do. And as far as your last comment, I would say that no tool worth using is "idiot proof"; a hammer, screwdriver and a saw are all very simple tools that are easy to use, but if placed in the wrong hands, they can lead to great damage.)
Maybe Splunk isn't the right tool for what you are trying to do. It sounds like you are looking for a database which can read in random text files and let users search them. Splunk is really focused on events. Sure it can do searches of data in random text files, but that isn't really its focus.
If you describe your requirements in a little more detail maybe we can help you find the right tool for what you are trying to do. (Maybe something like JackRabbit? http://jackrabbit.apache.org/ )
Or are you looking for something to monitor text messages and emails for keywords? Or something else?
If you have an event, something which occurs at a specific time and place in your systems, you won't find a better tool than Splunk to record, report, and alert on that event (IMO). It may not be the right sort of database you are looking for though.
I have not used JackRabbit myself. I started to look closely at it for a project that dematerialized before it even got rolling. It seems to be the tool of choice if you want to index a lot of documents. (which is different than events) I'd suggest looking for a python equivalent, since you are a python guy, but the project at http://www.pycr.org doesn't seem to have a lot of traction. (any, actually)
I think you understand. We have a crude tool (I wrote it -LOL) used to search many, many very large text files. If it was 5 times faster, it would be ideal or if it did indexing (I started working on one in Python) it would also be do the trick. "Events" are not as critical as matching strings and then looking for the time the "event" occurred. What else can you tell me about Jack Rabbit? and thanks for your help everyone
If you are using the file system change:
[fschange:/etc/app/logs/] sourcetype = misc_text
For monitoring the files/folders:
[monitor:///etc/app/logs/] sourcetype = misc_text
A great resource to find out more about the inputs.conf is: http://www.splunk.com/base/Documentation/4.1.4/admin/Inputsconf