If I were you, I would put these regexes into EXTRACTs in props.conf under their respective source/sourcetype stanza, instead of doing it via rex in the search. props.conf [aaa] EXTRACT-filename_a = Filename was \[(?<filename>[^\]]+)\] [bbb] EXTRACT-filename_b = uploaded file \[(?<filename>[^\]]+)\] [ccc] EXTRACT-filename_c = download file \[(?<filename>[^\]]+)\] Then you can make searches like; sourcetype=aaa OR sourcetype=bbb OR sourcetype=ccc | table filename EDIT: clarification and typos fixed. ... View more

locate the monitor stanza in inputs.conf (there can/will be more than one inputs.conf file), and make it like so; [monitor:///var/log/] index=abc sourcetype=xyz blacklist = /\.[^\\/]+\.swp$ /K ... View more

I didn't really understand you data, but the follwing rex will extract the username part of a domain\user type string. Assuming the field is called "domain_user" and contains the value acme\bob ... | rex field = domain_user "[^\\\\]+\\\\(?<user>.*)" should extract bob into the field user . /K EDIT: corrected the number of backslashes required. ... View more

There might be some confusion of terminology. Basically an index is a 'db', where the defaultdb is the main index. $SPLUNK_DB usually refers to the 'root' directory for where the indexes are stored (default paths are /opt/splunk/var/lib/splunk/ on *nix and c:\program files\splunk\var\lib\splunk on Win). You can set up new indexes, as it seems you have, either in the GUI or manually in indexes.conf. You can then route incoming event into this index, normally by setting the index=blahblah in inputs.conf under a monitor stanza. Moving a whole index is not particularly hard. Simplest is to stop splunk, move the index directory to the new location, edit indexes.conf to reflect the new location (you can use absolute paths, not just relative to $SPLUNK_DB ). Restart splunk. However, it is quite hard to move data (individual events) from an index to another once it's there, but I don't think that was your question. UPDATE: Hmm, I'm still not sure what your situation is; you say that you have created some indexes (index1, index2 etc) in the defaultdb. I'm starting to think that maybe you correctly created indexes, but stored them in the directory that belongs to 'main', i.e. /opt/splunk/var/lib/splunk/defaultdb/yourindex. If so, and if that is even possible without splunk complaining, it should be just as easy to move anyway. Just make sure that you don't move anything that belongs to defaultdb. All indexes should have the following subdirectories under it, colddb, db and thaweddb . Nothing more. The actual events within an index are stored in $SPLUNK_DB/indexname/db or colddb/bucketfolder/rawdata/journal.gz The index (I just love the naming conventions) that make the event data searchable is the $SPLUNK_DB/indexname/db or colddb/bucketfolder/*.tsidx file (or files). I have never tried to move anything out of this type of location in order to place it in another index. Perhaps it would be a good thing if you clarified in which way your events are 'in the wrong db'. /K EDIT. Typos, clarification, etc. ... View more

You should probably try something like [monitor://g:\blackberry enterprise server\logs\*\*MAGT*.txt] Strptime date variables are not supported in monitor-stanzas. /K ... View more

You should probably have a look at this section of the docs; http://docs.splunk.com/Documentation/Splunk/6.1.3/Data/Advancedsourcetypeoverrides It deals with sourcetypes and hosts, but you could just as easily use the method to rewrite the index. If your events normally go to the index 'blah' and you just want to re-route the 'varnishncsa' you'd do it like this; inputs.conf [monitor:///path/to/file] index=blah sourcetype=bob props.conf [bob] TRANSFORMS-chidx = reroute_varnish transforms.conf [reroute_varnish] REGEX = ^varnishncsa FORMAT = varnishncsa DEST_KEY = _MetaData:Index Or you could do it dynamically, i.e. re-route all events to an index that matches the first word/string in each event. Just make sure that the indexes actually exist first - they will not be dynamically created; props.conf [bob] TRANSFORMS-chidx = dyn_idx transforms.conf [dyn_idx] REGEX = ^(\S+)\s* FORMAT = $1 DEST_KEY = _MetaData:Index Haven't tried the last alternative, since that can be slightly unpredictable. UPDATE: So if the data is coming in via syslog, I guess you should do it in inputs.conf; [udp://514] sourcetype=access_combined_wcookie index=blah connection_host= ip OR dns. see the docs. no_appending_timestamp = true This is assuming that you don't have other types of data coming in via that port. In that case you configure Splunk to listen on a dedicated port just for this traffic (if you can configure your web servers to send to e.g. udp:10514). In props.conf you call for transformation; [access_combined_wcookie] TRANSFORMS-set_index = awc_change_index and in transforms.conf you do the REGEX, FORMAT and DEST_KEY as discussed above. See http://docs.splunk.com/Documentation/Splunk/6.1.3/Admin/Inputsconf /K ... View more

Well, it's not entirely clear what you want to achieve. Sorting and grouping is usually performed on fields . Say that you want to group all errors on the type of error ("Unexpected error", "Invalid date" or "SDP lookup failed" in your example), you need to extract this part of the message as a field (let's call it errType ) and whatever comes after it is extracted as errMsg . This can be done in config files or inline in the search query. We'll do the latter here. ERROR source="/home/logs/DataTransferService.log" | rex "\sERROR\s-\s(?<errType>[^\.\r\n:]+)(?<errMsg>.*)" | eval niceTime = strftime(_time, "%F %T") | stats list(niceTime) list(errMsg) by errType This should result in an output like Unexpected Error 2014-06-20 11:22:23 com.eMeter.PIPe.datatransferblahblahblahb 2014-06-21 12:23:34 some.other.errormessage Invalid date 2014-06-23 22:32:21 Blah Blah Blah error and so forth. Maybe something like that is what you're looking for? /K ... View more