You cannot have multiple REGEX parameters in transforms.conf for the same stanza. You almost have it correct with breaking this into 2 transforms, but they need to have unique names. So here's how you would split into 2 and call them from props.conf -----props.conf----- [mysourcetype] TRANSFORMS-foo = WindowsLogonEvent675_Part1, WindowsLogonEvent675_Part2 -----transforms.conf----- WindowsLogonEvent675_Part1] REGEX = (?msi)EventCode=4624.Account Name:\s(-) DEST_KEY = _TCP_ROUTING FORMAT = forwardauqldrv00mgt1ai [WindowsLogonEvent675_Part2] REGEX = (?msi)^EventCode=(632|4719|4728|4729|4670) DEST_KEY = _TCP_ROUTING FORMAT = forwardauqldrv00mgt1ai ... View more

Yes, this is possible. However, if you have 2 separate servers it may be best to keep both and have one distribute searches to the other. This way you are effectively searching both Splunk servers and get the added bonus of 2 servers sharing the work and executing in parallel. More on distributed search if this interests you: http://www.splunk.com/base/Documentation/latest/Admin/Whatisdistributedsearch. If, however, you are looking to re-purpose one of the servers and truly need to consolidate your datastore, then the process is similar to backing up your Splunk datastore, covered here: http://www.splunk.com/base/Documentation/latest/Admin/Backupindexeddata. This is the skeleton process (assuming you have enough storage): Redirect the incoming data stream to Splunk1 Shut down Splunk2 Roll the hot bucket on Splunk2 to grab the latest data Move all buckets to Splunk1 after ensuring bucket sequence ids are unique Repeat steps 3 and 4 for each index. Steps 1 and 2 are self-explanatory. For step 3, you can issue this command on the CLI: ./splunk _internal call /data/indexes/<index_name>/roll-hot-buckets –auth <admin_username>:<admin_password> For Step 4, on Splunk1 and Splunk2, look in $SPLUNK_HOME/var/lib/splunk/defaultdb/db $SPLUNK_HOME/var/lib/splunk/defaultdb/colddb The directories in these folders all have a unique sequence ID at the end of the directory name: db_#_#_id You need to ensure all the directories in Splunk1 and Splunk2 have a unique ID. Write a script or change the sequence ID manually if there are any duplicates between Splunk1 and Splunk2. Then move all the directories from Splunk2: $SPLUNK_HOME/var/lib/splunk/defaultdb/db Splunk2: $SPLUNK_HOME/var/lib/splunk/defaultdb/colddb to Splunk1: $SPLUNK_HOME/var/lib/splunk/defaultdb/colddb ... View more

Is your preference not to use Splunk as your 12-month datastore? Splunk can retain all or any data for as long as you want (provided you have adequate storage capacity). It is simple to set a time-based retention policy instructing Splunk to retain the data for no less than 12 months. If you want to retain the data outside of Splunk, then there is no way to configure the batch processor to index and not delete. Your original use of the monitor input is the better option in this case. Are you by chance using the Light Forwarder? If so, it has a setting to limit the size of output stream. In $SPLUNK_HOME/etc/apps/SplunkLightForwarder/default/limits.conf: [thruput] maxKBps = 256 This could be why you are seeing very slow uptake of the data in your monitored directory. You can set this higher to increase the output rate. Also, you might want to check the number of files in the monitor directory. Are they compressed? The number of files and whether they are compressed will also have an impact on the processing. ... View more

You are correct--the original data source can be removed after indexing. Splunk no longer requires it as a separate copy is now stored in the Splunk datastore. Having said that, you should ensure the appropriate protections are taken to protect and preserve the data in Spllunk should there be any software/hardware failures. ... View more

These are the 2 options I would try configuration files rex command in the search bar The easiest, but also most transient, option is to use rex command inline in your search. For example: sourcetype="multiline" | rex "CLOSE, loaded in (?<close_pe_rt>\S+)" | rex "FX_CLOSE, loaded in (?<fx_close_pe_rt>\S+)" | rex "XLA_ENV, loaded in (?<xla_env_pe_rt>\S+)" | rex "INTRADAY, loaded in (?<intraday_pe_rt>\S+)" | rex "CPTY_CREDIT, loaded in (?<cpty_credit_pe_rt>\S+)" Maybe there's a way to do this in one rex invocation, but I tried several things which didn't work. The other option is to add a few stanzas to props.conf and transforms.conf. For example, in props.conf: [multiline] REPORT-foo = mlFields in transforms.conf: [mlFields] REGEX = CLOSE, loaded in (\S+).* FX_CLOSE, loaded in (\S+).* XLA_ENV, loaded in (\S+).* INTRADAY, loaded in (\S+).* CPTY_CREDIT, loaded in (\S+) FORMAT = close_pe_rt::$1 fx_close_pe_rt::$2 xla_env_pe_rt::$3 intraday_pe_rt::$4 cpty_credit_pe_rt::$5 You could also try using the Interactive Field Extractor (IFX). ... View more

You can simply copy (or move) the directories where the summary indexes live to the new box in the same location. In your Splunk home directory where Splunk is installed look in var/lib/splunk. There should be directories for each of the summary indexes: summary_daydb summary_hourdb summary_minutedb To be safe, shut down Splunk then move the directories. Make sure your new Splunk instance is also stopped. Then you can start it up after the move completes. Also, be sure to create the indexes using SplunkWeb or transfer the entries from indexes.conf. ... View more