Hi Michael, You are right that universal forwarders cannot extract or transform the data you want to forward. If you just want to process raw data to meet your specific requirement and forward the tailored data to a destination without indexing it, you just need to install a heavy forwarder - no indexer needed. To deploy a heavy forwarder, you install the full Splunk instance as an indexer, but configure it to use the forwarder license: In Splunk Web, select Settings > Licensing from the menu. Click Change license group and choose Forwarder license. Click Save. To transform your raw data - e.g., rewrite "date:ipaddress" as events, define and use the transforming processor. Here is an example: props.conf [source::...\my.log] TRANSFORMS-dateip = dateip transforms.conf REGEX = .*(\d{1,2}\/\d{1,2}\/\d{4}).*(?:[0-9]{1,3}\.){3}[0-9]{1,3}) DEST_KEY = _raw FORMAT = $1:$2 The transforming processor captures dates ($1) and IP addresses ($2) from the original raw data and rewrites the _raw data using the captured data in the format you want. Hope this helps. Thanks! Hunter ... View more

If you have already ingested and indexed the events...why are you syslogging them at all? You already have them in Splunk, the greatest software on the planet. ... View more

try like : your_base_search|stats list("fields name Event code") as " Event code" by "fields name of text description" |.... ... View more

Another side note, this post from @Gilles http://unix.stackexchange.com/questions/10735/linux-allowing-an-user-to-listen-to-a-port-below-1024 provides three possible solutions for this. From my point of view the docs http://docs.splunk.com/Documentation/Splunk/6.3.2/installation/RunSplunkasadifferentornon-rootuser should be modified in this regard - I'll ping @docs ... View more

thanks. this is a start ... View more

this link can help you: http://docs.splunk.com/Documentation/Splunk/6.3.2/Knowledge/WhatisSplunkknowledge ... View more

If Splunk can't support this as a "production stable" functionality then it shouldn't be in the product IMHO. Regardless the questions asked were answered. Down vote all you want. We both know Splunk isn't designed to be a syslog forwarder. ... View more

I would say DO NOT do any conversion. Splunk's core feature is the fact that it can take data of varying formats and styles and, thanks to scheme-on-the-fly and field extraction, allow you to compare and manipulate fields that represent the same data, regardless of the format. Transforming prior only reverts back to the ETL world and manifests dependencies. Take advantage of what splunk does best. Over time you'll see that the format of the individual events is not important because you're manipulating fields and looking for trends and values of those fields. At that point you rarely look at the events because you use stats-type commands. ... View more

Allow me a slightly different take, though still echoing the general principles. In a smaller environment you can install and use syslog (either r-syslog or syslog-ng) on the same box where Splunk is installed. It's not ideal, but honestly the needs of syslog are small and as long as you log-rotate properly you shouldn't significant disk space requirements beyond what the data needs in Splunk. If you do that, you do not need a forwarder and can simply have Splunk read the files directly from the local filesystem. Simple and straightforward. ... View more

Yes of course.Start here to learn how to configure the application http://docs.splunk.com/Documentation/StreamApp/6.4.1/DeployStreamApp/AboutSplunkAppforStream http://docs.splunk.com/Documentation/StreamApp/6.4.1/User/ConfigureStreams ... View more

oh ok. like a SAN disk or something. Got it. Thanks. ... View more

I'd suggest you read through this and familiarize yourself with how Splunk manages index data for you. Data is stored in buckets, which go through multiple stages, namely from HOT (actively written to) to WARM (read-only) to COLD (read-only) to FROZEN (not searchable). You have full control over how long you keep data in each stage, either by specifying a time period or by limiting how much disk space each stage can consume. This is fully automatic, but it DOES NOT replace backing up your data, if you are concerned about data loss. Take a gander at this for best practices around backing up your index data and/or consider using index replication to guard against indexer/data loss. ... View more