About DaClyde

DaClyde · ‎01-26-2015

Ok, so I guess this answers question one: http://docs.splunk.com/Documentation/Splunk/latest/DistSearch/Forwardsearchheaddata Not sure how I kept missing that. Still looking for help with question two.

DaClyde · ‎01-26-2015

We have a slightly odd architecture as we have a single search head, a single indexer and multiple forwarders (for the sake of arguement, all running 6.2.1 after several upgrades from prior versions). All of the current documentation only seems to address scenarios of a search head pool and indexer clusters, but nothing seems to allow for just one of each server type. The issue I have is that the search head is indexing its own events (_audit, _internal and _introspection indexes are active), rather than forwarding them to the indexer. Is it just a simple matter of adding an outputs.conf to push all search head activity to the indexer? If so, what would I put in there? Also, is there a way to make the indexer's indexes appear in the "Indexes" page of the search head, instead of seeing a pile of empty search head indexes?

DaClyde · ‎08-14-2014

There is a 50MB difference in package size when downloading the Splunk installer vs the Universal Forwarder installer. If there is that much difference, they are not the same product. I'm just asking since they are distributed as two separate pieces of software, they should have separate release notes.

DaClyde · ‎08-14-2014

I would hope for something more explicit that specifically calls out the universal forwarder itself, and not just the general forwarding capability of Splunk as a whole. Our Information Assurance people tend to err on the side of "when in doubt, upgrade". I would love to have something that actually states whether or not the upgrade is necessary, specifically for the universal forwarder. Basically if we are not experiencing any functional issues with our universal forwarders, the only reason we would ever need to upgrade would be for security issues.

DaClyde · ‎08-14-2014

Since the Universal Forwarder is a separate package from the main Splunk install, could we please get a separate Release Notes page just for the forwarder so we can know definitively whether or not the forwarder has changed from one release to the next? If all that has changed is the version number, we would love not to waste time needlessly upgrading forwarders that don't need it. If we're running 6.0.4 forwarders, is there any technical reason to upgrade to 6.1.3 forwarders? This really needs to be spelled out, in no uncertain terms, with each universal forwarder release.

DaClyde · ‎01-06-2014

JAVA_HOME typically needs to point at least one folder deeper, C:/Program Files/Java/jre7.

DaClyde · ‎12-17-2013

We took a shot at hiring a consultant a couple of years ago, but the experience was disappointing and resulted in no payment. Having been through a poorly coordinated consultant experience, I would recommend the following steps if you are looking into this: 1) Have very clear goals for what you are hoping to accomplish and for what you need assistance from a consultant (server configuration, deployment scenarios, dashboard ideas). If you know what you want to do, it goes a long way toward helping both Splunk and the consultant reach your goals. Don't be afraid to draw pictures of what you want your dashboards or tables to look like. 2) If coordinating through Splunk professional services, make absolutely sure both Splunk AND the consultant are all on the same page as to what your environment and project involve, so the consultant should be able to hit the ground with at least a superficial idea of what you are trying to accomplish. We had no contact with our consultant beforehand and he arrived having absolutely no idea what we needed (having apparently received none of the information we provided to Splunk), and then spent half the week working for his full-time customer, and we got nothing out of it. We ended up not paying. If you can make a semi-coherent case for both of the above, you should be able to get more out of the experience than we did. Also, I'd say go through and watch a lot of the customer videos under the Resources section of the main Splunk.com website. Seeing how other organizations are using Splunk can help give you lots of ideas of what you might accomplish. We actually want to try hiring a consultant again once our management signs off, and chalk up the previous experience to just an unfortunate confluence of events.

DaClyde · ‎11-05-2013

Sure, that would be great.

DaClyde · ‎11-05-2013

That is correct.

DaClyde · ‎11-05-2013

It is enabled. I even deleted the index, re-created it, restarted the indexer, flushed the fishbucket on the forwarder and restarted that, but I'm still getting the same error. If I generate traffic on the forwarder, I can see new entries pop up in the metrics.log on the forwarder, so it's trying to work.

DaClyde · ‎11-05-2013

The timestamps look good on the file I manually imported. Searching all time, and even adding a latest=+10d doesn't find any stray data outside the one manual import.

DaClyde · ‎11-05-2013

To append to that, the adviis index does exist because I can see yesteday's log (that I manually imported through the Manager>Data Inputs page) if I do a "search index=adviis sourcetype=adviis" for the last 24 hours.

DaClyde · ‎11-05-2013

For now, my inputs.conf on the forwarder only has the monitor one stanza. I've added your recommended stanzas to my props and transforms on the indexer, but so far, no luck. Now I'm getting this message on the search head: Search peer SPLUNK402 has the following message: received event for unconfigured/disabled/deleted index='adviis' with source='source::F:\inetpub\logs\LogFiles\W3SVC1\iis_D20131105-001238183.log' host='host::Weeble' sourcetype='sourcetype::adviis' (1 missing total)

DaClyde · ‎11-05-2013

I am experiencing an issue where my universal forwarder (v5.0.4) is not forwarding my IIS Advanced Logs to the indexer. Here is the stanza from my inputs.conf [monitor://F:\inetpub\logs\LogFiles\W3SVC1\] disabled = false whitelist = iis_D(\d+)-(\d+).log sourcetype = adviis index = adviis I can tell it sees the logs because I get these entries in my metrics.log: 11-05-2013 14:28:01.726 -0600 INFO Metrics - group=per_index_thruput, series="adviis", kbps=0.012349, eps=0.161290, kb=0.382813, ev=5, avg_age=0.600000, max_age=3 11-05-2013 14:28:01.726 -0600 INFO Metrics - group=per_source_thruput, series="f:\inetpub\logs\logfiles\w3svc1\iis_d20131105-202617637.log", kbps=0.012349, eps=0.161290, kb=0.382813, ev=5, avg_age=0.600000, max_age=3 11-05-2013 14:28:01.726 -0600 INFO Metrics - group=per_sourcetype_thruput, series="adviis", kbps=0.012349, eps=0.161290, kb=0.382813, ev=5, avg_age=0.600000, max_age=3 And my splunkd.log shows it watching the folder: 11-05-2013 13:38:04.363 -0600 INFO TailingProcessor - Adding watch on path: F:\inetpub\logs\LogFiles\W3SVC1. So why is nothing showing up in my index? I can forward WinEventLogs and standard IIS logs with no issue between these two machines. I even manually imported one of these logs into the indexer just to make sure the "adviis" index and sourcetypes existed (I know that shouldn't be necessary). I've cleared the fishbucket multiple times, but these files just won't budge. UPDATE Shane's answer accounted for everything but my ineptitude and ignorance about how the search head relates to an indexer. From the Manager in the web interface, if you create an index, it is going to create the index on the machine that web UI represents. In my case, I was just creating the adviis index on the search head. No amount of forwarding to the indexer is going to find that index. To get everything lined up properly, I had to delete the adviis index from the search head and delete the corresponding adviis index folder from /Splunk/var/lib/splunk/. Then I deleted the assorted attempted copies of the same folder and indexes.conf files that had been generated. Then (after finally remembering the login for the indexer), I logged into the web UI on the indexer and created the adviis index through the Manager that way. Now all the bits and pieces were in place, I flushed the fishbucket on my forwarder and data started moving. So all of Shane's advice was correct, if I had created the index properly in the first place. Thank you Shane!

DaClyde · ‎10-03-2013

Thank you, I'll get my files cleaned up and keep an eye on that in the future. Now I understand the "DO NOT EDIT THIS FILE!" warning. I didn't think it made sense given the context.

DaClyde · ‎10-02-2013

From the top of the file: Copyright (C) 2005-2011 Splunk Inc. All Rights Reserved. Version 4.3.1 DO NOT EDIT THIS FILE! Please make all changes to files in $SPLUNK_HOME/etc/system/local. To make changes, copy the section/stanza you want to change from $SPLUNK_HOME/etc/system/default into ../local and edit there. This file contains possible attributes and values you can use to configure Splunk's web interface.

DaClyde · ‎10-02-2013

That took care if it, thank you. Mine was set to False. I can now access my data. Should the conf files all have the version of the current Splunk install? My web.conf listed 4.3.1 (I've gone through several upgrades on this machine).

DaClyde · ‎10-02-2013

I get the following error for all of my searches after upgrading from 5.0.4 to 6.0: Write access to the proxy endpoint is disabled. Anyone have any idea what this means? I'm just running on a standalone Win7 machine, no forwarders or anything exotic involved.

DaClyde · ‎09-18-2013

As best as I can make out, the only way to make this work (since there is no configurable option in the search head as to where its ETC folder resides) is to use a symbolic link and mount the copy of the etc folder on the shared storage to where the search head would normally expect the etc folder to reside. In Windows, there is now a mklink command for mounting a network share directly into the folder structure you want, similar to how *nix has always had. If your knowledge bundle is here: \\NAS\KB\etc\ On the Windows you would use mklink something like this: navigate to your %SPLUNK_HOME% location (something like C:\Program Files\Splunk) then execute the command from there: mklink /D etc "\\NAS\KB\etc\" So now, your C:\Program Files\Splunk\etc folder is actually a link to \\NAS\KB\etc, but the search head just sees it as a local folder (provided all the permissions are set properly to allow the search head to write to the shared storage). Someone please correct me if this is incorrect. This was the best I could work out since the Splunk documentation always assumes the end-user already knows how to do everything non-Splunk related.

DaClyde · ‎05-10-2012

You would need to use two different browsers (i.e. Internet Explorer and Firefox/Chrome/Safari), not two instances of the same browser (two Internet Explorer windows).

DaClyde · ‎05-10-2012

Ok I finally tracked down the culprit for the monitored folder. We had attempted to change the "sourcetype" to a manual setting, which Splunk couldn't interpret. Setting it back to "Automatic" cleared up the automated import issue. That setting changed sometime after our 4.2.5 upgrade but before the 4.3.1 upgrade. The single file import still shows a mess in the preview, but we got the automated import working again.

DaClyde · ‎05-10-2012

Posting this as an answer to have more characters to work with. The issue seems to be with the preview feature when attempting to manually import an individual file. In an attempt to see where this issue cropped up, I cleaned Splunk off my laptop, and re-installed 4.0.11. I created a data input watching an empty folder. When I dropped an evt file in there, it recognized it and parsed it properly into the index. So I upgraded from 4.0.11 to 4.1.8, cleared the indexes and tried it again. Success. So I upgraded from 4.1.8 to 4.2.5, and tried the same process, and again it read the file and properly showed the events. So I upgraded from 4.2.5 to 4.3.2 and it STILL recognizes the files if I drop them into my watched folder. However, neither 4.3.1, nor 4.3.2 have any idea what to do with an evt file if I attempt to import an individual file manually. In that scenario, I get the gibberish as I posted in the initial question.

DaClyde · ‎05-09-2012

We got stuck using 4.0.11 for a very long time, but during that time, it had no trouble importing exported Windows Event Logs in evt format. Now that we have upgraded through 4.2.5 to 4.3.2, Splunk will recognize Windows Event Log data from a forwarder, but when attempting to manually import an evt file, the preview show this: Failed to decode 2155 bytes: source::C:\workbench\logs\jkreceive.evt|host::SPLUNK01|preprocess-winevt| 0\x00\x00\x00LfLe\x1\x00\x00\x00\x1\x00\x00\x000\x00\x00\x00D\xFE\xFF\x00\x8F\xEE\x15\x00\xF4R\x15\x00l\xFE\xFF\x00\x00\x00\x00\x00\x00\x00\x00\x000\x00\x00\x00\xD8\x1\x00\x00LfLe\xF4R\x15\x00߱aO߱aO\x00\x00\x00\x00\x4\x00\x1\x00\x00\x00\x00\x00\x00\x00\x00\x00d\x00\x00\x00\x00\x00\x00\x00d\x00\x00\x00\x00\x00\x00\x00\xD2\x1\x00\x00J\x00K\x00R\x00e\x00c\x00e\x00i\x00v\x00e\x00\x00\x00J\x00T\x00D\x00I\x00-\x00R\x00S\x00A\x00-\x000\x009\x00\x00\x00F\x00i\x00l\x00e\x00 \x00r\x00e\x00c\x00e\x00i\x00v\x00e\x00d\x00 \x00f\x00r And all the "events" just get piled into a single day. Events from the local machine's event log are imported fine. I'm seeing this both on a Win7 laptop I'm using for testing 4.3.1, as well as a Windows 2008 server running 4.3.1. Back when we were running 4.0.11, we had a data input set for a particular directory, and we would just drop exported evt files in there and Splunk had no trouble consuming them. We need that capability back. It doesn't seem to matter what OS the evt files are from as I've tried it with files exported from XP, 2k3, 7 and 2k8 and get the same results for both evt and evtx files. What changed and how can we fix it?

DaClyde · ‎07-03-2011

Ah, that works much better. Thank you!

DaClyde · ‎07-01-2011

I'm working with a custom Windows EventLog that contains successful file upload events. I'm attempting to extract just the filenames from the full UNC path that is listed in the event. Most of the paths are about 3 folders deep, like this: Message=File received from user: user1 - file name: \\server\category\type\loca-tion\filename.zip size: 1157112 date: 5/17/2011 4:48:50 PM Message=File received from user: user1 - file name: \\server\category\type\lo-cation\filename.zip size: 1157112 date: 5/17/2011 4:48:50 PM Message=File received from user: user1 - file name: \\server\category\type\location\filename.zip size: 1157112 date: 5/17/2011 4:48:50 PM The field extraction in Splunk (we're stuck at 4.0.11 for now and won't be able to upgrade to 4.2 for at least a month), gives me a regex of this: (?im)^(?:[^\-]*\-){4}\w+\\(?P<FIELDNAME>[^ ]*)[ ] Unfortunately, this catches all the files in folders with hyphens, but misses all files in folders with no hyphen. What needs to be removed from this regex so it catches ALL of the files from the above sample structures? --edit-- For some reason the slashes all seem to be missing from my post. They show up in the preview, but most of them in the UNC paths are missing in the actual post. As a result, what I'm asking about probably makes no sense. Do we have to escape slashes in the posts here to get them to show up? Thanks!

Posts	130
Solutions	4
Karma Given	47
Karma Received	30
Member Since	‎10-22-2010

Online Status	Offline
Date Last Visited	‎03-11-2026 11:50 AM

"Layout undefined is not defined" in Dashboard Stu...

Possible to output a literal string with backslash...

Splunk App for Lookup File Editing - REST Handler ...

Searching for "-" in IIS logs?

Re-arranging Apps menu in Splunk 9.2.x

Where to reallocate CPU cores, SH or IDX?

How to keep users out of Search?

How to use a radio button to run different searche...

Cannot re-assign ownership of phantom orphaned sea...

regex help with field extraction

Re: How to configure search head to forward intern...

How to configure search head to forward internal d...

Re: Can we get a change log just for the Universal...

Re: Can we get a change log just for the Universal...

Can we get a change log just for the Universal For...

Re: Java home does not work

Re: Any good experience with Splunk consultants?

Re: Problem forwarding Advanced IIS Logs

Re: Problem forwarding Advanced IIS Logs

Re: Problem forwarding Advanced IIS Logs

Re: Problem forwarding Advanced IIS Logs

Re: Problem forwarding Advanced IIS Logs

Re: Problem forwarding Advanced IIS Logs

Problem forwarding Advanced IIS Logs

Re: "Write access to the proxy endpoint is disable...

Re: "Write access to the proxy endpoint is disable...

Re: "Write access to the proxy endpoint is disable...

"Write access to the proxy endpoint is disabled. "...

Re: Mounted Bundle path - Search Head

Re: Log into from different browsers using differe...

Re: After upgrade, Splunk can no longer read evt f...

Re: After upgrade, Splunk can no longer read evt f...

After upgrade, Splunk can no longer read evt files

Re: trouble extracting field, filenames from UNC p...

trouble extracting field, filenames from UNC paths

Join the Conversation