Hexx....I am running into an issue with SoS not being able to fill in the host to query table. Below is the question I posted even after reading your detailed steps here. Any thoughts on why the Indexing hosts do not populate the drop-down? Just getting a "dns" entry. Getting detailed data from the Search head where SoS is installed works like a charm, but pulling data from the Indexers is non-existant. My Splunk version is 4.3.2; SoS v2.1 and Sideview Utils have been tested with both v1.3.4 & v1.25. http://splunk-base.splunk.com/answers/52655/splunk-on-splunk-not-filling-in-server-to-query ... View more

I have an "error-string" and need to alert when I find it not only in the first 10 minute check; not only in the second 10 minute check; but when I run a third consecutive check and find the "error-string" in all three. Is this possible? I looked at alert throttling in 4.3.x, but that won't meet my needs. There must be some complex search string that allows for a check of "error-string" every 10 minutes, but don't alert until the third straight positive event. Thoughts? ... View more

I apologize for the Splunk formatting of my configuration settings....for some reason this UI is removing my "_" from all my configuration lines....even when typing manually versus copy in from a *nix window. So please be aware these are not typos but what happened when I copied the lines in. +++++++++++++++++++++++++++++++++++++++++ I have been working on many solutions to this line break issue and will turn to the Answer Gawds for help. The data I have looks like the following: 25 Apr 2012 14:38:39.430 [INFO] [WorkerThread45] [com.blah.accessor.LeadIntakeManagerAccessor] - RealTimeCall Bean message received is: <LiveCall> blah blah whole bunch of lines, but ending with the following LiveCall line blah blah </LiveCall> I have tried iterations of the following to get the data to break after the last line in each event....." </LiveCall> " but before the date which was listed first. +++++++++++++++++++++++++++++++++++++++++ Example 1 - (output from btool listed at very bottom) props.conf [source:///flocal/logs/netkernel/intakeform/emaillead.log] MAX_TIMESTAMP_LOOKAHEAD=24 LINE_BREAKER = "<\/LiveCall>" +++++++++++++++++++++++++++++++++++++++++ Example 2 [source:///flocal/logs/netkernel/intakeform/emaillead.log] MAX_TIMESTAMP_LOOKAHEAD=24 MUST_BREAK_AFTER = "<\/LiveCall>" +++++++++++++++++++++++++++++++++++++++++ Example 3 [source:///flocal/logs/netkernel/intakeform/emaillead.log] MAX_TIMESTAMP_LOOKAHEAD=24 ++++++++++++++++++++++++++++++++++++++ Output from '''splunk cmd btool props list --debug''' system [source:///flocal/logs/netkernel/intakeform/emaillead.log] system ANNOTATE_PUNCT = True system BREAK_ONLY_BEFORE = system BREAK_ONLY_BEFORE_DATE = True system CHARSET = UTF-8 system DATETIME_CONFIG = /etc/datetime.xml system HEADER_MODE = system LEARN_SOURCETYPE = true system LINE_BREAKER = <\/LiveCall> system LINE_BREAKER_LOOKBEHIND = 100 system MAX_DAYS_AGO = 2000 system MAX_DAYS_HENCE = 2 system MAX_DIFF_SECS_AGO = 3600 system MAX_DIFF_SECS_HENCE = 604800 system MAX_EVENTS = 256 system MAX_TIMESTAMP_LOOKAHEAD = 24 system MUST_BREAK_AFTER = system MUST_NOT_BREAK_AFTER = system MUST_NOT_BREAK_BEFORE = system SEGMENTATION = indexing system SEGMENTATION-all = full system SEGMENTATION-inner = inner system SEGMENTATION-outer = outer system SEGMENTATION-raw = none system SEGMENTATION-standard = standard system SHOULD_LINEMERGE = True system TRANSFORMS = system TRUNCATE = 10000 system maxDist = 100 +++++++++++++++++++++++++++++++++++++++++ So after trying many, many "sure things" Splunk has decided not to play nicely. Any thoughts on what I might have missed? ... View more

I have two versions of Splunk, v4.3.1 & v4.1.4 Indexing the same data, but only v4.3.1 indexes as a single line event, which is correct. What is the best method to determine what Props.conf file settings are causing the data to stay in multiline events, and not single as needed. I have used "splunk cmd btool props list --debug" I have used "splunk test sourcetype /tmp/testfile.txt" to understand more of what is being used for sourcetype. The data is being pushed from a client installation on a LWF, so I am depending on the Indexer to correctly recognize the timestamp. I have the following setup in default Props already: "BREAK_ONLY_BEFORE_DATE = True" and with this very straight forward data having the timestamp as the first characters it seems like it shouldn't be confused on how to break lines....but it does. My default installations are pretty generic and don't usually have to do weird stuff to make Splunk work. This is the exception. V4.1.4 - exhibits merged events [1334880312] PASSIVE HOST CHECK: blah.com;0;Host state confirmed by hostchecker - TCP OK - 0.115 second response time on port 22 <--- no break in events here --- it just formats weird online. [1334880312] HOST ALERT: blah.com;UP;HARD;1;Host state confirmed by hostchecker - TCP OK - 0.115 second response time on port 22 v4.3.1 - exhibits single line events (correct way to be indexed) [1334954305] SERVICE NOTIFICATION: theparser;blahblah.com;sinfo;CRITICAL;notify-by-parser;Disk_Space_[D:92.0%>=90%] Disk_Space_[E:97.0%>=90%] [1334954305] SERVICE NOTIFICATION: theparser;blahblah.com;sinfo;CRITICAL;notify-by-parser;Disk_Space_[E:82.0%>=80%] ... View more

I have a Lookup Table the returns data from all 32 columns, but many times data that is confirmed good in the assetlist.csv file comes back as "unknown". Here is a snippet of my header in assetlist.csv: logicalname,building,cabinet,cabinetslot,customerfacing,deploystatus Along with a snippet of the next line under the header: albia,e,ell_mfrm ax031,02,non-customer,production Something as simple as asking for the "building" or "cabinet" is returned as "unknown". Here is my search string: earliest=-1m | lookup assetlist logical_name OUTPUT building | stats count by building and from this search it returns "building 'unknown' count=55129 Any ideas or thoughts behind why some of the data populates but other data doesn't? I get the same incorrect results when I run this type of search: earliest=-1m | table building cabinet PS. When building the Lookup Table using the UI I checked the "automatic lookups" option so I get all the fields, all the time. Here is the first six columns from that part of the installation: assetlist logical_name AS host OUTPUTNEW building AS building cabinet AS cabinet cabinet_slot AS cabinet_slot customer_facing AS customer_facing deploy_status AS deploystatus This setup is as generic as possible and nothing tricked out to do anything fancy so I would think it would work "all the time". MasterOogway ... View more

Has anyone else experienced a deathly slow loading of the "BIG-IP Hostname" dropdown, or any other dropdown fields in the new "Splunk for F5 Security" App? We can search the data in normal display speeds, but once we move into the App and go do any of the ASM drop downs each field takes upwards of 5 minutes to load. The "asm_log" has been set correctly in the inputs.conf...which is really the only requirement to have this App run correctly from the Index server side. The F5 is pushing data correctly because we can test that in a normal search. Running v4.1.4 (2) Quad-Core processors; 48Gb Mem; (2) 146Gb SAS 10k HDD's; NAS storage for install & indexing. On all previous installations over nine other environments, we have NEVER had an issue running on NAS as it is tuned to the "T's" for our Oracle installations. Thoughts? Ideas? Anyone else experiencing this? ... View more

I am creating new Views for clients and would like to add in the "Splunk standard" results dropdown found on the right side of the results page. The one that allows you to specify 10,20,50 results. Thus changing the number of results provided. Also, where is the module that allows a you to add in this one to the new View: « prev 1 2 3 4 5 6 7 8 9 10 next »| Options... I have tried to leverage the ui_examples App but can't find some of the basic stuff like this listed. Thoughts? MasterOogway ... View more

I have what appears to be a simple monitor to watch for a specific file name with a regex to define the date stamped file. The file in question is named, /log/blahblah_0.0.0.653_9110.log On my LWF I have the following simple inputs.conf definition: [monitor:///log/blahblah_.*\.log] disabled=false sourcetype=search-log From ../splunkd.log I get the following error. INFO TailingProcessor - No configurations match, will ignore path='/log/blahblah_0.0.0.653_9110.log' I have also defined a monitor based on a whitelist with the same result. [monitor:///log] disabled=false whitelist = blahblah_.*.log$ sourcetype=search-log My question is, "why does Splunk not want to index this file? I have confirmed the Regex is defined correctly, so I believe that is not it. Most likely it is something VERY simple where I can't see the forest through the trees. Thoughts? MasterOogway ... View more

I am attempting to bring data together from servers sitting in GMT in line with the logs from servers sitting in CMT, but have been unsuccessful in setting the TZ within Props.conf on the index server for the affected hosts. My goal is to have all logs appear as if they are indexed within CST. Currently I have data that "appears in the future" due to GMT settings on each host. My host list looks like this: pd-mnsdfdh123 vn-mnkcdjkel2 vm-mnkdjke235 . . In order to capture all the host names affected I have added the following REGEX to my props.conf which captures the names, but doesnt' translate the TZ....at all. [host::\w{2}-mn.*] TZ=GMT Aslo tried without luck. TZ=GMT+0500 This appears to be pretty straight forward but for some reason the solution has escaped me to date. ... View more