I am trying to make sure I know how to configure an environment to ingest weblogs that are correctly parsed and I am running into trouble in that I am only getting 1 single event. I have used feedback provided to similar questions to build out my configurations. Note that the original intent of this exercise was to see what would the different effect be with two different props.conf. My weblog source is this on both forwarders: '<photo id="123" title="Birthday" format="jpg"> <owner id="1111">Jason</owner> <CreationDate>2009-11-06T02:22:37.063</CreationDate> <comments> <comment ownerid="112">Good pic!</comment> <comment ownerif="223">Happy birthday</comment> <comments> </photo> <photo id="123" title="Birthday" format="jpg"> <owner id="1111">Jason</owner> <CreationDate>2009-11-06T02:22:37.063</CreationDate> <comments> <comment ownerid="112">Good pic!</comment> <comment ownerif="223">Happy birthday</comment> <comments> </photo> <photo id="123" title="Birthday" format="jpg"> <owner id="1111">Jason</owner> <CreationDate>2009-11-06T02:22:37.063</CreationDate> <comments> <comment ownerid="112">Good pic!</comment> <comment ownerif="223">Happy birthday</comment> <comments> </photo>' My inputs.conf on FW1is this: '[monitor:///home/labuser/xmldata/] index=web sourcetype=xml disabled=false' My inputs.conf on FW2 is this so that I could figure out which props.conf works: '[monitor:///home/labuser/xmldata/] index=web2 sourcetype=xml2 disabled=false' My props.conf on FW1 is this: 'KV_MODE = xml LINE_BREAKER = () MUST_BREAK_AFTER = \ NO_BINARY_CHECK = 1 SHOULD_LINEMERGE = false TRUNCATE = 0 TIME_PREFIX = \ TIME_FORMAT = %Y-%m-%dT%H:%M:%S.%3N' My props.conf on FW2 is this: 'KV_MODE = xml LINE_BREAKER = ([\r\n]+)() MUST_BREAK_AFTER = \ NO_BINARY_CHECK = 1 SHOULD_LINEMERGE = false TRUNCATE = 0 TIME_PREFIX = \ TIME_FORMAT = %Y-%m-%dT%H:%M:%S.%3N' All the data in both web index and web2 index looks identical in Splunk that both index=web1 or index=web2 produce identical results in that I only get a single event back instead of multiple events. What am I doing wrong? ... View more

I have one Search Head(SH)/DS, one indexer, and one forwarder all on separate Centos Linux VMs. I cannot see any forwarder data on the SH or the indexer. If I run the following on the indexer index=_internal source=*metrics.log* tcpin_connections | stats count by sourceIp I get no result found. If I run the following on the indexer: index=_internal */pathwheremydataresides/* I get results so that path that is being monitored by the forwarder shows up in the internal log If I run the following on the indexer index=_internal source=*metrics.log* group=queue tcpout | stats count by name I get no result found. If I run the following on the forwarder $SPLUNK_HOME/bin/splunk search 'index=_internal source=*metrics.log* destHost | dedup destHost' I get "FATAL: Unable to read the job status" If I run the following on the forwarder ./splunk list monitor it successfully shows my path that I am monitoring. If I run a search on a different index that was previously set up with one shot data on that machine (not forwarder data) I get results. The indexes that I created on each of indexers are reserved for loading forwarded data only. I am not mixing data sources. Under the forward management panel, I can see all of my apps, all of my clients as well as my server classes correctly associating apps to clients All of the search peers are connected and healthy. All apps have been successfully deployed out. inputs.conf & outputs.conf look as expected on the forwarder inputs.conf look as expected on the indexer Any guidance? ... View more

If I have my outputs.conf file on all of my forwarders are configured to send all the data to all of the indexers what is the point or value of load balancing on the forwarder? I would think that autoLB would not need to equal true of all the data is going to all of the indexers. ... View more

(attempting 1 Indexer, +1 SH setup) For some reason I am not able to add a search peer. I tried two approaches as follows: Log into Splunk Web on the search head and click Settings at the top of the page. Click Distributed search in the Distributed Environment area. Click Search peers. On the Search peers page, select New. Specify the search peer, along with any authentication settings. Click Save. Repeat for each of the search head's search peers. It produces: Encountered the following error while trying to save: Error while sending public key to search peer: No route to host 2) Ran the following from the command line producing the same result: splunk add search-server https:// IndexerIPADDRES:8089 -auth admin:password -remoteUsername admin -remotePassword passremote I am able to ping the machines from each respective machine as well as ping google.com from each of the machine. I even checked the host level firewall and enable/permit the port in firewall using: sudo firewall-cmd --zone=public --add-port=8089/tcp --permanent and reload the config list: firewall-cmd --list-all Why I'm I getting this result? ... View more