Since our last update to 8.2.2.1 the index _internal contains lots of ERROR messages where we cannot get any information about it's meaning: ERROR ILightWeightSearchStringParser [4392 SchedulerThread] - still in inQueto=true Does anybody know this message and can give some information about it?   Thanks ... View more

We have problems collecting data from AWS-Cloudwatch logs with Splunk Addon for AWS, so we are looking for alternatives. On searching this forum and the internet at large, most advices go to Amazon Kinesis Firehose, only a few go to HEC and AWS Lamda. But the second seems much easier, only that all the information I found is either quite old or not complete. Does anyone have experience with HEC and AWS Lamda? Or can even provide some pros and cons for the alternatives? Thanks for any help. ... View more

Hello, we are using timeline in an embedded Report which is enclosed into a html-page calling several embedded reports. For every other report I can define the size of the report panel. Only the size of the timeline report always stays the same. When first searching splunk.answers I found a hint that this could be a bug in timeline style-sheet, but sorry cannot find it again. It showed that in the stylesheet the height is set fix to 130, as far as I remember. I also tried to file a case at splunk, but it seems I do not get the rights to do so (although I am in contact with splunk gb). Can somebody else who might have the same problem or is able to confirm this behavior file a case for this? Or maybe there is already a solution an I just could not find it. Thanks ... View more

Hello I need to stream access_logs from aws from different directories such as /directory/subdirectory1/subdirectory2/subdirectory3/various cryptic numbers/var/log/apache2/access.log. I would need something like /ecs/service/apache-tls10/PROD-apache-tls10/*/var/log/apache2/access.log because .*/var/log/apache2/access.log sends to many logs since there are other directories in the log_group starting with a different directory name which we do not need to index. Reading docs and splunk answers did not point me to a useful example explaining exactly what a 'stream matching regex' should look like. What ever I try I find this entry in internal: 2019-09-05 06:04:01,758 level=ERROR pid=19218 tid=MainThread logger=splunk_ta_aws.modinputs.inspector pos=util.py:call:163 | | message="Failed to execute function=run, error=Traceback (most recent call last): File "/opt/splunk/etc/apps/Splunk_TA_aws/bin/3rdparty/splunktalib/common/util.py", line 160, in __call_ return func(*args, **kwargs) File "/opt/splunk/etc/apps/Splunk_TA_aws/bin/splunk_ta_aws/modinputs/inspector/init.py", line 53, in run do_run() File "/opt/splunk/etc/apps/Splunk_TA_aws/bin/splunk_ta_aws/modinputs/inspector/init.py", line 30, in _do_run aiconf.AWSInspectorConf, "aws_inspector", logger) File "/opt/splunk/etc/apps/Splunk_TA_aws/bin/splunk_ta_aws/common/ta_aws_common.py", line 136, in get_configs tasks = conf.get_tasks() File "/opt/splunk/etc/apps/Splunk_TA_aws/bin/splunk_ta_aws/modinputs/inspector/aws_inspector_conf.py", line 60, in get_tasks _cleanup_checkpoints(tasks, config) File "/opt/splunk/etc/apps/Splunk_TA_aws/bin/splunk_ta_aws/modinputs/inspector/aws_inspector_conf.py", line 119, in _cleanup_checkpoints internals = store.get_state("internals") File "/opt/splunk/etc/apps/Splunk_TA_aws/bin/3rdparty/splunktalib/state_store.py", line 155, in get_state state = json.load(jsonfile) File "/opt/splunk/lib/python2.7/json/init.py", line 291, in load **kw) File "/opt/splunk/lib/python2.7/json/init_.py", line 339, in loads return _default_decoder.decode(s) File "/opt/splunk/lib/python2.7/json/decoder.py", line 364, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end()) File "/opt/splunk/lib/python2.7/json/decoder.py", line 382, in raw_decode raise ValueError("No JSON object could be decoded") ValueError: No JSON object could be decoded " can anyone provide an example for that please? ... View more

Hello, I have tried to implement row highlighting according to the example dashboard "Table Row Highlighting", but cannot make it work. After reading all answers I could fin in splunk>answers I now think that this might not work anymore in splunk 7.2. because even the example does not show row coloring according to the examples purpose. The table is all green. Does anybody have a solution to this? best regards Gesa ... View more

My scenario is: 1 Indexer (SPLUNK Enterprise 7.1.3) 1 Heavy Forwarder (SPLUNK Enterprise 7.1.2 1 Universal Forwarder (7.1.2) both forwarders forward /var/log/tomcat7/catalina.out to the same index with the same sourcetype at the indexer instance. Both files have the same structure containing 3 different types of events : [DEBUG] [15.10.2018] [10:40:04] [CustomerController] ... 2018-10-15 10:42:37,875 [ajp-bio-8009-exec-10] INFO... Oct 15, 2018 8:05:14 AM org.apache.catalina.startup.HostConfig... I want to index only the firs type. So on indexer side I made the following configuration entries : in props.conf : [source::/var/log/tomcat7/catalina.out] TRANSFORMS-set=setnull,setparsing in transforms.conf : [setnull] REGEX = . DEST_KEY = queue FORMAT = nullQueue [setparsing] REGES = ^\[[A_Z]*\] DEST_KEY = queue FORMAT = indexQueue The data from the Universal Forwarder is filtered as expected, only the first type events are indexed The data from the Heavy Forwarder is not filtered, all events are indexed. I restarted all the instances several times. I compared the outputs.conf from both forwarders, they are identical. Any ideas what the reason for the different behavior could be? Thanks! ... View more