Dear Team, This is my setup for analyzing log from S3: 1 - splunk enterprise 8.1 for standalone VM. 2 - S3 IAM role for bucket with logs. 3 - I installed Splunk Add-on for AWS 4 - for first run, everything is okay. However, i shutdown the VM, and increased the RAM for this VM. and here the problem start: Query from the health Check: Index=“_internal”  (host=“*”)   (sourcetype=aws:s3:log OR sourcetype=aws:logs:log OR sourcetype=aws:sqsbaseds3:log OR sourcetype=aws:description:log OR sourcetype=aws:cloudwatch:log)   (datainput=“*”)   level=ERROR                  message=“Failed to collect data through generic S3.” | fillnull value=“” ErrorCode, ErrorDetail                 | eval ErrorDetail = if((ErrorDetail == “” or ErrorDetail == “‘’“) and !isnull(message), message, ErrorDetail) Response: 2020-10-24 01:23:18,036 level=ERROR pid=25464 tid=Thread-7 logger=splunk_ta_aws.modinputs.generic_s3.aws_s3_data_loader pos=aws_s3_data_loader.py:index_data:91 | datainput=“bucket-log” bucket_name=“logs-storage” | message=“Failed to collect data through generic S3.” start_time=1603473783 job_uid=“f852cf4b-f1fe-4197-bf93-3494f3d2adb7" Traceback (most recent call last):   File “/opt/splunk/etc/apps/Splunk_TA_aws/bin/splunk_ta_aws/modinputs/generic_s3/aws_s3_data_loader.py”, line 86, in index_data     self._do_index_data()   File “/opt/splunk/etc/apps/Splunk_TA_aws/bin/splunk_ta_aws/modinputs/generic_s3/aws_s3_data_loader.py”, line 107, in _do_index_data     self.collect_data()   File “/opt/splunk/etc/apps/Splunk_TA_aws/bin/splunk_ta_aws/modinputs/generic_s3/aws_s3_data_loader.py”, line 153, in collect_data     self._discover_keys(index_store)   File “/opt/splunk/etc/apps/Splunk_TA_aws/bin/splunk_ta_aws/modinputs/generic_s3/aws_s3_data_loader.py”, line 233, in _discover_keys     for key in keys:   File “/opt/splunk/etc/apps/Splunk_TA_aws/bin/splunk_ta_aws/modinputs/generic_s3/aws_s3_common.py”, line 227, in get_keys     for key in keys:   File “/opt/splunk/etc/apps/Splunk_TA_aws/bin/splunk_ta_aws/modinputs/generic_s3/aws_s3_common.py”, line 196, in bucket_lister     encoding_type=encoding_type)   File “/opt/splunk/etc/apps/Splunk_TA_aws/bin/3rdparty/python3/solnlib/utils.py”, line 172, in wrapper     raise last_ex   File “/opt/splunk/etc/apps/Splunk_TA_aws/bin/3rdparty/python3/solnlib/utils.py”, line 159, in wrapper     return func(*args, **kwargs)   File “/opt/splunk/etc/apps/Splunk_TA_aws/bin/splunk_ta_aws/modinputs/generic_s3/aws_s3_common.py”, line 186, in get_all_keys     encoding_type=encoding_type)   File “/opt/splunk/etc/apps/Splunk_TA_aws/bin/3rdparty/python3/boto/s3/bucket.py”, line 474, in get_all_keys     ‘’, headers, **params)   File “/opt/splunk/etc/apps/Splunk_TA_aws/bin/3rdparty/python3/boto/s3/bucket.py”, line 412, in _get_all     response.status, response.reason, body) boto.exception.S3ResponseError: S3ResponseError: 400 Bad Request <?xml version=“1.0" encoding=“UTF-8”?> <Error><Code>ExpiredToken</Code><Message>The provided token has expired.</Message><Token-0>xxxx</Token-0><RequestId>aaaaaaa</RequestId><HostId>Ibbbbb</HostId></Error>   I would like to know what the root cause of this? and how to fix it? ... View more

Dear Team, I have cloudflare app setup and index has data. However, when i open the app from the menu, it show zero result. This is the search of one query: | tstats count from datamodel=cloudflare.cloudflare where Cloudflare.ClientCountry="*" Cloudflare.ClientDeviceType="*" Cloudflare.dest_ip="*" Cloudflare.dest_host="*" Cloudflare.uri_path="*" Cloudflare.http_user_agent="*" Cloudflare.status="*" Cloudflare.src_ip="" Cloudflare.OriginResponseStatus="200" Cloudflare.RayID="*" Cloudflare.WorkerSubrequest="*" Cloudflare.http_method="*" --> the result is 0. However, when i omit the rest and leave ony clientcountry field. I have data. I have my data model created and finished acceleration. What is the cause of that? ... View more

Dear Team,   I had an issue with splunk and had to follow this post: https://community.splunk.com/t5/All-Apps-and-Add-ons/Amazon-Web-Services-Add-on-s3-generic-error-TypeError-int-object/td-p/439118 to make my splunk works again. However, now Splunk will ingest logs from the beginning of everything. How do I make splunk to ingest logs from the last 7 days / 14 days. I'm pretty new with Splunk so I really appreciate every input from you guys. ... View more

Dear Team, I am trying to build a chart like this: - x-axis is the website name - y-axis is the number of request to that website - on top of the bar for Website, I want to display the ut_shannon value of that website. Is there anyway that we can do it? ... View more

Dear Team, What i am trying to achieve is like this: I have a lookup table with many subnets. I am trying to match the destination host IP with the suitable subnet from the table. | index dhcp | lookup IPtable ip as dest_host OUTPUT ip as dest_match The problem is the IP column in IPtable contains IP subnet only. How do I solve it? Any help would be much appreciated! ... View more

Good afternoon, everyone I'm looking for a solution for my idea like this: Today, I want to create a first baseline of domain requests that computers in our network send to our DNS server. I've created a search that gave me a unique list of domains(domain) + count of domain (Count). I want to use this result as a baseline. So tomorrow, I can run the same search that shows me new domains that our computer requests. 1 - compare new data with the baseline, if no new domain, the result is 0. If we have a new domain, print out the new domain only. This new domain will be added to the baseline as baseline-new. 2 - the day after tomorrow, the same search will run and give us new domain again. Is there anyway that we can achieve such result in Splunk? Below is my search for now: index=domain dns_server earliest = -30d latest=now | dedup domain | stats count as Count, values(domain) Thank you so much ... View more

I have two indexes. 1- dns log with source IP with _time field 2 - dhcp log with dhcp IP with _time field I figured out a way to match source IP of DNS Log and source IP of DHCP Log. However, what concerns me is the differences in time between these two indexes. Is there anyway to display the table as DNS_TIME, DHCP TIME (approximately 5m windows different), DHCP_hostname ??? I would really appreciate if anyone could help me with this! ... View more

I need to find a sequence of activity that always start with: http://abc.com/abc.html http://abc.com/end.xvz?.... so I tried to uses this search query: index=sourceindex | transaction maxspan=30s startswith=url="/abc.html" endswith=eval(match(url,"^\/\w+\.\w+") However, the return result is 0 event. Is there anyway that I can refine it? Thank you so much ... View more