Use Lambda function to first put logs into S3 in schedule, then you can do your regular S3 input. I was able to use python code from this repo to archive this: https://github.com/ryanholland/rdslogs_to_s3 ... View more

backend dbx2.log showed something fairly generic. It almost looks as if the interpreter couldn't find the proper connection stanza in the db_connections.conf file: 2016-10-27T20:25:52+0000 [ERROR] [dbxquery.py], line 41 : action=dbxquery_command_failure error=HTTP 404 Not Found -- In handler 'conf-db_connections': Could not find object id='MXProd' Traceback (most recent call last): File "/opt/splunk/etc/apps/splunk_app_db_connect/bin/dbxquery.py", line 39, in wrap return func(*args, **kwargs) File "/opt/splunk/etc/apps/splunk_app_db_connect/bin/dbxquery.py", line 256, in generate connection_info = self._retrieve_connection_info(user, session_key) File "/opt/splunk/etc/apps/splunk_app_db_connect/bin/dbxquery.py", line 147, in _retrieve_connection_info db, self.db_health = ci.get_connection(self.connection, user, session_key) File "/opt/splunk/etc/apps/splunk_app_db_connect/bin/dbx2/rest_api/connection_info.py", line 30, in get_connection dbm, conn = _get_conn_params(connection_name, service) File "/opt/splunk/etc/apps/splunk_app_db_connect/bin/dbx2/rest_api/connection_info.py", line 64, in _get_conn_params conn = ConnectionConf(splunk_service, connection_name).content File "/opt/splunk/etc/apps/splunk_app_db_connect/bin/dbx2/splunk_client/connection_conf.py", line 10, in __init__ client.Entity.__init__(self, service, path, **kwargs) File "/opt/splunk/etc/apps/splunk_app_db_connect/bin/dbx2/splunk_client/../../splunk_sdk-1.5.0-py2.7.egg/splunklib/client.py", line 872, in __init__ self.refresh(kwargs.get('state', None)) # "Prefresh" File "/opt/splunk/etc/apps/splunk_app_db_connect/bin/dbx2/splunk_client/../../splunk_sdk-1.5.0-py2.7.egg/splunklib/client.py", line 1011, in refresh self._state = self.read(self.get()) File "/opt/splunk/etc/apps/splunk_app_db_connect/bin/dbx2/splunk_client/../../splunk_sdk-1.5.0-py2.7.egg/splunklib/client.py", line 981, in get return super(Entity, self).get(path_segment, owner=owner, app=app, sharing=sharing, **query) File "/opt/splunk/etc/apps/splunk_app_db_connect/bin/dbx2/splunk_client/../../splunk_sdk-1.5.0-py2.7.egg/splunklib/client.py", line 738, in get **query) File "/opt/splunk/etc/apps/splunk_app_db_connect/bin/dbx2/splunk_client/../../splunk_sdk-1.5.0-py2.7.egg/splunklib/binding.py", line 286, in wrapper return request_fun(self, *args, **kwargs) File "/opt/splunk/etc/apps/splunk_app_db_connect/bin/dbx2/splunk_client/../../splunk_sdk-1.5.0-py2.7.egg/splunklib/binding.py", line 68, in new_f val = f(*args, **kwargs) File "/opt/splunk/etc/apps/splunk_app_db_connect/bin/dbx2/splunk_client/../../splunk_sdk-1.5.0-py2.7.egg/splunklib/binding.py", line 660, in get response = self.http.get(path, self._auth_headers, **query) File "/opt/splunk/etc/apps/splunk_app_db_connect/bin/dbx2/splunk_client/../../splunk_sdk-1.5.0-py2.7.egg/splunklib/binding.py", line 1150, in get return self.request(url, { 'method': "GET", 'headers': headers }) File "/opt/splunk/etc/apps/splunk_app_db_connect/bin/dbx2/splunk_client/../../splunk_sdk-1.5.0-py2.7.egg/splunklib/binding.py", line 1205, in request raise HTTPError(response) HTTPError: HTTP 404 Not Found -- In handler 'conf-db_connections': Could not find object id='MXProd' ... View more

thx for the reply. Yup, this is under the free license. I was trying to avoid going over the indexing limit by limiting the index size itself. But from your response it does look like I'll have to find ways to limiting the data input itself. ... View more