Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for
- About xiyangyang
xiyangyang
Path Finder
Member since:
02-17-2014
08-23-2022
Community Statistics
| Posts | 40 |
| Solutions | 3 |
| Karma Given | 0 |
| Karma Received | 5 |
| Member Since | 02-17-2014 |
Activity Feed
- Posted How to explain follow _audit log? on Splunk Search. 08-23-2022 12:40 AM
- Got Karma for Can a universal forwarder be restarted via REST API?. 11-12-2021 09:09 PM
- Posted Re: Splunk Add-on for AWS is not working, s3 generic input not indexing while other s3 generic inputs are working? on All Apps and Add-ons. 06-29-2021 11:12 PM
- Got Karma for Where can I find the ID of Dashboard?. 06-17-2021 12:22 PM
- Got Karma for Re: Where can I find the ID of Dashboard?. 06-17-2021 12:22 PM
- Posted Splunk App for AWS: Why the S3 buckets list does not show in the Splunk Add-on on All Apps and Add-ons. 01-18-2021 03:13 AM
- Got Karma for Re: Where can I find the ID of Dashboard?. 06-05-2020 12:48 AM
- Got Karma for Deploy *nux add-on files from Windows deployment server. 06-05-2020 12:47 AM
- Posted How splunk UF handle windows EventLog rotation? on Getting Data In. 06-30-2019 05:44 PM
- Tagged How splunk UF handle windows EventLog rotation? on Getting Data In. 06-30-2019 05:44 PM
- Tagged How splunk UF handle windows EventLog rotation? on Getting Data In. 06-30-2019 05:44 PM
- Tagged How splunk UF handle windows EventLog rotation? on Getting Data In. 06-30-2019 05:44 PM
- Posted Re: what is the difference between crcSalt and CHECK_METHOD=modtime? on Getting Data In. 06-25-2018 07:00 PM
- Posted what is the difference between crcSalt and CHECK_METHOD=modtime? on Getting Data In. 06-25-2018 06:07 PM
- Tagged what is the difference between crcSalt and CHECK_METHOD=modtime? on Getting Data In. 06-25-2018 06:07 PM
- Posted Re: How to run ps command via splunk web with build-in commond? on Splunk Search. 12-25-2017 11:08 PM
- Posted Can a universal forwarder be restarted via REST API? on Getting Data In. 12-21-2017 11:53 PM
- Tagged Can a universal forwarder be restarted via REST API? on Getting Data In. 12-21-2017 11:53 PM
- Posted Re: How to run ps command via splunk web with build-in commond? on Splunk Search. 12-21-2017 04:39 PM
- Posted How to run ps command via splunk web with build-in commond? on Splunk Search. 12-21-2017 12:11 AM
Topics I've Started
| Subject | Karma | Author | Latest Post |
|---|---|---|---|
| 0 | |||
| 0 | |||
| 0 | |||
| 0 | |||
| 1 | |||
| 0 | |||
| 0 | |||
| 0 | |||
| 1 | |||
| 0 |
01-26-2023
06:16 AM
3 Karma
Wanted to share some good news regarding this issue with the broader community - over the past year we worked with Splunk and Microsoft support and development teams to sort this out. In our organization, 99% of the corrupt events were when Windows was rebooting - a fix for this was identified and successfully tested in our environment. Remaining issue was related to an api error response from EventLog api and a fix for that is in the works per Splunk Dev and support. Both fixes are expected to make it into 9.1 (hopefully around .CONF23). And here's some recommendations that came from the collaborative work with Microsoft. Use 'Delayed Start' for the Splunk Forwarder service. Setup following service dependencies to reduce errors sc config EventLog depend=RpcSs sc config SplunkForwarder depend=EventLog
... View more
08-23-2022
05:09 AM
The underscore in an index or field name is just part of the name, however, names beginning with an underscore are reserved for use by Splunk. The _cd field gives the location of an event within an index. See https://docs.splunk.com/Documentation/SplunkCloud/8.2.2203/Knowledge/Usedefaultfields#_cd for details.
... View more
07-04-2021
10:57 PM
There is no improvement, but every time changing the configuration and restarted the Splunk service it will take an hour to resume the logs flow
... View more
01-18-2021
03:13 AM
I can get cloudwatch log and description successfull but not S3 buckets. My splunk EC2 has full access right to the s3 buckets but they are not listed in the Add-on setting page. I tried to get the s3 buckets from Add data, but I got internal logs like: "The last data ingestion itertion hasn't been completed yet" I attcheched the internal logs, hope soemone can help. ---------------internal logs----------------------------------------------2021-01-18 18:57:59,217 level=ERROR pid=11508 tid=Thread-4 logger=splunk_ta_aws.modinputs.generic_s3.aws_s3_data_loader pos=aws_s3_data_loader.py:index_data:91 | datainput="s3_test_122401" bucket_name="aws-sb-dev-ia0767-splunk-test" | message="Failed to collect data through generic S3." start_time=1610963333 job_uid="96ba7b5d-6dff-4d81-bc6b-b4581d4d3066" Traceback (most recent call last): File "/opt/splunk/etc/apps/Splunk_TA_aws/bin/splunk_ta_aws/modinputs/generic_s3/aws_s3_data_loader.py", line 86, in index_data self._do_index_data() File "/opt/splunk/etc/apps/Splunk_TA_aws/bin/splunk_ta_aws/modinputs/generic_s3/aws_s3_data_loader.py", line 107, in _do_index_data self.collect_data() File "/opt/splunk/etc/apps/Splunk_TA_aws/bin/splunk_ta_aws/modinputs/generic_s3/aws_s3_data_loader.py", line 153, in collect_data self._discover_keys(index_store) File "/opt/splunk/etc/apps/Splunk_TA_aws/bin/splunk_ta_aws/modinputs/generic_s3/aws_s3_data_loader.py", line 224, in _discover_keys bucket = self._get_bucket(credentials) File "/opt/splunk/etc/apps/Splunk_TA_aws/bin/splunk_ta_aws/modinputs/generic_s3/aws_s3_data_loader.py", line 370, in _get_bucket bucket = conn.get_bucket(self._config[asc.bucket_name]) File "/opt/splunk/etc/apps/Splunk_TA_aws/bin/3rdparty/python3/boto/s3/connection.py", line 509, in get_bucket return self.head_bucket(bucket_name, headers=headers) File "/opt/splunk/etc/apps/Splunk_TA_aws/bin/3rdparty/python3/boto/s3/connection.py", line 528, in head_bucket response = self.make_request('HEAD', bucket_name, headers=headers) File "/opt/splunk/etc/apps/Splunk_TA_aws/bin/splunk_ta_aws/common/boto2_s3_patch.py", line 12, in wrapper response = func(*args, **kwargs) File "/opt/splunk/etc/apps/Splunk_TA_aws/bin/3rdparty/python3/boto/s3/connection.py", line 671, in make_request retry_handler=retry_handler File "/opt/splunk/etc/apps/Splunk_TA_aws/bin/3rdparty/python3/boto/connection.py", line 1084, in make_request retry_handler=retry_handler) File "/opt/splunk/etc/apps/Splunk_TA_aws/bin/3rdparty/python3/boto/connection.py", line 1043, in _mexe raise ex File "/opt/splunk/etc/apps/Splunk_TA_aws/bin/3rdparty/python3/boto/connection.py", line 956, in _mexe request.body, request.headers) File "/opt/splunk/lib/python3.7/http/client.py", line 1244, in request self._send_request(method, url, body, headers, encode_chunked) File "/opt/splunk/lib/python3.7/http/client.py", line 1290, in _send_request self.endheaders(body, encode_chunked=encode_chunked) File "/opt/splunk/lib/python3.7/http/client.py", line 1239, in endheaders self._send_output(message_body, encode_chunked=encode_chunked) File "/opt/splunk/lib/python3.7/http/client.py", line 1026, in _send_output self.send(msg) File "/opt/splunk/lib/python3.7/http/client.py", line 966, in send self.connect() File "/opt/splunk/etc/apps/Splunk_TA_aws/bin/3rdparty/python3/boto/https_connection.py", line 119, in connect sock = socket.create_connection((self.host, self.port), self.timeout) File "/opt/splunk/lib/python3.7/socket.py", line 727, in create_connection raise err File "/opt/splunk/lib/python3.7/socket.py", line 716, in create_connection sock.connect(sa) socket.timeout: timed out 2021-01-18 18:57:53,442 level=INFO pid=11508 tid=Thread-13 logger=splunk_ta_aws.modinputs.generic_s3.aws_s3_data_loader pos=aws_s3_data_loader.py:_do_index_data:96 | datainput="s3_test_122401" bucket_name="aws-sb-dev-ia0767-splunk-test" | message="The last data ingestion iteration hasn't been completed yet." 2021-01-18 18:57:23,442 level=INFO pid=11508 tid=Thread-13 logger=splunk_ta_aws.modinputs.generic_s3.aws_s3_data_loader pos=aws_s3_data_loader.py:_do_index_data:96 | datainput="s3_test_122401" bucket_name="aws-sb-dev-ia0767-splunk-test" | message="The last data ingestion iteration hasn't been completed yet." 2021-01-18 18:56:53,442 level=INFO pid=11508 tid=Thread-13 logger=splunk_ta_aws.modinputs.generic_s3.aws_s3_data_loader pos=aws_s3_data_loader.py:_do_index_data:96 | datainput="s3_test_122401" bucket_name="aws-sb-dev-ia0767-splunk-test" | message="The last data ingestion iteration hasn't been completed yet." 2021-01-18 18:56:23,442 level=INFO pid=11508 tid=Thread-13 logger=splunk_ta_aws.modinputs.generic_s3.aws_s3_data_loader pos=aws_s3_data_loader.py:_do_index_data:96 | datainput="s3_test_122401" bucket_name="aws-sb-dev-ia0767-splunk-test" | message="The last data ingestion iteration hasn't been completed yet." 2021-01-18 18:55:53,442 level=INFO pid=11508 tid=Thread-13 logger=splunk_ta_aws.modinputs.generic_s3.aws_s3_data_loader pos=aws_s3_data_loader.py:_do_index_data:96 | datainput="s3_test_122401" bucket_name="aws-sb-dev-ia0767-splunk-test" | message="The last data ingestion iteration hasn't been completed yet." 2021-01-18 18:55:23,442 level=INFO pid=11508 tid=Thread-13 logger=splunk_ta_aws.modinputs.generic_s3.aws_s3_data_loader pos=aws_s3_data_loader.py:_do_index_data:96 | datainput="s3_test_122401" bucket_name="aws-sb-dev-ia0767-splunk-test" | message="The last data ingestion iteration hasn't been completed yet." 2021-01-18 18:54:53,444 level=INFO pid=11508 tid=Thread-13 logger=splunk_ta_aws.modinputs.generic_s3.aws_s3_data_loader pos=aws_s3_data_loader.py:_do_index_data:96 | datainput="s3_test_122401" bucket_name="aws-sb-dev-ia0767-splunk-test" | message="The last data ingestion iteration hasn't been completed yet." 2021-01-18 18:54:23,442 level=INFO pid=11508 tid=Thread-13 logger=splunk_ta_aws.modinputs.generic_s3.aws_s3_data_loader pos=aws_s3_data_loader.py:_do_index_data:96 | datainput="s3_test_122401" bucket_name="aws-sb-dev-ia0767-splunk-test" | message="The last data ingestion iteration hasn't been completed yet." 2021-01-18 18:53:53,442 level=INFO pid=11508 tid=Thread-13 logger=splunk_ta_aws.modinputs.generic_s3.aws_s3_data_loader pos=aws_s3_data_loader.py:_do_index_data:96 | datainput="s3_test_122401" bucket_name="aws-sb-dev-ia0767-splunk-test" | message="The last data ingestion iteration hasn't been completed yet." 2021-01-18 18:53:23,442 level=INFO pid=11508 tid=Thread-13 logger=splunk_ta_aws.modinputs.generic_s3.aws_s3_data_loader pos=aws_s3_data_loader.py:_do_index_data:96 | datainput="s3_test_122401" bucket_name="aws-sb-dev-ia0767-splunk-test" | message="The last data ingestion iteration hasn't been completed yet." 2021-01-18 18:52:53,442 level=INFO pid=11508 tid=Thread-13 logger=splunk_ta_aws.modinputs.generic_s3.aws_s3_data_loader pos=aws_s3_data_loader.py:_do_index_data:96 | datainput="s3_test_122401" bucket_name="aws-sb-dev-ia0767-splunk-test" | message="The last data ingestion iteration hasn't been completed yet." 2021-01-18 18:52:23,442 level=INFO pid=11508 tid=Thread-13 logger=splunk_ta_aws.modinputs.generic_s3.aws_s3_data_loader pos=aws_s3_data_loader.py:_do_index_data:96 | datainput="s3_test_122401" bucket_name="aws-sb-dev-ia0767-splunk-test" | message="The last data ingestion iteration hasn't been completed yet." 2021-01-18 18:51:53,442 level=INFO pid=11508 tid=Thread-13 logger=splunk_ta_aws.modinputs.generic_s3.aws_s3_data_loader pos=aws_s3_data_loader.py:_do_index_data:96 | datainput="s3_test_122401" bucket_name="aws-sb-dev-ia0767-splunk-test" | message="The last data ingestion iteration hasn't been completed yet." 2021-01-18 18:51:23,442 level=INFO pid=11508 tid=Thread-13 logger=splunk_ta_aws.modinputs.generic_s3.aws_s3_data_loader pos=aws_s3_data_loader.py:_do_index_data:96 | datainput="s3_test_122401" bucket_name="aws-sb-dev-ia0767-splunk-test" | message="The last data ingestion iteration hasn't been completed yet." 2021-01-18 18:50:53,442 level=INFO pid=11508 tid=Thread-13 logger=splunk_ta_aws.modinputs.generic_s3.aws_s3_data_loader pos=aws_s3_data_loader.py:_do_index_data:96 | datainput="s3_test_122401" bucket_name="aws-sb-dev-ia0767-splunk-test" | message="The last data ingestion iteration hasn't been completed yet." 2021-01-18 18:50:23,442 level=INFO pid=11508 tid=Thread-13 logger=splunk_ta_aws.modinputs.generic_s3.aws_s3_data_loader pos=aws_s3_data_loader.py:_do_index_data:96 | datainput="s3_test_122401" bucket_name="aws-sb-dev-ia0767-splunk-test" | message="The last data ingestion iteration hasn't been completed yet." 2021-01-18 18:49:53,442 level=INFO pid=11508 tid=Thread-13 logger=splunk_ta_aws.modinputs.generic_s3.aws_s3_data_loader pos=aws_s3_data_loader.py:_do_index_data:96 | datainput="s3_test_122401" bucket_name="aws-sb-dev-ia0767-splunk-test" | message="The last data ingestion iteration hasn't been completed yet." 2021-01-18 18:49:23,442 level=INFO pid=11508 tid=Thread-13 logger=splunk_ta_aws.modinputs.generic_s3.aws_s3_data_loader pos=aws_s3_data_loader.py:_do_index_data:96 | datainput="s3_test_122401" bucket_name="aws-sb-dev-ia0767-splunk-test" | message="The last data ingestion iteration hasn't been completed yet." 2021-01-18 18:48:53,545 level=INFO pid=11508 tid=Thread-4 logger=splunk_ta_aws.modinputs.generic_s3.aws_s3_data_loader pos=aws_s3_data_loader.py:_get_bucket:365 | datainput="s3_test_122401" bucket_name="aws-sb-dev-ia0767-splunk-test", start_time=1610963333 job_uid="96ba7b5d-6dff-4d81-bc6b-b4581d4d3066", phase="discover_key" | message="Create new S3 connection." 2021-01-18 18:48:53,545 level=INFO pid=11508 tid=Thread-4 logger=splunk_ta_aws.common.aws_credentials pos=aws_credentials.py:load:163 | datainput="s3_test_122401" bucket_name="aws-sb-dev-ia0767-splunk-test", start_time=1610963333 job_uid="96ba7b5d-6dff-4d81-bc6b-b4581d4d3066", phase="discover_key" | message="load credentials succeed" arn="arn:aws:sts::052086164386:assumed-role/splunk_ec2_access/i-000c869a7adf815b0" expiration="2021-01-18 15:23:20+00:00" 2021-01-18 18:48:53,444 level=INFO pid=11508 tid=Thread-4 logger=splunk_ta_aws.common.aws_credentials pos=aws_credentials.py:_load_source_credentials:195 | datainput="s3_test_122401" bucket_name="aws-sb-dev-ia0767-splunk-test", start_time=1610963333 job_uid="96ba7b5d-6dff-4d81-bc6b-b4581d4d3066", phase="discover_key" | message="fetch ec2 instance credentials" 2021-01-18 18:48:53,444 level=INFO pid=11508 tid=Thread-4 logger=splunk_ta_aws.common.aws_credentials pos=aws_credentials.py:load:156 | datainput="s3_test_122401" bucket_name="aws-sb-dev-ia0767-splunk-test", start_time=1610963333 job_uid="96ba7b5d-6dff-4d81-bc6b-b4581d4d3066", phase="discover_key" | message="begin loading credentials" aws_account="splunk_ec2_access" aws_iam_role=None 2021-01-18 18:48:53,444 level=INFO pid=11508 tid=Thread-4 logger=splunk_ta_aws.modinputs.generic_s3.aws_s3_data_loader pos=aws_s3_data_loader.py:_discover_keys:220 | datainput="s3_test_122401" bucket_name="aws-sb-dev-ia0767-splunk-test", start_time=1610963333 job_uid="96ba7b5d-6dff-4d81-bc6b-b4581d4d3066", phase="discover_key" | message="Start of discovering S3 keys." 2021-01-18 18:48:53,443 level=INFO pid=11508 tid=Thread-4 logger=splunk_ta_aws.modinputs.generic_s3.aws_s3_data_loader pos=aws_s3_data_loader.py:collect_data:143 | datainput="s3_test_122401" bucket_name="aws-sb-dev-ia0767-splunk-test", start_time=1610963333 job_uid="96ba7b5d-6dff-4d81-bc6b-b4581d4d3066" | message="Start processing" last_modified="2020-12-17T07:30:59.000Z" latest_scanned="" 2021-01-18 18:48:53,442 level=INFO pid=11508 tid=Thread-4 logger=splunk_ta_aws.modinputs.generic_s3.aws_s3_data_loader pos=aws_s3_data_loader.py:_do_index_data:105 | datainput="s3_test_122401" bucket_name="aws-sb-dev-ia0767-splunk-test", start_time=1610963333 job_uid="96ba7b5d-6dff-4d81-bc6b-b4581d4d3066" | message="Start processing."
... View more
Labels
- Labels:
-
troubleshooting
07-01-2019
09:54 AM
If you use the WinEventLog monitor (https://docs.splunk.com/Documentation/Splunk/latest/Admin/Inputsconf#Windows_Event_Log_Monitor) it shouldn't care about the log rolling. It doesn't actually care about the log file itself as it monitors the specific event log channel rather than the .evtx file.
... View more
06-25-2018
10:28 PM
こんにちは。
CHECK_METHOD=modtimeを設定した場合はファイルの更新時刻が更新されれば取得します。ただし、ファイル全件取得で、差分だけ取得するわけではありません。
＞crcSaltの場合、ファイルの中身が同じであれば、timestampだけ変更した場合、ファイルの取得できない認識です。
⇒crcSaltは目的が違うので認識が少し違います。ファイルの中身が同じでもファイルパスが違えば取得します。同じヘッダファイルを持つログ等、本来は違うファイルなのにchecksumが同じになってしまうような場合にファイルパスをchecksumに含むことで両方のログを取得できるようにcrcSaltは使うものです。
※平日の業務時間内であればSPSSで質問して頂いたほうが回答は早いと思います。
... View more
12-22-2017
12:59 AM
1 Karma
Hi @xiyangyang,
Yes, you can restart UF via REST API (ref doc. http://docs.splunk.com/Documentation/Splunk/7.0.1/RESTREF/RESTsystem#server.2Fcontrol.2Frestart )
curl -k -u admin:changeme https://localhost:8089/services/server/control/restart -X POST
If you want to run above command from remote server then you need to change default password for admin user otherwise you will get below error.
<?xml version="1.0" encoding="UTF-8"?>
<response>
<messages>
<msg type="WARN">Remote login has been disabled for 'admin' with the default password. Either set the password, or override by changing the 'allowRemoteLogin' setting in your server.conf file.</msg>
</messages>
</response>
I hope this helps.
Thanks,
Harshil
... View more
12-28-2017
01:58 PM
Hi - I added this post - If you find it useful, please upvote the answer, or add your own solution if you found another way!
https://answers.splunk.com/answers/606762/how-do-i-monitor-jbosstomcatapacheetc-and-raise-an.html
... View more
12-18-2017
03:38 AM
If your just looking to index the files when they change, you can use a normal monitor statement, and set CHECK_METHOD = entire_md5 in props.conf which will trigger Splunk to reindex the whole file each time it changes.
https://docs.splunk.com/Documentation/Splunk/7.0.1/Admin/Propsconf#File_checksum_configuration
... View more
05-07-2017
10:38 PM
Yes, the 2.txt was removed after it was indexed.
2.txt is indexed twice, and both of the sourcetype are shinsei_db_audit_utf8.
If I move "" from Line 6, [monitor://C:\Splunk\log_SME.log]. This phenomenon will not happen.
... View more
04-27-2017
06:40 PM
@xiyangyang - Did your answer provide a working solution to your question? If yes and you would like to close out your post, don't forget to click "Accept". Thank you.
... View more
05-11-2017
10:44 PM
You should click Accept on this answer to close the question.
... View more
10-29-2015
04:34 PM
Hi xiyangyang,
It turns out field ordering only happens on the search UI. Command line search does not do this ordering. If you'd like to control the order when you export to CSV, you can do a work around with an approach similar to the one used here : https://answers.splunk.com/answers/60017/search-jobs-export-changes-order-of-columns-unexpectedly.html
... View more
03-12-2015
02:42 AM
Splunk doesnt have this capability built in.
Your best bet would be to use dns in the case one fails.
If you your your windows DS is winds.mydomain.com and if your linux DS is nixds.mydomain.com, change DNS when one of these goes down. So if winds.mydomain.com goes down, change DNS to point to the nixds.mydomain.com server.
You need to maintain the contents of both serverclasses.conf on each DS however to account for failover.
For setting deploy-poll, I would create a custom app that sets this time. And then push this to your clients.
... View more
03-12-2015
06:29 AM
Just to add, you can also adjust polling interval to 1hour or more assuming you won't change config frequently. So that your DS can manage at least 2K forwarders.
... View more
09-26-2017
05:57 AM
Do you know if this issue has been solved?
... View more
12-24-2014
08:59 PM
If the names of the logs are truly random, and cant apply a whitelist / blacklist, then you really cant do much except the monitor the whole directory.
Whitelisting the filenames would be the best method to approach this.
If you are not looking at realtime monitoring of these files, then you might be better to work on a scripted solution that will move only the desired logs to a separate folder and have splunk monitor that directory.. That does require some scripting outside of Splunk though...
Is this a custom app?
... View more
Contact Me
| Online Status |
Offline
|
| Date Last Visited |
08-23-2022
04:46 AM
|
