Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
×
Are you a member of the Splunk Community?
Sign in or Register with your Splunk account to get your questions answered, access valuable resources and connect with experts!
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
- Find Answers
- :
- About jto13
jto13
Explorer
Member since:
02-09-2022
11-12-2024
Community Statistics
| Posts | 9 |
| Solutions | 0 |
| Karma Given | 2 |
| Karma Received | 0 |
| Member Since | 02-09-2022 |
Activity Feed
- Posted Re: REGEX not being applied even though it is working in REX on Splunk Enterprise. 11-12-2024 06:04 PM
- Karma Re: REGEX not being applied even though it is working in REX for PickleRick. 11-12-2024 06:01 PM
- Karma Re: REGEX not being applied even though it is working in REX for PickleRick. 11-11-2024 10:57 PM
- Posted Re: REGEX not being applied even though it is working in REX on Splunk Enterprise. 11-11-2024 10:31 PM
- Posted Re: REGEX not being applied even though it is working in REX on Splunk Enterprise. 11-10-2024 10:08 PM
- Posted Re: REGEX not being applied even though it is working in REX on Splunk Enterprise. 11-07-2024 07:01 PM
- Posted Re: REGEX not being applied even though it is working in REX on Splunk Enterprise. 11-07-2024 05:43 PM
- Posted Re: REGEX not being applied even though it is working in REX on Splunk Enterprise. 11-05-2024 07:19 PM
- Posted Re: REGEX not being applied even though it is working in REX on Splunk Enterprise. 11-04-2024 06:26 PM
- Posted REGEX not being applied even though it is working in REX on Splunk Enterprise. 11-04-2024 01:12 AM
- Tagged REGEX not being applied even though it is working in REX on Splunk Enterprise. 11-04-2024 01:12 AM
- Tagged REGEX not being applied even though it is working in REX on Splunk Enterprise. 11-04-2024 01:12 AM
- Tagged REGEX not being applied even though it is working in REX on Splunk Enterprise. 11-04-2024 01:12 AM
- Tagged REGEX not being applied even though it is working in REX on Splunk Enterprise. 11-04-2024 01:12 AM
- Posted How to search Indexes with count = 0 on Splunk Search. 02-10-2022 12:10 AM
Topics I've Started
| Subject | Karma | Author | Latest Post |
|---|---|---|---|
| 0 | |||
| 0 |
11-12-2024
06:04 PM
Hi Rick, Makes sense, thanks a lot for your help. 🙏
... View more
11-11-2024
10:31 PM
Hi Rick,
Ok understood, to sum it up as below:
The search-time extraction settings are much simpler and there is less load to our environment compared to the index-time extraction.
For our index-time extraction, there should be additional configurations as well in our props and transforms conf files and most likely that's why our existing ones didn't work.
We resolved it by moving the regex settings to the props.conf in our search heads (search-time extraction) manually in the /opt/splunk/etc/system/local directory as below:
[aws:elb:accesslogs]
EXTRACT-aws_elb_accesslogs = ^(?P<Protocol>\S+)\s+(?P<Timestamp>\S+)\s+(?P<ELB>\S+)\s+(?P<ClientPort>\S+)\s+(?P<TargetPort>\S+)\s+(?P<RequestProcessingTime>\S+)\s+(?P<TargetProcessingTime>\S+)\s+(?P<ResponseProcessingTime>\S+)\s+(?P<ELBStatusCode>\S+)\s+(?P<TargetStatusCode>\S+)\s+(?P<ReceivedBytes>\S+)\s+(?P<SentBytes>\S+)\s+\"(?P<Request>[^\"]+)\"\s+\"(?P<UserAgent>[^\"]+)\"\s+(?P<SSLCipher>\S+)\s+(?P<SSLProtocol>\S+)\s+(?P<TargetGroupArn>\S+)\s+\"(?P<TraceId>[^\"]+)\"\s+\"(?P<DomainName>[^\"]+)\"\s+\"(?P<ChosenCertArn>[^\"]+)\"\s+(?P<MatchedRulePriority>\S+)\s+(?P<RequestCreationTime>\S+)\s+\"(?P<ActionExecuted>[^\"]+)\"\s+\"(?P<RedirectUrl>[^\"]+)\"\s+\"(?P<ErrorReason>[^\"]+)\"\s+(?P<AdditionalInfo1>\S+)\s+(?P<AdditionalInfo2>\S+)\s+(?P<AdditionalInfo3>\S+)\s+(?P<AdditionalInfo4>\S+)\s+(?P<TransactionId>\S+)
Thank you for the help.
... View more
11-10-2024
10:08 PM
Hi Rick, Instead of props.conf and transforms.conf in the HF (index-time extraction), we have moved the regex settings to the props.conf in all of our search heads (search-time extraction) manually in the /opt/splunk/etc/system/local directory as below: props.conf [aws:elb:accesslogs] EXTRACT-aws_elb_accesslogs = ^(?P<Protocol>\S+)\s+(?P<Timestamp>\S+)\s+(?P<ELB>\S+)\s+(?P<ClientPort>\S+)\s+(?P<TargetPort>\S+)\s+(?P<RequestProcessingTime>\S+)\s+(?P<TargetProcessingTime>\S+)\s+(?P<ResponseProcessingTime>\S+)\s+(?P<ELBStatusCode>\S+)\s+(?P<TargetStatusCode>\S+)\s+(?P<ReceivedBytes>\S+)\s+(?P<SentBytes>\S+)\s+\"(?P<Request>[^\"]+)\"\s+\"(?P<UserAgent>[^\"]+)\"\s+(?P<SSLCipher>\S+)\s+(?P<SSLProtocol>\S+)\s+(?P<TargetGroupArn>\S+)\s+\"(?P<TraceId>[^\"]+)\"\s+\"(?P<DomainName>[^\"]+)\"\s+\"(?P<ChosenCertArn>[^\"]+)\"\s+(?P<MatchedRulePriority>\S+)\s+(?P<RequestCreationTime>\S+)\s+\"(?P<ActionExecuted>[^\"]+)\"\s+\"(?P<RedirectUrl>[^\"]+)\"\s+\"(?P<ErrorReason>[^\"]+)\"\s+(?P<AdditionalInfo1>\S+)\s+(?P<AdditionalInfo2>\S+)\s+(?P<AdditionalInfo3>\S+)\s+(?P<AdditionalInfo4>\S+)\s+(?P<TransactionId>\S+) This is working as of now, but it is weird that the props and transforms configurations wouldn't work since the regex are the same.
... View more
11-07-2024
07:01 PM
Hi Bhumi, Yes, it is from HF->indexer
... View more
11-07-2024
05:43 PM
Hi Rick, When you mean to search for the field::value, do you mean at the rex part or during search? Apologies if my wording was confusing but the rex part managed to work and we did see the fields when we just searched the index (index= index_name) using verbose mode. However, we did not manage to see those fields when just using the props and transforms.conf.
... View more
11-05-2024
07:19 PM
Hi Rick, Thanks for the info, understood on that. For now, we are trying to get it to work first to at least get the fields and maybe understand where we configured it wrong, would there be any problem with our props and transforms.conf that made it unable to work?
... View more
11-04-2024
06:26 PM
Hi Rick, Thanks for the response, but just wondering what would be the disadvantages of index-time extractions? Our search head is quite overloaded so we are making changes at the heavy forwarder side and trying to reduce the load by parsing it there. We have also tried to change the TRANSFORMS to EXTRACT instead in the props.conf and put the regex there as well but it is also not working even after restarting splunk for some reason, so we're wondering if any additional config lines are required.
... View more
11-04-2024
01:12 AM
Hi all,
We have ingested some logs using a heavy forwarder as below in /opt/splunk/etc/apps/test_inputs/local/:
inputs.conf
[monitor:///opt/splunk/test/test.log]
index=test
sourcetype=aws:elb:accesslogs
disabled=0
start_from=oldest
_meta = splunk_orig_fwd::splunkfwd_hostname
Props.conf
[aws:elb:accesslogs]
TRANSFORMS-aws_elb_accesslogs = aws_elb_accesslogs_extract_all_fields
Transforms.conf
[aws_elb_accesslogs_extract_all_fields]
REGEX = ^(?P<Protocol>\S+)\s+(?P<Timestamp>\S+)\s+(?P<ELB>\S+)\s+(?P<ClientPort>\S+)\s+(?P<TargetPort>\S+)\s+(?P<RequestProcessingTime>\S+)\s+(?P<TargetProcessingTime>\S+)\s+(?P<ResponseProcessingTime>\S+)\s+(?P<ELBStatusCode>\S+)\s+(?P<TargetStatusCode>\S+)\s+(?P<ReceivedBytes>\S+)\s+(?P<SentBytes>\S+)\s+\"(?P<Request>[^\"]+)\"\s+\"(?P<UserAgent>[^\"]+)\"\s+(?P<SSLCipher>\S+)\s+(?P<SSLProtocol>\S+)\s+(?P<TargetGroupArn>\S+)\s+\"(?P<TraceId>[^\"]+)\"\s+\"(?P<DomainName>[^\"]+)\"\s+\"(?P<ChosenCertArn>[^\"]+)\"\s+(?P<MatchedRulePriority>\S+)\s+(?P<RequestCreationTime>\S+)\s+\"(?P<ActionExecuted>[^\"]+)\"\s+\"(?P<RedirectUrl>[^\"]+)\"\s+\"(?P<ErrorReason>[^\"]+)\"\s+(?P<AdditionalInfo1>\S+)\s+(?P<AdditionalInfo2>\S+)\s+(?P<AdditionalInfo3>\S+)\s+(?P<AdditionalInfo4>\S+)\s+(?P<TransactionId>\S+)
Before we applied the props and transforms.conf, we have used the rex function to test the logs in the search head as below and the fields appeared when searched:
index=test sourcetype=aws:elb:accesslogs
| rex field=_raw "^(?P<Protocol>\S+)\s+(?P<Timestamp>\S+)\s+(?P<ELB>\S+)\s+(?P<ClientIP>\S+)\s+(?P<TargetIP>\S+)\s+(?P<RequestProcessingTime>\S+)\s+(?P<TargetProcessingTime>\S+)\s+(?P<ResponseProcessingTime>\S+)\s+(?P<ELBStatusCode>\S+)\s+(?P<TargetStatusCode>\S+)\s+(?P<ReceivedBytes>\S+)\s+(?P<SentBytes>\S+)\s+\"(?P<Request>[^\"]+)\"\s+\"(?P<UserAgent>[^\"]+)\"\s+(?P<SSLCipher>\S+)\s+(?P<SSLProtocol>\S+)\s+(?P<TargetGroupArn>\S+)\s+\"(?P<TraceId>[^\"]+)\"\s+\"(?P<DomainName>[^\"]+)\"\s+\"(?P<ChosenCertArn>[^\"]+)\"\s+(?P<MatchedRulePriority>\S+)\s+(?P<RequestCreationTime>\S+)\s+\"(?P<ActionExecuted>[^\"]+)\"\s+\"(?P<RedirectUrl>[^\"]+)\"\s+\"(?P<ErrorReason>[^\"]+)\"\s+(?P<AdditionalInfo1>\S+)\s+(?P<AdditionalInfo2>\S+)\s+(?P<AdditionalInfo3>\S+)\s+(?P<AdditionalInfo4>\S+)\s+(?P<TransactionId>\S+)"
However, when we ingested the logs as usual, the fields weren't extracted as per the rex during the search, is there anything missing or why the regex isn't being applied to the logs?
Appreciate if anyone has any advice on this.
Thank you in advance.
... View more
Labels
- Labels:
-
configuration
02-10-2022
12:10 AM
Dear Team,
I just want to use the simple search below to see which indexes are having zero count that day/week/whichever time period.
index= *
| stats count by index
| where count = 0
However, the search is not returning anything and if I remove the where count=0 it is only returning indexes with more than zero counts. How do I make sure that the indexes with count=0 are included?
Thank you.
Warm Regards.
... View more
Contact Me
| Online Status |
Offline
|
| Date Last Visited |
11-12-2024
09:34 PM
|
