All Posts

Find Answers
Ask questions. Get answers. Find technical product solutions from passionate members of the Splunk community.

All Posts

Hello Everyone, I have following splunk query, which I am trying to build for dropdown in dashboard. Basically 2 dropdowns, the 1st dropdown has got static value which is index names:  index_1 , ind... See more...
Hello Everyone, I have following splunk query, which I am trying to build for dropdown in dashboard. Basically 2 dropdowns, the 1st dropdown has got static value which is index names:  index_1 , index_2 , index_3 Based on the selected index,  I am trying to run the splunk query:   index="index_1" | eval hostname_pattern=case( index == "index_1","*-hostname_1", index == "index_2","*-hostname_2" ) | search hostname= hostname_pattern   the search always return empty. However if I run the direct query for index_1 or index_2 with its relevant hostname, it works and returns me results   index="index_1" | search hostname= "*-hostname_1"    For the sake of checking if my condition is working or not, I fed the output of eval case into table. And checked by passing relevant indexes (index_1 or index_2)   index="index_1" | eval hostname_pattern=case( index == "index_1","*-hostname_1", index == "index_2","*-hostname_2" ) | stats count by hostname_pattern | table hostname_pattern | sort hostname_pattern   returns *-hostname_1 Not sure how do we pass the hostname value based on selected index for search. Highly appreciate your help.
Hi Guys I have issue for the newly setup HF and UF. The windows UF’s logs are reaching the Indexers while the Linux UF are not. Communication is ok between LiNux UF and HF as observed using tcpdum... See more...
Hi Guys I have issue for the newly setup HF and UF. The windows UF’s logs are reaching the Indexers while the Linux UF are not. Communication is ok between LiNux UF and HF as observed using tcpdump. The linux UF is sending traffics and HF received and process it. can you help what needs to check on UF or HF?
hi @Xander13 , good for you, see next time! Ciao and happy splunking Giuseppe P.S.: Karma Points are appreciated by all the contributors
Thank you guys. Issue was resolved. There is NOEXEC restriction configured on the account in SUDOERS file.    
Hi @Xander13 , you're using the splunk user to run the upgrade and probably there are some files owned by root. You have two choices: run the upgrade by root, run by root the command "chown -R s... See more...
Hi @Xander13 , you're using the splunk user to run the upgrade and probably there are some files owned by root. You have two choices: run the upgrade by root, run by root the command "chown -R splunk:splunk /opt/splunk" and then run the upgrade by splunk user Ciao. Giuseppe
Hi @Xander13  the error - Error calling execve(): Permission denied was discussed in this post. could you pls check this once, thanks.  https://community.splunk.com/t5/Getting-Data-In/When-trying-... See more...
Hi @Xander13  the error - Error calling execve(): Permission denied was discussed in this post. could you pls check this once, thanks.  https://community.splunk.com/t5/Getting-Data-In/When-trying-to-start-Splunk-I-m-getting-an-quot-execve/m-p/119749    
yes, that is exactly what I want. I have one pipeline, that starts several pipelines and they all started from the same traceid. That is the reason I want to point to a specific time frame so I wil... See more...
yes, that is exactly what I want. I have one pipeline, that starts several pipelines and they all started from the same traceid. That is the reason I want to point to a specific time frame so I will see the exact trace that is related directly to this pipeline.
Thanks, but it didn't help, I still see all the traces even if I changed it to -5m or -1m the startTime
1.  index=abc source="/opt/src/datasource.tmp" | dedup _raw | table Servers | stats count(Servers) as Total 2.  index=abc source="/opt/src/datasource.tmp" | dedup _raw | table CompletedServers | s... See more...
1.  index=abc source="/opt/src/datasource.tmp" | dedup _raw | table Servers | stats count(Servers) as Total 2.  index=abc source="/opt/src/datasource.tmp" | dedup _raw | table CompletedServers | stats count(CompletedServers) as Completed As @PickleRick points out, these searches you posted reveal potentially deeper problems that is your data.  If there is a need to dedup _raw, you should try clean up data first.  Also, there should never be two separate index searches using the same source.  PickleRick already illustrated a single search to get the same counts.   Let me further point out that most likely, the two searches produce the exact same count if Servers and CompletedServers appear in the same events. But back to your original table ServerName             UpgradeStatus ==========         ============= Server1                     Completed Server2                     Completed Server3                     Completed Server4                     Completed Server5                     Completed Server6                     Completed Server7                     Pending Server8                     Pending Server9                     Pending Server10                  Pending Obviously, neither of your searches will provide those "Pending" ones.  When asking a question in a public forum, it is really important to explain your input and output.  It is obvious that you did not think @sainag_splunk's and my previous answers did not give you the solution because you didn't even have the table. Because If you did, either of our searches will have given you the table you needed. So, I venture to guess that the real question is how to derive the first table from the index data you have.  Once this table is formed, either of our suggestions would have given you the display you wanted.  Is this correct? Back to the problem of UpgradeStatus.  When I point out that your searches do not produce Pending values, the big question is: What is in CompletedStatus?  Does it give "Completed" for some ServerName, and "Pending" for others?  And what is the field name that gives you ServerName?  Is it Servers used in your first search? If both are true, and that ServerName and CompletedStatus appear in the same events, the solution is as simple as index=abc source="/opt/src/datasource.tmp" | stats dc(Servers) as count by CompletedStatus | eventstats sum(count) as total | eval count = count . " (" . round(count / total * 100) . "%)" | fields - total | transpose header_field=CompletedStatus | fields - column In other words, all that change from my previous answer is field names that I guess from the two meaningless searches. Here are my four commandments of asking answerable data analytics questions: Illustrate data input (in raw text, anonymize as needed), whether they are raw events or output from a search that volunteers here do not have to look at. Illustrate the desired output from illustrated data. Explain the logic between illustrated data and desired output without SPL. If you also illustrate attempted SPL, illustrate actual output and compare with desired output, explain why they look different to you if that is not painfully obvious. Volunteers are not your mind readers.  It is unfair to ask unanswerable questions here.
The following is the emulation to produce the mock data.  Play with it and compare with real data.   | makeresults | fields - _* | eval _raw = "{ \"somekey\": [ { \"AmiLaunchIndex\": 0, \... See more...
The following is the emulation to produce the mock data.  Play with it and compare with real data.   | makeresults | fields - _* | eval _raw = "{ \"somekey\": [ { \"AmiLaunchIndex\": 0, \"ImageId\": \"ami-08435a7e5e61bc00b\", \"InstanceId\": \"i-0b307f45f965d4222\", \"InstanceType\": \"r5b.2xlarge\", \"KeyName\": \"gsus-prod-net-01-key\", \"LaunchTime\": \"2024-07-19T15:42:09+00:00\", \"Monitoring\": { \"State\": \"disabled\" }, \"Placement\": { \"AvailabilityZone\": \"us-east-1b\", \"GroupName\": \"\", \"Tenancy\": \"default\" }, \"Platform\": \"windows\", \"PrivateDnsName\": \"ip-172-23-43-196.ec2.internal\", \"PrivateIpAddress\": \"172.23.43.196\", \"ProductCodes\": [], \"PublicDnsName\": \"\", \"State\": { \"Code\": 16, \"Name\": \"running\" }, \"StateTransitionReason\": \"\", \"SubnetId\": \"subnet-03aa1e87392c5e63d\", \"VpcId\": \"vpc-0e8954f5da98fe98d\", \"Architecture\": \"x86_64\", \"BlockDeviceMappings\": [ { \"DeviceName\": \"/dev/sda1\", \"Ebs\": { \"AttachTime\": \"2024-07-19T15:41:21+00:00\", \"DeleteOnTermination\": false, \"Status\": \"attached\", \"VolumeId\": \"vol-0d39da483194fdffa\" } }, { \"DeviceName\": \"/dev/xvdf\", \"Ebs\": { \"AttachTime\": \"2023-07-06T19:30:04+00:00\", \"DeleteOnTermination\": false, \"Status\": \"attached\", \"VolumeId\": \"vol-07813f09191a4af20\" } }, { \"DeviceName\": \"/dev/xvdg\", \"Ebs\": { \"AttachTime\": \"2023-07-06T19:30:15+00:00\", \"DeleteOnTermination\": false, \"Status\": \"attached\", \"VolumeId\": \"vol-05d213e8588431029\" } }, { \"DeviceName\": \"/dev/xvdh\", \"Ebs\": { \"AttachTime\": \"2023-07-06T19:29:54+00:00\", \"DeleteOnTermination\": false, \"Status\": \"attached\", \"VolumeId\": \"vol-08849724290ff4ff0\" } }, { \"DeviceName\": \"/dev/xvdi\", \"Ebs\": { \"AttachTime\": \"2023-07-06T19:29:54+00:00\", \"DeleteOnTermination\": false, \"Status\": \"attached\", \"VolumeId\": \"vol-0711192d14ffdc4ff\" } }, { \"DeviceName\": \"/dev/xvdj\", \"Ebs\": { \"AttachTime\": \"2023-07-06T19:30:25+00:00\", \"DeleteOnTermination\": false, \"Status\": \"attached\", \"VolumeId\": \"vol-0d8db3494a8c7254c\" } }, { \"DeviceName\": \"/dev/xvdk\", \"Ebs\": { \"AttachTime\": \"2023-07-06T19:30:16+00:00\", \"DeleteOnTermination\": false, \"Status\": \"attached\", \"VolumeId\": \"vol-0cba7f29aec3d9a35\" } }, { \"DeviceName\": \"/dev/xvdl\", \"Ebs\": { \"AttachTime\": \"2023-07-06T19:30:15+00:00\", \"DeleteOnTermination\": false, \"Status\": \"attached\", \"VolumeId\": \"vol-0810a5ac78ecf0e11\" } }, { \"DeviceName\": \"/dev/xvdm\", \"Ebs\": { \"AttachTime\": \"2023-07-06T19:30:15+00:00\", \"DeleteOnTermination\": false, \"Status\": \"attached\", \"VolumeId\": \"vol-0e4ec2b5f1bf4d871\" } }, { \"DeviceName\": \"/dev/xvdn\", \"Ebs\": { \"AttachTime\": \"2023-07-06T19:29:54+00:00\", \"DeleteOnTermination\": false, \"Status\": \"attached\", \"VolumeId\": \"vol-092a40b78baeeac7f\" } }, { \"DeviceName\": \"/dev/xvdo\", \"Ebs\": { \"AttachTime\": \"2023-07-06T19:29:55+00:00\", \"DeleteOnTermination\": false, \"Status\": \"attached\", \"VolumeId\": \"vol-0b1e2d02d509f3f5e\" } }, { \"DeviceName\": \"/dev/xvdp\", \"Ebs\": { \"AttachTime\": \"2023-07-06T19:30:15+00:00\", \"DeleteOnTermination\": false, \"Status\": \"attached\", \"VolumeId\": \"vol-01cabf285c719295f\" } } ], \"ClientToken\": \"terraform-20230706192857135700000004\", \"EbsOptimized\": true, \"EnaSupport\": true, \"Hypervisor\": \"xen\", \"IamInstanceProfile\": { \"Arn\": \"arn:aws:iam::189682858917:instance-profile/hig/WindowsEC2Role\", \"Id\": \"AIPASYKP2XOSSBXPPW5LL\" }, \"NetworkInterfaces\": [ { \"Attachment\": { \"AttachTime\": \"2023-07-06T19:28:58+00:00\", \"AttachmentId\": \"eni-attach-09192f2a4fbf5a17d\", \"DeleteOnTermination\": false, \"DeviceIndex\": 0, \"Status\": \"attached\", \"NetworkCardIndex\": 0 }, \"Description\": \"\", \"Groups\": [ { \"GroupName\": \"gsus-prod-net-01-cmdb-sg-2022051216484634970000000a\", \"GroupId\": \"sg-04908349d0f791aec\" }, { \"GroupName\": \"gsus-prod-net-01-sql-sg-20220512164845082300000004\", \"GroupId\": \"sg-0b214ef19d0a4c0f7\" }, { \"GroupName\": \"gsus-prod-net-01-app-4587-v2sql-sg-20230707183929669400000001\", \"GroupId\": \"sg-0c2f333cbfb09b29f\" }, { \"GroupName\": \"gsus-prod-net-01-win-sg-20220512164844866600000001\", \"GroupId\": \"sg-0257bb3846318d5fd\" } ], \"Ipv6Addresses\": [], \"MacAddress\": \"12:90:ce:a5:ca:a1\", \"NetworkInterfaceId\": \"eni-0086b400faa99451d\", \"OwnerId\": \"189682858917\", \"PrivateDnsName\": \"ip-172-23-43-196.ec2.internal\", \"PrivateIpAddress\": \"172.23.43.196\", \"PrivateIpAddresses\": [ { \"Primary\": true, \"PrivateDnsName\": \"ip-172-23-43-196.ec2.internal\", \"PrivateIpAddress\": \"172.23.43.196\" }, { \"Primary\": false, \"PrivateDnsName\": \"ip-172-23-43-163.ec2.internal\", \"PrivateIpAddress\": \"172.23.43.163\" }, { \"Primary\": false, \"PrivateDnsName\": \"ip-172-23-43-141.ec2.internal\", \"PrivateIpAddress\": \"172.23.43.141\" } ], \"SourceDestCheck\": true, \"Status\": \"in-use\", \"SubnetId\": \"subnet-03aa1e87392c5e63d\", \"VpcId\": \"vpc-0e8954f5da98fe98d\", \"InterfaceType\": \"interface\" } ], \"RootDeviceName\": \"/dev/sda1\", \"RootDeviceType\": \"ebs\", \"SecurityGroups\": [ { \"GroupName\": \"gsus-prod-net-01-cmdb-sg-2022051216484634970000000a\", \"GroupId\": \"sg-04908349d0f791aec\" }, { \"GroupName\": \"gsus-prod-net-01-sql-sg-20220512164845082300000004\", \"GroupId\": \"sg-0b214ef19d0a4c0f7\" }, { \"GroupName\": \"gsus-prod-net-01-app-4587-v2sql-sg-20230707183929669400000001\", \"GroupId\": \"sg-0c2f333cbfb09b29f\" }, { \"GroupName\": \"gsus-prod-net-01-win-sg-20220512164844866600000001\", \"GroupId\": \"sg-0257bb3846318d5fd\" } ], \"SourceDestCheck\": true, \"Tags\": [ { \"Key\": \"Description\", \"Value\": \"\" }, { \"Key\": \"hig-owner\", \"Value\": \"mahesh.vidyasagar@thehartford.com\" }, { \"Key\": \"Billing\", \"Value\": \"APP-4587\" }, { \"Key\": \"Repository\", \"Value\": \"https://github.thehartford.com/HIG/gs_tfe_peoplesoft_app4587\" }, { \"Key\": \"Schedule\", \"Value\": \"None\" }, { \"Key\": \"mssql_adou\", \"Value\": \"OU=SQL-GS,OU=Windows,OU=Cloud,OU=Servers,DC=ad1,DC=prod\" }, { \"Key\": \"child_module_name\", \"Value\": \"terraform-aws-ec2-db-mssql\" }, { \"Key\": \"mssql_instancename\", \"Value\": \"MSSQLSERVER\" }, { \"Key\": \"module_name\", \"Value\": \"terraform-aws-ec2-instance\" }, { \"Key\": \"Pipeline\", \"Value\": \"https://jenkinsci.thehartford.com/iac-1/job/IAC/job/GS/job/PeopleSoft-4587/job/gsus_4587_peoplesoft_aws_gsprod/\" }, { \"Key\": \"IMDSv2\", \"Value\": \"Enabled\" }, { \"Key\": \"IMDSv2 Date\", \"Value\": \"07/06/2023, 19:30:24\" }, { \"Key\": \"PatchGroup\", \"Value\": \"Multiple DB Node AZ2\" }, { \"Key\": \"module_version\", \"Value\": \"v4.0.6\" }, { \"Key\": \"BackupDBGroup\", \"Value\": \"ec2-sqldb-prod-silver\" }, { \"Key\": \"mssql_version\", \"Value\": \"2019\" }, { \"Key\": \"AppOwner\", \"Value\": \"Mahesh Vidyasagar\" }, { \"Key\": \"hig-planit-appid\", \"Value\": \"APP-4587-PROD\" }, { \"Key\": \"ServiceTier\", \"Value\": \"Silver\" }, { \"Key\": \"DBNodes\", \"Value\": \"Secondary\" }, { \"Key\": \"Hostname\", \"Value\": \"a4587ue1pwsag06\" }, { \"Key\": \"mssql_datavolumes\", \"Value\": \"4\" }, { \"Key\": \"mssql_installtype\", \"Value\": \"SAG\" }, { \"Key\": \"hig-data-classification\", \"Value\": \"CompanyConfidential\" }, { \"Key\": \"PatchingFlag\", \"Value\": \"Tag Applied\" }, { \"Key\": \"Environment\", \"Value\": \"prod\" }, { \"Key\": \"Owner\", \"Value\": \"HIG\" }, { \"Key\": \"hig-billing\", \"Value\": \"APP-4587\" }, { \"Key\": \"mssql_environment\", \"Value\": \"prod\" }, { \"Key\": \"Domain\", \"Value\": \"ad1.prod\" }, { \"Key\": \"AppName\", \"Value\": \"PeopleSoft\" }, { \"Key\": \"ThirdParty\", \"Value\": \"No\" }, { \"Key\": \"child_module_version\", \"Value\": \"v2.2.1\" }, { \"Key\": \"Hosted Region\", \"Value\": \"US\" }, { \"Key\": \"Name\", \"Value\": \"PeopleSoft-4587-PROD-sag-06\" }, { \"Key\": \"BackupGroup\", \"Value\": \"ec2-nobackup\" }, { \"Key\": \"LOB\", \"Value\": \"Global Specialty\" }, { \"Key\": \"Patching Group\", \"Value\": \"Multiple DB Node AZ1\" }, { \"Key\": \"backup_s3bucket\", \"Value\": \"gs-gsus-prod-net-01-sqlbackup-us-east-1\" }, { \"Key\": \"DynatraceMonitoring\", \"Value\": \"true\" }, { \"Key\": \"mssql_collation\", \"Value\": \"Latin1_General_BIN\" }, { \"Key\": \"DataClassification\", \"Value\": \"CompanyConfidential\" }, { \"Key\": \"AppId\", \"Value\": \"APP-4587\" }, { \"Key\": \"AppTier\", \"Value\": \"DB\" }, { \"Key\": \"hig-environment-type\", \"Value\": \"PROD\" }, { \"Key\": \"backup_s3bucketkms\", \"Value\": \"arn:aws:kms:us-east-1:189682858917:key/e4b94c9a-82ad-4322-98ec-48d610f6548b\" }, { \"Key\": \"Comments\", \"Value\": \"\" } ], \"VirtualizationType\": \"hvm\", \"CpuOptions\": { \"CoreCount\": 4, \"ThreadsPerCore\": 1 }, \"CapacityReservationSpecification\": { \"CapacityReservationPreference\": \"open\" }, \"HibernationOptions\": { \"Configured\": false }, \"MetadataOptions\": { \"State\": \"applied\", \"HttpTokens\": \"required\", \"HttpPutResponseHopLimit\": 1, \"HttpEndpoint\": \"enabled\", \"HttpProtocolIpv6\": \"disabled\", \"InstanceMetadataTags\": \"disabled\" }, \"EnclaveOptions\": { \"Enabled\": false }, \"PlatformDetails\": \"Windows\", \"UsageOperation\": \"RunInstances:0002\", \"UsageOperationUpdateTime\": \"2023-07-06T19:28:58+00:00\", \"PrivateDnsNameOptions\": { \"HostnameType\": \"ip-name\", \"EnableResourceNameDnsARecord\": false, \"EnableResourceNameDnsAAAARecord\": false }, \"MaintenanceOptions\": { \"AutoRecovery\": \"default\" }, \"CurrentInstanceBootMode\": \"legacy-bios\", \"someOtherStuff\": \"othervalue\", \"c7n:MatchedFilters\": [ \"tag:AccountType\", \"tag:Attributes\", \"tag:EnvironmentType\", \"tag:ImageBuilder\" ] }, { \"AmiLaunchIndex\": 1, \"someOtherStuff\": \"othervalue\", \"c7n:MatchedFilters\": [ \"tag:ApplicationFailoverGroup\", \"tag:AppTier\", \"tag:Attributes\", \"tag:DBNodes\", \"tag:rk_aws_native_account_id\", \"tag:rk_cluster_id\", \"tag:rk_component\", \"tag:rk_instance_class\", \"tag:rk_job_id\", \"tag:rk_managed\", \"tag:rk_object\", \"tag:rk_restore_source_region\", \"tag:rk_restore_timestamp\", \"tag:rk_source_snapshot_native_id\", \"tag:rk_source_vm_native_id\", \"tag:rk_source_vm_native_name\", \"tag:rk_taskchain_id\", \"tag:rk_user\", \"tag:rk_version\" ] } ] }" | spath ``` data emulation above ```          
To ask an answerable data analytics question, follow these golden rules; nay, call them the four commandments: Illustrate data input (in raw text, anonymize as needed), whether they are raw events ... See more...
To ask an answerable data analytics question, follow these golden rules; nay, call them the four commandments: Illustrate data input (in raw text, anonymize as needed), whether they are raw events or output from a search that volunteers here do not have to look at. Illustrate the desired output from illustrated data. Explain the logic between illustrated data and desired output without SPL. If you also illustrate attempted SPL, illustrate actual output and compare with desired output, explain why they look different to you if that is not painfully obvious. Let's check. Your sample data are JSON fragments, not a complete JSON object.  Even the last update, no one can tell how deep that structure goes.  In other words, which exact JSON path leads to the subnode c7n:MatchedFilters.  I understand that your event size is enormous so you cannot show a complete sample.  But you should show the complete structure of the object. The phrase "show the count and tagname in tabular format" in the title seems to be an attempt to describe desired output.  But there are a thousand ways to show count and tagname in tabular format.  What is even a "count" in your text? No description of logic. N/A Without these, you are effectively asking volunteers to read your mind.  That is not a fair ask. Now, based on your last update, we can see that c7n:MatchedFilters is not a top node in your raw data.  It is a subnode in a very large array.  Is this large array a top node of your raw data?  If not, how many upper paths are there before we reach c7n:MatchedFilters?  Without an exact answer to this, there is no good answer to your question.  In the following, I will assume "yes" to this, i.e., I will assume that somekey[c7n:MatchedFilters] is the correct path. (Or, in SPL flattened notation, somekey{}.c7n:MatchedFilters{}. I will further assume that "show the count and tagname in tabular format" means to count events grouped by the string following "tag:" in each value of c7n:MatchedFilters.  In other words, I assume that this table is what you look for: tagName count AccountType 1 AppTier 1 ApplicationFailoverGroup 1 Attributes 2 DBNodes 1 EnvironmentType 1 ImageBuilder 1 rk_aws_native_account_id 1 rk_cluster_id 1 rk_component 1 rk_instance_class 1 rk_job_id 1 rk_managed 1 rk_object 1 rk_restore_source_region 1 rk_restore_timestamp 1 rk_source_snapshot_native_id 1 rk_source_vm_native_id 1 rk_source_vm_native_name 1 rk_taskchain_id 1 rk_user 1 rk_version 1 The above count is based on the fragments you pasted.  I constructed the following as a valid JSON object to mimic the structure.  If this is not the correct structure, you need to go back to the four commandments and start over.     { "somekey": [ { "AmiLaunchIndex": 0, "ImageId": "ami-08435a7e5e61bc00b", "InstanceId": "i-0b307f45f965d4222", "InstanceType": "r5b.2xlarge", "KeyName": "gsus-prod-net-01-key", "LaunchTime": "2024-07-19T15:42:09+00:00", "Monitoring": { "State": "disabled" }, "Placement": { "AvailabilityZone": "us-east-1b", "GroupName": "", "Tenancy": "default" }, "Platform": "windows", "PrivateDnsName": "ip-172-23-43-196.ec2.internal", "PrivateIpAddress": "172.23.43.196", "ProductCodes": [], "PublicDnsName": "", "State": { "Code": 16, "Name": "running" }, "StateTransitionReason": "", "SubnetId": "subnet-03aa1e87392c5e63d", "VpcId": "vpc-0e8954f5da98fe98d", "Architecture": "x86_64", "BlockDeviceMappings": [ { "DeviceName": "/dev/sda1", "Ebs": { "AttachTime": "2024-07-19T15:41:21+00:00", "DeleteOnTermination": false, "Status": "attached", "VolumeId": "vol-0d39da483194fdffa" } }, { "DeviceName": "/dev/xvdf", "Ebs": { "AttachTime": "2023-07-06T19:30:04+00:00", "DeleteOnTermination": false, "Status": "attached", "VolumeId": "vol-07813f09191a4af20" } }, { "DeviceName": "/dev/xvdg", "Ebs": { "AttachTime": "2023-07-06T19:30:15+00:00", "DeleteOnTermination": false, "Status": "attached", "VolumeId": "vol-05d213e8588431029" } }, { "DeviceName": "/dev/xvdh", "Ebs": { "AttachTime": "2023-07-06T19:29:54+00:00", "DeleteOnTermination": false, "Status": "attached", "VolumeId": "vol-08849724290ff4ff0" } }, { "DeviceName": "/dev/xvdi", "Ebs": { "AttachTime": "2023-07-06T19:29:54+00:00", "DeleteOnTermination": false, "Status": "attached", "VolumeId": "vol-0711192d14ffdc4ff" } }, { "DeviceName": "/dev/xvdj", "Ebs": { "AttachTime": "2023-07-06T19:30:25+00:00", "DeleteOnTermination": false, "Status": "attached", "VolumeId": "vol-0d8db3494a8c7254c" } }, { "DeviceName": "/dev/xvdk", "Ebs": { "AttachTime": "2023-07-06T19:30:16+00:00", "DeleteOnTermination": false, "Status": "attached", "VolumeId": "vol-0cba7f29aec3d9a35" } }, { "DeviceName": "/dev/xvdl", "Ebs": { "AttachTime": "2023-07-06T19:30:15+00:00", "DeleteOnTermination": false, "Status": "attached", "VolumeId": "vol-0810a5ac78ecf0e11" } }, { "DeviceName": "/dev/xvdm", "Ebs": { "AttachTime": "2023-07-06T19:30:15+00:00", "DeleteOnTermination": false, "Status": "attached", "VolumeId": "vol-0e4ec2b5f1bf4d871" } }, { "DeviceName": "/dev/xvdn", "Ebs": { "AttachTime": "2023-07-06T19:29:54+00:00", "DeleteOnTermination": false, "Status": "attached", "VolumeId": "vol-092a40b78baeeac7f" } }, { "DeviceName": "/dev/xvdo", "Ebs": { "AttachTime": "2023-07-06T19:29:55+00:00", "DeleteOnTermination": false, "Status": "attached", "VolumeId": "vol-0b1e2d02d509f3f5e" } }, { "DeviceName": "/dev/xvdp", "Ebs": { "AttachTime": "2023-07-06T19:30:15+00:00", "DeleteOnTermination": false, "Status": "attached", "VolumeId": "vol-01cabf285c719295f" } } ], "ClientToken": "terraform-20230706192857135700000004", "EbsOptimized": true, "EnaSupport": true, "Hypervisor": "xen", "IamInstanceProfile": { "Arn": "arn:aws:iam::189682858917:instance-profile/hig/WindowsEC2Role", "Id": "AIPASYKP2XOSSBXPPW5LL" }, "NetworkInterfaces": [ { "Attachment": { "AttachTime": "2023-07-06T19:28:58+00:00", "AttachmentId": "eni-attach-09192f2a4fbf5a17d", "DeleteOnTermination": false, "DeviceIndex": 0, "Status": "attached", "NetworkCardIndex": 0 }, "Description": "", "Groups": [ { "GroupName": "gsus-prod-net-01-cmdb-sg-2022051216484634970000000a", "GroupId": "sg-04908349d0f791aec" }, { "GroupName": "gsus-prod-net-01-sql-sg-20220512164845082300000004", "GroupId": "sg-0b214ef19d0a4c0f7" }, { "GroupName": "gsus-prod-net-01-app-4587-v2sql-sg-20230707183929669400000001", "GroupId": "sg-0c2f333cbfb09b29f" }, { "GroupName": "gsus-prod-net-01-win-sg-20220512164844866600000001", "GroupId": "sg-0257bb3846318d5fd" } ], "Ipv6Addresses": [], "MacAddress": "12:90:ce:a5:ca:a1", "NetworkInterfaceId": "eni-0086b400faa99451d", "OwnerId": "189682858917", "PrivateDnsName": "ip-172-23-43-196.ec2.internal", "PrivateIpAddress": "172.23.43.196", "PrivateIpAddresses": [ { "Primary": true, "PrivateDnsName": "ip-172-23-43-196.ec2.internal", "PrivateIpAddress": "172.23.43.196" }, { "Primary": false, "PrivateDnsName": "ip-172-23-43-163.ec2.internal", "PrivateIpAddress": "172.23.43.163" }, { "Primary": false, "PrivateDnsName": "ip-172-23-43-141.ec2.internal", "PrivateIpAddress": "172.23.43.141" } ], "SourceDestCheck": true, "Status": "in-use", "SubnetId": "subnet-03aa1e87392c5e63d", "VpcId": "vpc-0e8954f5da98fe98d", "InterfaceType": "interface" } ], "RootDeviceName": "/dev/sda1", "RootDeviceType": "ebs", "SecurityGroups": [ { "GroupName": "gsus-prod-net-01-cmdb-sg-2022051216484634970000000a", "GroupId": "sg-04908349d0f791aec" }, { "GroupName": "gsus-prod-net-01-sql-sg-20220512164845082300000004", "GroupId": "sg-0b214ef19d0a4c0f7" }, { "GroupName": "gsus-prod-net-01-app-4587-v2sql-sg-20230707183929669400000001", "GroupId": "sg-0c2f333cbfb09b29f" }, { "GroupName": "gsus-prod-net-01-win-sg-20220512164844866600000001", "GroupId": "sg-0257bb3846318d5fd" } ], "SourceDestCheck": true, "Tags": [ { "Key": "Description", "Value": "" }, { "Key": "hig-owner", "Value": "mahesh.vidyasagar@thehartford.com" }, { "Key": "Billing", "Value": "APP-4587" }, { "Key": "Repository", "Value": "https://github.thehartford.com/HIG/gs_tfe_peoplesoft_app4587" }, { "Key": "Schedule", "Value": "None" }, { "Key": "mssql_adou", "Value": "OU=SQL-GS,OU=Windows,OU=Cloud,OU=Servers,DC=ad1,DC=prod" }, { "Key": "child_module_name", "Value": "terraform-aws-ec2-db-mssql" }, { "Key": "mssql_instancename", "Value": "MSSQLSERVER" }, { "Key": "module_name", "Value": "terraform-aws-ec2-instance" }, { "Key": "Pipeline", "Value": "https://jenkinsci.thehartford.com/iac-1/job/IAC/job/GS/job/PeopleSoft-4587/job/gsus_4587_peoplesoft_aws_gsprod/" }, { "Key": "IMDSv2", "Value": "Enabled" }, { "Key": "IMDSv2 Date", "Value": "07/06/2023, 19:30:24" }, { "Key": "PatchGroup", "Value": "Multiple DB Node AZ2" }, { "Key": "module_version", "Value": "v4.0.6" }, { "Key": "BackupDBGroup", "Value": "ec2-sqldb-prod-silver" }, { "Key": "mssql_version", "Value": "2019" }, { "Key": "AppOwner", "Value": "Mahesh Vidyasagar" }, { "Key": "hig-planit-appid", "Value": "APP-4587-PROD" }, { "Key": "ServiceTier", "Value": "Silver" }, { "Key": "DBNodes", "Value": "Secondary" }, { "Key": "Hostname", "Value": "a4587ue1pwsag06" }, { "Key": "mssql_datavolumes", "Value": "4" }, { "Key": "mssql_installtype", "Value": "SAG" }, { "Key": "hig-data-classification", "Value": "CompanyConfidential" }, { "Key": "PatchingFlag", "Value": "Tag Applied" }, { "Key": "Environment", "Value": "prod" }, { "Key": "Owner", "Value": "HIG" }, { "Key": "hig-billing", "Value": "APP-4587" }, { "Key": "mssql_environment", "Value": "prod" }, { "Key": "Domain", "Value": "ad1.prod" }, { "Key": "AppName", "Value": "PeopleSoft" }, { "Key": "ThirdParty", "Value": "No" }, { "Key": "child_module_version", "Value": "v2.2.1" }, { "Key": "Hosted Region", "Value": "US" }, { "Key": "Name", "Value": "PeopleSoft-4587-PROD-sag-06" }, { "Key": "BackupGroup", "Value": "ec2-nobackup" }, { "Key": "LOB", "Value": "Global Specialty" }, { "Key": "Patching Group", "Value": "Multiple DB Node AZ1" }, { "Key": "backup_s3bucket", "Value": "gs-gsus-prod-net-01-sqlbackup-us-east-1" }, { "Key": "DynatraceMonitoring", "Value": "true" }, { "Key": "mssql_collation", "Value": "Latin1_General_BIN" }, { "Key": "DataClassification", "Value": "CompanyConfidential" }, { "Key": "AppId", "Value": "APP-4587" }, { "Key": "AppTier", "Value": "DB" }, { "Key": "hig-environment-type", "Value": "PROD" }, { "Key": "backup_s3bucketkms", "Value": "arn:aws:kms:us-east-1:189682858917:key/e4b94c9a-82ad-4322-98ec-48d610f6548b" }, { "Key": "Comments", "Value": "" } ], "VirtualizationType": "hvm", "CpuOptions": { "CoreCount": 4, "ThreadsPerCore": 1 }, "CapacityReservationSpecification": { "CapacityReservationPreference": "open" }, "HibernationOptions": { "Configured": false }, "MetadataOptions": { "State": "applied", "HttpTokens": "required", "HttpPutResponseHopLimit": 1, "HttpEndpoint": "enabled", "HttpProtocolIpv6": "disabled", "InstanceMetadataTags": "disabled" }, "EnclaveOptions": { "Enabled": false }, "PlatformDetails": "Windows", "UsageOperation": "RunInstances:0002", "UsageOperationUpdateTime": "2023-07-06T19:28:58+00:00", "PrivateDnsNameOptions": { "HostnameType": "ip-name", "EnableResourceNameDnsARecord": false, "EnableResourceNameDnsAAAARecord": false }, "MaintenanceOptions": { "AutoRecovery": "default" }, "CurrentInstanceBootMode": "legacy-bios", "someOtherStuff": "othervalue", "c7n:MatchedFilters": [ "tag:AccountType", "tag:Attributes", "tag:EnvironmentType", "tag:ImageBuilder" ] }, { "AmiLaunchIndex": 1, "someOtherStuff": "othervalue", "c7n:MatchedFilters": [ "tag:ApplicationFailoverGroup", "tag:AppTier", "tag:Attributes", "tag:DBNodes", "tag:rk_aws_native_account_id", "tag:rk_cluster_id", "tag:rk_component", "tag:rk_instance_class", "tag:rk_job_id", "tag:rk_managed", "tag:rk_object", "tag:rk_restore_source_region", "tag:rk_restore_timestamp", "tag:rk_source_snapshot_native_id", "tag:rk_source_vm_native_id", "tag:rk_source_vm_native_name", "tag:rk_taskchain_id", "tag:rk_user", "tag:rk_version" ] } ] }   Basically, I took the valid element from your latest sample, and added the fragment containing the c7n:MatchedFilters key as a second element of top level key somekey. So, the second element does not have all the other keys as they are not of interest to this particular question. After this construction, I think I get a handle to why the basic technique doesn't work.  But let me explain the basic technique first.     This is assuming that Splunk search head already extracted all fields from raw data, which is the norm.  But your raw events are so huge, even that single valid array element you posted is >15KB.  Splunk has this default in limits.conf   [spath] # Number of characters to read from an XML or JSON event when # auto extracting. extraction_cutoff = 5000   You can increase this to be larger than the maximum size of your events.  But that may require a lot of RAM. Without touching extraction_cutoff, you can directly reach c7n:MatchedFilters with spath command.  Like this:   | spath path=somekey{} | spath input=somekey{} path=c7n:MatchedFilters{} output=tagName | stats count by tagName | eval tagName = replace(tagName, "tag:", "")    
OK. There are several issues with your searches and maybe your data. 1. dedup _raw - that means you're removing duplicates of _whole events_. Which implies you have those duplicate events (otherwise... See more...
OK. There are several issues with your searches and maybe your data. 1. dedup _raw - that means you're removing duplicates of _whole events_. Which implies you have those duplicate events (otherwise the command would be unnecessary). That is not a typical situation. Or you simply don't need this comman (or you meant something completely different by it) 2. The table command very rarely makes sense in the middle of the search. Additionally, it is a performance hit in a distributed environment because it moves all procesing to search head tier so you don't benefit from map-reduce distributed processing. Table can be used at the end to present results in tabular view. 3. If you want to count distinct values of a field do just that. Doing strange stuff with dedup can fail you miserably if you have multivalued fields. So again, if you want to count distinct values of those two fields (and so far you haven't said otherwise) the search with single stats command containing two aggregations is completely sufficient.  
Hello, I am following this tutorial to create a Splunk app using React on macOS Sonoma: https://splunkui.splunk.com/Toolkits/SUIT/AppTutorial However, I am not able to get it to work. The 'start' v... See more...
Hello, I am following this tutorial to create a Splunk app using React on macOS Sonoma: https://splunkui.splunk.com/Toolkits/SUIT/AppTutorial However, I am not able to get it to work. The 'start' view is simply not added to the app views on Splunk, even though they are there in the files in my app. I wasn't even able to launch the app before I set it to 'Visible' by going to 'Manage Apps' and editing its properties. It should have been visible because it is set as such under my app.conf. But after I launched it, I was redirected to the search page (image below). If I go to the URL http://localhost:8000/en-US/app/my-splunk-app/start, I get the 'Page not found' error page. Could someone please help me with this?
@PickleRick Thank you for your response and patience. 
Thank you @yuanliu  Had to modify a little to make it work | rename log.message as _raw | rex mode=sed "s/errors=(.+) fields=(.+)/errors=\"\1\" fields=\"\2\"/" | rex field=_raw "path=(?P<path... See more...
Thank you @yuanliu  Had to modify a little to make it work | rename log.message as _raw | rex mode=sed "s/errors=(.+) fields=(.+)/errors=\"\1\" fields=\"\2\"/" | rex field=_raw "path=(?P<path>.*) feedType=(?P<feedType>.*) sku=(?P<sku>.*) status=(?P<status>.*) errorCount=(?P<errorCount>.*) errors=(?P<errors>.*) fields=(?P<fields>.*)" | table path, feedType, sku, status, errorCount, errors, fields
1. I need to dedup servers in first query. Count only Unique Servers.  2. I need to consider all the Servers with duplicates in the second query and then CompletedStatus then count the completed.  ... See more...
1. I need to dedup servers in first query. Count only Unique Servers.  2. I need to consider all the Servers with duplicates in the second query and then CompletedStatus then count the completed.  |.   index=abc source="/opt/src/datasource.tmp" | dedup _raw | dedup Servers | table Servers | stats count(Servers) as Total ||.   index=abc source="/opt/src/datasource.tmp" | dedup _raw | table Servers, CompletedStatus | stats count(CompletedStatus) as Completed   So 2 queries are compulsory. 
But since transaction is one of the cursed commands you can - assuming Job is unique - do | stats min(_time) as start max(_time) as end by Job | eval duration=end-start
1. Each of your searches is unnecessarily complicated (and thus less efficient than it could be). You can cut the whole dedup and table part and get the same results. 2. Are you sure that count(Serv... See more...
1. Each of your searches is unnecessarily complicated (and thus less efficient than it could be). You can cut the whole dedup and table part and get the same results. 2. Are you sure that count(Servers) and count(CompletedServers) respectively is what you need? It might be - I don't know your use case - but typically it's either the general count of results or distinct count of a field. Counts of fields are rare the proper solution. But there are uses for it. Yours might be one of them so just asking if you know what you're doing. 3. You don't have to join anything. Assuming the counts are OK, you simply need one search to calculate both stats index=abc source="/opt/src/datasource.tmp" stats count(CompletedServers) as Completed count(Servers) as Total And that's it.
Hi Sir, I have two queries like below and I need to join both of queries and then divide Completed by Total and calculate percentage and display the percentage in bracket along with Completed count ... See more...
Hi Sir, I have two queries like below and I need to join both of queries and then divide Completed by Total and calculate percentage and display the percentage in bracket along with Completed count as shown in the above screenshot in panel. 1.  index=abc source="/opt/src/datasource.tmp" | dedup _raw | table Servers | stats count(Servers) as Total 2.  index=abc source="/opt/src/datasource.tmp" | dedup _raw | table CompletedServers | stats count(CompletedServers) as Completed    
Assuming the timestamp you want is in _time,  you could use transaction to get the duration | transaction Job If not, you could reassign the _time field to be the time you want.