All Posts

Find Answers
Ask questions. Get answers. Find technical product solutions from passionate members of the Splunk community.

All Posts

The following is the emulation to produce the mock data.  Play with it and compare with real data.   | makeresults | fields - _* | eval _raw = "{ \"somekey\": [ { \"AmiLaunchIndex\": 0, \... See more...
The following is the emulation to produce the mock data.  Play with it and compare with real data.   | makeresults | fields - _* | eval _raw = "{ \"somekey\": [ { \"AmiLaunchIndex\": 0, \"ImageId\": \"ami-08435a7e5e61bc00b\", \"InstanceId\": \"i-0b307f45f965d4222\", \"InstanceType\": \"r5b.2xlarge\", \"KeyName\": \"gsus-prod-net-01-key\", \"LaunchTime\": \"2024-07-19T15:42:09+00:00\", \"Monitoring\": { \"State\": \"disabled\" }, \"Placement\": { \"AvailabilityZone\": \"us-east-1b\", \"GroupName\": \"\", \"Tenancy\": \"default\" }, \"Platform\": \"windows\", \"PrivateDnsName\": \"ip-172-23-43-196.ec2.internal\", \"PrivateIpAddress\": \"172.23.43.196\", \"ProductCodes\": [], \"PublicDnsName\": \"\", \"State\": { \"Code\": 16, \"Name\": \"running\" }, \"StateTransitionReason\": \"\", \"SubnetId\": \"subnet-03aa1e87392c5e63d\", \"VpcId\": \"vpc-0e8954f5da98fe98d\", \"Architecture\": \"x86_64\", \"BlockDeviceMappings\": [ { \"DeviceName\": \"/dev/sda1\", \"Ebs\": { \"AttachTime\": \"2024-07-19T15:41:21+00:00\", \"DeleteOnTermination\": false, \"Status\": \"attached\", \"VolumeId\": \"vol-0d39da483194fdffa\" } }, { \"DeviceName\": \"/dev/xvdf\", \"Ebs\": { \"AttachTime\": \"2023-07-06T19:30:04+00:00\", \"DeleteOnTermination\": false, \"Status\": \"attached\", \"VolumeId\": \"vol-07813f09191a4af20\" } }, { \"DeviceName\": \"/dev/xvdg\", \"Ebs\": { \"AttachTime\": \"2023-07-06T19:30:15+00:00\", \"DeleteOnTermination\": false, \"Status\": \"attached\", \"VolumeId\": \"vol-05d213e8588431029\" } }, { \"DeviceName\": \"/dev/xvdh\", \"Ebs\": { \"AttachTime\": \"2023-07-06T19:29:54+00:00\", \"DeleteOnTermination\": false, \"Status\": \"attached\", \"VolumeId\": \"vol-08849724290ff4ff0\" } }, { \"DeviceName\": \"/dev/xvdi\", \"Ebs\": { \"AttachTime\": \"2023-07-06T19:29:54+00:00\", \"DeleteOnTermination\": false, \"Status\": \"attached\", \"VolumeId\": \"vol-0711192d14ffdc4ff\" } }, { \"DeviceName\": \"/dev/xvdj\", \"Ebs\": { \"AttachTime\": \"2023-07-06T19:30:25+00:00\", \"DeleteOnTermination\": false, \"Status\": \"attached\", \"VolumeId\": \"vol-0d8db3494a8c7254c\" } }, { \"DeviceName\": \"/dev/xvdk\", \"Ebs\": { \"AttachTime\": \"2023-07-06T19:30:16+00:00\", \"DeleteOnTermination\": false, \"Status\": \"attached\", \"VolumeId\": \"vol-0cba7f29aec3d9a35\" } }, { \"DeviceName\": \"/dev/xvdl\", \"Ebs\": { \"AttachTime\": \"2023-07-06T19:30:15+00:00\", \"DeleteOnTermination\": false, \"Status\": \"attached\", \"VolumeId\": \"vol-0810a5ac78ecf0e11\" } }, { \"DeviceName\": \"/dev/xvdm\", \"Ebs\": { \"AttachTime\": \"2023-07-06T19:30:15+00:00\", \"DeleteOnTermination\": false, \"Status\": \"attached\", \"VolumeId\": \"vol-0e4ec2b5f1bf4d871\" } }, { \"DeviceName\": \"/dev/xvdn\", \"Ebs\": { \"AttachTime\": \"2023-07-06T19:29:54+00:00\", \"DeleteOnTermination\": false, \"Status\": \"attached\", \"VolumeId\": \"vol-092a40b78baeeac7f\" } }, { \"DeviceName\": \"/dev/xvdo\", \"Ebs\": { \"AttachTime\": \"2023-07-06T19:29:55+00:00\", \"DeleteOnTermination\": false, \"Status\": \"attached\", \"VolumeId\": \"vol-0b1e2d02d509f3f5e\" } }, { \"DeviceName\": \"/dev/xvdp\", \"Ebs\": { \"AttachTime\": \"2023-07-06T19:30:15+00:00\", \"DeleteOnTermination\": false, \"Status\": \"attached\", \"VolumeId\": \"vol-01cabf285c719295f\" } } ], \"ClientToken\": \"terraform-20230706192857135700000004\", \"EbsOptimized\": true, \"EnaSupport\": true, \"Hypervisor\": \"xen\", \"IamInstanceProfile\": { \"Arn\": \"arn:aws:iam::189682858917:instance-profile/hig/WindowsEC2Role\", \"Id\": \"AIPASYKP2XOSSBXPPW5LL\" }, \"NetworkInterfaces\": [ { \"Attachment\": { \"AttachTime\": \"2023-07-06T19:28:58+00:00\", \"AttachmentId\": \"eni-attach-09192f2a4fbf5a17d\", \"DeleteOnTermination\": false, \"DeviceIndex\": 0, \"Status\": \"attached\", \"NetworkCardIndex\": 0 }, \"Description\": \"\", \"Groups\": [ { \"GroupName\": \"gsus-prod-net-01-cmdb-sg-2022051216484634970000000a\", \"GroupId\": \"sg-04908349d0f791aec\" }, { \"GroupName\": \"gsus-prod-net-01-sql-sg-20220512164845082300000004\", \"GroupId\": \"sg-0b214ef19d0a4c0f7\" }, { \"GroupName\": \"gsus-prod-net-01-app-4587-v2sql-sg-20230707183929669400000001\", \"GroupId\": \"sg-0c2f333cbfb09b29f\" }, { \"GroupName\": \"gsus-prod-net-01-win-sg-20220512164844866600000001\", \"GroupId\": \"sg-0257bb3846318d5fd\" } ], \"Ipv6Addresses\": [], \"MacAddress\": \"12:90:ce:a5:ca:a1\", \"NetworkInterfaceId\": \"eni-0086b400faa99451d\", \"OwnerId\": \"189682858917\", \"PrivateDnsName\": \"ip-172-23-43-196.ec2.internal\", \"PrivateIpAddress\": \"172.23.43.196\", \"PrivateIpAddresses\": [ { \"Primary\": true, \"PrivateDnsName\": \"ip-172-23-43-196.ec2.internal\", \"PrivateIpAddress\": \"172.23.43.196\" }, { \"Primary\": false, \"PrivateDnsName\": \"ip-172-23-43-163.ec2.internal\", \"PrivateIpAddress\": \"172.23.43.163\" }, { \"Primary\": false, \"PrivateDnsName\": \"ip-172-23-43-141.ec2.internal\", \"PrivateIpAddress\": \"172.23.43.141\" } ], \"SourceDestCheck\": true, \"Status\": \"in-use\", \"SubnetId\": \"subnet-03aa1e87392c5e63d\", \"VpcId\": \"vpc-0e8954f5da98fe98d\", \"InterfaceType\": \"interface\" } ], \"RootDeviceName\": \"/dev/sda1\", \"RootDeviceType\": \"ebs\", \"SecurityGroups\": [ { \"GroupName\": \"gsus-prod-net-01-cmdb-sg-2022051216484634970000000a\", \"GroupId\": \"sg-04908349d0f791aec\" }, { \"GroupName\": \"gsus-prod-net-01-sql-sg-20220512164845082300000004\", \"GroupId\": \"sg-0b214ef19d0a4c0f7\" }, { \"GroupName\": \"gsus-prod-net-01-app-4587-v2sql-sg-20230707183929669400000001\", \"GroupId\": \"sg-0c2f333cbfb09b29f\" }, { \"GroupName\": \"gsus-prod-net-01-win-sg-20220512164844866600000001\", \"GroupId\": \"sg-0257bb3846318d5fd\" } ], \"SourceDestCheck\": true, \"Tags\": [ { \"Key\": \"Description\", \"Value\": \"\" }, { \"Key\": \"hig-owner\", \"Value\": \"mahesh.vidyasagar@thehartford.com\" }, { \"Key\": \"Billing\", \"Value\": \"APP-4587\" }, { \"Key\": \"Repository\", \"Value\": \"https://github.thehartford.com/HIG/gs_tfe_peoplesoft_app4587\" }, { \"Key\": \"Schedule\", \"Value\": \"None\" }, { \"Key\": \"mssql_adou\", \"Value\": \"OU=SQL-GS,OU=Windows,OU=Cloud,OU=Servers,DC=ad1,DC=prod\" }, { \"Key\": \"child_module_name\", \"Value\": \"terraform-aws-ec2-db-mssql\" }, { \"Key\": \"mssql_instancename\", \"Value\": \"MSSQLSERVER\" }, { \"Key\": \"module_name\", \"Value\": \"terraform-aws-ec2-instance\" }, { \"Key\": \"Pipeline\", \"Value\": \"https://jenkinsci.thehartford.com/iac-1/job/IAC/job/GS/job/PeopleSoft-4587/job/gsus_4587_peoplesoft_aws_gsprod/\" }, { \"Key\": \"IMDSv2\", \"Value\": \"Enabled\" }, { \"Key\": \"IMDSv2 Date\", \"Value\": \"07/06/2023, 19:30:24\" }, { \"Key\": \"PatchGroup\", \"Value\": \"Multiple DB Node AZ2\" }, { \"Key\": \"module_version\", \"Value\": \"v4.0.6\" }, { \"Key\": \"BackupDBGroup\", \"Value\": \"ec2-sqldb-prod-silver\" }, { \"Key\": \"mssql_version\", \"Value\": \"2019\" }, { \"Key\": \"AppOwner\", \"Value\": \"Mahesh Vidyasagar\" }, { \"Key\": \"hig-planit-appid\", \"Value\": \"APP-4587-PROD\" }, { \"Key\": \"ServiceTier\", \"Value\": \"Silver\" }, { \"Key\": \"DBNodes\", \"Value\": \"Secondary\" }, { \"Key\": \"Hostname\", \"Value\": \"a4587ue1pwsag06\" }, { \"Key\": \"mssql_datavolumes\", \"Value\": \"4\" }, { \"Key\": \"mssql_installtype\", \"Value\": \"SAG\" }, { \"Key\": \"hig-data-classification\", \"Value\": \"CompanyConfidential\" }, { \"Key\": \"PatchingFlag\", \"Value\": \"Tag Applied\" }, { \"Key\": \"Environment\", \"Value\": \"prod\" }, { \"Key\": \"Owner\", \"Value\": \"HIG\" }, { \"Key\": \"hig-billing\", \"Value\": \"APP-4587\" }, { \"Key\": \"mssql_environment\", \"Value\": \"prod\" }, { \"Key\": \"Domain\", \"Value\": \"ad1.prod\" }, { \"Key\": \"AppName\", \"Value\": \"PeopleSoft\" }, { \"Key\": \"ThirdParty\", \"Value\": \"No\" }, { \"Key\": \"child_module_version\", \"Value\": \"v2.2.1\" }, { \"Key\": \"Hosted Region\", \"Value\": \"US\" }, { \"Key\": \"Name\", \"Value\": \"PeopleSoft-4587-PROD-sag-06\" }, { \"Key\": \"BackupGroup\", \"Value\": \"ec2-nobackup\" }, { \"Key\": \"LOB\", \"Value\": \"Global Specialty\" }, { \"Key\": \"Patching Group\", \"Value\": \"Multiple DB Node AZ1\" }, { \"Key\": \"backup_s3bucket\", \"Value\": \"gs-gsus-prod-net-01-sqlbackup-us-east-1\" }, { \"Key\": \"DynatraceMonitoring\", \"Value\": \"true\" }, { \"Key\": \"mssql_collation\", \"Value\": \"Latin1_General_BIN\" }, { \"Key\": \"DataClassification\", \"Value\": \"CompanyConfidential\" }, { \"Key\": \"AppId\", \"Value\": \"APP-4587\" }, { \"Key\": \"AppTier\", \"Value\": \"DB\" }, { \"Key\": \"hig-environment-type\", \"Value\": \"PROD\" }, { \"Key\": \"backup_s3bucketkms\", \"Value\": \"arn:aws:kms:us-east-1:189682858917:key/e4b94c9a-82ad-4322-98ec-48d610f6548b\" }, { \"Key\": \"Comments\", \"Value\": \"\" } ], \"VirtualizationType\": \"hvm\", \"CpuOptions\": { \"CoreCount\": 4, \"ThreadsPerCore\": 1 }, \"CapacityReservationSpecification\": { \"CapacityReservationPreference\": \"open\" }, \"HibernationOptions\": { \"Configured\": false }, \"MetadataOptions\": { \"State\": \"applied\", \"HttpTokens\": \"required\", \"HttpPutResponseHopLimit\": 1, \"HttpEndpoint\": \"enabled\", \"HttpProtocolIpv6\": \"disabled\", \"InstanceMetadataTags\": \"disabled\" }, \"EnclaveOptions\": { \"Enabled\": false }, \"PlatformDetails\": \"Windows\", \"UsageOperation\": \"RunInstances:0002\", \"UsageOperationUpdateTime\": \"2023-07-06T19:28:58+00:00\", \"PrivateDnsNameOptions\": { \"HostnameType\": \"ip-name\", \"EnableResourceNameDnsARecord\": false, \"EnableResourceNameDnsAAAARecord\": false }, \"MaintenanceOptions\": { \"AutoRecovery\": \"default\" }, \"CurrentInstanceBootMode\": \"legacy-bios\", \"someOtherStuff\": \"othervalue\", \"c7n:MatchedFilters\": [ \"tag:AccountType\", \"tag:Attributes\", \"tag:EnvironmentType\", \"tag:ImageBuilder\" ] }, { \"AmiLaunchIndex\": 1, \"someOtherStuff\": \"othervalue\", \"c7n:MatchedFilters\": [ \"tag:ApplicationFailoverGroup\", \"tag:AppTier\", \"tag:Attributes\", \"tag:DBNodes\", \"tag:rk_aws_native_account_id\", \"tag:rk_cluster_id\", \"tag:rk_component\", \"tag:rk_instance_class\", \"tag:rk_job_id\", \"tag:rk_managed\", \"tag:rk_object\", \"tag:rk_restore_source_region\", \"tag:rk_restore_timestamp\", \"tag:rk_source_snapshot_native_id\", \"tag:rk_source_vm_native_id\", \"tag:rk_source_vm_native_name\", \"tag:rk_taskchain_id\", \"tag:rk_user\", \"tag:rk_version\" ] } ] }" | spath ``` data emulation above ```          
To ask an answerable data analytics question, follow these golden rules; nay, call them the four commandments: Illustrate data input (in raw text, anonymize as needed), whether they are raw events ... See more...
To ask an answerable data analytics question, follow these golden rules; nay, call them the four commandments: Illustrate data input (in raw text, anonymize as needed), whether they are raw events or output from a search that volunteers here do not have to look at. Illustrate the desired output from illustrated data. Explain the logic between illustrated data and desired output without SPL. If you also illustrate attempted SPL, illustrate actual output and compare with desired output, explain why they look different to you if that is not painfully obvious. Let's check. Your sample data are JSON fragments, not a complete JSON object.  Even the last update, no one can tell how deep that structure goes.  In other words, which exact JSON path leads to the subnode c7n:MatchedFilters.  I understand that your event size is enormous so you cannot show a complete sample.  But you should show the complete structure of the object. The phrase "show the count and tagname in tabular format" in the title seems to be an attempt to describe desired output.  But there are a thousand ways to show count and tagname in tabular format.  What is even a "count" in your text? No description of logic. N/A Without these, you are effectively asking volunteers to read your mind.  That is not a fair ask. Now, based on your last update, we can see that c7n:MatchedFilters is not a top node in your raw data.  It is a subnode in a very large array.  Is this large array a top node of your raw data?  If not, how many upper paths are there before we reach c7n:MatchedFilters?  Without an exact answer to this, there is no good answer to your question.  In the following, I will assume "yes" to this, i.e., I will assume that somekey[c7n:MatchedFilters] is the correct path. (Or, in SPL flattened notation, somekey{}.c7n:MatchedFilters{}. I will further assume that "show the count and tagname in tabular format" means to count events grouped by the string following "tag:" in each value of c7n:MatchedFilters.  In other words, I assume that this table is what you look for: tagName count AccountType 1 AppTier 1 ApplicationFailoverGroup 1 Attributes 2 DBNodes 1 EnvironmentType 1 ImageBuilder 1 rk_aws_native_account_id 1 rk_cluster_id 1 rk_component 1 rk_instance_class 1 rk_job_id 1 rk_managed 1 rk_object 1 rk_restore_source_region 1 rk_restore_timestamp 1 rk_source_snapshot_native_id 1 rk_source_vm_native_id 1 rk_source_vm_native_name 1 rk_taskchain_id 1 rk_user 1 rk_version 1 The above count is based on the fragments you pasted.  I constructed the following as a valid JSON object to mimic the structure.  If this is not the correct structure, you need to go back to the four commandments and start over.     { "somekey": [ { "AmiLaunchIndex": 0, "ImageId": "ami-08435a7e5e61bc00b", "InstanceId": "i-0b307f45f965d4222", "InstanceType": "r5b.2xlarge", "KeyName": "gsus-prod-net-01-key", "LaunchTime": "2024-07-19T15:42:09+00:00", "Monitoring": { "State": "disabled" }, "Placement": { "AvailabilityZone": "us-east-1b", "GroupName": "", "Tenancy": "default" }, "Platform": "windows", "PrivateDnsName": "ip-172-23-43-196.ec2.internal", "PrivateIpAddress": "172.23.43.196", "ProductCodes": [], "PublicDnsName": "", "State": { "Code": 16, "Name": "running" }, "StateTransitionReason": "", "SubnetId": "subnet-03aa1e87392c5e63d", "VpcId": "vpc-0e8954f5da98fe98d", "Architecture": "x86_64", "BlockDeviceMappings": [ { "DeviceName": "/dev/sda1", "Ebs": { "AttachTime": "2024-07-19T15:41:21+00:00", "DeleteOnTermination": false, "Status": "attached", "VolumeId": "vol-0d39da483194fdffa" } }, { "DeviceName": "/dev/xvdf", "Ebs": { "AttachTime": "2023-07-06T19:30:04+00:00", "DeleteOnTermination": false, "Status": "attached", "VolumeId": "vol-07813f09191a4af20" } }, { "DeviceName": "/dev/xvdg", "Ebs": { "AttachTime": "2023-07-06T19:30:15+00:00", "DeleteOnTermination": false, "Status": "attached", "VolumeId": "vol-05d213e8588431029" } }, { "DeviceName": "/dev/xvdh", "Ebs": { "AttachTime": "2023-07-06T19:29:54+00:00", "DeleteOnTermination": false, "Status": "attached", "VolumeId": "vol-08849724290ff4ff0" } }, { "DeviceName": "/dev/xvdi", "Ebs": { "AttachTime": "2023-07-06T19:29:54+00:00", "DeleteOnTermination": false, "Status": "attached", "VolumeId": "vol-0711192d14ffdc4ff" } }, { "DeviceName": "/dev/xvdj", "Ebs": { "AttachTime": "2023-07-06T19:30:25+00:00", "DeleteOnTermination": false, "Status": "attached", "VolumeId": "vol-0d8db3494a8c7254c" } }, { "DeviceName": "/dev/xvdk", "Ebs": { "AttachTime": "2023-07-06T19:30:16+00:00", "DeleteOnTermination": false, "Status": "attached", "VolumeId": "vol-0cba7f29aec3d9a35" } }, { "DeviceName": "/dev/xvdl", "Ebs": { "AttachTime": "2023-07-06T19:30:15+00:00", "DeleteOnTermination": false, "Status": "attached", "VolumeId": "vol-0810a5ac78ecf0e11" } }, { "DeviceName": "/dev/xvdm", "Ebs": { "AttachTime": "2023-07-06T19:30:15+00:00", "DeleteOnTermination": false, "Status": "attached", "VolumeId": "vol-0e4ec2b5f1bf4d871" } }, { "DeviceName": "/dev/xvdn", "Ebs": { "AttachTime": "2023-07-06T19:29:54+00:00", "DeleteOnTermination": false, "Status": "attached", "VolumeId": "vol-092a40b78baeeac7f" } }, { "DeviceName": "/dev/xvdo", "Ebs": { "AttachTime": "2023-07-06T19:29:55+00:00", "DeleteOnTermination": false, "Status": "attached", "VolumeId": "vol-0b1e2d02d509f3f5e" } }, { "DeviceName": "/dev/xvdp", "Ebs": { "AttachTime": "2023-07-06T19:30:15+00:00", "DeleteOnTermination": false, "Status": "attached", "VolumeId": "vol-01cabf285c719295f" } } ], "ClientToken": "terraform-20230706192857135700000004", "EbsOptimized": true, "EnaSupport": true, "Hypervisor": "xen", "IamInstanceProfile": { "Arn": "arn:aws:iam::189682858917:instance-profile/hig/WindowsEC2Role", "Id": "AIPASYKP2XOSSBXPPW5LL" }, "NetworkInterfaces": [ { "Attachment": { "AttachTime": "2023-07-06T19:28:58+00:00", "AttachmentId": "eni-attach-09192f2a4fbf5a17d", "DeleteOnTermination": false, "DeviceIndex": 0, "Status": "attached", "NetworkCardIndex": 0 }, "Description": "", "Groups": [ { "GroupName": "gsus-prod-net-01-cmdb-sg-2022051216484634970000000a", "GroupId": "sg-04908349d0f791aec" }, { "GroupName": "gsus-prod-net-01-sql-sg-20220512164845082300000004", "GroupId": "sg-0b214ef19d0a4c0f7" }, { "GroupName": "gsus-prod-net-01-app-4587-v2sql-sg-20230707183929669400000001", "GroupId": "sg-0c2f333cbfb09b29f" }, { "GroupName": "gsus-prod-net-01-win-sg-20220512164844866600000001", "GroupId": "sg-0257bb3846318d5fd" } ], "Ipv6Addresses": [], "MacAddress": "12:90:ce:a5:ca:a1", "NetworkInterfaceId": "eni-0086b400faa99451d", "OwnerId": "189682858917", "PrivateDnsName": "ip-172-23-43-196.ec2.internal", "PrivateIpAddress": "172.23.43.196", "PrivateIpAddresses": [ { "Primary": true, "PrivateDnsName": "ip-172-23-43-196.ec2.internal", "PrivateIpAddress": "172.23.43.196" }, { "Primary": false, "PrivateDnsName": "ip-172-23-43-163.ec2.internal", "PrivateIpAddress": "172.23.43.163" }, { "Primary": false, "PrivateDnsName": "ip-172-23-43-141.ec2.internal", "PrivateIpAddress": "172.23.43.141" } ], "SourceDestCheck": true, "Status": "in-use", "SubnetId": "subnet-03aa1e87392c5e63d", "VpcId": "vpc-0e8954f5da98fe98d", "InterfaceType": "interface" } ], "RootDeviceName": "/dev/sda1", "RootDeviceType": "ebs", "SecurityGroups": [ { "GroupName": "gsus-prod-net-01-cmdb-sg-2022051216484634970000000a", "GroupId": "sg-04908349d0f791aec" }, { "GroupName": "gsus-prod-net-01-sql-sg-20220512164845082300000004", "GroupId": "sg-0b214ef19d0a4c0f7" }, { "GroupName": "gsus-prod-net-01-app-4587-v2sql-sg-20230707183929669400000001", "GroupId": "sg-0c2f333cbfb09b29f" }, { "GroupName": "gsus-prod-net-01-win-sg-20220512164844866600000001", "GroupId": "sg-0257bb3846318d5fd" } ], "SourceDestCheck": true, "Tags": [ { "Key": "Description", "Value": "" }, { "Key": "hig-owner", "Value": "mahesh.vidyasagar@thehartford.com" }, { "Key": "Billing", "Value": "APP-4587" }, { "Key": "Repository", "Value": "https://github.thehartford.com/HIG/gs_tfe_peoplesoft_app4587" }, { "Key": "Schedule", "Value": "None" }, { "Key": "mssql_adou", "Value": "OU=SQL-GS,OU=Windows,OU=Cloud,OU=Servers,DC=ad1,DC=prod" }, { "Key": "child_module_name", "Value": "terraform-aws-ec2-db-mssql" }, { "Key": "mssql_instancename", "Value": "MSSQLSERVER" }, { "Key": "module_name", "Value": "terraform-aws-ec2-instance" }, { "Key": "Pipeline", "Value": "https://jenkinsci.thehartford.com/iac-1/job/IAC/job/GS/job/PeopleSoft-4587/job/gsus_4587_peoplesoft_aws_gsprod/" }, { "Key": "IMDSv2", "Value": "Enabled" }, { "Key": "IMDSv2 Date", "Value": "07/06/2023, 19:30:24" }, { "Key": "PatchGroup", "Value": "Multiple DB Node AZ2" }, { "Key": "module_version", "Value": "v4.0.6" }, { "Key": "BackupDBGroup", "Value": "ec2-sqldb-prod-silver" }, { "Key": "mssql_version", "Value": "2019" }, { "Key": "AppOwner", "Value": "Mahesh Vidyasagar" }, { "Key": "hig-planit-appid", "Value": "APP-4587-PROD" }, { "Key": "ServiceTier", "Value": "Silver" }, { "Key": "DBNodes", "Value": "Secondary" }, { "Key": "Hostname", "Value": "a4587ue1pwsag06" }, { "Key": "mssql_datavolumes", "Value": "4" }, { "Key": "mssql_installtype", "Value": "SAG" }, { "Key": "hig-data-classification", "Value": "CompanyConfidential" }, { "Key": "PatchingFlag", "Value": "Tag Applied" }, { "Key": "Environment", "Value": "prod" }, { "Key": "Owner", "Value": "HIG" }, { "Key": "hig-billing", "Value": "APP-4587" }, { "Key": "mssql_environment", "Value": "prod" }, { "Key": "Domain", "Value": "ad1.prod" }, { "Key": "AppName", "Value": "PeopleSoft" }, { "Key": "ThirdParty", "Value": "No" }, { "Key": "child_module_version", "Value": "v2.2.1" }, { "Key": "Hosted Region", "Value": "US" }, { "Key": "Name", "Value": "PeopleSoft-4587-PROD-sag-06" }, { "Key": "BackupGroup", "Value": "ec2-nobackup" }, { "Key": "LOB", "Value": "Global Specialty" }, { "Key": "Patching Group", "Value": "Multiple DB Node AZ1" }, { "Key": "backup_s3bucket", "Value": "gs-gsus-prod-net-01-sqlbackup-us-east-1" }, { "Key": "DynatraceMonitoring", "Value": "true" }, { "Key": "mssql_collation", "Value": "Latin1_General_BIN" }, { "Key": "DataClassification", "Value": "CompanyConfidential" }, { "Key": "AppId", "Value": "APP-4587" }, { "Key": "AppTier", "Value": "DB" }, { "Key": "hig-environment-type", "Value": "PROD" }, { "Key": "backup_s3bucketkms", "Value": "arn:aws:kms:us-east-1:189682858917:key/e4b94c9a-82ad-4322-98ec-48d610f6548b" }, { "Key": "Comments", "Value": "" } ], "VirtualizationType": "hvm", "CpuOptions": { "CoreCount": 4, "ThreadsPerCore": 1 }, "CapacityReservationSpecification": { "CapacityReservationPreference": "open" }, "HibernationOptions": { "Configured": false }, "MetadataOptions": { "State": "applied", "HttpTokens": "required", "HttpPutResponseHopLimit": 1, "HttpEndpoint": "enabled", "HttpProtocolIpv6": "disabled", "InstanceMetadataTags": "disabled" }, "EnclaveOptions": { "Enabled": false }, "PlatformDetails": "Windows", "UsageOperation": "RunInstances:0002", "UsageOperationUpdateTime": "2023-07-06T19:28:58+00:00", "PrivateDnsNameOptions": { "HostnameType": "ip-name", "EnableResourceNameDnsARecord": false, "EnableResourceNameDnsAAAARecord": false }, "MaintenanceOptions": { "AutoRecovery": "default" }, "CurrentInstanceBootMode": "legacy-bios", "someOtherStuff": "othervalue", "c7n:MatchedFilters": [ "tag:AccountType", "tag:Attributes", "tag:EnvironmentType", "tag:ImageBuilder" ] }, { "AmiLaunchIndex": 1, "someOtherStuff": "othervalue", "c7n:MatchedFilters": [ "tag:ApplicationFailoverGroup", "tag:AppTier", "tag:Attributes", "tag:DBNodes", "tag:rk_aws_native_account_id", "tag:rk_cluster_id", "tag:rk_component", "tag:rk_instance_class", "tag:rk_job_id", "tag:rk_managed", "tag:rk_object", "tag:rk_restore_source_region", "tag:rk_restore_timestamp", "tag:rk_source_snapshot_native_id", "tag:rk_source_vm_native_id", "tag:rk_source_vm_native_name", "tag:rk_taskchain_id", "tag:rk_user", "tag:rk_version" ] } ] }   Basically, I took the valid element from your latest sample, and added the fragment containing the c7n:MatchedFilters key as a second element of top level key somekey. So, the second element does not have all the other keys as they are not of interest to this particular question. After this construction, I think I get a handle to why the basic technique doesn't work.  But let me explain the basic technique first.     This is assuming that Splunk search head already extracted all fields from raw data, which is the norm.  But your raw events are so huge, even that single valid array element you posted is >15KB.  Splunk has this default in limits.conf   [spath] # Number of characters to read from an XML or JSON event when # auto extracting. extraction_cutoff = 5000   You can increase this to be larger than the maximum size of your events.  But that may require a lot of RAM. Without touching extraction_cutoff, you can directly reach c7n:MatchedFilters with spath command.  Like this:   | spath path=somekey{} | spath input=somekey{} path=c7n:MatchedFilters{} output=tagName | stats count by tagName | eval tagName = replace(tagName, "tag:", "")    
OK. There are several issues with your searches and maybe your data. 1. dedup _raw - that means you're removing duplicates of _whole events_. Which implies you have those duplicate events (otherwise... See more...
OK. There are several issues with your searches and maybe your data. 1. dedup _raw - that means you're removing duplicates of _whole events_. Which implies you have those duplicate events (otherwise the command would be unnecessary). That is not a typical situation. Or you simply don't need this comman (or you meant something completely different by it) 2. The table command very rarely makes sense in the middle of the search. Additionally, it is a performance hit in a distributed environment because it moves all procesing to search head tier so you don't benefit from map-reduce distributed processing. Table can be used at the end to present results in tabular view. 3. If you want to count distinct values of a field do just that. Doing strange stuff with dedup can fail you miserably if you have multivalued fields. So again, if you want to count distinct values of those two fields (and so far you haven't said otherwise) the search with single stats command containing two aggregations is completely sufficient.  
Hello, I am following this tutorial to create a Splunk app using React on macOS Sonoma: https://splunkui.splunk.com/Toolkits/SUIT/AppTutorial However, I am not able to get it to work. The 'start' v... See more...
Hello, I am following this tutorial to create a Splunk app using React on macOS Sonoma: https://splunkui.splunk.com/Toolkits/SUIT/AppTutorial However, I am not able to get it to work. The 'start' view is simply not added to the app views on Splunk, even though they are there in the files in my app. I wasn't even able to launch the app before I set it to 'Visible' by going to 'Manage Apps' and editing its properties. It should have been visible because it is set as such under my app.conf. But after I launched it, I was redirected to the search page (image below). If I go to the URL http://localhost:8000/en-US/app/my-splunk-app/start, I get the 'Page not found' error page. Could someone please help me with this?
@PickleRick Thank you for your response and patience. 
Thank you @yuanliu  Had to modify a little to make it work | rename log.message as _raw | rex mode=sed "s/errors=(.+) fields=(.+)/errors=\"\1\" fields=\"\2\"/" | rex field=_raw "path=(?P<path... See more...
Thank you @yuanliu  Had to modify a little to make it work | rename log.message as _raw | rex mode=sed "s/errors=(.+) fields=(.+)/errors=\"\1\" fields=\"\2\"/" | rex field=_raw "path=(?P<path>.*) feedType=(?P<feedType>.*) sku=(?P<sku>.*) status=(?P<status>.*) errorCount=(?P<errorCount>.*) errors=(?P<errors>.*) fields=(?P<fields>.*)" | table path, feedType, sku, status, errorCount, errors, fields
1. I need to dedup servers in first query. Count only Unique Servers.  2. I need to consider all the Servers with duplicates in the second query and then CompletedStatus then count the completed.  ... See more...
1. I need to dedup servers in first query. Count only Unique Servers.  2. I need to consider all the Servers with duplicates in the second query and then CompletedStatus then count the completed.  |.   index=abc source="/opt/src/datasource.tmp" | dedup _raw | dedup Servers | table Servers | stats count(Servers) as Total ||.   index=abc source="/opt/src/datasource.tmp" | dedup _raw | table Servers, CompletedStatus | stats count(CompletedStatus) as Completed   So 2 queries are compulsory. 
But since transaction is one of the cursed commands you can - assuming Job is unique - do | stats min(_time) as start max(_time) as end by Job | eval duration=end-start
1. Each of your searches is unnecessarily complicated (and thus less efficient than it could be). You can cut the whole dedup and table part and get the same results. 2. Are you sure that count(Serv... See more...
1. Each of your searches is unnecessarily complicated (and thus less efficient than it could be). You can cut the whole dedup and table part and get the same results. 2. Are you sure that count(Servers) and count(CompletedServers) respectively is what you need? It might be - I don't know your use case - but typically it's either the general count of results or distinct count of a field. Counts of fields are rare the proper solution. But there are uses for it. Yours might be one of them so just asking if you know what you're doing. 3. You don't have to join anything. Assuming the counts are OK, you simply need one search to calculate both stats index=abc source="/opt/src/datasource.tmp" stats count(CompletedServers) as Completed count(Servers) as Total And that's it.
Hi Sir, I have two queries like below and I need to join both of queries and then divide Completed by Total and calculate percentage and display the percentage in bracket along with Completed count ... See more...
Hi Sir, I have two queries like below and I need to join both of queries and then divide Completed by Total and calculate percentage and display the percentage in bracket along with Completed count as shown in the above screenshot in panel. 1.  index=abc source="/opt/src/datasource.tmp" | dedup _raw | table Servers | stats count(Servers) as Total 2.  index=abc source="/opt/src/datasource.tmp" | dedup _raw | table CompletedServers | stats count(CompletedServers) as Completed    
Assuming the timestamp you want is in _time,  you could use transaction to get the duration | transaction Job If not, you could reassign the _time field to be the time you want.
My query returns these events, i need to compute the total time A was in this state and total time B was in this state. My thought is to subtract the TImestamp of the first A from the most recent A a... See more...
My query returns these events, i need to compute the total time A was in this state and total time B was in this state. My thought is to subtract the TImestamp of the first A from the most recent A and so on for B but cant figure out the right way to do this?   Timestamp Job Date LoggedTime Ready 1728092168.000000 A 10/4/2024 21:36:03 1 1728092163.000000 A 10/4/2024 21:35:50 1 1728092150.000000 A 10/4/2024 21:35:27 1 1728092127.000000 A 10/4/2024 21:35:16 1 1728090335.000000 B 10/4/2024 21:05:15 2 1728090315.000000 B 10/4/2024 21:05:03 2 1728090303.000000 B 10/4/2024 21:04:53 2 1728090293.000000 B 10/4/2024 21:04:31 2
| stats count(eval(status="error")) AS FailCount count(eval(status="planning")) AS TotalPlanned | eval percentageFailed=(FailCount/TotalPlanned)*10
This is not a complete event. If you don't want to provide that, perhaps you could tell us what fields have already been extracted?
If the same search on the same data ran within the same app (are you running both searches from the same app?) yields different results for two different users there must be some difference in config... See more...
If the same search on the same data ran within the same app (are you running both searches from the same app?) yields different results for two different users there must be some difference in configuration. It can be either due to one of the users having custom settings defined on a per user level or difference in permissions to the app the settings (probably either extractions or calculated fields) are defined in. Compare settings for relevant sourcetype with app and user context using btool.
Did you create any custom field extraction? If so, check if the field extraction's permissions are set to "global." It might currently be private to you, which could explain why only you're getting t... See more...
Did you create any custom field extraction? If so, check if the field extraction's permissions are set to "global." It might currently be private to you, which could explain why only you're getting the correct results.
No, I am using 9.1 and 9.3. However, I am experiencing the same problem on Red Hat 8. I will start a new thread as per your advice. 
Hi @whitecat001 ... this looks like a mistaken eval field assignment or table printing issue.  pls share with us your search query(remove any sensitive details) and/or the other user's search query.... See more...
Hi @whitecat001 ... this looks like a mistaken eval field assignment or table printing issue.  pls share with us your search query(remove any sensitive details) and/or the other user's search query. then troubleshooting this will become easy one, thanks. 
It may help to think of a subsearch like a macro.  Just as the contents of a macro replace the macro name in a query, so, too, do the results of a subsearch replace the subsearch text in the query.  ... See more...
It may help to think of a subsearch like a macro.  Just as the contents of a macro replace the macro name in a query, so, too, do the results of a subsearch replace the subsearch text in the query.  Therefore, it's important that the results of the subsearch make sense, semantically. In the example query, once the subsearch completes, Splunk tries to execute this index=abc status=error | stats count AS FailCount (( TotalPlanned=761 )) | eval percentageFailed=(FailCount/TotalPlanned)*100 which is not a valid query. One fix is to use the appendcols command with the subsearch index=abc status=error | stats count AS FailCount | appendcols [ search index=abc status=planning | stats count AS TotalPlanned | table TotalPlanned ] | eval percentageFailed=(FailCount/TotalPlanned)*100  
Out of curiosity - why foreach? mvmap() wouldn't suffice? I find foreach... a bit unsplunky