About roopeshetty

roopeshetty

our sourcetype=kafka_metrics, now i have modified the props.conf as below, but still the same issue. Do i need to change anything in transforms.conf as well?

roopeshetty

yes, source="kafka_metrics://kafka_metrics" as you can see here below;

roopeshetty

Hmm, not sure whats wrong with my setup, I have modified the props.conf as below and restarted the splunk service. Still its coming as a single event.

roopeshetty

yes I restarted it. below is the RAW data how its getting caputured. Below is a single event, but we need to split or break every line as a indivisual events . ----------------------------------------------------------------------------------------------------------------------------------- # HELP confluent_kafka_server_request_bytes The delta count of total request bytes from the specified request types sent over the network. Each sample is the number of bytes sent since the previous data point. The count is sampled every 60 seconds. # TYPE confluent_kafka_server_request_bytes gauge confluent_kafka_server_request_bytes{kafka_id="trtrtrt",principal_id="4343fgfg",type="ApiVersions",} 0.0 1684127220000 confluent_kafka_server_request_bytes{kafka_id="trtrtrt",principal_id="5656ghgh",type="ApiVersions",} 0.0 1684127220000 confluent_kafka_server_request_bytes{kafka_id="trtrtrt",principal_id="5656ghgh",type="Fetch",} 18000.0 1684127220000 confluent_kafka_server_request_bytes{kafka_id="trtrtrt",principal_id="4343fgfg",type="Metadata",} 0.0 1684127220000 confluent_kafka_server_request_bytes{kafka_id="trtrtrt",principal_id="5656ghgh",type="Metadata",} 0.0 1684127220000 # HELP confluent_kafka_server_response_bytes The delta count of total response bytes from the specified response types sent over the network. Each sample is the number of bytes sent since the previous data point. The count is sampled every 60 seconds. # TYPE confluent_kafka_server_response_bytes gauge confluent_kafka_server_response_bytes{kafka_id="trtrtrt",principal_id="4343fgfg",type="ApiVersions",} 0.0 1684127220000 confluent_kafka_server_response_bytes{kafka_id="trtrtrt",principal_id="5656ghgh",type="ApiVersions",} 0.0 1684127220000 confluent_kafka_server_response_bytes{kafka_id="trtrtrt",principal_id="5656ghgh",type="Fetch",} 5040.0 1684127220000 confluent_kafka_server_response_bytes{kafka_id="trtrtrt",principal_id="4343fgfg",type="Metadata",} 0.0 1684127220000 confluent_kafka_server_response_bytes{kafka_id="trtrtrt",principal_id="5656ghgh",type="Metadata",} 0.0 1684127220000 confluent_kafka_server_response_bytes{kafka_id="yuuyu",principal_id="hyyu44",type="ApiVersions",} 0.0 1684127220000 confluent_kafka_server_response_bytes{kafka_id="yuuyu",principal_id="hyyu44",type="Metadata",} 0.0 1684127220000 # HELP confluent_kafka_server_received_bytes The delta count of bytes of the customer's data received from the network. Each sample is the number of bytes received since the previous data sample. The count is sampled every 60 seconds. # TYPE confluent_kafka_server_received_bytes gauge confluent_kafka_server_received_bytes{kafka_id="yuyuyu",topic="tytytytuyu-processing-log",} 0.0 1684127220000 confluent_kafka_server_received_bytes{kafka_id="yuyuyu",topic="hhhh.kk.json.cs.order.fulfilment.auto.pricing.retry",} 553822.0 1684127220000 confluent_kafka_server_received_bytes{kafka_id="yuyuyu",topic="hhhh.kk.json.cs.order.yy",} 0.0 1684127220000 confluent_kafka_server_received_bytes{kafka_id="uiuiui",topic="topic_0",} 35385.0 1684127220000 # HELP confluent_kafka_server_sent_bytes The delta count of bytes of the customer's data sent over the network. Each sample is the number of bytes sent since the previous data point. The count is sampled every 60 seconds. # TYPE confluent_kafka_server_sent_bytes gauge confluent_kafka_server_sent_bytes{kafka_id="uiuiuiu",topic="_confluent-controlcenter-7-3-0-0-MonitoringMessageAggregatorWindows-THREE_HOURS-changelog",} 0.0 1684127220000 confluent_kafka_server_sent_bytes{kafka_id="yuyuyuy",topic="kafka-connector.connect-configs",} 0.0 1684127220000 confluent_kafka_server_sent_bytes{kafka_id="yuyuyuy",topic="kafka.replicator-configs",} 0.0 1684127220000 confluent_kafka_server_sent_bytes{kafka_id="yuyuyuy",topic="tytyty-log",} 0.0 1684127220000 confluent_kafka_server_sent_bytes{kafka_id="yuyuyuy",topic="success-lcc-trttytyty",} 0.0 1684127220000 # HELP confluent_kafka_server_received_records The delta count of records received. Each sample is the number of records received since the previous data sample. The count is sampled every 60 seconds. # TYPE confluent_kafka_server_received_records gauge confluent_kafka_server_received_records{kafka_id="yuyuyu",topic="tytytytuyu-processing-log",} 0.0 1684127220000 confluent_kafka_server_received_records{kafka_id="yuyuyu",topic="hhhh.kk.json.cs.order.fulfilment.auto.pricing.retry",} 24.0 1684127220000 confluent_kafka_server_received_records{kafka_id="yuyuyu",topic="hhhh.kk.json.cs.order.yy",} 0.0 1684127220000 confluent_kafka_server_received_records{kafka_id="uiuiui",topic="topic_0",} 126.0 1684127220000 # HELP confluent_kafka_server_sent_records The delta count of records sent. Each sample is the number of records sent since the previous data point. The count is sampled every 60 seconds. # TYPE confluent_kafka_server_sent_records gauge confluent_kafka_server_sent_records{kafka_id="yuyuyu",topic="tytytytuyu-processing-log",} 0.0 1684127220000 confluent_kafka_server_sent_records{kafka_id="yuyuyu",topic="hhhh.kk.json.cs.order.fulfilment.auto.pricing.retry",} 24.0 1684127220000 confluent_kafka_server_sent_records{kafka_id="yuyuyu",topic="hhhh.kk.json.cs.order.yy",} 0.0 1684127220000 confluent_kafka_server_sent_records{kafka_id="uiuiuiu",topic="_confluent-command",} 0.0 1684127220000 confluent_kafka_server_sent_records{kafka_id="uiuiuiu",topic="_confluent-controlcenter-7-3-0-0-AlertHistoryStore-changelog",} 0.0 1684127220000 confluent_kafka_server_sent_records{kafka_id="yuyuyuy",topic="kafka-connector.connect-configs",} 0.0 1684127220000 confluent_kafka_server_sent_records{kafka_id="yuyuyuy",topic="kafka.replicator-configs",} 0.0 1684127220000 confluent_kafka_server_retained_bytes{kafka_id="uiuiuiu",topic="_confluent-controlcenter-7-3-0-0-monitoring-aggregate-rekey-store-repartition",} 0.0 1684127220000 confluent_kafka_server_retained_bytes{kafka_id="uiuiuiu",topic="_confluent-controlcenter-7-3-0-0-monitoring-message-rekey-store",} 7.148088E7 1684127220000 confluent_kafka_server_retained_bytes{kafka_id="uiuiuiu",topic="_confluent-controlcenter-7-3-0-0-monitoring-trigger-event-rekey",} 5.0344769E7 1684127220000

roopeshetty

tried this but its not working.

roopeshetty

Hi Team, I am collecting metrics using API calls for every 5 minutes , but all the metrics are coming as a single event as attached screen shot I need to break these events as individuals (which ever events starting from text “confluent_kafka_”) . I have edited my props.conf as below but its not coming as expected still its coming as a single event. Can some one please guide me how to do it. [source::kafka_metrics://kafka_metrics] LINE_BREAKER = (confluent_kafka_)(\s) SHOULD_LINEMERGE = false

roopeshetty

Hi Team, I am collecting metrics using API calls for every 5 minutes , but all the metrics are coming as a single event as below for every 5 minutes. I have attadhed the screen shot here. confluent_kafka_server_request_bytes{kafka_id="tythtyt",principal_id="sa-r29997",type="Fetch",} 2092668.0 1683872880000 confluent_kafka_server_memory{kafka_id="yyyy",topic="host002.json.cs.tt.gg",} 0.0 1683872880000 I need to break these events as individuals (which ever events starting from text “confluent_kafka_”) . I have edited my props.conf as below but its not coming as expected still its coming as a single event. Can some one please guide me how to do it. [source::kafka_metrics://kafka_metrics] LINE_BREAKER = (confluent_kafka_)(\s) SHOULD_LINEMERGE = false

roopeshetty

Hi Team, I am collecting metrics using API calls for every 5 minutes , but all the metrics are coming as a single event as below for every 5 minutes. confluent_kafka_server_request_bytes{kafka_id="tythtyt",principal_id="sa-r29997",type="Fetch",} 2092668.0 1683872880000 confluent_kafka_server_request_bytes{kafka_id="tythtyt",principal_id="sa-9pyr8m",type="Metadata",} 1849.0 1683872880000 confluent_kafka_server_request_bytes{kafka_id="tythtyt",principal_id="sa-r29997",type="Metadata",} 66279.0 1683872880000 confluent_kafka_server_request_bytes{kafka_id="tythtyt",principal_id="u-09pr56",type="Metadata",} 0.0 1683872880000 confluent_kafka_server_response_bytes{kafka_id="rtrtt",principal_id="sa-y629ok",type="Fetch",} 5019.0 1683872880000 confluent_kafka_server_response_bytes{kafka_id="trtrt",principal_id="sa-8gg7jr",type="Metadata",} 0.0 1683872880000 confluent_kafka_server_memory{kafka_id="yyyy",topic="host002.json.cs.tt",} 1.0 1683872880000 confluent_kafka_server_memory{kafka_id="yyyy",topic="host002.json.cs.tt.enriched",} 1.0 1683872880000 confluent_kafka_server_memory{kafka_id="yyyy",topic="host002.json.cs.tt.fulfilment.auto",} 1.0 1683872880000 confluent_kafka_server_memory{kafka_id="yyyy",topic="host002.json.cs.tt.gg",} 0.0 1683872880000 I need to break these events as individuals (which ever events starting from text “confluent_kafka_”) . I have edited my props.conf as below but its not coming as expected still its coming as a single event. Can some one please guide me how to do it. [source::kafka_metrics://kafka_metrics] LINE_BREAKER = (confluent_kafka_)(\s) SHOULD_LINEMERGE = false

roopeshetty · ‎04-14-2023

Hi I know there are many splunk add on's available to collect azure monitor metrics which collects the logs using app id, client id, directory and secret key. My question is how these add on's actually authenticate and pulls these azure metrics, as azure these metrics can only be retrieved using bearer tokens. If we create bearer token in azure monitor its valid for only 24 hours. Actually we need to create some custom add ons to pull azure metrics, but unable to crack how to authenticate. Can some one please guide us.

roopeshetty · ‎03-24-2023

Hi We need to ingest only those events which starts with any of the below strings ; (please note its starts with not contains) create, drop, login, logout, alter, delete For example “ login success for user Peter” "create action success for user Martin" we have edited the transforms.conf as below [setnull] REGEX = . DEST_KEY = queue FORMAT = nullQueue [setparsing] REGEX = login|logout!|create|drop|alter|DELETE DEST_KEY = queue FORMAT = indexQueue But this is allowing all events which contains above strings. But our requirement is event should start with above strings; Could you please help us with a PCRE regex for above conditions on transforms.conf?

roopeshetty · ‎01-09-2023

Hi , We are getting lot of events into our Splunk which is filling up the our computer disk storage rapidly. Our search query is “index="cloud" sourcetype=sls_sourcetype” Upon investigating we decided that we need to capture only below events; Collect only below events Where field name “sql” contains below 4 values ; logout! login success! login failed! create Actually field name “sql” contains more than 100 values so we need to exclude all of them and capture only those events which has mentioned above 4 values. We tried below configurations in transforms.conf and props.conf files but none of them are giving desired results. Can some please help us with the correct settings as this is the first time we are working transforms.conf and props.conf settings. transforms.conf as below; [setnull] SOURCE_KEY=_raw REGEX = * DEST_KEY = queue FORMAT = nullQueue [setqueue] SOURCE_KEY=_raw REGEX = sql=log(out! |in success! |in failure!) DEST_KEY = queue FORMAT = indexQueue Props.conf as below; [sourcetype::sls_sourcetype] TRANSFORMS-set= setnull,setparsing

roopeshetty · ‎01-21-2022

thanks @johnhuang it worked.

roopeshetty · ‎01-21-2022

Hi Guys I have a query like this <query>| stats avg(CurrentConnections) as CC by host And the output is as below with multiple rows But we have a requirement to get all the results in a single row (all outputs are required but in a single row instead of multiple rows one after one) some thing like this; host CC server01 server02 server03 server04 server05 server06 368.333333333333 365.333333333333 345.333333333333 379.666666666666 356.333333333333 381.666666666666 Can someone please guide us how to do this?

roopeshetty · ‎12-21-2021

thanks a lot aasabatini. Its working now as expected. Thanks again.

roopeshetty · ‎12-21-2021

Hi Team, We are collecting data from Alibaba cloud through a heavy forwarder (using Alibaba add-ons) and pushing the data to our splunk cloud. But what we are seeing is its collecting all data from the Alibaba cloud which is huge in size, and upon validating it we realized that below events are making 80% of the whole events and it is not required to us. So we want to exclude below events (rule_result=pass and status=200) from being collected. We know this can be done by editing Props.conf File, but we have been trying it from long for it but we are not successful. Can someone please advise us how to edit this Props.conf file and get these below events (rule_result=pass and status=200) excluded from the heavy forwarder. index= alibaba source="alibaba:cloudfirewall" rule_result=pass index=alibaba source="alibaba:waf" status=200

roopeshetty · ‎11-22-2021

Hi Guys, We are using “Microsoft Azure App for Splunk” by getting inputs from “Splunk Add-on for Microsoft Cloud Services” and “Microsoft Azure Add-on for Splunk”. We have created the inputs for Audit logs and security center logs and we are getting the data in dashboards “Microsoft Azure App for Splunk” as expected. But one problem is we don’t get to see or select “Subscription” dropdown menu even though we are seeing data of multiple subscription on dashboards. In “Subscription” dropdown menu we are seeing a message as “Search produced no result”. Any idea why is like this? Or any bug in that app? Please suggest.

roopeshetty · ‎05-05-2021

Hi Guys, We can see there are 6 hosts which are sending bulk events (logs) to splunk. But we don’t know who is using these host events in spunk. Is there any way we can identify the searches, reports, alerts or dashboards where these hosts events are being used. The purpose is if no where these logs are being used then we can stop the forwarders from those hosts.

roopeshetty · ‎04-20-2021

Thanks Giuseppe, you fixed our issue instantly. really appreciate your help.

roopeshetty · ‎04-20-2021

Hi Guys, We have this query which will give the output as a table with 3 columns in it by name Servername, ServerIP and ServerLocation. | inputlookup WindowsTag.csv Now we are trying to add the Search input using a token , so modified the query as below; | inputlookup WindowsTag.csv | search Servername=$mytoken$ But here problem is when we search a text it will search only in column “Servername”, but we want to search for that text in all the available columns (Servername, ServerIP and ServerLocation) in that table. Can we do that? We tried tweaking the query in many ways but no luck. Can someone please guide us how to do this?

roopeshetty · ‎04-16-2021

Hi Guys, We have a alert setup for below query with condition when ever there is Number of Result greater than 0. <query>| table ServerName UserName SupportTeam This query runs once in a day and triggers the alerts perfectly as expected. But now the management wants some modifications in alerting condition in such a way that trigger a alert if any ServerName is present continuously in 3 runs then only it should trigger the alert. Example : If ServerName "APAC_Japan01” present in output of all 3 days query, then only it should alert for this ServerName, if it presents only on 1st and 2nd day output results then it should not trigger the alert. Can this be possible?

Posts	55
Solutions	0
Karma Given	0
Karma Received	1
Member Since	‎03-24-2016

Online Status	Offline
Date Last Visited	3 weeks ago

Why are events breaking using props conf?

multiple lines are coming as single event

single event for mutiple lines

Azure connection concern

What PCRE regex can we use in our transforms.conf?

How to solve setnull issue while filtering require...

result in single row

avoid some events

Azure app issue

search help

Re: events breaking using props conf

Re: events breaking using props conf

Re: events breaking using props conf

Re: events breaking using props conf

Re: events breaking using props conf

Why are events breaking using props conf?

multiple lines are coming as single event

single event for mutiple lines

Azure connection concern

What PCRE regex can we use in our transforms.conf?

How to solve setnull issue while filtering require...

Re: result in single row

result in single row

Re: avoid some events

avoid some events

Azure app issue

search help

Re: searching issue

searching issue

alerts query