@livehybrid  Hi thanks for response but from splunk I know I can use the way add on but client does not know about azure monitor logs can be send to event hubs or not they are providing logs in csv files so how to take it from here or should we ask them to send to event hubs to use addons or i was looking for a way around if anyone or you know to help them and me. ... View more

Thanks @kiran_panchavat  it helped alot I can look into this.  but if client denies for the azure credentials which have high chances ..! is there any other way also to do this ? ... View more

Hi @kiran_panchavat  thanks for replying  the concern is using the Splunk Add-on for Microsoft Cloud Services in splunk enterprise needs azure side configurations as well I think and I will be having no access to it as its client side and they will just provide us with the data only ? is it possible you can clear my doubt about add on services using azure config as well ... View more

Can you please check syntax and everything is correct? I have used the same thing in my terminal and after this I am writing command in my search and reporting search bar that is  index="mycloud" sourcetype="httpevent" | table message props.conf [source::http: LogStash] sourcetype = httpevent TRANSFORMS-00 = securelog_set_default_metadata TRANSFORMS-01 = securelog_override_raw transforms.conf [securelog_set_default_metadata] INGEST_EVAL = host = json_extract(_raw, "host.name") [securelog_override_raw] INGEST_EVAL = _raw = json_extract(_raw, "message") ... View more

so if i am getting below data This below using spath for message field: 2025-02-11 20:20:46.192 [com.bootstrapserver.runtim] DEBUG Stability run result com.cmp.bootstrapserver.runtime.internal.api.RStability@36464gf 2025-02-11 20:20:46 [com.bootstrapserver.runtim] DEBUG Stability run result :com.cmp.bootstrapserver.runtime.interndal.api.RStability@373638cgf This below using props and transforms for message field:   2025-02-11 20:20:46.192 [com.bootstrapserver.runtim] DEBUG Stability run result com.cmp.bootstrapserver.runtime.internal.api.RStability@36464gf 2025-02-11 20:20:46 [com.bootstrapserver.runtim] DEBUG Stability run result :com.cmp.bootstrapserver.runtime.interndal.api.RStability@373638cgf Both way got same data so is it correct or wrong? or it should be different after using transforms and props i need to know this first and if not then what? ... View more

Hi I check the fields are correct but i want to know that using spath for extraction from json and using props and transforms gives same result or not? As I am getting same message value that was unstructured earlier which was coming after using below statement in conf file   [securelog_override_raw] INGEST_EVAL = message := json_extract(_raw, "message") the value in message is still same unstructured with lots of messed up data like below do i have to separate this myself in this case using search queries? 2025-02-11 20:20:46.192 [com.bootstrapserver.runtim] DEBUG Stability run result : com.cmp.bootstrapserver.runtime.internal.api.RStability@36464gf 2025-02-11 20:20:46 [com.bootstrapserver.runtim] DEBUG Stability run result :com.cmp.bootstrapserver.runtime.interndal.api.RStability@373638cgf after spath same message came from message field and now using conf file with prof and tranforms its still the same. will it extract like this only?   ... View more

yes fields are correct but values coming out are same as what i was doing using spath statement. is there any difference i will get if iam using props and transofrms conf ... View more

Hi, I wrote this to check if its working or not .. props.conf [source::http:my LogStash] sourcetype = httpevent TRANSFORMS-00 = securelog_set_default_metadata TRANSFORMS-01 = securelog_override_raw transforms.conf [securelog_set_default_metadata] INGEST_EVAL = host := json_extract(_raw, "host.name") [securelog_override_raw] INGEST_EVAL = message := json_extract(_raw, "message") now which query i need to run in search reporting i wrote  index="" sourcetype=""  | table host,message,_raw ( with this i am only seeing the data which i was able to do with the spath in search query) but i did not got any extracted data its the same values iam getting what can be the reason or how to check if my props and transforms is working correctly. I see my results same non extracted can you provide some query to check or some information which i can refer? ... View more

Hi, I have a problem that i do not have props.conf and transforms.conf because  i do not have agent or forwarder we just getting data through HEC. so we do not have props and conf file and hec is bind to host that is indexer directly? what to do in this case or we have to create props.conf? ... View more

Ok got it after doing all these configuration when i will search my index with its source i will get my result in logs itself right?  ... View more

I was asking i have to use this below code same is this a constant predefined or i have to edit and  add my values ? for example: INGEST_EVAL = host := json_extract(_raw, "host.name")                                                                                              |                                                                                          my host name like host.ABC INGEST_EVAL = _raw := json_extract(_raw, "message")                                                                                   |                                                                  is _raw is predinfed or we need to add something here and same for message doe we have to add whole json ? in props.conf: [source::http:logstash] TRANSFORMS-00 = securelog_set_default_metadata TRANSFORMS-01 = securelog_override_raw in transforms.conf: [securelog_set_default_metadata] INGEST_EVAL = host := json_extract(_raw, "host.name") [securelog_override_raw] INGEST_EVAL = _raw := json_extract(_raw, "message")   ... View more

so the example you sent this same has to be set only in transform.conf have to change host.name right? ... View more

{"input":{"type":"container"},"message":"2025-01-27 18:45:51.546GMT+0000 gm [com.bootserver.runtim] DEBUG Stability run result : com.bootserver.runtime.internal.api.RStability@6373","kubernetes":{"namespace":"mg-prd","labels":{"service_app_my_com/gm-mongodb":"client","app_kubernetes_io/name":"gm-core","app_kubernetes_io/instance":"mg-prd-release-gm-core","service_app_my_com/gm-external-https":"server","service_app_my_com/gm-internal-https":"both","pod-template-hash":"5889b666","app_my_com/chart":"gm-core-0.1.1","app_kubernetes_io/part-of":"my-JKT","app_my_com/service":"mela","app_kubernetes_io/managed-by":"mela","app_kubernetes_io/component":"my-race","app_kubernetes_io/version":"2.0.002333","app_my_com/release":"mg-prod-release","service_app_my_com/fm-internal-bridge":"client","app_my_com/name":"gm-core"},"container":{"name":"gm-core"}},"host":{"name":"mg-prd"},"@timestamp":"2025-01-:25:31","environment":"pr","event":{"original":"2025-01-27 18:25:31.426GMT+0000 FM [com.my.bootserver.runtim] DEBUG Stability run result : com.my.bootserver.runtime.internal.api.RStability@6373"}} ... View more

Hi I think i was not able to explain you properly if you can have a look now in proper way and tell will be helpful as iam getting data in json format already so. I am getting  cloud logstash data and its sourcetype is httpevent now below is the output iam already getting in json format in search logs of splunk. @timestamp: 2025T19:31:30.615Z environment: dev event: { [+] } host: { [+] } input: { [+] } kubernetes: { [+] } message: +0000 FM [com.abc.cmp.event.message.base.Abs] DEBUG Receiver not ready, try other receivers or try later audit is disabled } so in message sometime iam getting above data and  some logs have some json data as well. and also above json data for above fields are also coming. Now i want to know that i have to use this data in my splunk for to have a structured data out of it how to do that so that i can use that data for my use ? ... View more

Which standard add on you are talking about or related add on where to get them from can you tell any name or example ... View more

ok last thing i want to know from all this solutions when i will perform it . Will i get  any new different data from what i am getting now because as of now i am getting timestamp, hostname, kubernetes.container name etc. ... View more

Hi I am confused like what in logs iam getting is this below: timestamp:  environment: event :{ json format under that orginal : java.lang.throwable and list of errors host:  loglevel: log file: message: java.lang. throwable  iam getting this above type of data in logs when i search the index and this is logstash data coming in splunk in json format now iam confused with what to do with this data is this the data coming is fine or i can further filter anything from this  and get some other output out of it which can be meaningful for me. if there is any way of it then please share. ... View more