topic Re: Has anyone used Kafka-Connect to Publish Appdynamics Data to Other Source? in Splunk AppDynamics

Has anyone used Kafka-Connect to Publish Appdynamics Data to Other Source?

Vaibhav_Vir_Sin — Sat, 01 Dec 2018 16:03:03 GMT

Experts,

There is an increase demand of APM data to our common data reservoir.

Currently we are extracting data via REST API from Appdynamics, transforming it in the meaningful format and then publishing it to data reservoir.

But has anyone tried any specific Kafka Connector to stream the AppDynamics data to other sources?

If yes, appreciate if you could share the details over here.

Regards,

Vaibhav

Re: Has anyone used Kafka-Connect to Publish Appdynamics Data to Other Source?

David_Hope — Fri, 07 Dec 2018 02:37:24 GMT

Hi Vaibhav,

You are currently extracting data via AppDynamics via our REST API.

Did you want to extract from the REST API->Kafka, or directly out of AppDynamics via Kafka? We don't currently support any other method than via the REST API.

Kind Regards,

David Hope - APM Architect

Re: Has anyone used Kafka-Connect to Publish Appdynamics Data to Other Source?

Vaibhav_Vir_Sin — Fri, 07 Dec 2018 03:26:39 GMT

Thanks David for your response.

I am referring Rest API -> Kafka and if anyone has used Kafka connect or
NiFi to integrate.

Currently I am using python script to extract data via rest API,
transforming the data and then publishing to my data lake via AMPS bus and
to be kafka(in simple terms using post API to put the data).

So was thinking if it is feasible and scalable to use any other option to
achieve the same.

Re: Has anyone used Kafka-Connect to Publish Appdynamics Data to Other Source?

Vaibhav_Vir_Sin — Fri, 14 Dec 2018 19:16:55 GMT

HEllo,

Anyone tried above activity by any means?

Re: Has anyone used Kafka-Connect to Publish Appdynamics Data to Other Source?

David_Hope — Fri, 14 Dec 2018 19:21:50 GMT

Hi Vaibhav

I'm very sorry that I haven't got back to you yet, I was hoping to have a little more information.

I will reach out to the internal AppD team to see if we know of anyone who has tried this. I personally unaware of this being tried. Let me do some more investigation.

Happy Holidays.

Kind Regards,

David

Re: Has anyone used Kafka-Connect to Publish Appdynamics Data to Other Source?

Daniel_Odievich — Fri, 14 Dec 2018 21:40:01 GMT

Vaibhav,

What kind of data are you exfiltrating with your scripts? Metric data? Events? Which ones?

Are you aware of the https://github.com/Appdynamics/AppDynamics.DEXTER/wiki toolkit? It wraps a lot of things you can't ordinarily get into and gives them to you in data warehouse format, ready to go into your data lake.

Re: Has anyone used Kafka-Connect to Publish Appdynamics Data to Other Source?

Vaibhav_Vir_Sin — Mon, 17 Dec 2018 12:47:23 GMT

Thanks for your response @Daniel.Odievich

I am subscribing metric data (event data we already getting from AppD-Netcool integration and Netcool data is already onboarded to our data lake).

YEs, I am aware of that but as i have a constraint of using external dependency (from organization security point of view), cannot use that product in Production. Moreover, I felt Dexter is useful for extracting data and potraying the detailed output. But my requirement is somewhat not just extraction but data publishing as well. Where I beleive products\utility like Kafka-Connector for Appdynamics would be of much help and meeting the requirements.

Hope you understand my pain area and looking for someone who already gone through this.

Regards,

vaibhav

Re: Has anyone used Kafka-Connect to Publish Appdynamics Data to Other Source?

Umervali_Niyama — Fri, 21 Dec 2018 11:31:45 GMT

I am also extracting data with python scripts. if any one has any thing on this please lwt mw know.

Re: Has anyone used Kafka-Connect to Publish Appdynamics Data to Other Source?

Umervali_Niyama — Thu, 07 Feb 2019 08:04:48 GMT

Any thoughts on this?

Re: Has anyone used Kafka-Connect to Publish Appdynamics Data to Other Source?

Amit_Jaiswal — Thu, 17 Sep 2020 14:13:26 GMT

Hi Vaibhav,

I also have a similar requirement to extract AppD data and publish into Data Lake.

Currently, trying to use DEXTER csvs to transform and upload to Data Lake.

But, don't know how to make it Real Time?

Are you able to do it Real Time ? or is there any progress in this since your last query ?

Please guide.

Re: Has anyone used Kafka-Connect to Publish Appdynamics Data to Other Source?

Vaibhav_Vir_Sin — Thu, 17 Dec 2020 05:17:14 GMT

No Amit - i am doing in old school way of batch jobs using REST API with 5 mins interval.

Also never heard Appd is working on providing such plugin or functionality where everybody is talking about data streaming.

Re: Has anyone used Kafka-Connect to Publish Appdynamics Data to Other Source?

Subhrajit_mohan — Tue, 22 Dec 2020 10:23:22 GMT

Hi,

I am new to this. Can you please help me to understand how do you use rest API to extract appdynamics data? If you have any information on this please do share.

Thanks,

Subhrajit

Re: Has anyone used Kafka-Connect to Publish Appdynamics Data to Other Source?

iamryan — Mon, 04 Jan 2021 18:18:04 GMT

Hi @Vaibhav Vir.Singh, maybe you can help out @Subhrajit.mohanty with this question.