topic Monitoring OpenAI usage in Splunk in Getting Data In

Monitoring OpenAI usage in Splunk

edalbanese — Thu, 15 Feb 2024 23:28:43 GMT

We are rolling out a customer service chatbot. Has anyone needed to collect the data such as input/output and logs between an chatbot and OpenAI to monitor it in Splunk? If so, what did you use to GDI? One other note; there is the possibility for customers to share images or video with the chatbot; wondering if anyone has tried to collect this type of data in Splunk?

Re: Monitoring OpenAI usage in Splunk

gcusello — Fri, 16 Feb 2024 06:00:11 GMT

Hi @edalbanese ,

I still didn't used it, but I saw a colleague that used the OB OpenAI ChatGPT App at https://splunkbase.splunk.com/app/6957 .

About images, for my knowledge, at the moment Splunk stores and searches only text files.

Ciao.

Giuseppe

Re: Monitoring OpenAI usage in Splunk

edalbanese — Fri, 16 Feb 2024 19:35:32 GMT

Thank you @gcusello When I look at the add-on you posted, it seems to be a tool that helps to use OpenAI, within Splunk, to request additional context/perspective on specific questions or monitors (e.g. "Is this malicious?") My goal is actually just to get the content into Splunk first. So, for example, a web app that has a chatbot built in. The chatbot might receive input in text (or image/video) from users and then it generates a response (e.g. "Here is the documentation/help file relevant to that particular request"). I want to store all the activity between the application/user and OpenAI for evaluating security or compliance concerns using Splunk searches. Any thoughts on that? I could ask the App Engineer to log the input and output via HEC I guess. But I'm wondering if others have started logging OpenAI calls/responses (or any LLM such as Anthropic or Cohere or Gemini etc.) into Splunk yet? Thanks!

Re: Monitoring OpenAI usage in Splunk

gcusello — Sat, 17 Feb 2024 05:52:37 GMT

Hi @edalbanese ,

I don't usually work with OpenAI, even if I'll do it in the next future for a new customer.

But I saw two italian Splunk Sales Engineers that did exactly what you are searching with this add on.

They showed this concept app in a meeting of the Italia Splunk User Group.

Ciao.

Giuseppe