Monitoring OpenAI usage in Splunk

[edalbanese]

We are rolling out a customer service chatbot. Has anyone needed to collect the data such as input/output and logs between an chatbot and OpenAI to monitor it in Splunk? If so, what did you use to GDI?   One other note; there is the possibility for customers to share images or video with the chatbot; wondering if anyone has tried to collect this type of data in Splunk?  

[gcusello]

Hi @edalbanese ,

I still didn't used it, but I saw a colleague that used the OB OpenAI ChatGPT App at https://splunkbase.splunk.com/app/6957 .

About images, for my knowledge, at the moment Splunk stores and searches only text files.

Ciao.

Giuseppe

[edalbanese]

Thank you @gcusello   When I look at the add-on you posted, it seems to be a tool that helps to use OpenAI, within Splunk, to request additional context/perspective on specific questions or monitors (e.g. "Is this malicious?") My goal is actually just to get the content into Splunk first. So, for example, a web app that has a chatbot built in. The chatbot might receive input in text (or image/video) from users and then it generates a response (e.g. "Here is the documentation/help file relevant to that particular request"). I want to store all the activity between the application/user and OpenAI for evaluating security or compliance concerns using Splunk searches. Any thoughts on that? I could ask the App Engineer to log the input and output via HEC I guess. But I'm wondering if others have started logging OpenAI calls/responses (or any LLM such as Anthropic or Cohere or Gemini etc.) into Splunk yet? Thanks!

[gcusello · Accepted solution]

Hi @edalbanese ,

I don't usually work with OpenAI, even if I'll do it in the next future for a new customer.

But I saw two italian Splunk Sales Engineers that did exactly what you are searching with this add on.

They showed this concept app in a meeting of the Italia Splunk User Group.

Ciao.

Giuseppe