Getting Data In

Forwarding Azure App Service Logs do SPlunk

mochocki
Explorer

I have an azure app service with CUSTOM text log files (stored locally in app service filesystem). How can I index them in splunk?
I was thinking about the following, but none was working:

  • using azure file storage (samba ports are blocked)
  • read logs in splunk via ftp (as far as I know impossible)
  • trying to install splunk forwarder (as far as I know possible only in azure VM, not app service)
Tags (2)
0 Karma

jkat54
SplunkTrust
SplunkTrust

You need to send your logs to app fabric, table, blob, or eventhub then pull the data using other Microsoft cloud services app from splunk. Note it doesnt support event hubs but you can send event hub to blob storage and read from there.

jkat54
SplunkTrust
SplunkTrust
0 Karma

Sukisen1981
Champion

the ftp is not a bad idea actually, try this app out if possible - https://splunkbase.splunk.com/app/3318/#/details or this - https://splunkbase.splunk.com/app/3534/ ?
Is it possible to call them over some sort of API service? Then you can rest ingest them in splunk

mochocki
Explorer

Thanks, I'll try.
What do you mean by "call them over some sort of API service"?

0 Karma

Sukisen1981
Champion

i mean can you read the logs through an bash/python/shell script? Then you could create a scripted input and index the output of the script into splunk

0 Karma

mochocki
Explorer

In fact none of these addons are good enough. FTP Receiver sets up local ftp server instead of reading logs from remote one. The other addon can only rean diagnostic logs.
Can you provide more info about the scripts? Do they run on splunk server? Can they work realtime? I have daily rolling text files but I would like to have them indexed realtime not only after they are rolled.
Do you have any examples of such script?

0 Karma

Sukisen1981
Champion

Hi
Firstly, you have to bear with me , I have 0 experience on azure and a newbie on AWS so I am probably not able to understand simple things in azure.
Please see this - https://docs.splunk.com/Documentation/Splunk/7.2.6/Data/MonitorWMIdata#Security_and_remote_access_co...

Can you open cmd on your local and curl into the remote machine to read the log files? If you can then we can always set up a script , key thing is NOT the indexing here, but how you connect from your local to your remote instance AND download the log info from the remote machine.
I suggest you google a bit on pyhton or shell or curl command/scripts on how to connect and get logs from a remote azure instance. After that its a cakewalk and I can guide you in that but firstly, can you (you have to, if your splunk is on a different instance than the remote azure instance) gather the logs from the remote instance to your local?

0 Karma
Get Updates on the Splunk Community!

New in Observability - Improvements to Custom Metrics SLOs, Log Observer Connect & ...

The latest enhancements to the Splunk observability portfolio deliver improved SLO management accuracy, better ...

Improve Data Pipelines Using Splunk Data Management

  Register Now   This Tech Talk will explore the pipeline management offerings Edge Processor and Ingest ...

3-2-1 Go! How Fast Can You Debug Microservices with Observability Cloud?

Register Join this Tech Talk to learn how unique features like Service Centric Views, Tag Spotlight, and ...