Getting Data In

Is there something that can be done about GitHub Cloud Log streaming to network limited environment?

Tristan9608
Engager

Hi, I'm trying to get the audit logs from github cloud into splunk instance which has limited network access.

the problem is that ip of github that sends the data to splunk often changes. 

Instead of granting access to the changed ip, which takes some time to get the approval, I'd like to install another splunk instance in the DMZ environment, where there are no limit to the network, and send or forward the data in to the splunk instance in the limited network.

GitHub needs Splunk http event collector in order to verify before sending data. So I'm guessing that only heavy forwarder(full splunk instance to my knowledge, right?) is available.

Is this something that can be done? If so, could you please let me know the steps or docs that I could reference?

Thank you in advance.

0 Karma
1 Solution

isoutamo
SplunkTrust
SplunkTrust

Hi

it sounds like you have found the solution for your issue.

Just install one or more HFs on your DMZ with load balancer before those. Then send HEC event to that LB address which forwards those to these HFs. 

You should add same HEC tokens to all HFs which you have behind LB. If your connection between HFs and indexers changed often and it takes time to fix it, don't remember to add enough big queues for HEC inputs and outputs to indexers! Also indexer discovery is nice feature to ensure automatic reconnection to indexers if you are using indexer cluster. If not, then you must manually modify outputs.conf when indexer(s) changed.

I propose that you should (read: must) use some git repository (or other RCS) to store configurations and update HFs from it.

More detailed level instructions you found from Splunk documentation and/or community. One option is ask help from your local Splunk Partner.

r. Ismo

View solution in original post

isoutamo
SplunkTrust
SplunkTrust

Hi

it sounds like you have found the solution for your issue.

Just install one or more HFs on your DMZ with load balancer before those. Then send HEC event to that LB address which forwards those to these HFs. 

You should add same HEC tokens to all HFs which you have behind LB. If your connection between HFs and indexers changed often and it takes time to fix it, don't remember to add enough big queues for HEC inputs and outputs to indexers! Also indexer discovery is nice feature to ensure automatic reconnection to indexers if you are using indexer cluster. If not, then you must manually modify outputs.conf when indexer(s) changed.

I propose that you should (read: must) use some git repository (or other RCS) to store configurations and update HFs from it.

More detailed level instructions you found from Splunk documentation and/or community. One option is ask help from your local Splunk Partner.

r. Ismo

Get Updates on the Splunk Community!

.conf24 | Day 0

Hello Splunk Community! My name is Chris, and I'm based in Canberra, Australia's capital, and I travelled for ...

Enhance Security Visibility with Splunk Enterprise Security 7.1 through Threat ...

 (view in My Videos)Struggling with alert fatigue, lack of context, and prioritization around security ...

Troubleshooting the OpenTelemetry Collector

  In this tech talk, you’ll learn how to troubleshoot the OpenTelemetry collector - from checking the ...