Is there something that can be done about GitHub Cloud Log streaming to network limited environment?

[Tristan9608]

Hi, I'm trying to get the audit logs from github cloud into splunk instance which has limited network access.

the problem is that ip of github that sends the data to splunk often changes. 

Instead of granting access to the changed ip, which takes some time to get the approval, I'd like to install another splunk instance in the DMZ environment, where there are no limit to the network, and send or forward the data in to the splunk instance in the limited network.

GitHub needs Splunk http event collector in order to verify before sending data. So I'm guessing that only heavy forwarder(full splunk instance to my knowledge, right?) is available.

Is this something that can be done? If so, could you please let me know the steps or docs that I could reference?

Thank you in advance.

[isoutamo · Accepted solution]

Hi

it sounds like you have found the solution for your issue.

Just install one or more HFs on your DMZ with load balancer before those. Then send HEC event to that LB address which forwards those to these HFs. 

You should add same HEC tokens to all HFs which you have behind LB. If your connection between HFs and indexers changed often and it takes time to fix it, don't remember to add enough big queues for HEC inputs and outputs to indexers! Also indexer discovery is nice feature to ensure automatic reconnection to indexers if you are using indexer cluster. If not, then you must manually modify outputs.conf when indexer(s) changed.

I propose that you should (read: must) use some git repository (or other RCS) to store configurations and update HFs from it.

More detailed level instructions you found from Splunk documentation and/or community. One option is ask help from your local Splunk Partner.

r. Ismo