Getting Data In

Is there something that can be done about GitHub Cloud Log streaming to network limited environment?

Tristan9608
Engager

Hi, I'm trying to get the audit logs from github cloud into splunk instance which has limited network access.

the problem is that ip of github that sends the data to splunk often changes. 

Instead of granting access to the changed ip, which takes some time to get the approval, I'd like to install another splunk instance in the DMZ environment, where there are no limit to the network, and send or forward the data in to the splunk instance in the limited network.

GitHub needs Splunk http event collector in order to verify before sending data. So I'm guessing that only heavy forwarder(full splunk instance to my knowledge, right?) is available.

Is this something that can be done? If so, could you please let me know the steps or docs that I could reference?

Thank you in advance.

0 Karma
1 Solution

isoutamo
SplunkTrust
SplunkTrust

Hi

it sounds like you have found the solution for your issue.

Just install one or more HFs on your DMZ with load balancer before those. Then send HEC event to that LB address which forwards those to these HFs. 

You should add same HEC tokens to all HFs which you have behind LB. If your connection between HFs and indexers changed often and it takes time to fix it, don't remember to add enough big queues for HEC inputs and outputs to indexers! Also indexer discovery is nice feature to ensure automatic reconnection to indexers if you are using indexer cluster. If not, then you must manually modify outputs.conf when indexer(s) changed.

I propose that you should (read: must) use some git repository (or other RCS) to store configurations and update HFs from it.

More detailed level instructions you found from Splunk documentation and/or community. One option is ask help from your local Splunk Partner.

r. Ismo

View solution in original post

isoutamo
SplunkTrust
SplunkTrust

Hi

it sounds like you have found the solution for your issue.

Just install one or more HFs on your DMZ with load balancer before those. Then send HEC event to that LB address which forwards those to these HFs. 

You should add same HEC tokens to all HFs which you have behind LB. If your connection between HFs and indexers changed often and it takes time to fix it, don't remember to add enough big queues for HEC inputs and outputs to indexers! Also indexer discovery is nice feature to ensure automatic reconnection to indexers if you are using indexer cluster. If not, then you must manually modify outputs.conf when indexer(s) changed.

I propose that you should (read: must) use some git repository (or other RCS) to store configurations and update HFs from it.

More detailed level instructions you found from Splunk documentation and/or community. One option is ask help from your local Splunk Partner.

r. Ismo

Get Updates on the Splunk Community!

The Splunk Success Framework: Your Guide to Successful Splunk Implementations

Splunk Lantern is a customer success center that provides advice from Splunk experts on valuable data ...

Splunk Training for All: Meet Aspiring Cybersecurity Analyst, Marc Alicea

Splunk Education believes in the value of training and certification in today’s rapidly-changing data-driven ...

Investigate Security and Threat Detection with VirusTotal and Splunk Integration

As security threats and their complexities surge, security analysts deal with increased challenges and ...