All Apps and Add-ons

Where to place configuration files for universal forwarder push?

SplunkDash
Motivator

Hello,

I have data coming in near real-time to a host (Linux)  where UF installed on it. It's a new push, objective is to send these events to SPLUNK indexer to view them from search head. Everything on place except I need to put new props.conf, inputs.conf, and transforms.conf files into that server. My question is where and how should I put those configuration files. Create a new folder local under etc/apps/ folder from CLI and copy all these 3 configuration files Or copy all these configuration files into ......etc/system/local folder.....or ....? Any recommendations will be highly appreciated. Thank you so much. 

Labels (2)
0 Karma
1 Solution

gcusello
SplunkTrust
SplunkTrust

Hi @SplunkDash,

if the Universal Forwarder is already installed and confgured to send data to Indexers and you have only to add a new input, you have to:

if apps are manually deployed to the Forwarder:

  • open an inputs.conf in anothe app,
  • add a stanza like this:
[monitor:///your_path/your_file.log]
index=your_index
sourcetype=your_sourcetype
disabled=0
  • restart Splunk on Forwarder.

if instead you deploy apps using the Deployment Server, you have to do the same thing on one inputs.conf that you can find in an app in $SPLUNK_HOME/etc/deployment/apps of the Deployment Server and the nforce app deploy on DS (splunk reload deploy-server)

Ciao.

Giuseppe

View solution in original post

gcusello
SplunkTrust
SplunkTrust

Hi @SplunkDash,

you should read the documentation or see the videos at 

https://www.splunk.com/en_us/resources/videos/getting-data-in-with-forwarders.html

https://docs.splunk.com/Documentation/Splunk/8.2.6/Data/Usingforwardingagents

In few words, you have to:

  • on your target server:
    • Install Universal Forwarder on the target server;
    • configure it to send data to the indexers, as described in the above links, you can use a CLI command or outputs.conf file;
    • configure your Universal Forwarder to ingest logs, as described in the above links, you can use a CLI command or inputs.conf file;
  • On your Indexer,
    • you have to create an index to receive logs,
    • enable logs receiving,
    • configure sourcetype to correctly parse you logs, if the logs you have to index are standard, probably there's an Add-On (in apps.splunk.com) already developed to correctly parse those logs.

I was very quick in description, but in the above link and video you can find all the details to do all.

About where to put conf files, my hont is never use $SPLUNK_HOME/etc/system/local, but always create a dedicated App (on Search Head) or a dedicate Add-On (On indexers or Forwarders).

The way to deploy apps to Forwarders could depend on the number of targets, but anyway I hint to use a Deployment Server (https://docs.splunk.com/Documentation/Splunk/8.2.6/Updating/Updateconfigurations).

If you have few tergets, probably the first time is longer than manually doing, but I hint to use DS to learn how to use it.

Ciao.

Giuseppe

SplunkDash
Motivator

@gcusello 

Thank you so much for your quick response, as always. Everything is setup and we are getting data for that host. This is a new push for a new set of near real time events coming in. I need to push those events so we can see those events from SH. How and where should I put my new configuration files to push those events? 

0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @SplunkDash,

if the Universal Forwarder is already installed and confgured to send data to Indexers and you have only to add a new input, you have to:

if apps are manually deployed to the Forwarder:

  • open an inputs.conf in anothe app,
  • add a stanza like this:
[monitor:///your_path/your_file.log]
index=your_index
sourcetype=your_sourcetype
disabled=0
  • restart Splunk on Forwarder.

if instead you deploy apps using the Deployment Server, you have to do the same thing on one inputs.conf that you can find in an app in $SPLUNK_HOME/etc/deployment/apps of the Deployment Server and the nforce app deploy on DS (splunk reload deploy-server)

Ciao.

Giuseppe

SplunkDash
Motivator

@gcusello 

Awesome,  thank you so much for you support as always..... working as expected.

0 Karma

SplunkDash
Motivator

@gcusello 

I have a quick question...can I also use props.conf file......follow the same rules...open one props.conf file within the same app folder and add

[my_sourcetype] 

..................

.................

 

Thank you so much again.

0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @SplunkDash,

yes, you can, but what's the purpose of this props.conf?

If you have to add many configurations, probably it's a better choice to create a new dedicated App or TA.

Ciao.

Giuseppe

0 Karma

SplunkDash
Motivator

@gcusello

I have use cases to send stream of data from SPLUNK to 3rd party servers on a continuous basis, is there any ways you can help me? Thank you so much in advance.

Tags (1)
0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @SplunkDash,

it's always better to create a new question instead using an old (and closed) question because less people will read your question and answer you!.

At first you can find in Community tens of questions ans answers to this scope, anyray to send logs to a third party system, you have to follow the instructions at https://docs.splunk.com/Documentation/Splunk/9.0.2/Forwarding/Routeandfilterdatad#Replicate_a_subset...

Remember that you have to do this on an Indexer or, if present on an Heavy Forwarder.

Ciao.

Giuseppe

0 Karma

SplunkDash
Motivator

@gcusello

Yes, I think you recommended me to create a new question before, but I wanted to have you in this conversation. I just opened a new conversation with the subject title "Send Data from SPLUNK to Third Party Servers", your participation would be highly appreciated. Thank you!

Tags (1)
0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @SplunkDash,

Thank you for your esteem, in this case, next time add a post to your message with the link to the new question and I will certainly try to answer you, but with a new question you can get more help also from others as well.

Ciao.

Giuseppe

0 Karma

SplunkDash
Motivator
0 Karma

SplunkDash
Motivator

@gcusello 

On sending data to Third Party Server, SPLUNK is recommending 2 approaches (Syslog and TCP) based on the SPLUNK Resources I got from the link you provided. I have 2 questions:

1) Is it possible to use REST API to send data to Third Part Server (I couldn't find SPLUNK Recommendation to send data to Third Party Server Using REST API)

2) Using TCP port, what configuration we need to have in receiving server... besides configuring receiver port...Like how port is receiving files, how they see the files at receiving end point, what additional setup they need to have at the receiving end.

 

Thank you so much, appreciated your support as always. 

Tags (1)
0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @SplunkDash,

I'm not an expert of REST API, but You could create a script that extracts data from Splunk using REST APi and send them to a third party, probably it's more efficient than syslog.

I cannot help you in developing but you shoull be able to find how to extract data from Splunk using REST API.

About TCP, you could try to use a Forwarder to send data to a third party but I never tested it, I always used syslogs.

Ciao.

Giuseppe

SplunkDash
Motivator

@gcusello 

Thank you so much for your quick response, as always.

Another question based on your recommendation. How, would I send Data to Third Paty Server after I extract Data from SPLUNK using REST API? Is it possible to send that extracted data to Third party server using REST API? Thank you again!

0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @SplunkDash,

as I said, you have to create a program or a script that extract data from Splunk using Splunk REST API and writes the same data into ne third party system using its API.

In few words, you have to create your own connector!

As I said I always used syslog.

Ciao.

Giuseppe

SplunkDash
Motivator

Hello @gcusello 

I have a couple of questions on Using REST API to send data to third party server. Is it going to be easily supported by SPLUNK or we need to create an "App"  and also a lot of configurations would require to be done at the client side? Your recommendations will be highly appreciated as always. Thank you!

Tags (1)
0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @SplunkDash,

search in splunkbase if there's an app with this integration, but I suppose that you have to manually do it.

Ciao.

Giuseppe

0 Karma

SplunkDash
Motivator

Hello @gcusello,

I know you are an expert on Syslogs. Our client is planning to send us CEF formatted events to our syslog servers. Do you know what are the differences between regular syslog events and CEF Syslogs? Thank you so much for your support and any recommendation will be highly appreciated, as always. 

 

Tags (1)
0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @SplunkDash ,

as always, I hint to open a new question instead using an old one to have more quickers and probably better answers from more people.

Anyway, the only difference is the format, to use in the TA: there are some tAs that use only CEF format, other that use other formats and some that use both formats.

See in the TA that you need to use (releted to the technology) which format it waits for.

Ciao.

Giuseppe

SplunkDash
Motivator

Hello @gcusello 

I posted a new question under this link if you like to participate:

dbConnect Driver for Connection Type DB2 - Splunk Community

Thank you in advance.

 

 

Tags (1)
0 Karma
Get Updates on the Splunk Community!

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...