All Apps and Add-ons

UF Push and Where to place Configuration Files?

SplunkDash
Motivator

Hello,

I have data coming in near real-time to a host (Linux)  where UF installed on it. It's a new push, objective is to send these events to SPLUNK indexer to view them from search head. Everything on place except I need to put new props.conf, inputs.conf, and transforms.conf files into that server. My question is where and how should I put those configuration files. Create a new folder local under etc/apps/ folder from CLI and copy all these 3 configuration files Or copy all these configuration files into ......etc/system/local folder.....or ....? Any recommendations will be highly appreciated. Thank you so much. 

Labels (2)
0 Karma
1 Solution

gcusello
Esteemed Legend

Hi @SplunkDash,

if the Universal Forwarder is already installed and confgured to send data to Indexers and you have only to add a new input, you have to:

if apps are manually deployed to the Forwarder:

  • open an inputs.conf in anothe app,
  • add a stanza like this:
[monitor:///your_path/your_file.log]
index=your_index
sourcetype=your_sourcetype
disabled=0
  • restart Splunk on Forwarder.

if instead you deploy apps using the Deployment Server, you have to do the same thing on one inputs.conf that you can find in an app in $SPLUNK_HOME/etc/deployment/apps of the Deployment Server and the nforce app deploy on DS (splunk reload deploy-server)

Ciao.

Giuseppe

View solution in original post

gcusello
Esteemed Legend

Hi @SplunkDash,

you should read the documentation or see the videos at 

https://www.splunk.com/en_us/resources/videos/getting-data-in-with-forwarders.html

https://docs.splunk.com/Documentation/Splunk/8.2.6/Data/Usingforwardingagents

In few words, you have to:

  • on your target server:
    • Install Universal Forwarder on the target server;
    • configure it to send data to the indexers, as described in the above links, you can use a CLI command or outputs.conf file;
    • configure your Universal Forwarder to ingest logs, as described in the above links, you can use a CLI command or inputs.conf file;
  • On your Indexer,
    • you have to create an index to receive logs,
    • enable logs receiving,
    • configure sourcetype to correctly parse you logs, if the logs you have to index are standard, probably there's an Add-On (in apps.splunk.com) already developed to correctly parse those logs.

I was very quick in description, but in the above link and video you can find all the details to do all.

About where to put conf files, my hont is never use $SPLUNK_HOME/etc/system/local, but always create a dedicated App (on Search Head) or a dedicate Add-On (On indexers or Forwarders).

The way to deploy apps to Forwarders could depend on the number of targets, but anyway I hint to use a Deployment Server (https://docs.splunk.com/Documentation/Splunk/8.2.6/Updating/Updateconfigurations).

If you have few tergets, probably the first time is longer than manually doing, but I hint to use DS to learn how to use it.

Ciao.

Giuseppe

SplunkDash
Motivator

@gcusello 

Thank you so much for your quick response, as always. Everything is setup and we are getting data for that host. This is a new push for a new set of near real time events coming in. I need to push those events so we can see those events from SH. How and where should I put my new configuration files to push those events? 

0 Karma

gcusello
Esteemed Legend

Hi @SplunkDash,

if the Universal Forwarder is already installed and confgured to send data to Indexers and you have only to add a new input, you have to:

if apps are manually deployed to the Forwarder:

  • open an inputs.conf in anothe app,
  • add a stanza like this:
[monitor:///your_path/your_file.log]
index=your_index
sourcetype=your_sourcetype
disabled=0
  • restart Splunk on Forwarder.

if instead you deploy apps using the Deployment Server, you have to do the same thing on one inputs.conf that you can find in an app in $SPLUNK_HOME/etc/deployment/apps of the Deployment Server and the nforce app deploy on DS (splunk reload deploy-server)

Ciao.

Giuseppe

SplunkDash
Motivator

@gcusello 

Awesome,  thank you so much for you support as always..... working as expected.

0 Karma

SplunkDash
Motivator

@gcusello 

I have a quick question...can I also use props.conf file......follow the same rules...open one props.conf file within the same app folder and add

[my_sourcetype] 

..................

.................

 

Thank you so much again.

0 Karma

gcusello
Esteemed Legend

Hi @SplunkDash,

yes, you can, but what's the purpose of this props.conf?

If you have to add many configurations, probably it's a better choice to create a new dedicated App or TA.

Ciao.

Giuseppe

0 Karma

SplunkDash
Motivator

@gcusello

I have use cases to send stream of data from SPLUNK to 3rd party servers on a continuous basis, is there any ways you can help me? Thank you so much in advance.

Tags (1)
0 Karma

gcusello
Esteemed Legend

Hi @SplunkDash,

it's always better to create a new question instead using an old (and closed) question because less people will read your question and answer you!.

At first you can find in Community tens of questions ans answers to this scope, anyray to send logs to a third party system, you have to follow the instructions at https://docs.splunk.com/Documentation/Splunk/9.0.2/Forwarding/Routeandfilterdatad#Replicate_a_subset...

Remember that you have to do this on an Indexer or, if present on an Heavy Forwarder.

Ciao.

Giuseppe

0 Karma

SplunkDash
Motivator

@gcusello 

Thank you so much as always. I need to creat a new props due to the complexity of data structure and use cases.
What is the best way to creat a new dedicated app from CLI…it’s a completely new push. Can I creat a new app folder from CLI and copy the content of any existing  app there and modify its props and inputs configuration files? Thank you so much again.

0 Karma

gcusello
Esteemed Legend

Hi @SplunkDash,

you can use the approach you like and you better know.

I usually create an app by CLI and manually create the folders and the files.

If you want to be more sure, you could use the App builder App on your Splunk Enterprise to check this TA bfore deploying it.

There's only one point of attention I'd highlight: if you have to deploy this app to Linux targets, don't create it on Windows because there could be permissions issues.

Ciao.

Giuseppe

SplunkDash
Motivator

@gcusello 

I think that might be the smartest approach. Let me try this, thank you so much, appreciate your support in these efforts as always.

0 Karma

aasabatini
Motivator

Hi @SplunkDash 

yes the best pratices are create a small app to put your configuration.

be careful props and trasforms doesn't works on the UF, if you want use that file you have to install HF.

Regards

Alessandro

 

“The answer is out there, Neo, and it’s looking for you, and it will find you if you want it to.”

SplunkDash
Motivator

Hello,

Thank you so much for your quick reaponse....oh ok ..yes agree ..need to have HF....one question, how I would create new app from CLI.... go to the /etc/apps folder, then.........mkdir NewappName,  and then under that mkdir local, and then copy all configuration files under local folder. Your recommendation will be highly appreciated. Thank you so much again.

0 Karma

aasabatini
Motivator

Hi @SplunkDash 

 

there are many ways to manage this

first one:

use a deployment server (reccomended)

go on the /opt/splunk/etc/deployment-apps/ and create your small app

mkdir -p small_app/local

and put your conf file on the local folder

go on the deployment server web interface and crate a class server to push your new app.

connect in ssh on the HF and create a small app (not reccomended)

quick tip

if you are not familiar with the cli you can use this app to manage conf by UI

https://splunkbase.splunk.com/app/4353/

Regards

“The answer is out there, Neo, and it’s looking for you, and it will find you if you want it to.”

SplunkDash
Motivator

@aasabatini 

Thank you so much again:  let me try your this recommendation and get back to you.

use a deployment server (reccomended)

go on the /opt/splunk/etc/deployment-apps/ and create your small app

mkdir -p small_app/local

and put your conf file on the local folder

Thank you!

 

0 Karma
Get Updates on the Splunk Community!

How I Instrumented a Rust Application Without Knowing Rust

As a technical writer, I often have to edit or create code snippets for Splunk's distributions of ...

Splunk Community Platform Survey

Hey Splunk Community, Starting today, the community platform may prompt you to participate in a survey. The ...

Observability Highlights | November 2022 Newsletter

 November 2022Observability CloudEnd Of Support Extension for SignalFx Smart AgentSplunk is extending the End ...