Monitoring Splunk

Splunk integartion with Flat file

mukhan1
Explorer

Hello Team,

I have a .log flat file this file give us the data whenever we open and run command it give us some logs, now i am integrating this .log file with Splunk but it is not integrating. I ran following command to integrate it,
"/splunk/bin ---> ./splunk add monitor [file name]" it give me message that file has been added to monitor list. 

However i don't see this file on my Splunk, further if i have this file on Splunk how it will takes data from it whenever we run any command, also this .log file doesn't store data in any other directory whenever we close the file data disappears. Please note the OS im using is Sun Solaris 

0 Karma

PickleRick
SplunkTrust
SplunkTrust

OK. First things first.

1) Do you have _anything_ ingested from this forwarder? Check your _internal index for any logs coming from this UF

2) If you didn't specify a destination index, the forwarder will be trying to send the data to the default "main" index - it's not the best idea.

3) Check the output of

splunk list inputstatus

and

splunk list monitor

And verify if that file is being read by your forwarder

0 Karma

mukhan1
Explorer

@gcusello thanks for your reply, i have checked the connection by telnet the Splunk it is successfully connected, also cross checked it by adding other path of log files. It is adding successfully. 

I have added the file path manually but still file is not showing on splunk GUI. Further going through the doc you provided hope it will help.

0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @mukhan1,

ok, perform also the check I hinted to verify connection because telnet is important but it isn't the only check to perform: you could have an open connectin but you could not correctly configure outputs.conf in your Forwarder!

let me know if you solved or if I can help you more.

Ciao.

Giuseppe

P.S.: Karma Points are appreciated 😉

0 Karma

mukhan1
Explorer

Hey @gcusello,

Already checked the outputs.conf file it is working fine, i don't think this issue is related to outputs.conf because if the issue is with outputs.conf then other path of logs also failed to send logs however im receiving logs from same Host but unable to fetch .logs file into Splunk.

This file is actually have code in it, whenever i open this file and run command then it will give me some logs against the command i run. I want to ingest those logs into Splunk. Please remember once the file is close the data will wiped also. No other records of these logs.

I think Splunk doesn't support this type of file ingestion with Splunk.  

0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @mukhan1,

I'm confident that you can read this file with Splunk: Splunk can read every kind of text file!

Check if the path and filename i the stanza header is correct and then check if the user you're using to run Splunk can read that file.

Ciao.

Giuseppe

0 Karma

mukhan1
Explorer

@gcusello just for ur understanding ,
No, this not .txt file this is the flat file

0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @mukhan1,

a flat file, for my knowledge is a text file continously updated.

If your flat file is a text file continously updated Splunk can read it.

Ciao.

Giuseppe

0 Karma

mukhan1
Explorer

@gcusello  yes you're correct splunk can integrate any text file but my issue is that i have .log file namely as "F.JBASE.JED.AUDIT.LOG" this is the file name this file is not a text file

0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @mukhan1,

have you an API to extract the content of this file?

If yes, you could develop a script that periodically extract the logs and writes them in a text file readable from Splunk or directly in Splunk.

Splunk developed a connector (e.g. for wineventlog) to extract not text files.

Ciao.

Giuseppe

 

Ciao.

Giuseppe

0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @mukhan1,

at first read:

https://docs.splunk.com/Documentation/Splunk/latest/Data/Monitorfilesanddirectories 

https://lantern.splunk.com/Splunk_Platform/Getting_Started/Getting_data_into_Enterprise 

then check if the connection between the Forwarder and Splunk is open running a simple search on Splunk:

index=_internal host=<your_forwarder_host>

if you have events the connectin is established, if not you have primarly to configure the connection.

If the connectin is ok, then, you should have in $SPLUNK_HOME/etc/system/local and inputs.conf file.

In this file you should have a stanza that starts with [monitor://yourfile]

take the path you have after monitor:// and run ls -la your path to see if your monitor stanza really reache the file to monitor.

The issue could be have that the path isn't correct or that the user you're using to run Splunk hasn't the grants on that folder.

Manually modify the inputs.conf stanza and restart Splunk on the Forwarder.

Ciao.

Giuseppe

Get Updates on the Splunk Community!

Build Your First SPL2 App!

Watch the recording now!.Do you want to SPL™, too? SPL2, Splunk's next-generation data search and preparation ...

Exporting Splunk Apps

Join us on Monday, October 21 at 11 am PT | 2 pm ET!With the app export functionality, app developers and ...

[Coming Soon] Splunk Observability Cloud - Enhanced navigation with a modern look and ...

We are excited to introduce our enhanced UI that brings together AppDynamics and Splunk Observability. This is ...