Deployment Architecture

How to monitor a file on a remote unix machine via inputs.conf ?

Hemnaath
Motivator

Hi Team,

Currently we need to add additional monitoring stanza to monitor the audit files that are in the remote UNIX nodes. These are the files "wtmpx utmpx, wtmp and btmp" that needs to be included in the monitoring stanza along with the below inputs.conf stanza.

wtmpx and utmpx - These two files present under /var/adm/ directory.
wtmp and btmp - These two files present under /var/log/ directory.

inputs stanza details:

[monitor:///var/adm]
whitelist=(\.log|log$|wtmpx|utmpx|message)
index=nix
disable=0

inputs stanza details:

 [monitor:///var/log]
whitelist=(\.log|log$|secure|message|auth|wtmp|btmp|cron$|\.out)
blacklist=(lastlog)
index=nix
disable=0

Kindly let me know whether above input stanza is correct to fetch the newly added files from the remote machine and also it will be really helpful if you guide me what is the purpose of using $ and .out in the inputs.conf stanza.

thanks in advance

0 Karma
1 Solution

nickhills
Ultra Champion

Do you see any errors - I suspect if you look in your logs you may see warnings about these files being binary .
normally one uses a tool to read the contents of these files - so my initial guess is that could be your issue.

Edit: updated answer to include my comment
Take a look at this post: https://answers.splunk.com/answers/5844/can-i-splunk-my-wtmp-files.html

The data in wtmp (etc) is used by the who and last tools, and is not directly readable (as they are binary)
There are two good solutions in that post which explain how you can monitor the relevant data

If my comment helps, please give it a thumbs up!

View solution in original post

0 Karma

nickhills
Ultra Champion

Do you see any errors - I suspect if you look in your logs you may see warnings about these files being binary .
normally one uses a tool to read the contents of these files - so my initial guess is that could be your issue.

Edit: updated answer to include my comment
Take a look at this post: https://answers.splunk.com/answers/5844/can-i-splunk-my-wtmp-files.html

The data in wtmp (etc) is used by the who and last tools, and is not directly readable (as they are binary)
There are two good solutions in that post which explain how you can monitor the relevant data

If my comment helps, please give it a thumbs up!

View solution in original post

0 Karma

Hemnaath
Motivator

Hi Nickhill, I am unable to see Error message but could see the Warning message in splunk when I search index="_internal" host=test01 log_level=WARN

01-16-2018 08:06:47.560 -0500 WARN FileClassifierManager - The file '/var/adm/utmpx' is invalid. Reason: binary

01-16-2018 08:06:46.560 -0500 WARN FileClassifierManager - The file '/var/adm/wtmpx' is invalid. Reason: binary

Kindly let me know how to monitor these files in splunk.

0 Karma

nickhills
Ultra Champion

Sure - thats the message i was expecting.

Take a look at this post: https://answers.splunk.com/answers/5844/can-i-splunk-my-wtmp-files.html

The data in wtmp (etc) is used by the who and last tools, and is not directly readable (as they are binary)

There are two good solutions in that post which explain how you can monitor the relevent data

If my comment helps, please give it a thumbs up!
0 Karma

Hemnaath
Motivator

thanks Nickhill for the update, but now I am not sure how to write a scripted input to monitor this file and index into splunk by following the below approach.

1) Set up a scripted input calling a shell script that executes "who" or "last" with the options you need and that will index the generated output. This is the simplest approach.

So can you please guide me how to create an scripted inputs for my requirement please.

0 Karma

Hemnaath
Motivator

Hi Nickhill, can you guide me on how to create an scripted inputs for getting the data file into splunk.

thanks in advance.

0 Karma

nickhills
Ultra Champion

I would open another ticket for the question "how do I collect the results of who/last on linux"
As its a more of a useful heading for this issue.
Hopefully I have answered your question on what $ means, and "why" your collection is not working

If my comment helps, please give it a thumbs up!
0 Karma

Anam
Community Manager
Community Manager

@hemnaath

Was @nickhillscpl able to answer your question "on what $ means, and "why" your collection is not working"? If he was, please click accept on his answer to resolve the post. If not please provide more information that people can use to help troubleshoot further.

Thanks

0 Karma

Hemnaath
Motivator

Hi asiddique, Nickhillscpl was able to clarify the doubts both of my questions. So I am accepting this answer.

But I am not sure how to get this is done using the link shared by nickhillscpl. If you need me to open a new question for the same, I can do that.

https://answers.splunk.com/answers/5844/can-i-splunk-my-wtmp-files.html

thanks in advance.

0 Karma

493669
Super Champion

Hi @Hemnaath
in whitelist you need to provide regex to match filename which you want to monitor

The "$" anchors the regular expression to the end of the line.For ex. |log$ will check if your filename ends with log
.out specifies if filename extension is 'out' like filename.out
refer this document https://docs.splunk.com/Documentation/SplunkCloud/6.6.3/Data/Whitelistorblacklistspecificincomingdat...

as per splunk docs,
you need to write disabled instead of disable

disabled = [0|1]
* Whether or not the input is enabled.
* Defaults to 0 (enabled).
0 Karma

Hemnaath
Motivator

Hi, thanks for you support on this but I am unable to monitor the "wtmpx|utmpx" file from the path /var/adm/ on the remote host. But we could monitor other files ".log|log$|message" are being ingested into splunk .

Similarly for the second stanza also we are unable to monitor the "wtmp|btmp" files from the path /var/log/ on the remote host. But we could monitor other files ".log|log$|secure|message|auth|cron$|.out" are being ingested into splunk .

Kindly guide me how to fix this issue, we need to pull the wtmpx, utmpx, wtmp,btmp in splunk.

0 Karma

493669
Super Champion

what is the extension of these wtmpx, utmpx, wtmp,btmp files?

0 Karma

Hemnaath
Motivator

Hi wtmpx,utmpx are the data files which does not have any extension and I could see some warning message in the splunkd.log .

01-16-2018 08:06:47.560 -0500 WARN FileClassifierManager - The file '/var/adm/utmpx' is invalid. Reason: binary

01-16-2018 08:06:46.560 -0500 WARN FileClassifierManager - The file '/var/adm/wtmpx' is invalid. Reason: binary

0 Karma

493669
Super Champion

The NO_BINARY_CHECK is a props.conf configuration, and so you will want to create a stanza in props like:

 [<sourcetype_name>]
 NO_BINARY_CHECK = true
0 Karma

mayurr98
Super Champion

hey I think your first regex seems okay but the second is not. do you want to monitor cron.log and cron.out as well? for the second one .Also there are syntax problems. Refer the one I have given.

[monitor:///var/adm]
whitelist =   wtmpx\.log$|utmpx\.log$|message\.log$
index = nix
disabled =0

[monitor:///var/log]
whitelist = (secure|message|auth|wtmp|btmp|cron)(\.log$|\.out$)
index = nix
disabled = 0

Also $ means end of string
If you want to learn more about regex use this link https://regex101.com/

let me know if it helps!

0 Karma

Hemnaath
Motivator

Hi Mayurrr98, thanks for you support on this but I am unable to monitor the "wtmpx|utmpx" file from the path /var/adm/ on the remote host. But we could monitor other files ".log|log$|message" are being ingested into splunk . Similarly for the second stanza also we are unable to monitor the "wtmp|btmp" files from the path /var/log/ on the remote host. But we could monitor other files ".log|log$|secure|message|auth|cron$|.out" are being ingested into splunk .

Since already we are able to get the other files "message|auth|cron" etc info in splunk, I did not change the regex, just added the new file name in the stanza along with other file name.

Kindly guide me how to fix this issue, we need to pull the wtmpx, utmpx, wtmp,btmp in splunk.

0 Karma

mayurr98
Super Champion

did you use wtmpx\.log$|utmpx\.log$|message\.log$ to monitor your first stanza ?

0 Karma

Hemnaath
Motivator

Hi wtmpx,utmpx are the data files which does not have any extension and I could see some warning message in the splunkd.log .

01-16-2018 08:06:47.560 -0500 WARN FileClassifierManager - The file '/var/adm/utmpx' is invalid. Reason: binary

01-16-2018 08:06:46.560 -0500 WARN FileClassifierManager - The file '/var/adm/wtmpx' is invalid. Reason: binary.

So let me know whether, I can use the new regex which you had mentioned in the comment.
thanks in advance.

0 Karma

mayurr98
Super Champion

yes try that and for binary
check this link and my accepted solution on this
https://answers.splunk.com/answers/610499/why-uf-think-my-file-is-binary.html#comment-611591

0 Karma

Hemnaath
Motivator

Hi Mayurr, I tried by creating a test app and pushed the below configuration for the test machine and it was fetching the some data but the data was not in a readable format.

Inputs.conf
[monitor:///var/adm/wtmpx]
index = unix
sourcetype = unix:host:wtmpx

Props.conf

[unix:host:wtmpx]
CHARSET = AUTO
NO_BINARY_CHECK = true

And the output in splunk console :

\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00ts/1pts/1\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00No\x00\x00\x00\x00\x00\x00\x00Z^>Q\x00 P

Kindly let me know how to correct this.

0 Karma

mayurr98
Super Champion

I guess the data is binary you need to put appropriate CHARSET

0 Karma
Did you miss .conf21 Virtual?

Good news! The event's keynotes and many of its breakout sessions are now available online, and still totally FREE!