Installation

Heavy Forwarder cannot forward events

malekmo
Contributor

Hello,

I have a HF running in Linux machine. I have root access to that machine using sudo bash  as sudo - splunk or su - splunk is  not allowing me to get root access. But, when I copy files to the folders  where monitor command pointing to pickup the files,  it is not forwarding events to the SPLUNK indexer since I cannot see those events within SPLUNK. However, when I type chown -R splunk: splunk/opt/splunk and then restart SPLUNK, it's working as expected, that means I can see those events within SPLUNK. So, every time when I copy  files within HF folders, I need to use chown command and restart SPLUNK to make them available within SPLUNK. Is there anyway this can be resolved that I don't need to type chown command and restart SPLUNK to forward events.  Thank you so much.

Labels (2)
Tags (1)
0 Karma
1 Solution

PickleRick
Champion

sudo might be restricted to some selected commands. But from the root user it should be possible to

su -s /bin/bash splunk

 

View solution in original post

PickleRick
Champion

Short answer - no, you can't help the fact that files have wrong ownership/permissions. That's what the whole permission system is for.

Long answer - in general, you shouldn't copy files into /opt/splunk. The proper approach would be to write the log files normally to - for example - /var/log/somewhere or /opt/your_service/var/log and add a monitor input to splunk reading directly from there. Then you should make sure that splunk user has access to those files (possibly by means of proper umasks, group membership and acls).

malekmo
Contributor

Hello Picklerick,

Thank you for your reply. 

Let me explain a little more how I copy the source files. I create app and use  GUI feature "Install app from file"  to pull the source files into SPLUNK HF opt/splunk/etc/apps/TA-my_sourcefile  folder and then copy those source files from that folder to /opt/splunk/var/log/sourcefiles  and add a monitor input to SPLUNK reading directly from there.  Only problem now how would I make sure to have access as SPLUNK user since sudo bash   is giving me root user access (ie, whoami shows only root) and su/sudo - splunk is not working for me in that Linux machine. Is there any other ways I can have SPLUNK user access? Thank you again.

0 Karma

isoutamo
SplunkTrust
SplunkTrust
Sounds like user splunk hasn’t login shell / rights. Can you try sudo -u splunk bash if that is working?

PickleRick
Champion

sudo might be restricted to some selected commands. But from the root user it should be possible to

su -s /bin/bash splunk

 

View solution in original post

malekmo
Contributor

Hello,

Yes.....it's working as expected...I got the access as a splunk user.....thank you so much to all of you, appreciated.

0 Karma

malekmo
Contributor

Hello,

Thank you ....appreciated......but, sudo -u splunk bash not working.

0 Karma

isoutamo
SplunkTrust
SplunkTrust
You can see with "sudo -l" what you can do with it.

malekmo
Contributor

Hello,

Thank you so much, it's giving be access as a root user....but, my issue to get access as splunk user.

0 Karma

PickleRick
Champion

That sounds way too overcomplicated. Why do you do it like that? Apps are not meant as a way of uploading files to ingest 😲

If your event-generating solution is not on the same host as your HF, why aren't you using UF or sending events via other means (syslog, HEC)?

0 Karma

malekmo
Contributor

Yes, it is really very complicated and time consuming......but some of the things we don't control......

0 Karma

PickleRick
Champion

It's simply confusing since you apparently have CLI access (with permission to run sudo bash ) so you have quite "wide" access to the machine. Furthermore, if you can install apps, you also have quite high-privileged access to the splunk itself. So it's very unusual to do it this way.

There are way more efficient ways to onboard data. Why don't you set the monitor a "static" file or directory and update it periodicaly with scp/sftp/whatever?

0 Karma

malekmo
Contributor

Hello,

Yes, agree!

I can see those events using index=_internal (X OR Y) host=zzzzz, but when I use index=X......I can't...getting error message "Insufficient permission to read file ='/opt/splunk/var/folder. Looks like I can see the events but SPLUNK apps cannot.  Thank you so much, any help will be highly appreciated.

In regard to complexity....we receive files from 5 different sources by Email, and then transform them using python scripts based on our requirements, and then pull them into our Linux server using app and then copy from the app folder to /opt/splunk/var/folder.........we are in a process of automatic this system....it's an interim solution.

 

0 Karma

PickleRick
Champion

If you have to pull the data using a script why not make it into a scripted/modular input and run it from within the splunk  service?

That seems more consistent with overall splunk architecture.

malekmo
Contributor

Hello,

Thank you so much....Yes, agree....but, we perform transformation process in different server/computer at this stage....and then pull the data using app...as I mentioned.  

0 Karma
Did you miss .conf21 Virtual?

Good news! The event's keynotes and many of its breakout sessions are now available online, and still totally FREE!