topic Re: How do you make a regex to remove a string from file path in inputs.conf? in Splunk Search

How do you make a regex to remove a string from file path in inputs.conf?

meet_vadaria — Tue, 27 Nov 2018 22:32:29 GMT

I want to change a source by removing a "hostname" from file path (string) using inputs.conf

Currently, the source is /var/log/syslog/<hostname>/filename.

What I am looking for is /var/log/syslog/filename

How do I remove "hostname" string from path? I need to add it to inputs.conf

thanks in advance.

Re: How do you make a regex to remove a string from file path in inputs.conf?

ddrillic — Tue, 27 Nov 2018 22:53:57 GMT

@meet_vadaria, which path would you like to monitor? If it's /var/log/syslog/filename can't you just specify it as such in input.conf?

Re: How do you make a regex to remove a string from file path in inputs.conf?

meet_vadaria — Tue, 27 Nov 2018 22:57:33 GMT

filename is dynamic. I want to modify source of event to this. splunk forwarder by default shows file path which is /var/log/syslog// with hostname and filename dynamic. I want it to be modified to /var/log/syslog/. need to remove from file path.

Re: How do you make a regex to remove a string from file path in inputs.conf?

ddrillic — Tue, 27 Nov 2018 23:05:27 GMT

At the level of input.conf, you can use the * and ... wildcards. If needed, you can change source later during index time.

Re: How do you make a regex to remove a string from file path in inputs.conf?

meet_vadaria — Tue, 27 Nov 2018 23:23:36 GMT

I think you are not getting my point. here is example of my input.conf on splunk forwarder.

[monitor:///var/log/syslog//]
host_segment = 4
sourcetype = xxx

this input.conf monitors all log files sitting under /var/log/syslog// directory. let's say I have 3 log files there as foo.log, bar.log, xxx.log

now my default source in splunk for this logs(events) will be /var/log/syslog//foo.log or specific log file

I want to change it to /var/log/syslog/foo.log. I believe I can do it in input.conf by using regex. I guess I can have a regex to remove string from /var/log/syslog//foo.log and it will be right way to do in input.conf

I don't know how to do it in indexing time.

Re: How do you make a regex to remove a string from file path in inputs.conf?

richgalloway — Wed, 28 Nov 2018 03:55:13 GMT

You should be able to do it with transforms. See transforms.conf.example.

[hostoverride]
REGEX = (\/var\/log\/syslog\/)[^\/]+(\/filename)
FORMAT = host::$1$2
DEST_KEY = MetaData:Host

Re: How do you make a regex to remove a string from file path in inputs.conf?

meet_vadaria — Tue, 29 Sep 2020 22:12:31 GMT

I think this block is to change Host filed of events. I am trying to change Source field. I don't see any documents regarding that. I found splunk documents mentioning I can change source type with transforms.conf and props.conf but not source.

any idea if we can change Source using transforms.conf and props.conf?

props.conf
[source::udp:514]
TRANSFORMS-changesourcetype = set_sourcetype_my_log_for_some_hosts

http://docs.splunk.com/Documentation/Splunk/5.0/Data/Advancedsourcetypeoverrides

Re: How do you make a regex to remove a string from file path in inputs.conf?

meet_vadaria — Wed, 28 Nov 2018 18:40:05 GMT

I am trying to override source field of syslog. I think given block will help me to override host field. i tried for source by modifying little bit. still can't get it working.

input.conf
[monitor:///var/log/syslog/]
host_segment = 4
sourcetype = xxx

transforms.conf
[sourceoverride]
REGEX = (\/var\/log\/syslog\/)[^\/]+(\/filename)
FORMATE = source::$1$2
DEST_KEY = MetaData:Source

props.conf
[source::/var/log/syslog/]
TRANSFORMS-changesource = sourceoverride

I have config as above.

Re: How do you make a regex to remove a string from file path in inputs.conf?

richgalloway — Thu, 29 Nov 2018 14:07:22 GMT

Sorry for misreading your question. Your modifications look good. Be careful, however, of the REGEX string. Make sure it matches your file paths. "filename", for example, is taken from your question, but is probably not part of the file path. You should modify it accordingly.