Splunk Search

How do you make a regex to remove a string from file path in inputs.conf?

Engager

I want to change a source by removing a "hostname" from file path (string) using inputs.conf

Currently, the source is /var/log/syslog/<hostname>/filename.

What I am looking for is /var/log/syslog/filename

How do I remove "hostname" string from path? I need to add it to inputs.conf

thanks in advance.

0 Karma

SplunkTrust
SplunkTrust

You should be able to do it with transforms. See transforms.conf.example.

[hostoverride]
REGEX = (\/var\/log\/syslog\/)[^\/]+(\/filename)
FORMAT = host::$1$2
DEST_KEY = MetaData:Host
---
If this reply helps you, an upvote would be appreciated.
0 Karma

Engager

I am trying to override source field of syslog. I think given block will help me to override host field. i tried for source by modifying little bit. still can't get it working.

input.conf
[monitor:///var/log/syslog/]
host_segment = 4
sourcetype = xxx

transforms.conf
[sourceoverride]
REGEX = (\/var\/log\/syslog\/)[^\/]+(\/filename)
FORMATE = source::$1$2
DEST_KEY = MetaData:Source

props.conf
[source::/var/log/syslog/]
TRANSFORMS-changesource = sourceoverride

I have config as above.

0 Karma

SplunkTrust
SplunkTrust

Sorry for misreading your question. Your modifications look good. Be careful, however, of the REGEX string. Make sure it matches your file paths. "filename", for example, is taken from your question, but is probably not part of the file path. You should modify it accordingly.

---
If this reply helps you, an upvote would be appreciated.
0 Karma

Engager

I think this block is to change Host filed of events. I am trying to change Source field. I don't see any documents regarding that. I found splunk documents mentioning I can change source type with transforms.conf and props.conf but not source.

any idea if we can change Source using transforms.conf and props.conf?

transforms.conf
[setsourcetypemylogforsomehosts]
REGEX = :\d\d\s+(?:\d+\s+|(?:user|daemon|local.?).\w+\s+)[?(host1|host2|host3)[\w.-]]?\s
FORMAT = sourcetype::mylog
DEST
KEY = MetaData:Sourcetype

props.conf
[source::udp:514]
TRANSFORMS-changesourcetype = setsourcetypemylogforsomehosts

http://docs.splunk.com/Documentation/Splunk/5.0/Data/Advancedsourcetypeoverrides

0 Karma

Ultra Champion

@meet_vadaria, which path would you like to monitor? If it's /var/log/syslog/filename can't you just specify it as such in input.conf?

0 Karma

Engager

filename is dynamic. I want to modify source of event to this. splunk forwarder by default shows file path which is /var/log/syslog// with hostname and filename dynamic. I want it to be modified to /var/log/syslog/. need to remove from file path.

0 Karma

Ultra Champion

At the level of input.conf, you can use the * and ... wildcards. If needed, you can change source later during index time.

0 Karma

Engager

I think you are not getting my point. here is example of my input.conf on splunk forwarder.

[monitor:///var/log/syslog//]
host_segment = 4
sourcetype = xxx

this input.conf monitors all log files sitting under /var/log/syslog// directory. let's say I have 3 log files there as foo.log, bar.log, xxx.log

now my default source in splunk for this logs(events) will be /var/log/syslog//foo.log or specific log file

I want to change it to /var/log/syslog/foo.log. I believe I can do it in input.conf by using regex. I guess I can have a regex to remove string from /var/log/syslog//foo.log and it will be right way to do in input.conf

I don't know how to do it in indexing time.

0 Karma