Getting Data In

Input data from directory sourcetype changing

zitacheung
Engager

Hello every one,

I am a new user of the splunk.

I have facing a problem that input the log file from directory.

In that directoy every 15 mins a new log file will be generate.

I use the Manager -> Data Input -> Files & directories to input file.

suppose I define a new sourcetype "ABC" in the first of input.

I found that every 15 mins the new files is generate and import in the splunk. A new sourcetype is create ABC-1 ...ABC-12....ABC-256.

Is there anything I can do to prevent the sourcetype keep increase?

Tags (1)
0 Karma
1 Solution

jbsplunk
Splunk Employee
Splunk Employee

There is a good solution for this documented on answers here:

http://splunk-base.splunk.com/answers/10154/sourcetype-would-increment

I would start with trying to implement a similar solution and see if it works for your situation. If it doesn't, let us know and we can try to offer additional information.

View solution in original post

jbsplunk
Splunk Employee
Splunk Employee

There is a good solution for this documented on answers here:

http://splunk-base.splunk.com/answers/10154/sourcetype-would-increment

I would start with trying to implement a similar solution and see if it works for your situation. If it doesn't, let us know and we can try to offer additional information.

zitacheung
Engager

Thanks jbsplunk,
I have one more question, is it normal that the number of wrong sorcetype keep increasing as more file be recognised

0 Karma

jbsplunk
Splunk Employee
Splunk Employee

You can't surgically delete data from Splunk. The data will be aged out via your retention policy, or you can clean all the data from the index in which these sourcetypes exist. But, after data has been indexed, you can't alter the metadata or delete specific records yourself.

zitacheung
Engager

Thanks I can make all the new input log file become the same sourcetype.
But how can I change the sourcetype of log that already imported.
And how can I remove the wrong sourcetype(ABC-1...ABC-300)?

0 Karma

zitacheung
Engager

Thanks for helping me. I have try to modify the props.conf and the transforms.conf but I cannot fix it,
the file i change locate at "C:\Program Files\Splunk\etc\system\local"
the code I use is

[source::C:\logfile*]
TRANSFORMS-Bandwidth=fix_Bandwidth


[fix_Bandwidth]
REGEX = .
FORMAT = sourcetype::BandWidth
DEST_KEY = MetaData:Sourcetype

But I still cannot get it.And in the link it mention I can tag the wrong sourcetype to the correct one. May I know where I can do it??
Many Thanks.

Zita

0 Karma
Get Updates on the Splunk Community!

Security Highlights | November 2022 Newsletter

 November 2022 2022 Gartner Magic Quadrant for SIEM: Splunk Named a Leader for the 9th Year in a RowSplunk is ...

Platform Highlights | November 2022 Newsletter

 November 2022 Skill Up on Splunk with our New Builder Tech Talk SeriesCan you build it? Yes you can! *play ...

Splunk Education - Fast Start Program!

Welcome to Splunk Education! Splunk training programs are designed to enable you to get started quickly and ...