Splunk Search

How would I assign 1 sourcetype 2 different indexes?

SplunkDash
Motivator

Hello,

How I would assign one source type to two different indexes, one after another. As an example: I assigned sourcetype =win:syslog to index=winsyslog_test on January/20/2022. Now I need to assign sourcetype=win:syslog to index=win_syslog. I have 2 issues:

1. How I would  assign   sourcetype=win:syslog to index=winsyslog_test and index=win_syslog under this condition?

2. If I assign sourcetype=win:syslog to index=win_syslog, all of the events sourcetype=win:syslog (with index=winsyslog_test) have since January/20/2022 also show up under index=win_syslog sourcetype=win:syslog?

Any help will be highly appreciated. Thank you! 

Labels (1)
0 Karma
1 Solution

gcusello
SplunkTrust
SplunkTrust

Hi @SplunkDash,

at first it's strange to have different names between prod and test because you have to manually modify each search when you pass from test to prod!

I'm not sure, it's easier to copy the files in another folder or create a symbolic link.

Anyway, your inpus.conf should have also the file name in the monitor command:

[monitor://E:\path1\your_file.log]
sourcetype=win:syslog
index=winsyslog_test
disabled = 0
crcSalt = <SOURCE>

[monitor://E:\path2\your_file.log]
sourcetype=win:syslog
index=win_syslog
disabled = 0
crcSalt = <SOURCE>

Ciao.

Giuseppe

View solution in original post

SplunkDash
Motivator

Hello @gcusello 

Awesome!

Thank you so much for your support, truly appreciated, as always.

0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @SplunkDash,

at first, one question: why do you want to do this?

if you index twice a log, you pay twice the license, why?

Anyway, you can have the same sourcetype in all the indexes you like, there's no problem, you have only to remember this in your searches.

If you want to know how to index twice the same log in two indexes with the same sourcetype, you have to:

  • copy the file in another path or create a symbolic link,
  • use two different inputs stanzas,
  • both with crcSalt=<SOUCE> and the same sourcetype, but with different index.

Ciao.

Giuseppe

0 Karma

SplunkDash
Motivator

@gcusello 

 

Thank you again and very good question.

We have Test and Prod environments and they have 2 different access roles.

Regarding files, we don't have access to source servers, but our UF installed there and we use that server as deployment client. Do you think following going to work:

inputs.conf

[monitor://E:\path]

sourcetype=win:syslog

 index=winsyslog_test

disabled = 0

crcSalt = <SOURCE>

[monitor://E:\path]

sourcetype=win:syslog

index=win_syslog

disabled = 0

crcSalt = <SOURCE>

0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @SplunkDash,

at first it's strange to have different names between prod and test because you have to manually modify each search when you pass from test to prod!

I'm not sure, it's easier to copy the files in another folder or create a symbolic link.

Anyway, your inpus.conf should have also the file name in the monitor command:

[monitor://E:\path1\your_file.log]
sourcetype=win:syslog
index=winsyslog_test
disabled = 0
crcSalt = <SOURCE>

[monitor://E:\path2\your_file.log]
sourcetype=win:syslog
index=win_syslog
disabled = 0
crcSalt = <SOURCE>

Ciao.

Giuseppe

SplunkDash
Motivator

@gcusello 

Would it be possible to use same path? No ways we can use different path names, is there any ways we can use same path? Thank you!

0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @SplunkDash,

could you make a copy of the file, eventually with a symbolic link?

Otherwise, Splunk doesn't index twice a log also using crcSalt.

Ciao.

Giuseppe

SplunkDash
Motivator

@gcusello 

Probably no.

But, if we don't use crcSalt, would it be possible then?

0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @SplunkDash,

crcSail is the only way to foce Splunk to index twice an already indexed file, but not with the same name!

You have to find a different solution, e.g. extract from the Indexers the logs to send to test environment, but it isn't so easy.

If it's only for test, could you connect the test Search Head to the Production Indexers? in this way, you have the production data but you can modify as you like the test dashboards.

Ciao.

Giuseppe

jotne
Builder

One more question.  Why can you not have the data in one index?  Should be easy to search from various apps.

jotne
Builder

So in short, you would like to save the same data to two different index, but with same sourcetype?  (It may using double the license )

SplunkDash
Motivator

Hello,

Thank you so much, appreciated your quick response. 

Yes, that is correct plus need to make sure, new index with the same sourcetype has the data since January/20/2022.  

Regarding license: yes, we have the double license. If this is the only requirement , then I just need to use following inputs.conf file to achieve this goal.

inputs.conf

[monitor://E:\path]

sourcetype=win:syslog

 index=winsyslog_test

disabled = 0

crcSalt = <SOURCE>

[monitor://E:\path]

sourcetype=win:syslog

index=win_syslog

disabled = 0

crcSalt = <SOURCE>

Thank you again!

0 Karma

SplunkDash
Motivator

Hello,

Thank you again and very good question.

We have Test and Prod environments and they have 2 different access roles.

0 Karma

SplunkDash
Motivator

Hello @gcusello

Thank you again for your support, truly appreciate it.

I have a one question:

Are there any ways we can find list of the source types with the time those have been created and group by index names? I have following command, but just it is giving me list of source types with the dates those have been created, not with the index names. 

|metadata type=sourcetypes  index=*| convert ctime(firstTime)

Thank you so much again.

0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @SplunkDash,

it's easy, as you did, to have all the sourcetipes for each index.

It's possible to have the first event for each sourcetype in each index, it isn't possible to have the sourcetype's creation date.

to have the above information, please try this:

| metasearch index=*
| stats earliest(_time) AS earliest BY index sourcetype
| eval earliest=strftime(earliest,"%Y-%m-%d %H:%M:%S")

Ciao.

Giuseppe

SplunkDash
Motivator

Hello @gcusello,

I am planning to install ARUBA TA; do you have any recommendation. I will also submit same question separately in SPLUNK community but wanted to make sure you have this question. Any recommendations will be highly appreciated. Thank you!

 

0 Karma

gcusello
SplunkTrust
SplunkTrust

HI @SplunkDash,

thank you for your consideration!

I haven't any experience on this environment, the only general hint I can give is to read with attention the documentation before to start and to find the correct Ad-On to use because there are three different ones.

Have a good day and see next time!

Please accept one answer for the other people of Community

Ciao and happy splunking

Giuseppe

P.S.: Karma Points are appreciated 😉

Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...