Getting Data In

How to move data from one sourcetype to another sourcetype?

vishal_bandavad
Explorer

Hi,

I have sourcetype "abc" which has few days data. Now we have decided to modified the sourcetype name to def . I have made the required changes and data is coming to the new sourcetype. I just want to know how I could copy old sourcetype data to the new one.

Kindly help.

Tags (2)
0 Karma

DUThibault
Contributor

To change the indexed source and/or sourcetype after the fact, you need to export the raw data, delete it from the index, then re-import it. This is explained here: https://answers.splunk.com/answers/39756/change-source-and-sourcetype.html

In summary:

1) Export the logs with incorrect sourcetype so you have the raw, original logs:

 splunk search "index=myindex sourcetype=wrong_source_type" -maxout 0 -output rawdata > raw.logs

(Optionally also specify earliest=<timestamp> latest=<timestamp> -preview 0 etc.)

2) Delete the logs with incorrect sourcetype:

 index=myindex sourcetype=wrong_source_type | delete

(Use the same arguments as for the search)
(You have to add the 'delete' role to your account before doing this)

3) Re-index the raw logs:

 splunk add oneshot raw.logs -host myhost -index myindex -sourcetype correct_sourcetype -rename-source correct_source

(Without -rename-source , the new index would have source=raw.logs )

0 Karma

ppablo
Retired

Hi @vishal_bandavade

Can you revisit this post and confirm if the answer below solved your question? If yes, please resolve the post by clicking "Accept" directly below @Jeremiah's answer. Don't forget to upvote any responses in the thread that were helpful for you.

0 Karma

Jeremiah
Motivator

Take a look at sourcetype renaming. You can rename your abc events to def. This doesn't reindex or move the data, it just renames the sourcetype at search time.

http://docs.splunk.com/Documentation/Splunk/6.3.3/Data/Renamesourcetypes

In your props.conf do the following:

[abc]
rename=def

masonmorales
Influencer

^ @vishal_bandavade This. Your only other option is to delete and re-index the data.

MuS
SplunkTrust
SplunkTrust

and to add two important notes:

The indexed events still contain the original source type name. The renaming occurs only at search time. Also, renaming the source type does only that; it does not fix any problems with the indexed format of your event data caused by assigning the wrong source type in the first place.

and

Data from a renamed source type will only use the search-time configuration for the target source type 
0 Karma

somesoni2
SplunkTrust
SplunkTrust

Further,
The renamed sourcetype will not work for searches that queries tsidx /metadata files, such as "| metadata" OR "| tstats" etc.

0 Karma
Get Updates on the Splunk Community!

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...