Getting Data In

How to move data from one sourcetype to another sourcetype?

vishal_bandavad
Explorer

Hi,

I have sourcetype "abc" which has few days data. Now we have decided to modified the sourcetype name to def . I have made the required changes and data is coming to the new sourcetype. I just want to know how I could copy old sourcetype data to the new one.

Kindly help.

Tags (2)
0 Karma

DUThibault
Contributor

To change the indexed source and/or sourcetype after the fact, you need to export the raw data, delete it from the index, then re-import it. This is explained here: https://answers.splunk.com/answers/39756/change-source-and-sourcetype.html

In summary:

1) Export the logs with incorrect sourcetype so you have the raw, original logs:

 splunk search "index=myindex sourcetype=wrong_source_type" -maxout 0 -output rawdata > raw.logs

(Optionally also specify earliest=<timestamp> latest=<timestamp> -preview 0 etc.)

2) Delete the logs with incorrect sourcetype:

 index=myindex sourcetype=wrong_source_type | delete

(Use the same arguments as for the search)
(You have to add the 'delete' role to your account before doing this)

3) Re-index the raw logs:

 splunk add oneshot raw.logs -host myhost -index myindex -sourcetype correct_sourcetype -rename-source correct_source

(Without -rename-source , the new index would have source=raw.logs )

0 Karma

ppablo
Retired

Hi @vishal_bandavade

Can you revisit this post and confirm if the answer below solved your question? If yes, please resolve the post by clicking "Accept" directly below @Jeremiah's answer. Don't forget to upvote any responses in the thread that were helpful for you.

0 Karma

Jeremiah
Motivator

Take a look at sourcetype renaming. You can rename your abc events to def. This doesn't reindex or move the data, it just renames the sourcetype at search time.

http://docs.splunk.com/Documentation/Splunk/6.3.3/Data/Renamesourcetypes

In your props.conf do the following:

[abc]
rename=def

masonmorales
Influencer

^ @vishal_bandavade This. Your only other option is to delete and re-index the data.

MuS
Legend

and to add two important notes:

The indexed events still contain the original source type name. The renaming occurs only at search time. Also, renaming the source type does only that; it does not fix any problems with the indexed format of your event data caused by assigning the wrong source type in the first place.

and

Data from a renamed source type will only use the search-time configuration for the target source type 
0 Karma

somesoni2
Revered Legend

Further,
The renamed sourcetype will not work for searches that queries tsidx /metadata files, such as "| metadata" OR "| tstats" etc.

0 Karma
Get Updates on the Splunk Community!

How to Monitor Google Kubernetes Engine (GKE)

We’ve looked at how to integrate Kubernetes environments with Splunk Observability Cloud, but what about ...

Index This | How can you make 45 using only 4?

October 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with this ...

Splunk Education Goes to Washington | Splunk GovSummit 2024

If you’re in the Washington, D.C. area, this is your opportunity to take your career and Splunk skills to the ...