topic Re: Data copied from one index to another index in Splunk Search

How to copy data from one index to another index?

welcome — Thu, 17 Aug 2023 18:02:36 GMT

What will be the query to copy all data from one index to another index in splunk ,we are using splunk for jenkins logs

Re: Data copied from one index to another index

gcusello — Thu, 17 Aug 2023 07:02:43 GMT

there's no reasons to copy events from an index to another: indexes arent database's tables, they are silos in which store events with the same access rights and the same retention.

Why do you want to copy events from an index into another?

Anyway, there isn't a copy command, you can make a copy of the entire events or of some extracted fields from an index to another using the collect command (https://docs.splunk.com/Documentation/Splunk/9.1.0/SearchReference/Collect) that's usually used for Summary indexes.

Ciao.

Giuseppe

Re: Data copied from one index to another index

ITWhisperer — Thu, 17 Aug 2023 07:05:04 GMT

You can use the collect command to write to another index, but first I would question, why would you want to do this? What is it that you are trying to do that means you need a copy of the data in another index?

Re: Data copied from one index to another index

welcome — Thu, 17 Aug 2023 07:08:26 GMT

Just for the copy

Re: Data copied from one index to another index

welcome — Thu, 17 Aug 2023 07:13:04 GMT

I am using collect command but some data is missing, can you give me proper query

Re: Data copied from one index to another index

ITWhisperer — Thu, 17 Aug 2023 07:16:15 GMT

Which data is missing? Is it whole events or some fields in all events or some fields in some events? What characterises the missing data?

Re: Data copied from one index to another index

gcusello — Thu, 17 Aug 2023 07:17:05 GMT

Hi @welcome

please try something like this:

index=your_orig_index | table _time _raw | collect index=new_index

but there isn't any reason to do this!

ciao.

Giuseppe

Re: Data copied from one index to another index

welcome — Fri, 18 Aug 2023 11:45:46 GMT

Not all events only some events, suppose when we see in events section we have some fields ,in that we can see build related fileds ,we cant get the exact number of build fileds ,Job urls and job names etc.... as in previous index ,can you please give me proper answer

Re: Data copied from one index to another index

ITWhisperer — Fri, 18 Aug 2023 11:44:50 GMT

You keep asking for a proper answer, but you haven't really given a proper question, or at least one that can be answered in a meaningful way.

What events did you start with?

What events have you copied?

Which events did not get copied?

How did you do the copy?

Once we know what it is you are dealing with, and what you have already tried, we might be able to advise you further.

Re: Data copied from one index to another index

welcome — Tue, 22 Aug 2023 06:38:49 GMT

This is the query i have tried: index=A source=sourceA host=hostA | collect index=B source=sourceA sourcetype=sourcetypeA host=hostA

when I select for specific time range suppose from april1 to april 30, suppose there are 2 lakh events in indexA but I am getting only 1 lakh events into indexB ,I don't know why some events are missing.

Re: Data copied from one index to another index

ITWhisperer — Tue, 22 Aug 2023 07:02:51 GMT

There is possibly a limit to the number of events you can add to the summary index at any one time. Try running from Apr 1 to Apr 15, and then again from Apr 16 to Apr 30, or even Apr 1 to Apr 10, Apr 11 to Apr 20 and Apr 21 to Apr 30