topic Forwarding one instance to another in Getting Data In

Forwarding one instance to another

Drainy — Wed, 08 Jun 2011 10:24:36 GMT

Hi,

I have a splunk server setup on an internal network which has differing numbers of machines all using either syslog or the splunk forwarder to send on event or log information to the main server.
This network is not accessible externally so I have setup an external server.
Inside the internal one I then set it to forward everything and to keep a copy on itself, thus hopefully creating two servers containing identical data.

It has kind of worked. The internal one and external one are pretty much perfectly synced and have all the latest details, however the external one only has events going back as far as 3pm yesterday? The internal server has events going back to the middle of last week.

I did leave it overnight to try a reboot this morning to see if it was any different but it is still the same, does anyone have any ideas why this might be?

Re: Forwarding one instance to another

sdwilkerson — Wed, 08 Jun 2011 12:00:58 GMT

Draineh,

Are you saying the external server now is updated since the earliest event it received, or that it always maintains just a day (or so) of data in it?

When you configure a data clone, it will only send data from the time you set it up (its a copy not a sync). So if you did this after 3PM on the day in question, you will have nothing before that time but should have everything since.

Best,
Sean

Re: Forwarding one instance to another

Drainy — Wed, 08 Jun 2011 12:04:09 GMT

Thanks for the reply. That may well be the case, I have alot happening in here so I find it difficult maintaining what time I have started things. Is it safe/possible if I was to copy and paste the db folders for the indexes from the internal server to the external server? I basically have some log data on the internal server which I need to be on the external server for some people to work with

Re: Forwarding one instance to another

sdwilkerson — Wed, 08 Jun 2011 12:18:04 GMT

Draineh,

If now you are asking about bucket migration, yes that is possible. Since Splunk uses a flatfile data structure, you can just move the buckets around as necessary from system to system.
You do want to be sure of a few things:

The bucket isn't a hot bucket (roll the bucket first, if so)
That splunk isn't currently using the bucket when you move it
That the destination system doesn't have a duplicate bucket name already

Here are some high-level instructions: http://www.splunk.com/wiki/Community:MoveIndexes

Sean