Sunday, February 22, 2009

Speed up a copy over the network

I have to copy all datafiles of a database over the network between two of my VMs.
The datafiles are big, all together, it is around 150Gb. My network is slow and busy (an average of 5 or 6Mb/s).
The standard scp command line copy one 1gb datafile in 6 minutes (5'52'') :

Sun Feb 22 15:39:57 CET 2009
scp root@ .
Sun Feb 22 15:45:49 CET 2009
ls -lrt
-rw-r----- 1 root root 1040195584 Feb 22 15:45 ./pclarge.dbf

How to speed up the copy ?
I found a way to reduce that time to 34 seconds for this 1Gb datafile. Since the servers are mine, everything is allowed .
But only one command line :
Sun Feb 22 15:45:49 CET 2009
Sun Feb 22 15:46:23 CET 2009
ls -lrt
-rw-r--r-- 1 root root 1040195584 Feb 22 15:46 ./pclarge.dbf

Any idea ?

update : thank you for the suggestions, I have to test them.
Currently, I used the following command, I don't say it is the best, but at least, was not bad according to the down time :
ssh root@ "gzip -c < /oradata/DMOFSCM9/datafiles/pclarge.dbf -"|gunzip -c > /oradata/DMOFSCM9/datafiles/pclarge.dbf


Anonymous said...

Maybe scp -C to enable compression?

Anonymous said...

Have you tried few other tools like Fast Data Transfer?