All-time way to copy large amount of files to qnap

Innovate yourself to united states of america and other members here, or share your own product reviews, suggestions, and tips and tricks of using QNAP products.

qnbobby
Starting out
Posts: 15
Joined: Sun Jan 08, 2017 12:28 am

Best fashion to copy big amount of files to qnap

Finally setup my Qnap 451+ and now I need to copy big amount of files to information technology. The number of files is on the club of 100k+. How should I go almost doing this? Don't want to use windows copy & paste equally it will take days I guess. I take Beyond Compare, would that be good option or Qnap has something better. I am reading and see mention of this File Station but tin it easily copy that many files without any issues? Any other suggestions? Thanks in advance.


User avatar

razormoon
Easy as a breeze
Posts: 465
Joined: Fri Feb 13, 2015 5:05 am
Location: Denver, CO

Re: Best mode to copy big amount of files to qnap

Post by razormoon » Sunday Jan 08, 2017 1:xiii pm

qnbobby wrote:Finally setup my Qnap 451+ and at present I need to copy large amount of files to it. The number of files is on the order of 100k+. How should I go about doing this? Don't want to use windows re-create & paste as it will take days I guess. I have Across Compare, would that be expert pick or Qnap has something better. I am reading and see mention of this File Station but tin information technology easily copy that many files without any problems? Any other suggestions? Thanks in advance.

Have yous done any test copies? What makes you believe that the transfer is deadening?
With such an incredible amount of information at that place isn't anything that would brand it faster, it is what it is. If there is a noticeable "slowness", then nosotros'll exist more than happy to troubleshoot with y'all.

:!: TVS-871-i7-16G 5.0.0.1932 Build 20220129 , KODI, WSE 2019
:?: 1 10 KINGSTON SNV325S2 equally 2mb block enshroud, WDC WD40EFRX as RAID5, i x WDC WD40EFRX every bit iSCSI
:idea: APC PRO 1500 S
:-0 WIKI SUPPORT

"Zilch is impossible. Just expensive, illegal or both."


qnbobby
Starting out
Posts: 15
Joined: Dominicus Jan 08, 2017 12:28 am

Re: Best way to re-create big corporeality of files to qnap

Post past qnbobby » Sun Jan 08, 2017 one:19 pm

I haven't copied annihilation still, I am simply asking what is the all-time program to copy that many files. Does Qnap has something built in or I should use external programs similar BeyondCompare or rsysnc. This is my start fourth dimension with Qnap if that matters.


User avatar

dolbyman
Guru
Posts: 26537
Joined: Sat Feb 12, 2011 2:11 am
Location: Vancouver BC , Canada

Re: Best mode to copy large amount of files to qnap

Post past dolbyman » Lord's day Jan 08, 2017 ane:28 pm

use robocopy if your are pushing files to the qnap


User avatar

razormoon
Piece of cake as a breeze
Posts: 465
Joined: Fri February 13, 2015 5:05 am
Location: Denver, CO

Re: Best way to copy large amount of files to qnap

Post by razormoon » Sunday Jan 08, 2017 1:58 pm

qnbobby wrote:I oasis't copied annihilation yet, I am just asking what is the best plan to copy that many files. Does Qnap has something built in or I should utilise external programs like BeyondCompare or rsysnc. This is my first fourth dimension with Qnap if that matters.

Ah. Yes, of form. I apologize.

I had set my folders up before my QNAP was delivered. In that example, I mirrored the NAS' share folder. With that method, I mirrored the share of my NAS (/share) on my Windows client (C:\NAS\share) and copied everything into it (C:\NAS\share\Public, C:\NAS\share\Multimedia) and sent information technology over with one drag and drop.
You may heed dolbyman'southward suggestion with robocopy...you tin practice a batch command.
In every example, gear up to get out the client PC on overnight.
Information technology'll take a while, but well worth the wait. :D

If you do use my method, yous tin can move files (as opposed to copy) from where they currently reside on your PC to the C:\NAS\share and it won't have long to just drag over.

:!: TVS-871-i7-16G 5.0.0.1932 Build 20220129 , KODI, WSE 2019
:?: 1 x KINGSTON SNV325S2 as 2mb block cache, WDC WD40EFRX as RAID5, i x WDC WD40EFRX every bit iSCSI
:idea: APC PRO 1500 Due south
:-0 WIKI SUPPORT

"Zippo is incommunicable. Only expensive, illegal or both."


ja666
Know my way around
Posts: 183
Joined: Sat Nov 29, 2014 5:21 pm

Re: Best way to copy large amount of files to qnap

Post by ja666 » Sun January 08, 2017 four:24 pm

Final days I did similar (multiple of 40k+ files) task of rebuilding TS-x69L folders from network backup, so my suggestions:

* Avoid FTP protocol as FTP may screw-up sooner or later (once copy speed fails from 81MB/due south to 150kBs with troublesome file i.e. unknown filename coding folio, FTP initial transfer speed never builds-up again), go for CIFS/SMB protocol (in Win environs);
* You lot may want to dismantle your HDD from PC and adhere it to NAS directly with eSata ribbon cablevision, but if that HDD is your system disk (not data only disk) y'all'd ameliorate avoid that;
* When uploading files from PC to NAS I apply Total Commander, with TC 'verify' option (which sadly doubles the copying time);
* Movement is more risky (source may disappear when target may not build-up) than plain re-create , I believe unsuccessful FTP move did loose some of the files (with 'foreign' unsupported codepage names) (if it wasn't fill-in of the backup);
* Later on copy I synchronize folders with Total Commander to check if source and target folders are identical, if so I delete a source whenn needed.

-----

qnbobby wrote: The number of files is on the guild of 100k+


User avatar

schumaku
Guru
Posts: 43648
Joined: Mon Jan 21, 2008 4:41 pm
Location: Kloten (Zurich), Switzerland -- Skype: schumaku
Contact:

Re: Best way to copy large amount of files to qnap

Post by schumaku » Dominicus Jan 08, 2017 ix:11 pm

ja666 wrote:* Avoid FTP protocol as FTP may spiral-up sooner or subsequently (once copy speed fails from 81MB/due south to 150kBs with troublesome file i.e. unknown filename coding folio, FTP initial transfer speed never builds-upwards once again), become for CIFS/SMB protocol (in Win environment)

Five past one. Windows does make utilize of UTF8 folder and file proper noun encoding, the very same decent FTP clients are using, and QTS does support for folders and file names. If at that place are folders and file names - however these came to a Windows or U**10 like system (ie. past the many (if not all) file name encoding ignorant archivers and dearchivers - exist on the source side, information technology's not really to blame the whatever protocol ... in fact more the protocol client pushing potentially invalid folder or file names blindly _after_ declaring that information technology _is_ UTF8.


User avatar

lithiumus
Starting out
Posts: 45
Joined: Tue Sep 27, 2016 11:22 am

Re: All-time fashion to copy large amount of files to qnap

Post by lithiumus » Wednesday January 11, 2017 one:xv pm

I just did nigh 15tb of information transfer using File Station and I didn't have any problems. I had multiple drives continued using USB three.0 cables ordinarily transferring at to the lowest degree two drives at once and sometimes 3 drives to the Qnap. I didn't have whatever pregnant bug with File Station. My larger media files transferred at almost ~60+ Mb/s merely the smaller files were slower sometimes downwardly to 20 Mb/s. I did them in 1-3Tb chunks and in some cases I had almost 200k files in a single groundwork transfer. No issues.

There may be faster ways that i didn't try just I wanted to control the transfers and run across the progress.

The only problem I had with the File Station was when I tried uploading from my laptop to the qnap via the network using the spider web upload office while I was already transferring from the direct continued USB drives. It would either process extremely slowly or would hang during the upload prep.


derron
New hither
Posts: 4
Joined: Saturday Apr 20, 2013 four:31 am

Re: Best way to copy large amount of files to qnap

Mail service by derron » Fri Jun xxx, 2017 6:15 pm

Could I connect the NAS HDD (ExtFS) directly to PC (internal SATA) if I have a No-Raid Configuration (1 Bay NAS)?
Or should all file transfers done by the use of the NAS?


User avatar

schumaku
Guru
Posts: 43648
Joined: Mon January 21, 2008 iv:41 pm
Location: Kloten (Zurich), Switzerland -- Skype: schumaku
Contact:

Re: Best way to re-create big amount of files to qnap

Post by schumaku » Fri Jun 30, 2017 7:33 pm

derron wrote:Could I connect the NAS HDD (ExtFS) direct to PC (internal SATA) if I accept a No-Raid Configuration (1 Bay NAS)?

No. The HDD would be re-partitioned to QNAP QTS requirements for internal HDDs, your information would be lost.

Every bit per your forum profile, the TS-x19P Two does non back up exFAT at all, on newer devices it's supported but requires an ad-on license.


derron
New hither
Posts: iv
Joined: Sat Apr 20, 2013 4:31 am

Re: All-time fashion to copy large amount of files to qnap

Post by derron » Fri Jun thirty, 2017 ix:xi pm

schumaku wrote:

derron wrote:Could I connect the NAS HDD (ExtFS) directly to PC (internal SATA) if I take a No-Raid Configuration (i Bay NAS)?

No. The HDD would be re-partitioned to QNAP QTS requirements for internal HDDs, your data would be lost.

Every bit per your forum profile, the TS-x19P Two does not support exFAT at all, on newer devices it's supported just requires an ad-on license.

Fifty-fifty though I installed Paragon extFS? Considering I tin can read the whole file construction and tin can write so far. I'thousand curious when I plug in the drive later.


User avatar

dolbyman
Guru
Posts: 26537
Joined: Sat Feb 12, 2011 2:eleven am
Location: Vancouver BC , Canada

Re: All-time manner to copy large corporeality of files to qnap

Post by dolbyman » Fri Jun 30, 2017 10:52 pm

yous installed paragon on the x19 qnap?!


derron
New here
Posts: 4
Joined: Sat Apr 20, 2013 4:31 am

Re: Best manner to re-create large amount of files to qnap

Post by derron » Sat Jul 01, 2017 12:45 am

dolbyman wrote:you installed paragon on the x19 qnap?!

Information source is a windows pc.


User avatar

dolbyman
Guru
Posts: 26537
Joined: Sat Feb 12, 2011 ii:xi am
Location: Vancouver BC , Canada

Re: All-time way to copy large amount of files to qnap

Mail by dolbyman » Sat Jul 01, 2017 1:39 am

so where does the paragon come into play ? ... Windows reads exFat natively and the QNAP volition not at all. And so if you lot stick the exFat drive in a Windows system, you should be able to transfer the files to the QNAP without a problem


derron
New here
Posts: iv
Joined: Sat April xx, 2013 4:31 am

Re: Best manner to re-create large corporeality of files to qnap

Post by derron » Sabbatum Jul 01, 2017 2:31 am

I thought that Windows can't handle the QNAP Partition. And there are some (non all) folders in my share binder that I can't open while connected direct, but dorsum in the QNAP everthing is fine.