WatchDirectory Forum
http://www.watchdirectory.net/cgi-bin/yabb25/YaBB.pl
watchDirectory forums >> Wx Remote Control Center >> File transfer paused at x,999, and running really
http://www.watchdirectory.net/cgi-bin/yabb25/YaBB.pl?num=1484165871

Message started by GRM on Jan 11th, 2017 at 9:17pm

Title: File transfer paused at x,999, and running really
Post by GRM on Jan 11th, 2017 at 9:17pm
I am new to WatchDirectory, and I am running a transfer of a 42GB file. This file has 3 subfolders, and in one of the subfolders there are 150,000 files.

The transfer rate is ridiculously slow, but that is a story for another thread. My question is, is there a setting that causes the process to slow or stop when it reaches 1,000 files?

Whenever the transfer reaches x,999 files, it seems to pause for 3-4 minutes before continuing on at x,000.

The size of the file does not seem to matter, as I have observed this several times with different size files.

Can anything be done to address that?

Also, is there anything that can be done to increase the throughput rate (similar to adding connections to Filezilla transfers).

Title: Re: File transfer paused at x,999, and running really
Post by Gert on Jan 12th, 2017 at 7:33am
Hi,

I am not sure why there would be a pause at 1000 files, there is noting in WatchDirectory doing this so I must assume it is something on the server side (FTP).

About the upload speed, you are correct WD uses a single upload thread. Currently there is not much you can do about it.
One thing we found out.... Some modems (firewalls?) give preference (speed!) to certain known file types (for example *.mp4 files) and lower the speed of other file types. "Quality of Service" is a typical name used for this setting. If you use the option "upload files with a .partial extension, rename when complete" it will always be an unknown file type.....

To simulate multiple upload threads....
stop the task
right-click it and select "Copy" from the menu that pops up to create an identical task
Use the "Filter Events" settings on both tasks so these tasks work on different sets of files.
For example, task-1 only uploads files that have names starting with A, B, C or D
task-2 only uploads files having names that do NOT start with A, B, C or D (see attached picture for this one)
Obviously this will depend on the naming of your files so be creative.
12-1-2017_7-32-27.png (67 KB | 76 )

Title: Re: File transfer paused at x,999, and running really
Post by GRM on Jan 12th, 2017 at 2:47pm
Thank you for the quick response.

WatchDirectory Forum » Powered by YaBB 2.5.2!
YaBB Forum Software © 2000-2017. All Rights Reserved.