WatchDirectory home page
WatchDirectory Startseite (Deutsche Version)
Site WatchDirectory (Français)
  Welcome, Guest. Please Login or Register
YaBB - Yet another Bulletin Board
   
  HomeHelpSearchLoginRegister  
 
Page Index Toggle Pages: 1
File transfer paused at x,999, and running really (Read 444 times)
GRM
YaBB Newbies
*
Offline



Posts: 2
File transfer paused at x,999, and running really
Jan 11th, 2017 at 9:17pm
 
I am new to WatchDirectory, and I am running a transfer of a 42GB file. This file has 3 subfolders, and in one of the subfolders there are 150,000 files.

The transfer rate is ridiculously slow, but that is a story for another thread. My question is, is there a setting that causes the process to slow or stop when it reaches 1,000 files?

Whenever the transfer reaches x,999 files, it seems to pause for 3-4 minutes before continuing on at x,000.

The size of the file does not seem to matter, as I have observed this several times with different size files.

Can anything be done to address that?

Also, is there anything that can be done to increase the throughput rate (similar to adding connections to Filezilla transfers).
Back to top
 
 
IP Logged
 
Gert
YaBB Administrator
*****
Offline



Posts: 2217
The Netherlands
Re: File transfer paused at x,999, and running really
Reply #1 - Jan 12th, 2017 at 7:33am
 
Hi,

I am not sure why there would be a pause at 1000 files, there is noting in WatchDirectory doing this so I must assume it is something on the server side (FTP).

About the upload speed, you are correct WD uses a single upload thread. Currently there is not much you can do about it.
One thing we found out.... Some modems (firewalls?) give preference (speed!) to certain known file types (for example *.mp4 files) and lower the speed of other file types. "Quality of Service" is a typical name used for this setting. If you use the option "upload files with a .partial extension, rename when complete" it will always be an unknown file type.....

To simulate multiple upload threads....
stop the task
right-click it and select "Copy" from the menu that pops up to create an identical task
Use the "Filter Events" settings on both tasks so these tasks work on different sets of files.
For example, task-1 only uploads files that have names starting with A, B, C or D
task-2 only uploads files having names that do NOT start with A, B, C or D (see attached picture for this one)
Obviously this will depend on the naming of your files so be creative.
Back to top
 


Gert Rijs - gert (at) gdpsoftware (dot) com
Blog: http://blog-en.gdpsoftware.com/
End Alzheimer's: http://www.alz.org&&...
WWW WWW GdPSoftware  
IP Logged
 
GRM
YaBB Newbies
*
Offline



Posts: 2
Re: File transfer paused at x,999, and running really
Reply #2 - Jan 12th, 2017 at 2:47pm
 
Thank you for the quick response.
Back to top
 
 
IP Logged
 
Page Index Toggle Pages: 1