WatchDirectory Forum
http://www.watchdirectory.net/cgi-bin/yabb25/YaBB.pl
watchDirectory forums >> Plugins >> Zipping large files not working.
http://www.watchdirectory.net/cgi-bin/yabb25/YaBB.pl?num=1242937322

Message started by mpc on May 21st, 2009 at 10:22pm

Title: Zipping large files not working.
Post by mpc on May 21st, 2009 at 10:22pm
I'm having an issue with zipping large files. Files that are less than about 5gbs compress ok. But when I try to compress a file that is more than 5gbs, the Zip plugin service stops. I get the following Windows Application Event Errors:
Event Type:      Error
Event Source:      watchDirectory:2005-SQL-Archive-ZIP
Event Category:      None
Event ID:      1
Date:            5/21/2009
Time:            2:04:24 PM
User:            MPSSF2\watchDirectory
Computer:      MPC-IT
Description:
onefile
------------------------------------------------------------------
Event Type:      Error
Event Source:      watchDirectory:2005-SQL-Archive-ZIP
Event Category:      None
Event ID:      1
Date:            5/21/2009
Time:            2:04:24 PM
User:            MPSSF2\watchDirectory
Computer:      MPC-IT
Description:
Plugin returned errorcode -1. Error message=No further info. watchDirectory will abort
------------------------------------------------------------------------------------------------------
Event Type:      Error
Event Source:      watchDirectory:2005-SQL-Archive-ZIP
Event Category:      None
Event ID:      1
Date:            5/21/2009
Time:            2:04:24 PM
User:            MPSSF2\watchDirectory
Computer:      MPC-IT
Description:
Plugin returned errorcode -1. Error message=No further info. watchDirectory will abort
--------------------------------------------------------------------------------------------------------

Has anybody seen this before and or have a solution to the problem?
Thanks
Brian


Title: Re: Zipping large files not working.
Post by Gert on May 22nd, 2009 at 12:44pm
Hi,

Large files *should* not be an issue for the ZIP task. I have to admit we do not currently test with large files. I will see what we can do to make this work.

Gert

Title: Re: Zipping large files not working.
Post by Gert on Jun 5th, 2009 at 2:09pm
This has been fixed in the latest beta:
http://www.watchdirectory.net/cgi-bin/yabb2/YaBB.pl?num=1235818031

Title: Re: Zipping large files not working.
Post by mpc on Jun 22nd, 2009 at 6:57pm
Thanks for getting this fixed so quickly.
FYI: I have been able to zip up files as large as 94gb so far. I have some that are 200+ gb I will let ya know if I have any issues with these.

Thanks
Brian

Title: Re: Zipping large files not working.
Post by Gert on Jul 16th, 2009 at 1:48pm
The beta has now been released as v4.5.9
I expect zipping and unzipping will have no real limits anymore. There was a bug in both plugins that caused them to "loose position" for truly large files (~4gb seems to be the limit, but it also depended on the amount of free memory). The current version reads "chunks" of the file to compress and writes them to the ZIP immediately.
This new technique might cause slightly less compression on large files.

Obviously, if your file-system doesn't allow big files, WD does not support them. NTFS supports terabyte sized files (but we are not testing :) ).

Title: Re: Zipping large files not working.
Post by mpc on Jan 28th, 2010 at 8:52pm
I upgrade to 4.6 and  now I'm having problems zipping large files again. I'm getting the following.:
openzip ChilkatLog:
 OpenZip:
   DllDate: Mar 21 2009
   UnlockPrefix: GDPSOFTWARZIP
   Username: watchdirectory
   Component: Visual C++ 6.0
   zipFilename: WSS_Content_IAT_backup.zip
   OemCodePage: 437
   ZipStructure:
     FirstSignature: 0x4034b50
     FirstSigAddr: 0
     File access error: did not read full requested amount of data.
     curAddr64: 4911716953
     Failed to read Zip (10)
   Failed.

Date/Time: 1/28/2010 12:21:10 PM
Computer: MPC-IT



[watchDirectory:2005-SQL-ArcSite-ZIP] ERROR Plugin returned errorc...
Plugin returned errorcode -1. Error message=No further info. watchDirectory will abort
Date/Time: 1/28/2010 12:21:10 PM

Title: Re: Zipping large files not working.
Post by Gert on Jan 29th, 2010 at 8:20am
Hi,

You can downgrade here (4.5.9):
http://www.watchdirectory.net/downloads/SetupWatchDirectoryVersion459.exe
or (4.5.10):
http://www.watchdirectory.net/downloads/SetupWatchDirectoryVersion4510.exe


Do you always get this error or do you get it on some zips?

Title: Re: Zipping large files not working.
Post by Gert on Jan 29th, 2010 at 8:34am
By the way....

I just tested v4.6 with an 8GB file and it zipped fine. Also, the zip task had only the following changes for 4.6:
* new copyright statement (include 2010)
* it now writes the new zip filename to the history db

I can not imagine how this would break functionality of the zip task.

Is it possible another program was writing to the file being zipped?

Title: Re: Zipping large files not working.
Post by mpc on Feb 1st, 2010 at 6:00pm
The zipping was working fine prior to the upgrade. Seems I'm having this problem with only the really large files. Large = 100gb and larger. I will continue to monitor and debug ans much as I can.

WatchDirectory Forum » Powered by YaBB 2.5.2!
YaBB Forum Software © 2000-2017. All Rights Reserved.