WatchDirectory Forum
http://www.watchdirectory.net/cgi-bin/yabb25/YaBB.pl
watchDirectory forums >> Plugins >> FileAge Monitoring (Large # of Folders/Files)
http://www.watchdirectory.net/cgi-bin/yabb25/YaBB.pl?num=1381333375

Message started by montgomery_oh on Oct 9th, 2013 at 5:42pm

Title: FileAge Monitoring (Large # of Folders/Files)
Post by montgomery_oh on Oct 9th, 2013 at 5:42pm
I setup a task to delete files older than 30 days folloing these instructions:
http://www.watchdirectory.net/wdhelp/help/detect_old_files.html

Events:
FILENEW

How to run
Run as a service
Service automatically starts
Hide the run window

Monitoring Method
Trigger: 30 Days
Calculate: Latest of Creation/Modified
Poll: 1440 minutes
Remember triggered files (and tried without)

I have around 850K files in over 200 subdirectories.  I am filtering based on relative directory resulting in about 20 subdirectories.  These 20 subdirectories may contain up to 60K files. Each directory should contain about 1500 files that are less than 30 days resulting in 58K files deleted.  Then on a daily basis it should delete about 48 files from each subdirectory.

I turned on "Write Debug info..." and the task runs for a while, deletes some files and then gets stuck at a DMFA - Waiting for timeout

It appears to run fine if I start at a lower subdirectory.

How does it step through the directories?  I was anticipating alphabetical order and then deleting each file in that directory older than 30 days and then moving on to the next folder meeting the filter criteria.  Is there anything I can change to have it run smoother and finish (aside from breaking up the task for individual subdirectories)?  Do you know of any limitations regarding file counts?  

Title: Re: FileAge Monitoring (Large # of Folders/Files)
Post by Gert on Oct 10th, 2013 at 7:29am
Hi,

Please do NOT set the option to remember triggered files.

When you get the "waiting for timeout" message, it means the task completed scanning the directory and is waiting for the "poll directory every x minutes" to elapse. So it has checked every file.

When you enable "debug", it also reports the timestamps of all checked files to the history.db so you could check them.


WatchDirectory Forum » Powered by YaBB 2.5.2!
YaBB Forum Software © 2000-2017. All Rights Reserved.