WatchDirectory Forum - Using WD for load sharing

jimaus

Junior Member

Offline

Posts: 59
Australia

Using WD for load sharing
Apr 15^th, 2008 at 12:07am

Gert,
We use 2 Acrobat Distiller Servers to create pdf's of our newspaper pages, we would like to be able to share the load between the 2 servers.
Our Newspaper production system produces postscript files and drops them into a folder structure for conversion to pdf by distiller.
I would like to be able to poll an input folder and its associated output folders, one for each of the distiller servers, if the output folders are empty, WD will copy the file to one of the folders, if the output folders contain files already, WD not copy until it finds one of the folders empty, then copy.
This implies that WD is able to copy to one of 2+ locations, it may choose the same output folder each time if the load is light and only use the 2nd output folder when the load gets heavier.

Hope you can understand this,

Cheers
Jim

Back to top

IP Logged

Gert

YaBB Administrator

Offline

Posts: 2338
The Netherlands

Re: Using WD for load sharing
Reply #1 - Apr 15^th, 2008 at 7:42am

Hello Jim,

The following (untested!) batch file should do the trick:
Code:

@echo off
rem -- settings for this script --
SET GDPUTIL=%WD_INSTDIR%\gdputil.exe
rem     if you have more than 2 distiller processes, create a DISTFOLDER3 variable
SET DISTFOLDER1=D:\Testing\Target\d1
SET DISTFOLDER2=D:\Testing\Target\d2
rem     if all DISTFOLDERx already contain EPS files,
rem     this batch file will sleep SLEEPTIME seconds before it tries again
SET SLEEPTIME=2
rem -- end settings for this script --

rem make sure we are called for an eps file
IF /i "%WD_FILE_E%" NEQ "EPS" GOTO :EOF

:TryAgain
SET COUNTFOLDER=%DISTFOLDER1%
Call :CountEpsFiles
rem if you have more than 2 distiller processes, dup the next 4 lines for %DISTFOLDER3%
IF %COUNT% GEQ 1 (
	SET COUNTFOLDER=%DISTFOLDER2%
	Call :CountEpsFiles
)

IF %COUNT% GEQ 1 (
   rem both distiller folders already contain eps files
   rem sleep a bit and try again
   "%GDPUTIL%" -sleep %SLEEPTIME%
   GOTO :TryAgain
)

rem one of the distiller folders is empty
COPY "%WD_FILE%" "%COUNTFOLDER%"
IF %ERRORLEVEL% EQU 0 DEL "%WD_FILE%"
GOTO :EOF



rem -- subroutine to count the number of EPS files
rem -- in directory %COUNTFOLDER%
rem -- the result is stored in variable %COUNT%
:CountEpsFiles
SET COUNT=0
For %%i in ("%COUNTFOLDER%\*.eps") do Call :CountAddOne
GOTO :EOF

:CountAddOne
SET /A COUNT=%COUNT% + 1
GOTO :EOF

Run it using the batch file plugin of WD ( http://www.watchdirectory.net/wdhelp/plugins/wdopAutoRunBat.html ) to monitor the folder where the EPS files are created. If they aren't EPS files (but PS for example), you will need to change the script a bit more (eps -> ps in several places).

Gert

Back to top

Gert Rijs - gert (at) gdpsoftware (dot) com
Blog: http://blog-en.gdpsoftware.com/
End Alzheimer's: http://www.alz.org&&

WWW

IP Logged

jimaus Junior Member Offline Posts: 59 Australia	Re: Using WD for load sharing Reply #2 - Apr 15^th, 2008 at 8:47am Hi Gert, Thanks for the quick response, initial testing shows that it operates as requested. I will give it a run tomorrow on a busy queue. Will let you know how it goes. Cheers Jim
Back to top	IP Logged

Gert

YaBB Administrator

Offline

Posts: 2338
The Netherlands

Re: Using WD for load sharing
Reply #3 - Apr 16^th, 2008 at 3:05pm

Jim,

I have created a small variation on the above script that might be better suitable for you. This version will copy the file to the distiller folder that currently has the lowest number of files in it.
Code:

@echo off
rem -- settings for this script --
rem     if you have more than 2 distiller processes, create a DISTFOLDER3 variable
SET DISTFOLDER1=D:\Testing\Target\d1
SET DISTFOLDER2=D:\Testing\Target\d2
rem -- end settings for this script --

rem make sure we are called for an eps file
IF /i "%WD_FILE_E%" NEQ "EPS" GOTO :EOF

SET LOWEST=%DISTFOLDER1%
SET LOWCOUNT=99999

SET COUNTFOLDER=%DISTFOLDER1%
Call :CountEpsFiles
rem if you have added a third distiller, dup the following 2 lines for %DISTFOLDER3%
SET COUNTFOLDER=%DISTFOLDER2%
Call :CountEpsFiles

rem copy the file to the distiller with the lowest number of files
COPY "%WD_FILE%" "%LOWEST%"
IF %ERRORLEVEL% EQU 0 DEL "%WD_FILE%"
GOTO :EOF



rem -- subroutine to count the number of EPS files
rem -- in directory %COUNTFOLDER%
:CountEpsFiles
SET COUNT=0
For %%i in ("%COUNTFOLDER%\*.eps") do Call :CountAddOne

rem -- if this folder has less files as the previous lowest
rem -- setup this folder to be the lowest
IF %COUNT% LEQ %LOWCOUNT% (
   SET LOWEST=%COUNTFOLDER%
   SET LOWCOUNT=%COUNT%
)
GOTO :EOF

:CountAddOne
SET /A COUNT=%COUNT% + 1
GOTO :EOF

Gert

Back to top

Gert Rijs - gert (at) gdpsoftware (dot) com
Blog: http://blog-en.gdpsoftware.com/
End Alzheimer's: http://www.alz.org&&

WWW

IP Logged

jimaus Junior Member Offline Posts: 59 Australia	Re: Using WD for load sharing Reply #4 - Apr 24^th, 2008 at 6:22am Gert, Thanks I will give it a try. Cheers Jim
Back to top	IP Logged

kenwshmt2

YaBB Newbies

Offline

Posts: 5

Re: Using WD for load sharing
Reply #5 - May 22^nd, 2010 at 8:22pm

Can anyone think of a way to mod this to pull from a sorce directory and distribute onto as many as 20 directories? the directories will be emptied on completion. basically 'if dir empty, copy next file into it' , but only one file at a time.

One processing root, and the directories in that.
The destinations are cluttered, so monitoring those wouldn't be productive.

Back to top

IP Logged

Gert YaBB Administrator Offline Posts: 2338 The Netherlands	Re: Using WD for load sharing Reply #6 - May 23^rd, 2010 at 7:34am Hello Ken, Can you contact me by email with a more detailed description of what you want? I am not sure what you're asking exactly. Gert
Back to top	Gert Rijs - gert (at) gdpsoftware (dot) com Blog: http://blog-en.gdpsoftware.com/ End Alzheimer's: http://www.alz.org&& WWW IP Logged

kenwshmt2

YaBB Newbies

Offline

Posts: 5

Re: Using WD for load sharing
Reply #7 - May 23^rd, 2010 at 4:02pm

sorce dir with files- file1.m2p, file2.m2p,file3.m2p

check the emptyness of the processing directories
move one file to each processing directory if its available for it, but don't move a file there if a file is already there.

wait and scan, once a minute is often enough, until those are processed and the directory emptied, then move the next files into the new empty directories.

All the batch does is move files into the directories, it doesnt have to remove them once there.

Back to top

IP Logged

Gert YaBB Administrator Offline Posts: 2338 The Netherlands	Re: Using WD for load sharing Reply #8 - May 24^th, 2010 at 2:28pm I think the first script in this thread should do the trick. You will need to remove the checks for "EPS" or "PS" files, but it should work.
Back to top	Gert Rijs - gert (at) gdpsoftware (dot) com Blog: http://blog-en.gdpsoftware.com/ End Alzheimer's: http://www.alz.org&& WWW IP Logged

kenwshmt2 YaBB Newbies Offline Posts: 5	Re: Using WD for load sharing Reply #9 - May 25^th, 2010 at 12:38am the first script worked for what i was attempting.. now its distributing to 13 directories.
Back to top	IP Logged

WatchDirectory home page WatchDirectory Startseite (Deutsche Version) Site WatchDirectory (Français)
		Welcome, Guest. Please Login or Register