WatchDirectory home page
WatchDirectory Startseite (Deutsche Version)
Site WatchDirectory (Franais)
  Welcome, Guest. Please Login or Register
YaBB - Yet another Bulletin Board
   
  HomeHelpSearchLoginRegister  
 
Page Index Toggle Pages: 1
watchDirectory hangs website and network access (Read 20289 times)
mmarx82
YaBB Newbies
*
Offline



Posts: 11
watchDirectory hangs website and network access
Sep 2nd, 2010 at 5:19pm
 
I'm testing watchDirectory 4.6.2 on a Windows 2003 standard edition server, The server is running 2 Intel Xeon E5335 Processors with 4 GB of Ram.
Additionally the server runs IIS, FTP, WSUS 3.2 on it. I'm the only user with direct network access to it.

I've setup 1 task to run as a service, I had it doing something else but to eliminate the possiblity of the first subtask being the culprit I removed it only to find I still have the issue. So now all it's doing is what you see in the attached image.

The problem is when watchDirectory does detect the new files in the monitored Directory and then it starts to run the configured tasks we cannot access our hosted website or get access to the network drives on this server. The funny part is network and cpu utilization are less than 5-10% during this time so you would think there shouldn't be an issue but there is. After the tasks end everything is fine again. Any suggestions or need any more info?

Back to top
 

Capture.JPG (50 KB | )
Capture.JPG
 
IP Logged
 
Dirk
YaBB Administrator
*****
Offline



Posts: 658
South Germany
Re: watchDirectory hangs website and network acces
Reply #1 - Sep 2nd, 2010 at 8:32pm
 
Hi mmarx82,

can you please tell us if you see the same behavior when the task is *not* running as a service (started "normal, F5")?
Back to top
 

Viele Gre / Best regards
Dirk - GdP Software

dirk [at] gdpsoftware [dot] com
http://blog-de.gdpsoftware.com
Webseite: http://www.gdpsoftware.com
WWW WWW  
IP Logged
 
mmarx82
YaBB Newbies
*
Offline



Posts: 11
Re: watchDirectory hangs website and network acces
Reply #2 - Sep 2nd, 2010 at 8:42pm
 
Ok, that was better, I still noticed a network access hickup though it was much better (98% better).
The website though was still affected, causing navigation on the site to hang 30-60 seconds.
Back to top
 
 
IP Logged
 
Dirk
YaBB Administrator
*****
Offline



Posts: 658
South Germany
Re: watchDirectory hangs website and network acces
Reply #3 - Sep 2nd, 2010 at 8:53pm
 
is it also a terminal server? Under which user account was the task running and how much more non- WatchDirectory services are running under the same account?
Back to top
 

Viele Gre / Best regards
Dirk - GdP Software

dirk [at] gdpsoftware [dot] com
http://blog-de.gdpsoftware.com
Webseite: http://www.gdpsoftware.com
WWW WWW  
IP Logged
 
mmarx82
YaBB Newbies
*
Offline



Posts: 11
Re: watchDirectory hangs website and network acces
Reply #4 - Sep 2nd, 2010 at 8:54pm
 
No other WatchDirectory Services, logged in locally as domain admin.
Back to top
 
 
IP Logged
 
mmarx82
YaBB Newbies
*
Offline



Posts: 11
Re: watchDirectory hangs website and network acces
Reply #5 - Sep 2nd, 2010 at 9:02pm
 
I should also note that this server was completely reloaded about 2 weeks ago and is fully updated.
Back to top
 
 
IP Logged
 
Dirk
YaBB Administrator
*****
Offline



Posts: 658
South Germany
Re: watchDirectory hangs website and network acces
Reply #6 - Sep 2nd, 2010 at 9:06pm
 
I am asking under which account your service was running, not which user was logged it at server console. Please create a new domain user in AD, switch the task to run as service but do not start the task. Open you service configuration menu ( services.msc ) and search your task, its named "watchdirectory:[name of your task]". Double click it, change to the second tab and change the user account under which the task will be running from "local system" into your newly created domain user. Make sure that this user has enough rights for the things you want to do.
Start the task ( or start the service) and let me know what the performance is doing now.
Back to top
 

Viele Gre / Best regards
Dirk - GdP Software

dirk [at] gdpsoftware [dot] com
http://blog-de.gdpsoftware.com
Webseite: http://www.gdpsoftware.com
WWW WWW  
IP Logged
 
mmarx82
YaBB Newbies
*
Offline



Posts: 11
Re: watchDirectory hangs website and network acces
Reply #7 - Sep 2nd, 2010 at 9:16pm
 
The service was running under the system account, it is now running under a domain admin account doing the steps you described.

The performance was the same as the first time and I even got "not responding" on the watchDirectory Control Center app this time.
Back to top
 
 
IP Logged
 
Dirk
YaBB Administrator
*****
Offline



Posts: 658
South Germany
Re: watchDirectory hangs website and network acces
Reply #8 - Sep 2nd, 2010 at 9:28pm
 
please do not use admin account, use a dedicated new account. I cannot promise you that this will solve your problem, we need to experiment a little bit.
Is the server running as terminal server too? (i am not talking about about admin rdp access)
Note that we have hundert of installations on W2k3 servers (including my own) without a problem. We need to find out what is different at your server. Can you tell me a little bit about your hardware? Type of server, type of hhd's, controller, raid?
Back to top
 

Viele Gre / Best regards
Dirk - GdP Software

dirk [at] gdpsoftware [dot] com
http://blog-de.gdpsoftware.com
Webseite: http://www.gdpsoftware.com
WWW WWW  
IP Logged
 
mmarx82
YaBB Newbies
*
Offline



Posts: 11
Re: watchDirectory hangs website and network acces
Reply #9 - Sep 2nd, 2010 at 9:50pm
 
ok, RDP and Remote Assistance is turned off.
The server is a Gateway E-9520T
LSI MegaRAID 8308ELP RAID controller (SCSI)
Running Raid 5
Has 2 partitions (Drives C, D)
Drive C - 50 GB (39.1 GB free)
Drive D - 1.31 TB (1.07 TB free)
Has Intel Pro/1000 NIC - Driver Date: 3/26/2010 - Version: 9.13.41.0
There is a NIC driver feature that affects our 2008 servers Network Access that has to be disabled (Receive-Side Scaling), went ahead already and did that on this server even though it's not a 2008 server. Doesn't make a difference though on or off with this issue.

Using Sophos for AV, program and system firewall is off right now.

I installed watchDirectory to Drive C, but the configurations are saved to Drive D. (For backup purposes, We backup D not C)

The wmv file I'm testing with right now is 780MB.

Just tried a dedicated new account, still receiving the same issue.
Back to top
 
 
IP Logged
 
mmarx82
YaBB Newbies
*
Offline



Posts: 11
Re: watchDirectory hangs website and network acces
Reply #10 - Sep 2nd, 2010 at 9:54pm
 
No not running as terminal server either.
Back to top
 
 
IP Logged
 
Dirk
YaBB Administrator
*****
Offline



Posts: 658
South Germany
Re: watchDirectory hangs website and network acces
Reply #11 - Sep 2nd, 2010 at 10:10pm
 
thanks for your info. Actually I am out of ideas. Are you copying your wmv locally or to a network destination? If network, please try copy locally just for testing.
Please also stop virus scanner for wmv files just to see if something changes.
I am writing from Germany, it is late night here, we will continue next day.
Back to top
 

Viele Gre / Best regards
Dirk - GdP Software

dirk [at] gdpsoftware [dot] com
http://blog-de.gdpsoftware.com
Webseite: http://www.gdpsoftware.com
WWW WWW  
IP Logged
 
mmarx82
YaBB Newbies
*
Offline



Posts: 11
Re: watchDirectory hangs website and network acces
Reply #12 - Sep 2nd, 2010 at 10:21pm
 
the copy is a local copy

Folder Structure is

Videos
-Upload
-Videos

Anything dropped in the Upload folder will be copied/renamed (appending the date to the filename) and moved to the other videos folder. When this gets to working properly I'm going to add the convert video subtask back in there so the non-wmv files will be converted to mp4 with h264 encoding. Which btw, is a feature request of mine (converting video). I'm currently using HandbrakeCLI to do that with your "Start A Program" pluggin.

My workday is about to end anyways, then I have a 4 day weekend so back to testing on Tuesday. Thanks for trying.
Back to top
 
 
IP Logged
 
Gert
YaBB Administrator
*****
Offline



Posts: 2225
The Netherlands
Re: watchDirectory hangs website and network acces
Reply #13 - Sep 3rd, 2010 at 8:12am
 
Hi,

I strongly suspect this is caused by your virus scanner. Most scanners allow you to exclude complete directories from scanning, so that may help.

I see you use "Copy" as the first subtask and after that "Delete".
A "Copy" will normally copy bit-by-bit to its destination (a process that may take a lot of time), however, if you select the option to "[X] Delete after copy" and "[X] Implement as move" the copy process does not copy the bits of the file, but only changes the directory entry of the file (only when moving on the same disk). If it can do a Move instead of Copy, it should take less than a millisecond.

Hope this helps a bit.
Back to top
 

Gert Rijs - gert (at) gdpsoftware (dot) com
Blog: http://blog-en.gdpsoftware.com/
End Alzheimer's: http://www.alz.org&&...
WWW WWW GdPSoftware  
IP Logged
 
Dirk
YaBB Administrator
*****
Offline



Posts: 658
South Germany
Re: watchDirectory hangs website and network acces
Reply #14 - Sep 3rd, 2010 at 8:39am
 
I agree with Gert about virus scanner. Please stop it for a test. Note that the scanner will maybe scan twice: on reading and on writing the file. Please also try to copy this large file manually (without Watchdirectory) and with activated scanner. You should see a similar slow performance.

You are talking about MegaRAID 8308ELP: SAS or SATA? Does your controller have "write cache" enabled?
Back to top
 

Viele Gre / Best regards
Dirk - GdP Software

dirk [at] gdpsoftware [dot] com
http://blog-de.gdpsoftware.com
Webseite: http://www.gdpsoftware.com
WWW WWW  
IP Logged
 
mmarx82
YaBB Newbies
*
Offline



Posts: 11
Re: watchDirectory hangs website and network acces
Reply #15 - Sep 7th, 2010 at 4:02pm
 
I can copy the file manually and the problem not occur, disabling AV didn't help. It's SAS and the write cache was disabled so I enabled it to no avail.

What has seemed to help is I unistalled watchDirectory and re-installed it, this time using the default configuration location. The issues are still there but much more tolerable. It's more of a hickup now than a hang.

I also noticed for the website that the html pages don't experience the problem it's just the asp and aspx pages. Our site is mostly asp which is why I didn't notice that before.
Back to top
 
 
IP Logged
 
mmarx82
YaBB Newbies
*
Offline



Posts: 11
Re: watchDirectory hangs website and network acces
Reply #16 - Sep 7th, 2010 at 4:13pm
 
Is it possible to limit the number of CPU's or cores watchDirectory runs on? I would like to try lowering it by 1.
Back to top
 
 
IP Logged
 
Gert
YaBB Administrator
*****
Offline



Posts: 2225
The Netherlands
Re: watchDirectory hangs website and network acces
Reply #17 - Sep 7th, 2010 at 6:00pm
 
You can use the windows task manager to set the "Processor Affinity".

Start the task and right-click wdrun.exe in task manager.
Back to top
 

Gert Rijs - gert (at) gdpsoftware (dot) com
Blog: http://blog-en.gdpsoftware.com/
End Alzheimer's: http://www.alz.org&&...
WWW WWW GdPSoftware  
IP Logged
 
mmarx82
YaBB Newbies
*
Offline



Posts: 11
Re: watchDirectory hangs website and network acces
Reply #18 - Sep 7th, 2010 at 6:23pm
 
Duh, I can't say for sure but I think that's what the doctor ordered. Right now it's fine.
Back to top
 
 
IP Logged
 
Gert
YaBB Administrator
*****
Offline



Posts: 2225
The Netherlands
Re: watchDirectory hangs website and network acces
Reply #19 - Sep 7th, 2010 at 6:28pm
 
mmm, very strange. It sounds like your disk (raid) doesn't appreciate multiple accesses by different processes. You should run into the same issue if you run other software that does heavy disk access I guess.

If I am correct, the processor affinity set on Task Manager is NOT saved by Windows. This thread has a few solutions (I didn't try them): http://www.tomshardware.co.uk/forum/179198-10-processor-affinity-automatically
Back to top
 

Gert Rijs - gert (at) gdpsoftware (dot) com
Blog: http://blog-en.gdpsoftware.com/
End Alzheimer's: http://www.alz.org&&...
WWW WWW GdPSoftware  
IP Logged
 
Page Index Toggle Pages: 1