max number of task in progress
Message boards :
Number crunching :
max number of task in progress
Message board moderation
Author | Message |
---|---|
Send message Joined: 8 Apr 15 Posts: 15 Credit: 15,562 RAC: 0 |
need to set a max number of tasks in progress per host. just found one host with well over 600 task on it, 4 cores each task takes 30 minutes to run 12 hour deadline and... looks like you will be getting a lot of them back, starting at about 08:00 utc on 9 April 2015.... going on that you can not set the flop bound(?) on these tasks yet. then max in progress would be a big help.. thanks.. |
Send message Joined: 8 Apr 15 Posts: 15 Credit: 15,562 RAC: 0 |
just aborted over 900 off 3 host. hope they got the server set up correctly to resend them to new homes to get processed... your welcome... |
Send message Joined: 16 Mar 15 Posts: 219 Credit: 14,859 RAC: 0 |
Hello, We are working to solve it. And yes, the aborted task are resend to complete them, thank you for your attention. |
Send message Joined: 8 Apr 15 Posts: 3 Credit: 380 RAC: 0 |
Hello, in config.xml add <max_wus_in_progress>x</max_wus_in_progress> x is the value for WUs * NCPUs |
Send message Joined: 16 Mar 15 Posts: 219 Credit: 14,859 RAC: 0 |
Hello, Thank you! we have change it one hour ago! we hope you will see that change. We are a small team and these first days are plenty of work! |
Send message Joined: 8 Apr 15 Posts: 15 Credit: 15,562 RAC: 0 |
Hello, thanks, what did you set x to? I need to know, there maybe a problem with the count if you are running BU with ASIC attached to the same host got 2 host each with 14 ASIC on them running BU and this project, one has over 900 DENIS tasks on it and the other has over 700 DENIS tasks. may have to go back to the boinc dev for a fix. if it is counting the ASIC as cpus (cores) anybody else seeing this? thanks.. |
Send message Joined: 8 Apr 15 Posts: 3 Credit: 380 RAC: 0 |
Hello, For the ASICs you set it in cc_config.xml with <count>1</count>, the rest of the max_wus_in_progress is set in BU only for miners independend from this project. |
Send message Joined: 8 Apr 15 Posts: 15 Credit: 15,562 RAC: 0 |
reb, I know what I am trying to figure out is did I just run into a case of the boinc client counting the asic not only as coproc but also as cores so I need to know what this projects set as x for the max number of denis task * ncpu in progress thanks |
Send message Joined: 16 Mar 15 Posts: 219 Credit: 14,859 RAC: 0 |
reb, I know Hello, Now we setup the max_wus_in_progress to 3. I hope this information can be helpful for you. If you have any other question please ask to us! best regards, Joel. |
Send message Joined: 8 Apr 15 Posts: 15 Credit: 15,562 RAC: 0 |
reb, I know thanks, it does I'm going to let it run for now the way it is, any not started by deadline will get auto aborted. at 3 per ncpu I should have max of 9 on one host and 6 on the other, if I end up with (3 cores + 14 miners) * 3 task max = 51 and (2 cores + 14 miners) * 3 task max = 48 tasks then yes there is a bug. stay tuned, same bat-time, same bat-channel I'll let you know...... |
Send message Joined: 8 Apr 15 Posts: 15 Credit: 15,562 RAC: 0 |
ok, look like it was one of those one off hic ups things. has the right number of task per core. (I'm having a priest come over and do an exorcism on the computers because they must be possessed) :) thanks..... There are more things in heaven and earth, Horatio, Than are dreamt of in your philosophy. - Hamlet (1.5.167-8), Hamlet to Horatio my age > Event 1 "Do not meddle in the affairs of Wizards, for they are subtle and quick to anger." |
Send message Joined: 8 Apr 15 Posts: 11 Credit: 101,960 RAC: 0 |
Now that the project seems stable (well, it is for me anyway), I wonder if the number of tasks in progress limit could be raised. Perhaps to something like 20 WU per core? Would enable us (or at least me!) to build up a bit of a cache - Just enough work for 12 or 24 hours would be great. |
Send message Joined: 9 Apr 15 Posts: 1 Credit: 100,138 RAC: 0 |
A slightly different problem is now occurring on my computers. I am below the max in progress limit, yet I am getting only a couple of tasks at a time and not enough to bring it up to the limit (either DENIS max or Boinc queue). Consequently my secondary projects jump in and fill my queues to the max. Thus I get 3 30-min tasks for DENIS followed by 4 6-hour tasks from a secondary project. It could be a scheduler bug in 7.4.42 as it is new enough that I am not sure how it has changed from previous version. But more likely I see this as an issue of the server not sending enough. It may be that the server is overestimating the WU processing time and thus sending out too few WUs. On a simple test I just did. I increased my work buffer by 1 hour per core on a 4-core machine. The server then sent me only 3 30-min tasks. Shortly there after I got 2 more tasks at 1-min intervals. Again indicating not enough WUs are sent out per request. |
Send message Joined: 16 Mar 15 Posts: 219 Credit: 14,859 RAC: 0 |
Hi! We have increase de number of WUs in progress. We are developing the new version of software which will run 3x and 10x larger simulations, we hope that with the new parameter and the future software version the traffic would be the correct. Thank you for your help! PD: The max number of WUs has been set to 10. |
Send message Joined: 10 Apr 15 Posts: 8 Credit: 2,436,769 RAC: 0 |
Nice ! Now i can go to sleep, knowing that iám not running empty on wu´s, during while absent at night! :) EDIT (Got only 2 pr core earlier.. so this is a great improvement ) regards morgan |
Send message Joined: 9 Apr 15 Posts: 12 Credit: 44,521 RAC: 0 |
Thank you very much :) Looking forward to the longer work units. No rush though. :) I'm enjoying the short ones. |
Send message Joined: 9 Apr 15 Posts: 11 Credit: 116,882,448 RAC: 2 |
With the optimised applications the tasks are being crunched very quickly and continuos uploads/download occur, which stresses the servers and in some cases produce errors. I don't know what would be preferable/easier for the project, to increase the 10 wus per thread download limit or to make wus larger. The later is my favorite. |
Send message Joined: 10 Apr 15 Posts: 20 Credit: 19,361,104 RAC: 1 |
I think we could make some change in setting for connection and bigger wu as Trotador saying. When i take a look at my host they got almost same amount connection as finished task/day. Got over 4200 times per day for one host with 320 task stored. As soon on is completed it´s uploads and then download a new task every time. Takes about 2 min for size 400 and 6 min for size 1000 to complete. As mention in the other tread amount of task in /results.php? should have a limit. Amount of task or cleared by time. A progressbar would be awesome to if possible. |