𝕏

Hard drive space crisis solved // Crisis de espacio en el disco duro resuelta

Message boards : News : Hard drive space crisis solved // Crisis de espacio en el disco duro resuelta
Message board moderation

To post messages, you must log in.

AuthorMessage
Profile Jesús Carro
Project administrator
Project developer
Project scientist
Avatar

Send message
Joined: 18 Mar 15
Posts: 271
Credit: 494,851
RAC: 52
Message 2289 - Posted: 13 Mar 2024, 9:20:04 UTC

Dear volunteers,
This weekend we have broken a new record for computing Teraflops (since we are in this new stage of the project). This is thanks to the pace of sending results that you have been doing in recent days and being more stable in the generation of work packages.

Although we were monitoring it, the coincidence of this peak with the database backup on the weekend has caused the hard drive to become saturated sooner than expected. We have been working since Monday on moving data out of the server and even other processes, but as we gained space it kept filling up with results that were pending upload.

This morning, for the first time in two days, more tasks have been validated than have been received and little by little normality will return. At first, it doesn't seem like anything has been lost and we are already submitting work again.

We have expanded the space to receive results and are working on automating some of the tasks to ensure we can continue to grow.

Thank you all very much, if we have this type of problems it is because of the great support you give us, we will continue working to grow by your side.

Sorry for not answering in the forums, and thank you very much for be aware of the evolution.

All the best,
Jesus.

=====================================================

Estimados voluntarios:
Este fin de semana hemos batido un nuevo record de Teraflops de computación (desde que estamos en esta nueva etapa del proyecto). Esto es gracias al ritmo de envío de resultados que estáis haciendo en los últimos días y de estar más estables en la generación de paquetes de trabajo.

Aunque ibamos monitorizándolo, la coincidencia de este pico con la copia de seguridad de la base de datos en el fin de semana ha hecho que se saturara el disco duro antes de lo esperado. Hemos estado trabajando desde el lunes en mover datos fuera del servidor e incluso otros procesos, pero según ganábamos espacio se seguía llenando con los resultados que estaban pendientes de subir.

Esta mañana, por primera vez en dos días se han validado más tareas de las que se han recibido y poco a poco se irá retomando la normalidad. En principio, no parece que se haya perdido nada y ya estamos enviando trabajo de nuevo.

Hemos ampliado el espacio para recibir resultados y estamos trabajando en automatizar algunas de las tareas para garantizar que podemos seguir creciendo.

Muchas gracias a todos, si tenemos este tipo de problemas es por el gran apoyo que nos dais, seguiremos trabajando para crecer a vuestro lado.

Disculpad que no haya contestado en los foros, muchas gracias por estar tan pendientes de la evolución.

Un saludo,
Jesús.
Jesús Carro
Universidad San Jorge
@InSilicoHeart
ID: 2289 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
TPCBF

Send message
Joined: 11 Oct 23
Posts: 24
Credit: 2,921,958
RAC: 7,120
Message 2290 - Posted: 13 Mar 2024, 14:52:23 UTC - in response to Message 2289.  
Last modified: 13 Mar 2024, 14:52:34 UTC

Thanks for the update!
But due to bad experience at the World Community Grid (which, with their handling of their projects, has probably driven a lot of folks over here in recent months, including me), PLEASE, even if things go bad, which is something that can always happen, try to post some update quickly about what is going on. And if the forum is taken down as well, put it up on your FB site or X.
Honest bad news is by the vast majority better received than no news at all.

thanks again,

Ralf
ID: 2290 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
as1981

Send message
Joined: 24 Jul 23
Posts: 6
Credit: 79,641
RAC: 298
Message 2291 - Posted: 13 Mar 2024, 18:21:15 UTC

Thank you for the update.
ID: 2291 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Jean-David Beyer

Send message
Joined: 6 Mar 23
Posts: 31
Credit: 1,552,212
RAC: 3,685
Message 2292 - Posted: 13 Mar 2024, 18:22:59 UTC - in response to Message 2290.  

But due to bad experience at the World Community Grid (which, with their handling of their projects, has probably driven a lot of folks over here in recent months, including me), PLEASE, even if things go bad, which is something that can always happen, try to post some update quickly about what is going on. And if the forum is taken down as well, put it up on your FB site or X.
Honest bad news is by the vast majority better received than no news at all.


I am one of those who came here from WCG, due to their two-years of downtime. Then pretend to be up, but the only tasks I get are MCM1, and nothing from the other four. And even MCM is down a lot of the time.
I also signed up for Universe and MilkyWay. But Universe seems to be going to only the Python stuff and he other only GPU that I do not allow. So I resigned from those two (yesterday).

(But for me, even FB is bad as I cannot login. They do not like my password, but when I go through the motions to change the password, they send me a 6-digit access code to prove I am at my e-mail address. but they do not accept the access code they send me. I seem to be banned as there is no way to speak to anyone: no telephone number, no working e-mail address, nothing. None of this is the problem of Denis, though.)
ID: 2292 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Greg_BE

Send message
Joined: 2 Aug 22
Posts: 38
Credit: 837,265
RAC: 3,005
Message 2293 - Posted: 13 Mar 2024, 19:34:00 UTC - in response to Message 2292.  
Last modified: 13 Mar 2024, 19:42:01 UTC

But due to bad experience at the World Community Grid (which, with their handling of their projects, has probably driven a lot of folks over here in recent months, including me), PLEASE, even if things go bad, which is something that can always happen, try to post some update quickly about what is going on. And if the forum is taken down as well, put it up on your FB site or X.
Honest bad news is by the vast majority better received than no news at all.


I am one of those who came here from WCG, due to their two-years of downtime. Then pretend to be up, but the only tasks I get are MCM1, and nothing from the other four. And even MCM is down a lot of the time.
I also signed up for Universe and MilkyWay. But Universe seems to be going to only the Python stuff and he other only GPU that I do not allow. So I resigned from those two (yesterday).

(But for me, even FB is bad as I cannot login. They do not like my password, but when I go through the motions to change the password, they send me a 6-digit access code to prove I am at my e-mail address. but they do not accept the access code they send me. I seem to be banned as there is no way to speak to anyone: no telephone number, no working e-mail address, nothing. None of this is the problem of Denis, though.)




If you want a non BOINC GPU/CPU program to run for science, then download Folding@Home. They often run stuff that was designed by Baker Lab and take stuff from many other sources that are looking for a home in the DC world. Don't think their tech requirements are to strict.

Look at your DM regarding FB.
ID: 2293 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Greg_BE

Send message
Joined: 2 Aug 22
Posts: 38
Credit: 837,265
RAC: 3,005
Message 2294 - Posted: 13 Mar 2024, 19:41:40 UTC - in response to Message 2289.  

Jesus,

Kind of a good thing for your project to grow so rapidly, just not so good on our end.

Can I suggest two possible ways to get the word out quickly to us when you have an unexpected event happen? Someone mentioned X/Twitter. That is one good place to start and the other is to send out a broadcast notice like we see when you do these news messages. Just drop us a couple of lines saying what is going on so we know you are actively trying to fix the problem.

Continued success with your project!
ID: 2294 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
rampowers

Send message
Joined: 31 Jul 23
Posts: 1
Credit: 1,382,079
RAC: 6,435
Message 2296 - Posted: 14 Mar 2024, 3:03:01 UTC

Lack of hard drive space is a good problem to have. I am also from WCG and will second what was said above my message; please make sure you let people know quickly about issues and I'm sure you'll keep a lot of us :)
ID: 2296 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Yavanius
Avatar

Send message
Joined: 20 May 15
Posts: 50
Credit: 336,397
RAC: 755
Message 2297 - Posted: 14 Mar 2024, 4:00:32 UTC - in response to Message 2290.  

Thanks for the update!
But due to bad experience at the World Community Grid (which, with their handling of their projects, has probably driven a lot of folks over here in recent months, including me), PLEASE, even if things go bad, which is something that can always happen, try to post some update quickly about what is going on. And if the forum is taken down as well, put it up on your FB site or X.



DENIS doesn't have nearly the number of folks WCG SUPPOSEDLY has. It's a small project in terms of resources. Keep that in mind. The irony is that Jesus is quick to respond to things as soon as he aware of them. I have let him know a few times when the project was down for an extended period and he's been pretty quick to take care of things once he received the message. He's also been fairly responsive to inquiries in the forums.

I still run a little WCG, although since they keep running out of work or it's allocated to other OS, my systems don't hit them often. It use to be my main project as cancer is a close to home interest. But it's all turned out too little too late. The icing on at all is turning down volunteers and then be unable to report in / respond to the volunteers.

Yav
ID: 2297 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Yavanius
Avatar

Send message
Joined: 20 May 15
Posts: 50
Credit: 336,397
RAC: 755
Message 2298 - Posted: 14 Mar 2024, 4:14:24 UTC - in response to Message 2293.  



If you want a non BOINC GPU/CPU program to run for science, then download Folding@Home. They often run stuff that was designed by Baker Lab and take stuff from many other sources that are looking for a home in the DC world. Don't think their tech requirements are to strict.





Folding heavily favors GPU in preference and in terms of credit and while not exclusively GPU, that has been their focus for a whiles. If you're a GPU cruncher then Folding is definitely a good project to run.

Folding@home let's you choose a preference on what to work on, but ultimately the workload determines what you receive. Even within the categories, there are different studies going on so what your computer is working on one day may not be the same next week even if it's the same category (and the same study may not be both on CPU & GPU). For some folks, not being to specifically crunch for your preferred category may be a deal breaker.

Neocortix spent a lot of effort helping Folding@home (got to avoid calling it Stanford as it's not Stanford anymore) develop a new Android client then just decided to abandon it. (Neocortix also worked with Rosetta for the Android platform.) The project seems to be ignoring posts in regards to Android support. Their love is GPU.
ID: 2298 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
TPCBF

Send message
Joined: 11 Oct 23
Posts: 24
Credit: 2,921,958
RAC: 7,120
Message 2299 - Posted: 15 Mar 2024, 2:55:13 UTC - in response to Message 2297.  

DENIS doesn't have nearly the number of folks WCG SUPPOSEDLY has. It's a small project in terms of resources. Keep that in mind. The irony is that Jesus is quick to respond to things as soon as he aware of them. I have let him know a few times when the project was down for an extended period and he's been pretty quick to take care of things once he received the message. He's also been fairly responsive to inquiries in the forums.
Yeah, it is a smaller project, but it seems it gained quite a number of crunchers in the last few month. And usually, since I joined here in October, he has indeed been reasonably quick with answering. Just the last weekend it took a while to get the notice that they were out of drive space. The bad thing is that in such a case, everything goes down, including the forum...
I still run a little WCG, although since they keep running out of work or it's allocated to other OS, my systems don't hit them often. It use to be my main project as cancer is a close to home interest. But it's all turned out too little too late. The icing on at all is turning down volunteers and then be unable to report in / respond to the volunteers.
Well, I am still being moderated/censored on the WCG forum for speaking my mind and calling them out for telling even outright lies.
However,...
That they "keep running out of work" is highly exaggerated. Most of the time, WUs are coming just fine, at least if you are running with a default cache and not trying to hoard days worth of WUs.
And with the turning down of volunteers, that is something I can to some degree understand. Being situated in a healthcare network, they are likely to have some level of restriction like HIPAA here in the USA as to who can access what. But then there should also be a way to grant monitoring access to a couple of vetted IT professionals among the volunteers And a lot of the issues that arise are self-inflicted.
Yes, for me cancer research also hits close to home, though I am one of the VERY lucky ones that my stomach cancer was detected very early and was immediately operable and hasn't shown up now in 13 years. That was for me the reason why, after Rosetta had some serious issues back in those days, I ended up joining WCG. And when things are not going well at WCG, I chose DENIS as a backup project an in 4 1/2 months I made it (per BOINC stats) close to the Top 100 list of contributors....

But the main issue with WCG is their utter lack of communication. That's what I mentioned in my first reply in this thread to Jesus. Even reporting honestly some bad/not so good news is better than no info at all or just some platitudes...

Ralf
ID: 2299 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Schizm

Send message
Joined: 3 Mar 24
Posts: 1
Credit: 364,236
RAC: 1,789
Message 2300 - Posted: 15 Mar 2024, 13:12:25 UTC

Dear fellow volunteers,

I am not a spokesperson, but as a member of Dutch Power Cows i would like to introduce you to our annual stampede; an event where we usually pick one project to band together and focus on for one full month. This year Denis@home got picked on a fairly short notice. It was communicated to us that there might not be enough work to process at this stage of the project. In our naivity we were thinking;"let's hope we have enough to last the month." while forgetting the impact we had on other projects in the past.

Day 1 took us 2 hours to completely empty the queue....
....so i think we might have had a bit of an impact here too.

The day after we started our stampede we got a message stating testing different iterations required manual intervention. So i want to stress this to all of you: the DENIS team was communicating well, already working WAY harder than they normally would have to just to keep the WU's coming, and on top of that they had to deal with this saturated drive issue. This week must have been quite the adventure.

Therefore i would like to apologize to you all for any inconvenience our cows may have caused; they can get a bit rowdy sometimes when they're hungry.

Kind regards and happy crunching,
ID: 2300 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Greg_BE

Send message
Joined: 2 Aug 22
Posts: 38
Credit: 837,265
RAC: 3,005
Message 2301 - Posted: 15 Mar 2024, 20:36:04 UTC - in response to Message 2299.  

Raf- wife had hormonal breast cancer. Benign. I had a rare testicular cancer that stumped even the university types until dissection. It was a super rare tumor deep inside. It is less than 1% of the US population that gets this. It was also benign. Her mother had a rare form of abdominal cancer but it resisted all forms of standard treatment. A trial showed some promise, but not enough to stay in it. So that is why I take interest in cancer research. I was with Rosetta for a long time, but due to changes in their program they don't produce enough work.

Jose here with the heart is something new and exciting and he cranks out so much work, there is always something. Ok, when the cows came it killed the queue, but thats a good thing.
ID: 2301 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Yavanius
Avatar

Send message
Joined: 20 May 15
Posts: 50
Credit: 336,397
RAC: 755
Message 2303 - Posted: 15 Mar 2024, 23:53:14 UTC - in response to Message 2299.  

We should digress discussing this in the news topic lest we turn DENIS forums into WCG...

It is nice to see DENIS moving through being full-time active again. We're not there yet, but last few weeks there's been a fair amount of work often available.
ID: 2303 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Jesús Carro
Project administrator
Project developer
Project scientist
Avatar

Send message
Joined: 18 Mar 15
Posts: 271
Credit: 494,851
RAC: 52
Message 2309 - Posted: 19 Mar 2024, 11:33:15 UTC

Hi!
Thank you for your messages. Yes, you are right, I should post it before in other platforms. We can not send a message to the Boinc client if the server is down, because they are send when we crate a post in the News forum and we indicate it.
I should reactivate the different social profiles, at least for this situations.

Thank you,
Jesús
Jesús Carro
Universidad San Jorge
@InSilicoHeart
ID: 2309 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
TPCBF

Send message
Joined: 11 Oct 23
Posts: 24
Credit: 2,921,958
RAC: 7,120
Message 2312 - Posted: 20 Mar 2024, 0:09:42 UTC - in response to Message 2309.  

I should reactivate the different social profiles, at least for this situations.
Thanks, that would be very much appreciated if such a situation arises again (hopefully not, <frantically knocking on wood>)...

Ralf ;-)
ID: 2312 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote

Message boards : News : Hard drive space crisis solved // Crisis de espacio en el disco duro resuelta