:) That one with the large negative iteration numbers ... looks like a v1.07 task, where they had problems where the iteration number was being stored in a variable that was too small (hence the large negative numbers).

You need to cancel your v1.07 tasks, and get some v1.08 tasks.

Both were 1.07; both now aborted.

I have a bunch of V1.03 with 1d17h and 81%.

BETA_23071039_8000_0173_0 looped on me (on the aforementioned Win 7 laptop), so I aborted it. This machine is also running two other Beta units. One has also looped, but I'm going to see if the second loop finishes correctly. The other unit has not looped, and I'm hoping that somehow it finishes gracefully.

I have 2 machines, one with Fedora ans one with Windows 7
Fedora machine has no problems with rebooting,Windows machine restarts for 0 after a reboot (with Beta work)

---

LZ Loon
The Most Handsome Man on the Interweb (TM)

I used SysInternals Process Monitor to check what the beta application does in the slot directory during a checkpoint. The sequence is as follows:

1. File "denis_state" is not found.
   
2. Creates the file "temp" with "Desired Access: Generic Write, Read Attributes" and "ShareMode: Read, Write". Successful with "OpenResult: Overwritten". Does this twice.
   
3. Writes the "CP" line to "stderr.txt".
   
4. Writes 1280 bytes to "temp" then successfully closes it once.
   
5. Attempts to create "temp" with "Desired Access: Read Attributes, Delete, Synchronize", "Disposition: Open", "ShareMode: Read, Write, Delete". This causes a sharing violation (repeated on 5 further attempts) because the file was opened twice with "ShareMode: Read, Write", only closed once and the new open attempt is trying to add "Delete" to "ShareMode".
   

The obvious solution to this is to remove the spurious open of "temp".

---

"The ultimate test of a moral society is the kind of world that it leaves to its children." - Dietrich Bonhoeffer

I used SysInternals Process Monitor to check what the beta application does in the slot directory during a checkpoint. The sequence is as follows:

1. File "denis_state" is not found.
   
2. Creates the file "temp" with "Desired Access: Generic Write, Read Attributes" and "ShareMode: Read, Write". Successful with "OpenResult: Overwritten". Does this twice.
   
3. Writes the "CP" line to "stderr.txt".
   
4. Writes 1280 bytes to "temp" then successfully closes it once.
   
5. Attempts to create "temp" with "Desired Access: Read Attributes, Delete, Synchronize", "Disposition: Open", "ShareMode: Read, Write, Delete". This causes a sharing violation (repeated on 5 further attempts) because the file was opened twice with "ShareMode: Read, Write", only closed once and the new open attempt is trying to add "Delete" to "ShareMode".
   

The obvious solution to this is to remove the spurious open of "temp".

Thanks for the info, we are trying different strategies to checkpoint to avoid the problems we have. Maybe this solution is no the correct. We are modifing a bit more.

Best regards, Joel.

Last night, I decided to check whether beta 1.03 could recover from checkpoints, so I shut down BOINC and then did a Windows restart on one of my computers. The task started up again, could not find the checkpoint file, and restarted from the beginning. I do not expect it to finish by the deadline.

For what it's worth, I run Linux and if I select a task in boinc manager and look at the properties, one of the things it lists is the cpu time at the last checkpoint. If that's blank, I know that it has not checkpointed.

I've also set no new work for DENIS on all my systems until I hear that these problems have been fixed.

---

Seems you have to PM a mod to get a post deleted.

---

LZ Loon
The Most Handsome Man on the Interweb (TM)

Some things I've seen on other BOINC projects:

1. Two separate checkpoint files per task, so that if the task is interrupted in the middle of writing a checkpoint, it can resume from the other one instead. This needs a method for determining which of the files has the more recent checkpoint, in case both contain valid checkpoints.

2. Something identifying which checkpoint is written placed at the beginning, which must match something at the end to indicate that the checkpoint is complete. Whatever is used should not match any other checkpoints from the same task.

3. Requests from the project for users to deliberately interrupt BOINC during beta tests, to check whether the checkpoints are working properly.

Windows 10 has at least one update most Tuesday nights, with delaying them difficult, so you may soon see many more checkpoint resume failures from Windows 10 users.

I've got also an never ending result on a machine that is shut down every day.
GD_jcarro_20160714202331000000_ThirdSimulations_SteadyState8000Schmidt98_conf_1983.xml
Result: 38377099
Carro-Rodriguez-Laguna-Pueyo Epicardial Model (Carro et al. 2011) for human ventricular cells v1.08
On starting the computer at morning the result started calculation again from round 20%.
System Windows 7

Looks like there was a checkpoint lost (extract from <stderr_txt>)
Line 112 : LoadFromCP: it = 5956108503.000000
Line 263 : LoadFromCP: it = 6800888101.000000
Line 805 : LoadFromCP: it = 0.000000
Line 1217: LoadFromCP: it = 2471613195.000000

---

Matthias

Last night, I decided to check whether beta 1.03 could recover from checkpoints, so I shut down BOINC and then did a Windows restart on one of my computers. The task started up again, could not find the checkpoint file, and restarted from the beginning. I do not expect it to finish by the deadline.

It looks like beta 1.03 does NOT fix the endless restarts problem.

A few lines from stderr.txt showing this:

Doing CP It:3396376837.000000
Doing CP It:3402929064.00000000:35:28 (2660): called boinc_finish(0)
MName:CRLP2011_EPI
MID:6

At least this time it created a file named temp, AFTER it restarted.

Task 38385075

On my other computer with a beta 1.03 task in progress, there is a temp file, but it's empty - 0 bytes.

47.338% progress, so it should have created many checkpoints by now.

Task 38385601

Microsoft does not appear to have forced any Tuesday night Windows 10 updates requiring a Windows restart this week, so it may still complete without trying to resume from a checkpoint.

A suggested feature for the next beta: For the first few checkpoints, report whether the checkpoint file even exists before trying to write the checkpoint, and if so, how many bytes it contains. You could also have the first few checkpoints report whether they were successful in writing to the checkpoint file.

Hi!

We are thinking different ways to ensure checkpoints. As you have said, a double checkpoint file as a backup and check the size of the file is one of the ways that are on the table.

Writing into disk is one of the most "expensive" things in the execution of the simulation, this is why at the beginning we have tried to simplify this part. But with those ultra-long WUs it could be necessary.

Best regards, Joel.

The beta 1.03 task under Windows 10 also started the infinite reruns.

Doing CP It:3980371292.000000
Doing CP It:3988241479.00000010:58:46 (8512): called boinc_finish(0)
MName:CRLP2011_EPI
MID:6

Even though the temp file was previously present and not empty, it could not find it after this restart.

Your next beta version appears to need to be able to tell whether the checkpoint file is not present, present but empty, present and not empty but with the contents not usable, and some other error in reading it.

The beta 1.03 task under Windows Vista started another rerun. When should I shut this down by aborting it? The temp file is still present but only 0 bytes.

Do you have a way to make your next batch of Windows tasks use a much smaller number of iterations, so they can be aimed at testing whether the Windows version of the application shuts down properly? Also, they should send any remaining temp file back to you, so you can check if it's even in the right format.

I believe I've read that some compilers allow creating files in such a way that they will automatically be deleted when the application that created them ends. You may need to inspect the Windows application to see if it does this.

Also, does the Windows application close the checkpoint file after it finishes writing a checkpoint? If not, I'd expect the checkpoint file to be more likely to be lost, since it might be smaller than the hard drive block size, and therefore still in cache but not yet written to the hard drive.

Writing into disk is one of the most "expensive" things in the execution of the simulation, this is why at the beginning we have tried to simplify this part. But with those ultra-long WUs it could be necessary.

I will buy an SSD!

Writing into disk is one of the most "expensive" things in the execution of the simulation, this is why at the beginning we have tried to simplify this part. But with those ultra-long WUs it could be necessary.

I will buy an SSD!

I already have an SSD in each of my desktops. However, the one for my Windows Vista computer does not have a Windows Vista driver available. You may want to check for a similar problem before you buy any.

I will buy an SSD!

I already have an SSD in each of my desktops. However, the one for my Windows Vista computer does not have a Windows Vista driver available. You may want to check for a similar problem before you buy any.

My pc are Win10
And i will use Acronis to clone disk to ssd....

In comparison with the L1/L2/L3-Caches of modern CPUs, writing to an HDD or SSD will still be "expensive".

Another restart for beta 1.03 under Windows 10.

Doing CP It:3986402711.00000016:47:06 (6160): called boinc_finish(0)
MName:CRLP2011_EPI
MID:6

I'm about to abort this task.