RNA World (béta)

[fzs600]

Donc voila j'ai réussi a en toucher une. 

25 Dec 2016, 18:58:45 UTC    14 Jan 2017, 18:58:45 UTC    In progress    ---    ---    ---    cmsearch VM (VirtualBox) 1.0.2 v1.18 (vbox64)

0,863% après 19H de calcul.

Merci. 

[Maurice Goulois]

Pour l'instant j'ai laissé tomber

A chaque fois que je chope une ut, la bécane, où elle tourne,  explose avant qu'elle termine.

[JeromeC]

Sans aucune traduction ni adaptation je vous livre brut de fonderie des échanges que j'avais eu en novembre quand j'avais eu cette UT (pas eu d'autres depuis) en PM avec un fort sympathique teuton qui m'avait expliqué sa technique de backup / contrôle / restore d'une UT RNA.

Je n'ai pas eu l'occasion d'expérimenter tout ça et je ne sais pas si j'aurai le courage de le faire le jour où j'en aurai une...

------------------------------------------

I am hardcore. So ... the following steps, are going to be hardcore. Totally not recommended, unless you are an advanced user that knows exactly what you're doing, and am familiar with every instructions. It's also a fun way to get yourself more familiar with what's going on, behind the scenes with how BOINC manages the VMs, and also inside the VMs.

What I typically do is the following --- it's a bit dangerous, but if you follow the steps exactly, it shouldn't make your task fail. It is risky, though. Use lots of caution, if you decide to attempt!

----- Gracefully suspend and exit BOINC
- Suspend BOINC
- Monitor Task Manager, for all CPU activity to stop
- Exit BOINC
- Monitor Task Manager, for all VBoxHeadless* and VBoxWrapper* processes to gracefully exit
- Monitor Task Manager, for boinc.exe to gracefully exit
- In Task Manager, find VBoxSVC.exe, and use "End Task" on it

----- Optionally backup your Data folder
- Find your BOINC data folder
- Save a copy of it somewhere else, just in case. I do this every time I'm testing an OS upgrade or a VirtualBox upgrade.

----- Clone the VM
- Launch "Oracle VM VirtualBox". Note: I NEVER manually launch this, while BOINC is running. I make sure BOINC is all closed, before manually running this.
- If BOINC is NOT installed as a service, you should already see the machine in the list
- If BOINC is installed as service, use "Machine -> Add" to add the machine using the slot's .vbox file.
- Right-click the machine, and use Clone. Make sure to use "Full Clone" and "Current machine state".
- I typically have 3-4 weekly backups, so I append a number. My clones usually end with a name like "Clone 52".
- If BOINC is installed as a service, right-click the machine you added, and choose "Remove", and then "Remove Only". It's VERY important that you DO NOT choose "Delete all files"!

----- Inspect the clone
- Select the clone
- On the clone's snapshots tab, Right-click the top node, and choose "Restore snapshot", making sure to uncheck "Create a snapshot of the current state" before clicking Restore.
- Now right-click the clone, and choose Start. If you get a submenu next to Start, choose "Normal Start".
- In the new VM window, if it's all black, hit CTRL to "wake" the VM's display.
- Inspect the display for correctness. Nothing should look abnormal or look like an error.
- Hit CTRL+C, which will leave the cmsearch task running, but give you a command prompt.
- Type "top" in all lower-case, and hit enter. This launches the linux task manager.
- "cmsearch" should be near the top of the list. Make a note of it's "TIME+" column. This is the number of hours it has logged.
- Close the clone's window, and choose "Power off the machine", making sure to uncheck "Restore current snapshot".
- Close "Oracle VM VirtualBox", and monitor Task Manager, for VBoxSVC.exe to gracefully exit.

----- Determine if the VM is making progress
- Usually you can look at the progress.txt file in the slot folder, but if it's already 98.765%, then you can't rely on that.
- If you're worried that you may have screwed something up, make sure to go OFFLINE (by disabling all network adapters), before launching BOINC. This is so that the projects can't be notified of any task failures!
- Launch BOINC
- Resume BOINC, and let the tasks run for a couple hours.
- Repeat all of these steps, from the beginning (starting with Suspend BOINC), in order to get a 2nd clone to compare against.
- If all goes well, your second clone should show, in "top", that it has logged more time, than the first clone.

----- BOINC Data Recovery (if needed)
- If you had a problem while you were offline, and have a backup data folder, you can use it to recover.
- While BOINC is closed, move your current data folder to somewhere else
- Copy your backup data folder, to be the same folder location that you usually use for your data folder
- Test BOINC again, while offline.
- Keep testing until things look great.
- Then, while BOINC is closed, you can re-enable all your network adapters, then restart BOINC.


-----------------

JeromeC wrote:
- cloning the VM : excellent, I didn't know this could be done that way

Yes. Again, just be absolutely sure that BOINC is closed, the BOINC processes gracefully exit, then you kill VBoxSvc.exe, before you launch VirtualBox VM Manager to manually play with or clone VMs.

JeromeC wrote:
- looking at the console status and the slot text file : interesting, but this can be done on the actual running task, you are not obliged to see this in the clone, right ?

In BOINC, I only use "Show Console" to get the VM's forecast estimate or to see that it still looks okay without errors. I only ever hit "Ctrl" to wake the VM, and I never type anything else. And that's what I recommend - Never typing anything into that "Show Console" window!

To monitor status, you can look at the BOINC Progress %, which in most cases matches the progress.txt file, but... Christian coded things a bit poorly, such that, it can get to 100%, then fall back to 98.765%, and not get any new values into progress.txt. So, when that happens, usually the task is still healthy, and still updating the timestamp of progress.txt, but you have to "investigate using top within a clone" if you want to see if the VM is still logging useful time with the cmsearch application.

JeromeC wrote:
"- If you're worried that you may have screwed something up, make sure to go OFFLINE (by disabling all network adapters), before launching BOINC. This is so that the projects can't be notified of any task failures!"
What do you mean, if the task hasn't crashed yet but you suspect it could ? because there are some stuff looking like errors in the console ? anyway if the task has crashed you agree it's already too late ? (most probably already notified to the project)

I do some pretty crazy things while running RNA World tasks, like ... testing newer versions of Windows 10 Insider Preview builds (I'm on the "Fast Ring", but VirtualBox has problems quite often there), and testing newer versions of Oracle VirtualBox itself.

So ... For instance, if I get a new Windows 10 Build, it's standard practice for me to suspend BOINC, shutdown BOINC, install new Windows 10 build, BOINC will start up suspended, close BOINC, disconnect the network, make a backup copy of the BOINC folder, start BOINC, verify if tasks will run successfully .... and if tasks fail, I can close BOINC, restore my data folder, keep the RNA World tasks suspended, reconnect network, start BOINC.

In fact, when VirtualBox has problems with new Windows 10 Insider Builds (like 14971, released 2 days ago) ... I actually have a separate "Windows 10 Release" partition I can use, and the data folder is on a partition that they share. So, tonight for instance, I'll: uninstall BOINC from Insider partition, reboot into Release partition, install BOINC in Release partition, and start crunching RNA World tasks again from that partition.

JeromeC wrote:
- restore : your process would restore all projects backup in the data directory, since i run several projects all the time, so I couldn't do that... I'd need to be able to restore only RNA task... (meaning not only RNA data directory but being able to edit and modify various xms boinc status files... I did that in the past but it was very complicated and risky)

The backup process is applicable for the ENTIRE data folder. You are basically: going offline, then taking a snapshot of that entire folder, then manipulating it any way you think you need to, testing it, restoring if if you screw it up, retesting it, then if you're confident it's correct, going online.

I am a BOINC Alpha tester, and I too am connected to many projects -- in fact, I'm attached to ~60, and routinely do work for ~15. What you are not understanding is: When you go offline, then take a backup of the data folder, then ... anything you do to that folder (any work, any file modifications, any task failures, anything) is NOT communicated to the projects. So, if I had gone offline, and made a copy of my data folder, I could for instance, start BOINC, abort all the tasks for my ~15 projects. And then realize "oh that's not what I wanted", then close BOINC, restore the data folder, restart BOINC, verify the tasks are working again, close BOINC, enable the network adapters, then start BOINC. .... Basically, as soon as BOINC is running with the network enabled, you "invalidate" your offline backups, so you better be very sure things are exactly how you want them, before you allow BOINC to talk to the projects

[jm91160]

Je viens de choper une unité. Je suis à 2,6% après 48h de calcul... Aucune chance de finir avant le 21 mars.... 
Un conseil? je garde? je jette l'éponge?

[fzs600]

Je viens de choper une unité. Je suis à 2,6% après 48h de calcul... Aucune chance de finir avant le 21 mars.... 
Un conseil? je garde? je jette l'éponge?

Il faut ABSOLUMENT la garder jusqu'au bout, même si elle part en erreur de calcul tu aura des crédits.

La mienne est a 22% après 42 jours de calcul. 

[modesti]

C'est étonnant que ta deadline soit aussi courte. J'en ai chopé une fin janvier avec une deadline fin juillet.
Avancement: 10% après 2j10h et qq (temps restant théorique 11j14h)

[mamouth]

La deadline se met à jour automatiquement si j'ai bonne mémoire

à vérifier

[fzs600]

C'est étonnant que ta deadline soit aussi courte. J'en ai chopé une fin janvier avec une deadline fin juillet.
Avancement: 10% après 2j10h et qq (temps restant théorique 11j14h)

Une deadline courte c'est ce permet à UT d'être en priorité haute comme ça elle se calcule en permanence sans s'arrêter

[modesti]

La deadline se met à jour automatiquement si j'ai bonne mémoire

à vérifier

Il me semble aussi qu'elle est prolongée automatiquement mais uniquement sur le site du projet, pas dans le BM.

[JeromeC]

Oui c'est ça, il faut avoir la foi, ou aller voir sur le site du projet

Ca doit faire plusieurs éons que j'ai pas eu de RNA... et j'ai toujours eu des expériences compliquées avec RNA.

[jm91160]

Merci pour vos réponses et vos encouragements 

[fzs600]

UT a 80 % après 152 jours de calcul.  je continue. 

[naz]

bravo!

[JeromeC]

Les fois où j'ai réussi à tenir plusieurs dizaines de jours (c'est à dire quelques fois sur de nombreuses années) elles ont fini en erreur de calcul, ça fait tellement longtemps que j'ai plus eu de tâche... au début il distribuait des tâches "normales", j'avais quand même réussi à atteindre 160000 points... mais au début il avait pas de points de reprises, avant de passer à VB, si je me souviens bien.

[fzs600]

Ut que je calcul est partie de très très nombreuses fois en erreur et beaucoup d'abandon ( en tout 78 )

Quorum de 2 : le deuxième a calculer UT est  TBone un membre de SETI.Germany.

[modesti]

Félicitations fzs et

J'aimerais bien ne serait-ce que récupérer une nouvelle tâche...

[fzs600]

UT a 80 % après 152 jours de calcul.  je continue. 

Voila ut est partie en erreur.

22 Jan 2018, 6:00:05 UTC    8 Sep 2018, 23:33:47 UTC    Error while computing    19,824,448.16    19,622,510.00    59,473.34    cmsearch VM (VirtualBox) 1.0.2 v1.19 (vbox64)

Mais ca crédite pas mal quand meme. 

J'attend la prochaine avec impatience. 

[JeromeC]

229 jours de calcul

[modesti]

j'en voudrais bien une comme ça...

[JeromeC]

Oui enfin bon si c'est juste pour le plaisir d'avoir des crédits et que ça serve à 0 côté scientifique...

[modesti]

Il me semble que l'admin avait dit que même si les tâches partent en sucette, il y a des bouts qui sont utilisables. C'est pour ça qu'il accorde des crédits.

[fzs600]

Il me semble que l'admin avait dit que même si les tâches partent en sucette, il y a des bouts qui sont utilisables. C'est pour ça qu'il accorde des crédits.

Oui mème si UT part en erreur ça sert a quelque chose et ça crédite. 

[modesti]

Au fait, un grand MERCI à toi, fzs.

Grâce à tes points, l'AF est à nouveau rentrée dans les points au FB sur ce projet: https://statseb.boinc-af.org/top_FB.py?projet=110

[fzs600]

Au fait, un grand MERCI à toi, fzs.

Grâce à tes points, l'AF est à nouveau rentrée dans les points au FB sur ce projet: https://statseb.boinc-af.org/top_FB.py?projet=110

[JeromeC]

Je dirais même plus, tu es le seul de toute l'AF avec un RAC positif sur ce projet

En revanche tu vas avoir du boulot pour rattraper Frederic Abussan