Alisa Neeman @ on
What does submit do on abort?
I am testing submit on an application that saves state for a restart if a run on a cluster is aborted. The state is saved in a bunch of files.
I am trying to determine how big the run directory is going to be if a run fails or is aborted and whether it will fill the user’s quota.
While I can see the size of the state files the app generates, I am wondering if there is some duplication of that in the submit log.
What does submit do on abort? What goes into submit.log?
0 Like 0 Dislike
Steven Clark @ on
Submit relays the abort attempt form the HUB machine to the remote host using signals. What happens on the remote site is dependent on the batch queuing system being used. As an example in a job submitted to PBS is aborted then the qdel command will issued to terminate it. Similar actions are taken for other batch systems.
After the job is terminated remotely created files are returned to the HUB machine in whatever state the application left them. In most cases these files are not of much use because the application does not perform any checkpoint operations.
An attempt is made to apply the user disk quota on the remote host. In case of rapid multiple job submission however quota limit may not be completely enforceable. In such cases some files will be lost on return to the HUB machine.
.submit.log typically contains only error messages and ideally would be empty.
Reply Report abuse
Please login to answer the question.