Job states¶
Every job shows its state on the job page and in the Jobs list. Until a job starts running, the state tells you who is holding it: PoreStore, while it gets the job ready, or the hardware, while the job waits for a machine to come free.
What each state means¶
| State | Who holds the job | What's happening | What moves it on |
|---|---|---|---|
| pending | PoreStore | The job has been created and is waiting to be picked up. This usually lasts a moment. | PoreStore picks it up and starts preparing it. |
| queued | PoreStore | PoreStore has accepted the job and is getting it ready to send. A job that changes its image, such as a cleaning or a trim saved over the image, also waits here while another job is still writing to the same image. | PoreStore sends the job to its hardware. |
| queued, not started | The hardware | The job has been sent to its hardware and waits there for a machine to start it, behind other jobs or for hardware that's in short supply. | A machine starts it. With Auto, PoreStore moves a waiting job to other hardware that's free. |
| processing | The hardware | The job is running. | It finishes, fails or reaches its time limit. |
| completed | The result is saved and shown on the job page. | Nothing: a completed job never changes. | |
| failed | The job stopped with an error. The job page says why, with technical details underneath. | Fix the cause and Run Again. | |
| cancelled | You cancelled the job, or it waited or ran far longer than expected and PoreStore stopped it. | Run Again to try once more. |
A job that waits for a machine shows as queued everywhere: on its job page, in the Jobs list and its status filter, on Home, in an image's jobs and in the workflow grid. It shows as processing once a machine starts it. For SEM View and X-Ray View, the panel on the job page also says Queued, not started and why it waits.
Waiting and time limits¶
- Auto never leaves a job waiting for one machine. It sends the job to the smallest tier that fits, and while the job waits, moves it to any tier of the same size or larger that has a machine free. See Compute and hardware.
- A tier you chose yourself makes the job wait for that tier. On 48 GB GPU ยท RTX 8000, a job waits up to an hour for a GPU to come free, then fails.
- Hardware in short supply can keep a job waiting for minutes or hours. A job that can't start in time to finish within a day fails.
- A job that waits or runs far longer than its tool should take is stopped and marked cancelled or failed, and the job page says why.
Automatic retry¶
If a job fails while running, PoreStore tries it once more on its own, and the job shows queued again while it does. It fails only if the second attempt fails too. A job is not retried when a check before the work finds that the image or the settings can't be run, because a second attempt would fail the same way; the job page then says what to change.
Cancelling¶
Cancel job on the job page stops a job that hasn't started: one that's pending, queued, or queued, not started while it waits for hardware. Once a job is running it finishes on its own; the button is gone. A 3D Visualization or ParaView Server session is different: it runs until you select Shut down server or it reaches its maximum session duration, and closing the 3D viewer's tab also ends a 3D Visualization session. See 3D Visualization.
Batches¶
A batch launches one job per image, and each job goes through these states on its own. Some may complete while others still wait for hardware.