Monitoring jobs

You’ll feel a bit helpless submitting a job and then just waiting for the output to appear, so it’s important to be able to track and monitor the jobs you’ve submitted to the scheduler. The most useful command for tracking the status of submitted jobs is qstat. Try submitting a job as shown on the previous page and then run:

qstat
job-ID  prior   name       user         state submit/start at     queue  slots
------------------------------------------------------------------------------
<...more...>
6293704 0.00000 Exam8611-S skhodaei     qw    07/17/2026 10:16:45           80
6293706 0.00000 Exam8611-S skhodaei     qw    07/17/2026 10:17:36           80
6293723 0.00000 Rscript    pbreheny     qw    07/17/2026 10:42:13            1

Assuming you submitted qstat immediately after submitting your job, you’ll be at the end of the list, and your “state” will be qw, which stands for “queued and waiting”. This means that the batch scheduler is still in the process of deciding where to run the job and allocating resources to do so. After a few seconds, your job will transition to a state of r, meaning “running”. For the most part, these are the only two states you will see, although you may happen to catch your job in a t state, meaning that it is in the process of transitioning to a compute node. There are also various error states that your job could enter if something goes wrong, such as Eqw — if you see anything other than qw, r, or t, it’s an indication that something has gone wrong.

Running qstat with no options will tell you about all the jobs currently submitted to the entire Argon cluster. This is sometimes useful to see, but typically information overload. To see only your jobs, you can submit:

qstat -u pbreheny

(obviously, replacing pbreheny with your own HawkID). Alternatively, you can look at all the jobs submitted to a certain queue:

qstat -q BIOSTAT

One final monitoring command that may be useful is qhost, which provides more information about the processor and memory usage of specific hosts (qstat tells you which host your job is running on; it doesn’t in the above example because our job wasn’t running, it was queued and waiting):

qhost -h argon-itf-bx48-21.hpc

In particular, this command can be useful as a way to see if the node you are working on is running out of memory.

Deleting jobs: qdel

For various reasons, you may wish to delete a job from the queue (usually, because you realize there is a mistake in your code or your qsub command). By running qstat, you can learn the ID for the job you submitted (6293723 in the example above). To kill it, simply use the qdel command:

qdel 6293723

You will then get a confirmation message confirming that you have deleted the job.