FAQ

A few cluster-specific quirks and policy questions that come up often enough to be worth gathering here, rather than as asides in the step-by-step pages.

Why is my job crashing mysteriously with no warning?

This is usually because you’re requesting more memory than you’ve been allocated. If the code you have written tries to load, say, a 1 GB dataset into memory and the scheduler has only allocated 500 MB to your job, then it will die, instantly and unceremoniously.

How do I request more memory?

On Argon, memory is allocated in proportion to cores: requesting -pe smp 5 doesn’t just get you 5 cores, it gets you roughly 5 times the memory of a single-core request. This isn’t universal to all HPC systems — it’s just how Argon happens to be configured. But if your job is mysteriously crashing without any syntax errors, try increasing the number of cores. If the problem goes away, it probably means that your previous efforts were running out of memory.

How many jobs can I run?

We want to maximize productive use of the BIOSTAT queue while sharing it fairly. Two things to keep in mind:

  1. When using qlogin, request only what you need (qlogin -pe smp 2 is more polite than qlogin), and don’t leave interactive sessions logged in.
  2. Fairness depends on the jobs you submit, not just the scheduler. 10,000 jobs of 2 minutes each cost another user at most a 2-minute wait; 500 jobs of 13 hours each would lock up the entire BIOSTAT queue for at least 13 hours.

This is a rough guide that we recommend for Biostatistics students in the department:

During periods of low usage (e.g., winter break), it’s fine to exceed these limits, but be ready to throttle back if asked — the more resources you use, the greater your responsibility to make sure your work isn’t impacting others.

When should I use a queue other than BIOSTAT?