Computing at Scale

When a problem can no longer run on a desktop or laptop, use a large-scale computing resource.

Find a large scale resource that matches your needs.

Resources will require an account request or application.

To log in, you will need an account name and password, and the name of the login computer.

A job consists of a computational task, usually defined by input data and a piece of software, producing output data.

Most large scale systems consist of a head node for logging in and submitting jobs, where jobs are performed on worker nodes.

A batch scheduler controls where and when jobs run on the worker nodes.

In order to submit a job, you need to write a submit description file.

Jobs may go through various states in the queue, and can be monitored by a queue command.

It is important to follow your resource’s policies and procedures.

Have a data management plan in place before you start computing.

Always run a test job before submitting a full scale job.

To test a new job, use an interactive session beore submitting.

You can use log, standard output, and standard error information to determine why jobs fail.

Large-scale compute systems are powerful, but only if you ask questions and get the help you need.