Apple Compressor (4.0) - Manual (Page 221)

Chapter 8 Use Apple Qmaster to set up a distributed processing system 221
•
Multiple service nodes: In general, more service nodes mean more processing power. If your
workflow is mainly I/O bound, configure the number of services so that reads and writes do
not exceed your network and storage capabilities. For CPU-bound workflows, set the number
of services so that all available resources on the machine will be used. If the computing load,
per job, is closer to the network load, having a smaller number of service nodes per cluster
may be more efficient. If you’re using the Apple Qmaster distributed processing system with
an application other than Shake or Compressor, consult the application’s user manual for
information about how to optimize the number of service nodes.
•
Storage device: A storage device, such as a remote disk or group of disk arrays, can be used as
cluster scratch storage, which is a place for short-term storage of temporary data generated by
the cluster controller, clients, and service nodes. (You set the scratch storage location in the
Apple Qmaster Sharing window of Compressor. For more information, see
Advanced pane
of the Apple Qmaster Sharing window on page 254 and Use cluster storage on page 232.)
Alternatively, a storage area network (for example, Xsan) can be used as a final destination for
the files after they’re processed.
To maximize your resources, consider using some computers for more than one distributed
processing function:
•
Service node and cluster controller: In a small setup, one of the service nodes in a cluster can
also act as the cluster controller so that it performs both functions. However, in a cluster of
many service nodes, the processing load required for the cluster controller may be so high
that it becomes inefficient to use one computer as both a service node and a cluster controller.
•
Client computer and cluster controller or service node: You can also set up a client computer to
act as a cluster controller or service node in a cluster. However, keep in mind that the more
available processing power a computer has, the faster it can manage or process jobs.
How the Apple Qmaster system distributes batches
The cluster controller determines the most efficient use of the cluster resources based on the
availability of each service node and by the types of jobs in the batch. Certain jobs can be
segmented; others cannot.
•
If the job can be subdivided into data segments: Each segment is processed in parallel on the
service nodes in the cluster. For example, for a render batch, the cluster controller can divide
the jobs into groups (segments).
•
If the batch can be subdivided into tasks: Different tasks are run on different service nodes. For
example, for a render batch, the cluster controller can subdivide the rendering work into
different processing tasks.