| |
| |
Resource Management System
| The Quadrics Resource Management System (RMS) provides production parallel supercomputing facilities for UNIX. When used together with the QsNet data network, RMS gives supercomputer performance and quality of service on a cluster of commodity nodes. Supported platforms include HP AlphaServer SC™ systems and Linux clusters constructed from PentiumŪ, ItaniumŪ, Alpha and Opteron nodes. |
Why choose RMS?
Single point administration - command line and graphics interfaces save time and boost efficiency.
Resource scheduling - enables you to optimize use of large numbers of high CPU count nodes.
Parallel program runtime support - fast program start-up and full clean up.
System-wide event handling - single consistent interface to events across the whole cluster.
Seamless integration with industry standard tools.
Proven track record with 24 x 7 operational capability |
RMS Partitions and Scheduling
 The key to achieving high-levels of performance on a large-scale parallel application is dedicating resources (CPUs, memory, network bandwidth and local I/O capability) to its execution. RMS enables a system administrator to efficiently manage these resources to achieve maximum performance. Nodes can be configured into mutually exclusive sets known as partitions that each provide a specific system service. For example, your system could have an interactive partition for conventional UNIX processes and program development, a sequential batch partition and a parallel partition running the RMS gang scheduler. Free cycles on the interactive partition could be consumed by sequential batch jobs running from a low priority queue. Plus the system could be configured to allow certain users to run high-priority interactive jobs during working hours. |
As the administrator of an RMS system, you can control how the nodes are configured into partitions, how this changes with shift, who can access each partition and the levels of resource they consume. You can suspend, resume or cancel jobs and alter their priority. All of this can be done direct from the command line or from the Pandora GUI.The RMS scheduler supports both space and time sharing, allocating CPUs to jobs in strict priority order. Scheduling decisions are applied to all processes in a parallel program at the same time. The RMS scheduler optimizes process to CPU assignments for high CPU count SMP nodes connected via multi-rail QsNet data networks. The RMS scheduler assigns CPUs so as to maximize the use of QsNet global broadcast and synchronization hardware, exploit locality amongst processes that can run on the same node and minimize resource fragmentation.Where the CPUs on one node are shared between multiple jobs, RMS runs the processes of each job in a processor set guaranteeing access to the allocated CPUs.
Scheduling parameters and access controls are applied on a per-partition basis, providing full control over use of resources. The system administrator can set timelimits, timeslice intervals, minimum job sizes and default memory limits on a per partition basis. The maximum job size (CPUs and memory) and the default priority can be controlled on a per-user or per-project basis as can the total number of CPUs in use at any point in time. |
Parallel Programs under RMS
Parallel programs under RMS consist of a controlling process prun and a number of application processes distributed over the nodes of a partition. Each process is executed by dedicated CPUs. You choose how many are required and how they are distributed over multi-CPU nodes.
Parallel programs are started by prun requesting CPUs from the scheduler. When they are allocated, the RMS daemon running on each node executes a support process rmsloader for the user. This process starts the applications processes and routes their stdio streams to and from prun. The parallel program completes when all its application processes have exited or one or more have been killed. If an application process is killed the loader process runs a core file analysis script that provides analysis of the program fault. RMS also supports pre-allocation of resources for a sequence of parallel jobs. The command allocate can be used to make a single request for the resources required by a sequence of jobs. Once CPUs are allocated, the jobs will run in quick succession.When a job is completed and its CPUs are freed, RMS makes sure that the nodes are left clean and ready for the next job. All remaining processes are killed and site-specific prologue and epilogue scripts are executed. | |  |
| |  |
RMS Scalability
RMS is designed to be totally scalable, running systems ranging from just a few nodes to as many as 4096. The RMS daemons are connected in a tree-like structure, following that of the machine itself. New commands are passed down this tree to a minimal range of nodes - reducing the impact of job startup and scheduling operations on performance. Statistics and status information are fed back up the tree to the RMS management daemons and stored in an SQL database.RMS leads the way in scalable parallel job startup; simple commands are executed in seconds on even the largest machines. |
RMS System Monitoring
The RMS daemons continually monitor system availability and performance. The condition of every node in the system is stored in the RMS database together with a range of other vital statistics including; CPU utilization, memory availability, I/O rates, temperature data, error rates, power supply and fan status data. |
RMS Event Handling
RMS includes a cluster wide event handling mechanism. Events are generated on system incidents such as node crash, component failure or when environmental limits are exceeded. They can also be generated when an application service fails or a file-system exceeds its limit. Processes interested in these events register a handler script, which will be run by the eventmgr daemon if the event occurs. Events are promoted to Pandora and may also be escalated to an external SNMP management system. |
Pandora
The RMS management GUI, Pandora, provides a secure, single point of control for the whole system. Pandora is a Java application that runs on a PC or as an X application from a server node in the RMS system. Pandora provides a series of views, each showing an aspect of the system state:
The machine view shows the physical layout of the machine, environmental data on each of its components and status information on field replaceable components.
The network view displays QsNet status information. It provides the framework for executing QsNet diagnostics and visualizing their output.
The node management view shows the state of each node in the machine. It is used to boot, halt, reset and power cycle nodes and attach or monitor their consoles.
The partition view provides control over and visibility of the system's workload.
SQL shell allows administrators access to the RMS database.
Many of the Pandora views have detail frames that allow you to 'dive' on a particular component. For example, the switch module environment frame shows details of the power supply, fan and temperature data in each QsNet switch module. In addition, the Pandora Toolkit provides a simple, easy-to-use XML interface for extracting data from the RMS database, generating views and updating them as the system changes. | |  |
Integration with Third Party Products
RMS is integrated with industry standard products from Etnus and Platform Computing. TotalView™is a full-featured, source-level, graphical 'debugger' for applications written in C, C++, Fortran, HPF, and mixed source/assembler codes. It is a multi-process, multi-thread debugger that supports multiple parallel programming paradigms including MPI and OpenMP. Integration of the industry leading workload scheduler LSF® into RMS provides a productive and familiar environment for execution of parallel programs with jobs being submitted to partitions from interactive or batch queues. RMS stores all system statistics in an SQL database. An application interface is provided for CPU allocation and job launch. These interfaces are available to system administrators and end users to integrate RMS with local tool sets. Open PBS has been modified to integrate with the RMSŪ API to provide batch scheduling on Alpha Server™ SC systems, as well as Linux clusters. The modified version of Open PBS is capable of non-contiguous allocations, tying queues to partitions, and job control using RMS. Source is currently available through the Pittsburgh Supercomputing Center. |
RMS Product Availability
RMS is available for AlphaServer™ SC systems running Tru64 UNIX™ and for PentiumŪ, ItaniumŪ Alpha and Opteron™ Linux clusters. RMS is licensed to end-users on a per-CPU basis using the FLEXlm license manager. Binary distributions, documentation and a demo license key are all available from the Quadrics web site www.quadrics.com. Quadrics provides third level support to OEM customers and direct support to end-users. Quadrics also provides additional services including system configuration, installation planning, performance tuning, second level support and custom software development. |
Notes
RMS brochure
QsNet performance is dependent upon the host PCI interface. Performance figures given in this document are indicative of what can be achieved, but do not represent a commitment for any particular system.
TotalView™ is a trademark of Etnus LLC.
LSF™ is a trademark of Platform Computing Inc.
PentiumŪ and ItaniumŪ are trademarks of Intel Corporation.
Opteron™ is a trademark of Advanced Micro Devices.
All other trademarks are the property of their respective owners.
|
> Legal
| |
|