W. Stallings. Cap. 17 - Parallel Processing

W. Stallings. Cap. 17 - Parallel Processing

1. Types of Parallel Processor Systems

   • Single instruction, single data (SISD) stream: A single processor executes a single instruction stream to operate on data stored in a single memory. Uniprocessors fall into this category.
     • Single instruction, multiple data (SIMD) stream: A single machine instruc- tion controls the simultaneous execution of a number of processing elements on a lockstep basis. Each processing element has an associated data memory, so that instructions are executed on different sets of data by different proces- sors. Vector and array processors fall into this category, and are discussed in Section 18.7.
       • Multiple instruction, single data (MISD) stream: A sequence of data is trans- mitted to a set of processors, each of which executes a different instruction sequence. This structure is not commercially implemented.
         • Multiple instruction, multiple data (MIMD) stream: A set of processors si- multaneously execute different instruction sequences on different data sets. SMPs, clusters, and NUMA systems fit into this category.

2. Parallel Organizations

   • Parallel Organizations

3. SYMMETRIC MULTIPROCESSORS

   • 1. There are two or more similar processors of comparable capability.
     • 2. These processors share the same main memory and I/O facilities and are inter- connected by a bus or other internal connection scheme, such that memory
       • 3. All processors share access to I/O devices, either through the same channels or through different channels that provide paths to the same device.
         • 4. All processors can perform the same functions (hence the term symmetric).
           • 5. The system is controlled by an integrated operating system that provides inter- action between processors and their programs at the job, task, file, and data element levels

4. Organization

5. Multiprocessor Operating System Design Considerations

   • • Simultaneous concurrent processes: OS routines need to be reentrant to allow several processors to execute the same IS code simultaneously. With mul- tiple processors executing the same or different parts of the OS, OS tables and management structures must be managed properly to avoid deadlock or invalid operations.
     • • Scheduling: Any processor may perform scheduling, so conflicts must be avoided. The scheduler must assign ready processes to available processors.
       • • Synchronization: With multiple active processes having potential access to shared address spaces or shared I/O resources, care must be taken to provide effective synchronization. Synchronization is a facility that enforces mutual exclusion and event ordering.
         • • Memory management: Memory management on a multiprocessor must deal with all of the issues found on uniprocessor machines, as is discussed in Chapter 8. In addition, the operating system needs to exploit the available

6. The MESI Protocol

7. Implicit and Explicit Multithreading

   • Process: An instance of a program running on a computer. A process embod- ies two key characteristics:
     • Process switch: An operation that switches the processor from one process to another, by saving all the process control data, registers, and other information for the first and replacing them with the process information for the second.2
       • Thread: A dispatchable unit of work within a process. It includes a processor context (which includes the program counter and stack pointer) and its own data
         • Thread switch: The act of switching processor control from one thread to an- other within the same process.

8. MULTITHREADING AND CHIP MULTIPROCESSORS

9. CLUSTERS

   • Absolute scalability: It is possible to create large clusters that far surpass the power of even the largest standalone machines. A cluster can have tens, hun- dreds, or even thousands of machines, each of which is a multiprocessor.
     • Incremental scalability: A cluster is configured in such a way that it is possible to add new systems to the cluster in small increments. Thus, a user can start out with a modest system and expand it as needs grow, without having to go through a major upgrade in which an existing small system is replaced with a larger system.
       • High availability: Because each node in a cluster is a standalone computer, the failure of one node does not mean loss of service. In many products, fault tolerance is handled automatically in software.
   • Cluster Configurations

10. Cluster Computer Architecture

    • • Single entry point: A user logs onto the cluster rather than to an individual computer.
      • Single file hierarchy: The user sees a single hierarchy of file directories under the same root directory.
        • Single control point: There is a default workstation used for cluster manage- ment and control.
          • Single virtual networking: Any node can access any other point in the cluster, even though the actual cluster configuration may consist of multiple intercon- nected networks. There is a single virtual network operation.

Imágenes y archivos del mapa