Parallel Processing

Parallel Processing

1. 1) Multiple Processor Organizations

   • Types of Parallel Processor Systems
     • SIMD
       • Single instruction, multiple data stream
         • Vector processor fall into this category
         • Array processors fall into this category
     • MISD
       • Multiple instruction, single data stream
         • This structure is not commercially implemented
     • SISD
       • Single instruction, single data stream
         • Uniprocessors fall into this category.
     • MIMD
       • Multiple instruction, multiple data stream
         • Shared memory (tightly coupled)
           • Symmetric multiprocessor (SMP)
           • Nonuniform memory access (NUMA)
         • Distributed memory (loosely coupled)
           • Clusters

2. 2) Symmetric Multiprocessors

   • Organization
     • Tightly coupled multiprocessing
       • The most common organization for personal computers, workstations, and servers is the time-shared bus.
       • Single-processor system uses a bus interconnection that consists of control, address, and data lines.
   • Multiprocessor Operating System Design Considerations
     • An SMP manages processor and other computer resources
     • Among the key design issues:
       • Simultaneous concurrent processes
       • Scheduling
       • Memory management
       • Reliability and fault tolerance

3. 3) Cache Coherence and the MESI Protocol

   • Software Solutions
     • Attempt to avoid the need for additional hardware circuitry and logic by relying on the compiler and OS to deal with the problem
   • Hardware Solutions
     • Are generally referred to as cache coherence protocols.
   • MESI Protocol
     • Invalidate-based cache coherence protocol, one of the most common protocols which support write-back caches.

4. 4) Multithreading and Chip Multiprocessors

   • Approaches to Explicit Multithreading
     • Principals approaches to multithreading
       • Blocked multithreading
       • Interleaved multithreading
       • Simultaneous multithreading
       • Chip multiprocessing
   • Implicit and Explicit Multithreading
     • Concepts of software threads in a multiprogrammed operating system
       • Process
       • Process switch
       • Thread
       • Thread switch

5. 5) Clusters

   • Blade Servers
     • A common implementation of the cluster approach is the blade server
   • Benefits that can be achieved with clustering
     • High availability
     • Incremental scalability
     • Superior price/performance
     • Absolute scalability
   • Operating System Design Issues
     • Failure management
     • Load balancing
     • Parallelizing computation

6. 6) Nonuniform Memory Access

   • Organization
     • Each CPU has near and far memory
     • Each processor accesses its main memory faster.
   • NUMA Pros and Cons
     • 5) Can deliver effective performance at higher levels of parallelism
     • 3) Shared memory model
     • 2) Load distribution
     • 4) Concurrent I / O operations
     • 1) Multi CPU and Multiprocessing
     • (-)A CCNUMA does not transparently look like an SMP
   • Motivation
     • The processor limit in an SMP
     • Coherency is maintained in software rather than hardware.

7. 7) Cloud Computing

   • Cloud Computing Elements
     • Broad network access
     • On- demand self- service
     • Resource pooling
     • Measured service
     • Rapid elasticity
   • Cloud Computing Reference Architectur
     • Cloud broker
       • Service aggregation
       • Service intermediation
       • Service arbitrage
     • Cloud consumer
     • Cloud provider
     • Cloud auditor
     • Cloud carrier