Chapter 3: A Top-Level view of Computer function And Inter-Connection (2621)

Chapter 3: A Top-Level view of Computer function And Inter-Connection (2621)

1. At a top level, a computer consists of CPU (central processing unit), memory, and I/O components

2. we can characterize a computer system by describing (1) the external behavior of each component, that is, the data and control signals that it exchanges with other components, and (2) the interconnection structure and the controls required to manage the use of the interconnection structure

3. 3.1 Computer Components

   • von Neumann architecture is based on three key concepts:
     • Data and instructions are stored in a single read–write memory.
     • The contents of this memory are addressable by location, without regard to the type of data contained there.
     • Execution occurs in a sequential fashion (unless explicitly modified) from one instruction to the next.
   • general-purpose hardware, the system accepts data and control signals and produces results
     • instead of rewiring the hardware for each new program, the programmer merely needs to supply a new set of control signals
       • How shall control signals be supplied? The answer is simple but subtle. The entire program is actually a sequence of steps
       • For each step, a new set of control signals is needed.
       • To distinguish this new method of programming, a sequence of codes or instructions is called software.
       • two major components of the system: an instruction interpreter and a module of general-purpose arithmetic and logic functions. These two constitute the CPU.
     • I/O components.
       • Data and instructions must be put into the system
       • A means of reporting results is needed
     • main memory
       • there must be a place to temporarily store both instructions and data
   • CPU exchanges data with memory.
     • makes use of two internal (to the CPU) registers: a memory address register (MAR), which specifies the address in memory for the next read or write, and
     • a memory buffer register (MBR), which contains the data to be written into memory or receives the data read from memory.
   • I/O address register (I/OAR) specifies a particular I/O device
     • I/O buffer register (I/OBR) is used for the exchange of data between an I/O module and the CPU
   • A memory module consists of a set of locations, defined by sequentially numbered addresses. Each location contains a binary number that can be interpreted as either an instruction or data.

4. 3.2 Computer Function

   • processor reads (fetches) instructions from memory one at a time and executes each instruction. Program execution consists of repeating the process of instruction fetch and instruction execution
     • The two steps are referred to as the fetch cycle and the execute cycle
   • The processing required for a single instruction is called an instruction cycle
   • Instruction Fetch and Execute
     • In a typical processor, a register called the program counter (PC) holds the address of the instruction to be fetched next
     • The fetched instruction is loaded into a register in the processor known as the instruction register (IR)
     • processor interprets the instruction and performs the required action. In general, these actions fall into four categories:
       • ■ Processor-memory: Data may be transferred from processor to memory or from memory to processor.
       • ■ Processor-I/O: Data may be transferred to or from a peripheral device by transferring between the processor and an I/O module.
       • ■ Data processing: The processor may perform some arithmetic or logic operation on data.
       • ■ Control: An instruction may specify that the sequence of execution be altered. For example, the processor may fetch an instruction from location 149, which specifies that the next instruction be from location 182. PC is adjusted to the next location
     • The instruction format provides 4 bits for the opcode, so that there can be as many as 24 = 16 different opcodes, and up to 212 = 4096 (4K) words of memory can be directly addressed.
     • ■ Instruction address calculation (iac): Determine the address of the next instruction to be executed. For example, if each instruction is 16 bits long and memory is organized into 16-bit words, then add 1 to the previous address. If, instead, memory is organized as individually addressable 8-bit bytes, then add 2 to the previous address.
   • Interrupts
     • computers provide a mechanism by which other modules (I/O, memory) may interrupt the normal processing of the processor
     • Interrupts are provided primarily as a way to improve processing efficiency
     • When the external device becomes ready to be serviced—that is, when it is ready to accept more data from the processor—the I/O module for that external device sends an interrupt request signal to the processor
     • processor responds by suspending operation of the current program, branching off to a program to service that particular I/O device, known as an interrupt handler
     • the processor and the operating system are responsible for suspending the user program and then resuming it at the same point.
     • If an interrupt is pending, the processor does the following: ■ It suspends execution of the current program being executed and saves its context. This means saving the address of the next instruction to be executed (current contents of the program counter) and any other data relevant to the processor’s current activity.
     • ■ It sets the program counter to the starting address of an interrupt handler routine.
     • Figures 3.7b and 3.10b assume that the time required for the I/O operation is relatively short: less than the time to complete the execution of instructions between write operations in the user program. In this case, the segment of code labeled code segment 2 is interrupted. A portion of the code (2a) executes (while the I/O operation is performed) and then the interrupt occurs (upon the completion of the I/O operation). After the interrupt is serviced, execution resumes with the remainder of code segment 2 (2b)
   • Multiple interrupt
     • Two approaches can be taken to dealing with multiple interrupts.
       • The first is to disable interrupts while an interrupt is being processed. A disabled interrupt simply means that the processor can and will ignore that interrupt request signal
         • This approach is nice and simple, as interrupts are handled in strict sequential order (Figure 3.13a)
         • The drawback to the preceding approach is that it does not take into account relative priority or time-critical needs
       • A second approach is to define priorities for interrupts and to allow an interrupt of higher priority to cause a lower-priority interrupt handler to be itself interrupted (Figure 3.13b)
         • second approach, consider a system with three I/O devices: a printer, a disk, and a communications line, with increasing priorities of 2, 4, and 5, respectively
   • I/O Function
     • An I/O module (e.g., a disk controller) can exchange data directly with the processor
     • the processor can also read data from or write data to an I/O module.
     • In some cases, it is desirable to allow I/O exchanges to occur directly with memory
       • In such a case, the processor grants to an I/O module the authority to read from or write to memory, so that the I/O-memory transfer can occur without tying up the processor.
       • This operation is known as direct memory access (DMA)

5. 3.3 Interconnection Structures

   • The collection of paths connecting the various modules is called the interconnection structure
   • Figure 3.15 suggests the types of exchanges that are needed by indicating the major forms of input and output for each module type
     • ■ Memory: Typically, a memory module will consist of N words of equal length. Each word is assigned a unique numerical address (0, 1, c , N-1). A word of data can be read from or written into the memory.
     • ■ I/O module: From an internal (to the computer system) point of view, I/O is functionally similar to memory. There are two operations; read and write. refer to each of the interfaces to an external device as a port and give each a unique address. external data paths for the input and output of data with an external device. Finally, an I/O module may be able to send interrupt signals to the processor.
     • ■ Processor: The processor reads in instructions and data, writes out data after processing, and uses control signals to control the overall operation of the system. It also receives interrupt signals.
     • This list defines the data to be exchanged
   • The interconnection structure must support the following types of transfers:
     • ■ Memory to processor: The processor reads an instruction or a unit of data from memory.
     • ■ I/O to processor: The processor reads data from an I/O device via an I/O module.
     • ■ Processor to I/O: The processor sends data to the I/O device.
     • ■ I/O to or from memory: For these two cases, an I/O module is allowed to exchange data directly with memory, without going through the processor, using direct memory access.
     • ■ Processor to memory: The processor writes a unit of data to memory.
   • the most common are (1) the bus and various multiple-bus structures, and (2) pointto-point interconnection structures with packetized data transfer.

6. 3.4 Bus Interconnection

   • bus structures are still commonly used for embedded systems, particularly microcontrollers.
   • A bus is a communication pathway connecting two or more devices.
   • it is a shared transmission medium.
     • Hence only one device at a time can successfully transmit.
   • Typically, a bus consists of multiple communication pathways, or lines
     • For example, an 8-bit unit of data can be transmitted over eight bus lines.
   • Each line is capable of transmitting signals representing binary 1 and binary 0.
   • A bus that connects major computer components (processor, memory, I/O) is called a system bus
     • A system bus consists, typically, of from about fifty to hundreds of separate lines. Each line is assigned a particular meaning or function
     • on any bus the lines can be classified into three functional groups (Figure 3.16): data, address, and control lines
   • The data lines provide a path for moving data among system modules. These lines, collectively, are called the data bus
     • The data bus may consist of 32, 64, 128, or even more separate lines, the number of lines being referred to as the width of the data bus.
     • each line can carry only one bit at a time, the number of lines determines how many bits can be transferred at a time.
     • width of the data bus is a key factor in determining overall system performance.
   • The address lines are used to designate the source or destination of the data on the data bus
     • For example, if the processor wishes to read a word (8, 16, or 32 bits) of data from memory, it puts the address of the desired word on the address lines
     • the width of the address bus determines the maximum possible memory capacity of the system.
   • The control lines are used to control the access to and the use of the data and address lines.
     • Control signals transmit both command and timing information among system modules. Timing signals indicate the validity of data and address information
     • Typical control lines include: ■ Memory write: causes data on the bus to be written into the addressed location. ■ Memory read: causes data from the addressed location to be placed on the bus. ■ I/O write: causes data on the bus to be output to the addressed I/O port. ■ I/O read: causes data from the addressed I/O port to be placed on the bus. ■ Transfer ACK: indicates that data have been accepted from or placed on the bus. ■ Bus request: indicates that a module needs to gain control of the bus. ■ Bus grant: indicates that a requesting module has been granted control of the bus. ■ Interrupt request: indicates that an interrupt is pending. ■ Interrupt ACK: acknowledges that the pending interrupt has been recognized. ■ Clock: is used to synchronize operations. ■ Reset: initializes all modules.

7. 3.5 point-to-point Interconnect

   • The principal reason driving the change from bus to point-to-point interconnect was the electrical constraints encountered with increasing the frequency of wide synchronous buses
   • significant characteristics of QPI and other point-to-point interconnect schemes:
     • ■ Multiple direct connections: Multiple components within the system enjoy direct pairwise connections to other components. This eliminates the need for arbitration found in shared transmission systems.
     • ■ Layered protocol architecture: As found in network environments, such as TCP/IP-based data networks, these processor-level interconnects use a layered protocol architecture, rather than the simple use of control signals found in shared bus arrangements.
     • ■ Packetized data transfer: Data are not sent as a raw bit stream. Rather, data are sent as a sequence of packets, each of which includes control headers and error control codes.
     • Example: QuickPath Interconnect (QPI)
   • The QPI links (indicated by the green arrow pairs in the figure) form a switching fabric that enables data to move throughout the network. Direct QPI connections can be established between each pair of core processors. If core A in Figure 3.17 needs to access the memory controller in core D, it sends its request through either cores B or C, which must in turn forward that request on to the memory controller in core D.
   • QPI is used to connect to an I/O module, called an I/O hub (IOH). The IOH acts as a switch directing traffic to and from I/O devices
   • the link from the IOH to the I/O device controller uses an interconnect technology called PCI Express (PCIe)
   • QPI is defined as a four-layer protocol architecture
     • ■ Physical: Consists of the actual wires carrying the signals, as well as circuitry and logic to support ancillary features required in the transmission and receipt of the 1s and 0s. The unit of transfer at the Physical layer is 20 bits, which is called a Phit (physical unit). ■ Link: Responsible for reliable transmission and flow control. The Link layer’s unit of transfer is an 80-bit Flit (flow control unit). ■ Routing: Provides the framework for directing packets through the fabric. ■ Protocol: The high-level set of rules for exchanging packets of data between devices. A packet is comprised of an integral number of Flits.
   • QPI Physical Layer
     • The QPI port consists of 84 individual links grouped as follows. Each data path consists of a pair of wires that transmits data one bit at a time; the pair is referred to as a lane. There are 20 data lanes in each direction (transmit and receive), plus a clock lane in each direction
     • QPI is capable of transmitting 20 bits in parallel in each direction. The 20-bit unit is referred to as a phit
     • At 20 bits per transfer, that adds up to 16 GB/s, and since QPI links involve dedicated bidirectional pairs, the total capacity is 32 GB
     • The form of transmission on each lane is known as differential signaling, or balanced transmission. With balanced transmission, signals are transmitted as a current that travels down one conductor and returns on the other
     • The binary value depends on the voltage difference. Typically, one line has a positive voltage value and the other line has zero voltage, and one line is associated with binary 1 and one line is associated with binary 0
     • QPI is known as low-voltage differential signaling (LVDS).
     • Another function performed by the physical layer is that it manages the translation between 80-bit flits and 20-bit phits using a technique known as multilane distribution.
   • QPI Link Layer
     • performs two key functions: flow control and error control
     • Each flit consists of a 72-bit message payload and an 8-bit error control code called a cyclic redundancy check (CRC)
     • A flit payload may consist of data or message information. The data flits transfer the actual bits of data between cores or between a core and an IOH. The message flits are used for such functions as flow control, error control, and cache coherence
     • The flow control function is needed to ensure that a sending QPI entity does not overwhelm a receiving QPI entity by sending data faster than the receiver can process the data and clear buffers for more incoming data
     • To control the flow of data, QPI makes use of a credit scheme. During initialization, a sender is given a set number of credits to send flits to a receiver. Whenever a flit is sent to the receiver, the sender decrements its credit counters by one credit. Whenever a buffer is freed at the receiver, a credit is returned to the sender for that buffer. Thus, the receiver controls that pace at which data is transmitted over a QPI link.
     • Occasionally, a bit transmitted at the physical layer is changed during transmission, due to noise or some other phenomenon. The error control function at the link layer detects and recovers from such bit errors, and so isolates higher layers from experiencing bit errors.
     • 1. As mentioned, each 80-bit flit includes an 8-bit CRC field. The CRC is a function of the value of the remaining 72 bits. On transmission, A calculates a CRC value for each flit and inserts that value into the flit. 2. When a flit is received, B calculates a CRC value for the 72-bit payload and compares this value with the value of the incoming CRC value in the flit. If the two CRC values do not match, an error has been detected. 3. When B detects an error, it sends a request to A to retransmit the flit that is in error. However, because A may have had sufficient credit to send a stream of flits, so that additional flits have been transmitted after the flit in error andbefore A receives the request to retransmit. Therefore, the request is for A to back up and retransmit the damaged flit plus all subsequent flits.
   • QPI Routing Layer
     • The routing layer is used to determine the course that a packet will traverse across the available system interconnects.
     • Routing tables are defined by firmware and describe the possible paths that a packet can follow
   • QPI Protocol Layer
     • the packet is defined as the unit of transfer.
     • One key function performed at this level is a cache coherency protocol, which deals with making sure that main memory values held in multiple caches are consistent. A typical data packet payload is a block of data being sent to or from a cache.

8. 3.6 PCI express

   • peripheral component interconnect (PCI) is a popular high-bandwidth, processor-independent bus that can function as a mezzanine or peripheral bus.
   • a new version, known as PCI Express (PCIe) has been developed. PCIe, as with QPI, is a point-to-point interconnect scheme intended to replace bus-based schemes such as PCI.
   • A key requirement for PCIe is high capacity to support the needs of higher data rate I/O devices, such as Gigabit Ethernet
   • Another requirement deals with the need to support time-dependent data streams
     • such as video-ondemand and audio redistribution
   • PCI Physical and Logical Architecture
     • A root complex device, also referred to as a chipset or a host bridge, connects the processor and memory subsystem to the PCI Express switch fabric comprising one or more PCIe and PCIe switch devices.
     • root complex acts as a buffering device, to deal with difference in data rates between I/O controllers and memory and processor components.
     • translates between PCIe transaction formats and the processor and memory signal and control requirements.
     • PCIe links from the chipset may attach to the following kinds of devices that implement PCIe:
       • ■ Switch: The switch manages multiple PCIe streams.
       • ■ PCIe endpoint: An I/O device or controller that implements PCIe, such as a Gigabit ethernet switch
       • ■ Legacy endpoint: Legacy endpoint category is intended for existing designs that have been migrated to PCI Express, and it allows legacy behaviors such as use of I/O space and locked transactions
         • PCI Express endpoints are not permitted to require the use of I/O space at runtime and must not use locked transactions.
           • negative impacts on system performance and robustness.
       • ■ PCIe/PCI bridge: Allows older PCI devices to be connected to PCIe-based systems.
     • PCIe interactions are defined using a protocol architecture. The PCIe protocol architecture encompasses the following layers
       • ■ Physical: Consists of the actual wires carrying the signals, as well as circuitry and logic to support ancillary features required in the transmission and receipt of the 1s and 0s
       • ■ Data link: Is responsible for reliable transmission and flow control. Data packets generated and consumed by the DLL are called Data Link Layer Packets (DLLPs)
       • ■ Transaction: Generates and consumes data packets used to implement load/ store data transfer mechanisms and also manages the flow control of those packets between the two components on a link. Data packets generated and consumed by the TL are called Transaction Layer Packets (TLPs).
     • PCIe Physical Layer
       • PCIe is a point-to-point architecture. Each PCIe port consists of a number of bidirectional lanes
       • A PCI port can provide 1, 4, 6, 16, or 32
       • Data are distributed to the four lanes 1 byte at a time
         • At each physical lane, data are buffered and processed 16 bytes (128 bits) at a time. Each block of 128 bits is encoded into a unique 130-bit codeword for transmission; this is referred to as 128b/130b encoding.
       • unlike QPI, PCIe does not use its clock line to synchronize the bit stream.
         • it is necessary for the receiver to be synchronized with the transmitter, so that the receiver knows when each bit begins and ends
         • drift between the clocks used for bit transmission and reception of the transmitter and receiver, errors may occur
         • To compensate for the possibility of drift, PCIe relies on the receiver synchronizing with the transmitter based on the transmitted signal. As with QPI, PCIe uses differential signaling over a pair of wires.
         • Synchronization can be achieved by the receiver looking for transitions in the data and synchronizing its clock to the transition
         • consider that with a long string of 1s or 0s using differential signaling, the output is a constant voltage over a long period of time. Under these circumstances, any drift between the clocks of transmitter and receiver will result in loss of synchronization between the two.
           • overcoming the problem of a long string of bits of one value is scrambling. Scrambling, which does not increase the number of bits to be transmitted, is a mapping technique that tends to make the data appear more random.
           • scrambling tends to spread out the number of transitions so that they appear at the receiver more uniformly spaced, which is good for synchronization.
           • For more discussion of scrambling, see Appendix E
       • PCIe 3.0, each group of 128 bits of input is mapped into a 130-bit block by adding a 2-bit block sync header.
         • The value of the header is 10 for a data block and 01 for what is called an ordered set block, which refers to a link-level information block
       • Figure 3.24 illustrates the use of scrambling and encoding. Data to be transmitted are fed into a scrambler. The scrambled output is then fed into a 128b/130b encoder, which buffers 128 bits and then maps the 128-bit block into a 130-bit block
         • At the receiver, a clock is synchronized to the incoming data to recover the bit stream. This then passes through a serial-to-parallel converter to produce a stream of 130-bit blocks.
         • each transmission of a block of data over a PCI link begins and ends with an 8-bit framing sequence intended to give the receiver time to synchronize with the incoming physical layer bit stream
     • PCIe Transaction Layer
       • The transaction layer (TL) receives read and write requests from the software above the TL and creates request packets for transmission to a destination via the link layer
       • Most transactions use a split transaction technique
         • A request packet is sent out by a source PCIe device, which then waits for a response, called a completion packet
         • Each packet has a unique identifier that enables completion packets to be directed to the correct originator.
         • split transaction technique, the completion is separated in time from the request, in contrast to a typical bus operation in which both sides of a transaction must be available to seize and use the bus.
           • Between the request and the completion, other PCIe traffic may use the link.
           • The TL packet format supports 32-bit memory addressing and extended 64-bit memory addressing
           • Packets also have attributes such as “no-snoop,”,“relaxedordering,” and “priority,” which may be used to optimally route these packets through the I/O subsystem.
           • TL supports four address spaces: ■ Memory: The memory space includes system main memory. It also includes PCIe I/O devices. Certain ranges of memory addresses map into I/O devices. ■ I/O: This address space is used for legacy PCI devices, with reserved memory address ranges used to address legacy I/O devices. ■ Configuration: This address space enables the TL to read/write configuration registers associated with I/O devices. ■ Message: This address space is for control signals related to interrupts, error handling, and power management.
       • tlp packet assembly PCIe transactions are conveyed using transaction layer packets
         • A TLP originates in the transaction layer of the sending device and terminates at the transaction layer of the receiving device.
         • Upper layer software sends to the TL the information needed for the TL to create the core of the TLP, which consists of the following fields: ■ Header: The header describes the type of packet and includes information needed by the receiver to process the packet, including any needed routing information. The internal header format is discussed subsequently. ■ Data: A data field of up to 4096 bytes may be included in the TLP. Some TLPs do not contain a data field. ■ ECRC: An optional end-to-end CRC field enables the destination TL layer to check for errors in the header and data portions of the TLP.
     • PCIe Data Link Layer
       • The purpose of the PCIe data link layer is to ensure reliable delivery of packets across the PCIe link. The DLL participates in the formation of TLPs and also transmits DLLPs.
       • data link layer packets
         • Data link layer packets originate at the data link layer of a transmitting device and terminate at the DLL of the device on the other end of the link
         • three important groups of DLLPs used in managing a link: flow control packets, power management packets, and TLP ACK and NAK packets.
         • Power management packets are used in managing power platform budgeting. Flow control packets regulate the rate at which TLPs and DLLPs can be transmitted across a link. The ACK and NAK packets are used in TLP processing
       • transaction layer packet processing
         • DLL adds two fields to the core of the TLP created by the TL
         • a 16-bit sequence number and a 32-bit link-layer CRC (LCRC)
         • the two fields added by the DLL are processed at each intermediate node on the way from source to destination.
         • When a TLP arrives at a device, the DLL strips off the sequence number and LCRC fields and checks the LCRC. There are two possibilities:
           • 1. If no errors are detected, the core portion of the TLP is handed up to the local transaction layer. If this receiving device is the intended destination, then the TL processes the TLP. Otherwise, the TL determines a route for the TLP and passes it back down to the DLL for transmission over the next link on the way to the destination. 2. If an error is detected, the DLL schedules an NAK DLL packet to return back to the remote transmitter. The TLP is eliminated. When the DLL transmits a TLP, it retains a copy of the TLP. If it receives an NAK for the TLP with this sequence number, it retransmits the TLP. When it receives an ACK, it discards the buffered TLP.

Images and files in the map