Chapter 11: Reliability Engineering (3705)

Chapter 11: Reliability Engineering (3705)

1. 11.1 Availability and reliability

   • Availability and reliability can both be expressed as probabilities. If the availability is 0.999, this means that, over some time period, the system is available for 99.9% of that time. If, on average, 2 inputs in every 1000 result in failures, then the reliability, expressed as a rate of occurrence of failure, is 0.002.
   • More precise definitions of availability and reliability are:
     • Reliability The probability of failure-free operation over a specified time, in a given environment, for a specific purpose.
       • System reliability is not an absolute value—it depends on where and how that system is used.
         • High use VS Low use makes a difference in how reliability is perceived
       • A technical definition of failure is behavior that does not conform to the system’s specification. However, there are two problems with this definition:
         • Software specifications are often incomplete or incorrect, and it is left to software engineers to interpret how the system should behave
         • No one except system developers reads software specification documents. Users may therefore anticipate that the software should behave in one way when the specification says something completely different.
         • Failure is therefore not something that can be objectively defined. Rather, it is a judgment made by users of a system
     • Availability The probability that a system, at a point in time, will be operational and able to deliver the requested services.
   • Most inputs do not lead to system failure.
   • Faults that affect the reliability of the system for one user may never show up under someone else’s mode of working.
   • The availability of a system does not just depend on the number of system failures, but also on the time needed to repair the faults that have caused the failure
     • Long downtime, seldomly
     • short downtime, often
     • 10 min downtime over a critical time window might be more disruptive than 1 hour downtime in another period
   • Depending on the system, reliability and dependability might be more or less important

2. 11.2 Reliability requirements

   • system dependability does not just depend on good engineering. It also requires attention to detail when the system requirements are derived and the specification of software requirements
   • Those dependability requirements are of two types:
     • Functional requirement
     • Non-functional requirements
   • overall reliability of a system depends on the hardware reliability, the software reliability, and the reliability of the system operators
   • Reliability metrics
     • Three metrics may be used to specify reliability and availability:
       • Probability of failure on demand (POFOD)
         • a failure will occur when a demand is made.
         • POFOD should be used in situations where a failure on demand can lead to a serious system failure
       • Rate of occurrence of failures (ROCOF)
         • probable number of system failures that are likely to be observed relative to a certain time period
         • MTTF is the average number of time units between observed system failures
           • If the absolute time between failures is important, you may specify the reliability as the mean time to failures (MTTF).
         • ROCOF should be used when demands on systems are made regularly rather than intermittently
       • Availability (AVAIL)
         • probability that a system will be operational when a demand is made for service
   • Non-functional reliability requirements
     • specifications of the required reliability and availability of a system using one of the reliability metrics (POFOD, ROCOF, or AVAIL)
     • Quantitative reliability specification is useful in a number of ways:
       • The process of deciding the required level of the reliability helps to clarify what stakeholders really need
       • It provides a basis for assessing when to stop testing a system
       • It is a means of assessing different design strategies intended to improve the reliability of a system
       • If a regulator has to approve a system before it goes into service
     • To avoid incurring excessive and unnecessary costs, it is important that you specify the reliability that you really need rather than simply choose a very high level of reliability for the whole system
       • three guidelines when specifying reliability requirements:
         • Specify the availability and reliability requirements for different types of failure
           • lower probability of high-cost failures than failures that don’t have serious consequences.
         • Specify the availability and reliability requirements for different types of system service
         • Think about whether high reliability is really required
   • Functional reliability specification
     • use a combination of fault-avoidance, fault-detection, and fault-tolerance techniques
     • specify how the system should provide fault avoidance, detection, and tolerance.
     • There are four types of functional reliability requirements:
       • Checking requirements
         • checks on inputs to the system to ensure that incorrect or out-of-range inputs are detected
       • Recovery requirements
         • geared to helping the system recover after a failure has occurred
       • Redundancy requirements
         • ensure that a single component failure does not lead to a complete loss of service
       • Process requirements
         • fault-avoidance requirements, which ensure that good practice is used in the development process
     • Functional reliability specification, therefore, involves analyzing the non-functional requirements, assessing the risks to reliability and specifying system functionality to address these risks.

3. 11.3 Fault-tolerant architectures

   • Fault tolerance is a runtime approach to dependability in which systems include mechanisms to continue in operation, even after a software or hardware fault has occurred and the system state is erroneous.
   • The simplest realization of a dependable architecture is in replicated servers, where two or more servers carry out the same task.
     • This replicated server approach is widely used for transaction processing systems where it is easy to maintain copies of transactions to be processed.
     • Replicated servers provide redundancy but not usually diversity.
   • I describe three architectural patterns that have been used in fault-tolerant systems
   • Protection systems
     • A protection system is a specialized system that is associated with some other system.
     • Figure 11.6 illustrates the relationship between a protection system and a controlled system. The protection system monitors both the controlled equipment and the environment. If a problem is detected, it issues commands to the actuators to shut down the system or invoke other protection mechanisms such as opening a pressurerelease valve. Notice that there are two sets of sensors. One set is used for normal system monitoring and the other specifically for the protection system. In the event of sensor failure, backups are in place that will allow the protection system to continue in operation. The system may also have redundant actuators.
     • A protection system only includes the critical functionality that is required to move the system from a potentially unsafe state to a safe state
     • The advantage of this architectural style is that protection system software can be much simpler than the software that is controlling the protected process.
     • The only function of the protection system is to monitor operation and to ensure that the system is brought to a safe state in the event of an emergency.
   • Self monitoring
     • A self-monitoring architecture (Figure 11.7) is a system architecture in which the system is designed to monitor its own operation and to take some action if a problem is detected.
     • Computations are carried out on separate channels, and the outputs of these computations are compared
     • To be effective in detecting both hardware and software faults, self-monitoring systems have to be designed so that:
       • The hardware used in each channel is diverse. In practice, this might mean that each channel uses a different processor type to carry out the required computations,
       • The software used in each channel is diverse.
     • In the Airbus flight control system, each of the flight control computers carries out the computations in parallel, using the same inputs. The outputs are connected to hardware filters that detect if the status indicates a fault and, if so, that the output from that computer is switched off
   • N-version programming
     • Self-monitoring architectures are examples of systems in which multiversion programming is used to provide software redundancy and diversity.
     • This notion of multiversion programming has been derived from hardware systems where the notion of triple modular redundancy (TMR) has been used for many years to build systems that are tolerant of hardware failures (Figure 11.9)
     • The output from each unit is passed to an output comparator that is usually implemented as a voting system. This system compares all of its inputs, and, if two or more are the same, then that value is output.
     • Using a common specification, the same software system is implemented by a number of teams. These versions are executed on separate computers. Their outputs are compared using a voting system, and inconsistent outputs or outputs that are not produced in time are rejected
     • this approach is only used in systems where it is impractical to provide a protection system that can guard against safety-critical failures.
   • Software diversity
     • All of the above fault-tolerant architectures rely on software diversity to achieve fault tolerance
     • The company that is procuring the system may include explicit diversity policies that are intended to maximize the differences between the system versions. For example:
       • By including requirements that different design methods should be used.
       • By stipulating that the programs should be implemented using different programming language
       • By requiring the use of different tools and development environments for the system
       • By requiring different algorithms to be used in some parts of the implementation.
       • Ideally, the diverse versions of the system should have no dependencies and so should fail in completely different ways

4. 11.4 Programming for reliability

   • A list of eight good practice guidelines is shown in Figure 11.11. They can be applied regardless of the particular programming language used
   • Guideline 1: Control the visibility of information in a program
     • control access to the variables and data structures that you use. Program components should only be allowed access to data that they need for their implementation
   • Guideline 2: Check all inputs for validity
     • You should therefore always check the validity of inputs as soon as they are read from the program’s operating environment.
       • Range checks
       • Size checks
       • Representation checks
       • Reasonableness checks
   • Guideline 3: Provide a handler for all exceptions
     • Exceptions may be caused by hardware or software conditions. When an exception occurs, it must be managed by the system.
   • Guideline 4: Minimize the use of error-prone constructs
     • For example, you should try to avoid using floating-point numbers because the precision of floating point numbers is limited by their hardware representation. Comparisons of very large or very small numbers are unreliable.
     • dynamic storage allocation where you explicitly manage storage in the program
   • Guideline 5: Provide restart capabilities
     • In all of these types of systems (long transaction from start to end of process), you should provide a restart capability that is based on keeping copies of data collected or generated during processing
   • Guideline 6: Check array bounds
     • the lack of bound checking leads to security vulnerabilities, such as buffer overflow, which I discuss in Chapter 13. More generally, it introduces a system vulnerability that can lead to system failure
   • Guideline 7: Include timeouts when calling external components
     • you should always include timeouts when calling external components. A timeout is an automatic assumption that a called component has failed and will not produce a response.
   • Guideline 8: Name all constants that represent real-world values
     • You should always include a section in your program in which you name all realworld constant values that are used.

5. 11.5 Reliability measurement

   • To assess the reliability of a system, you have to collect data about its operation. The data required may include:
     • The number of system failures given a number of requests for system services. This is used to measure the POFOD and applies irrespective of the time over which the demands are made.
     • The time or the number of transactions between system failures plus the total elapsed time or total number of transactions. This is used to measure ROCOF and MTTF.
     • The repair or restart time after a system failure that leads to loss of service. This is used in the measurement of availability.
   • There are four stages in the statistical testing process:
     • You start by studying existing systems of the same type to understand how these are used in practice.
       • Your aim is to define an operational profile to determine probabilities
     • You then construct a set of test data that reflect the operational profile
     • You test the system using these data and count the number and type of failures that occur.
     • After you have observed a statistically significant number of failures, you can compute the software reliability and work out the appropriate reliability metric value.
   • The principal difficulties that arise are due to:
     • Operational profile uncertainty
     • High costs of test data generation
     • Statistical uncertainty when high reliability is specified
     • Recognizing failure
   • Operational profiles
     • The operational profile of a software system reflects how it will be used in practice. It consists of a specification of classes of input and the probability of their occurrence.
     • Developing an accurate operational profile is certainly possible for some types of system, such as telecommunication systems, that have a standardized pattern of use.

6. Introduction

   • Brian Randell, a pioneer researcher in software reliability, defined a fault–error–failure model (Randell 2000) based on the notion that human errors cause faults; faults lead to errors, and errors lead to system failures. He defined these terms precisely:
     • Human error or mistake
     • System fault
     • System error
     • System failure
   • System faults do not necessarily result in system errors, and system errors do not necessarily result in system failures:
     • Not all code in a program is executed.
     • Errors are transient. A state variable may have an incorrect value caused by the execution of faulty code. However, before this is accessed and causes a system failure, some other system input may be processed that resets the state to a valid value
     • The system may include fault detection and protection mechanisms.
   • The distinction between faults, errors, and failures leads to three complementary approaches that are used to improve the reliability of a system:
     • Fault avoidance
       • avoid design and programming errors and so minimize the number of faults introduced into the system
     • Fault detection and correction
       • Verification and validation processes
     • Fault tolerance
   • The cost of finding and removing the remaining faults in a software system rises exponentially as program faults are discovered and removed
     • accepting faults is that, if and when the system fails, it is cheaper to pay for the consequences of failure than it would be to discover and remove the faults before system delivery
     • The social and political acceptability of system failure must also be taken into account.

Bilder und Dateien der Mindmap