System software reliability

Contents 1 Introduction ...................................................................................................... 1 1.1 The Need for System Software Reliability ................................................ 1 1.2 Software-related Problems ........................................................................3 1.3 Software Reliability Engineering .............................................................. 5 1.4 Future Problems in the Twenty-first Century ............................................ 5 1.5 Further Reading .........................................................................................6 1.6 Problems ....................................................................................................7 2 System Reliability Concepts............................................................................. 9 2.1 Reliability Measures ................................................................................10 2.2 Common Distribution Functions ............................................................. 16 2.3 A Generalized Systemability Function .................................................... 32 2.3.1 Systemability Definition............................................................... 33 2.3.2 Systemability Calculations ........................................................... 33 2.4 System Reliability with Multiple Failure Modes..................................... 41 2.4.1 Reliability Calculations ................................................................. 42 2.4.2 An Application of Systems with Multiple Failure Modes .............. 48 2.5 Markov Processes ....................................................................................50 2.6 Counting Processes .................................................................................62 2.6.1 Poisson Processes ........................................................................... 63 2.6.2 Renewal Processes.......................................................................... 64 2.6.3 Quasi-renewal Processes ................................................................ 66 2.6.4 Non-homogeneous Poisson Processes ............................................ 69 2.7 Further Reading .......................................................................................71 2.8 Problems ..................................................................................................71 3 Theory of Estimation...................................................................................... 77 3.1 Point Estimation ......................................................................................77 3.2 Maximum Likelihood Estimation Method .............................................. 79 3.3 Maximum Likelihood Estimation with Censored Data ........................... 86 3.3.1 Parameter Estimate with Multiple-censored Data .......................... 86 xii Contents 3.3.2 Confidence Intervals of Estimates .................................................. 88 3.3.3 Applications.................................................................................... 89 3.4 Statistical Change-point Estimation Methods.......................................... 91 3.4.1 Application: A Software Model with a Change Point .................... 95 3.5 Goodness of Fit Techniques .................................................................... 96 3.5.1 Chi-squared Test............................................................................. 96 3.5.2 Kolmogorov-Smirnov d Test .......................................................... 98 3.6 Least Squared Estimation ........................................................................98 3.7 Interval Estimation ................................................................................100 3.7.1 Confidence Intervals for the Normal Parameters.......................... 100 3.7.2 Confidence Intervals for the Exponential Parameters................... 102 3.7.3 Confidence Intervals for the Binomial Parameters ....................... 104 3.7.4 Confidence Intervals for the Poisson Parameters ......................... 106 3.8 Non-parametric Tolerance Limits.......................................................... 106 3.9 Sequential Sampling .............................................................................. 107 3.10 Bayesian Methods..................................................................................113 3.11 Further Reading .....................................................................................118 3.12 Problems ................................................................................................119 4 Software Development Lifecycle and Data Analysis ................................ 121 4.1 Introduction ...........................................................................................121 4.2 Software vs Hardware Reliability.......................................................... 122 4.3 Software Reliability and Testing Concepts............................................ 124 4.4 Software Lifecycle ................................................................................127 4.5 Software Development Process and its Applications ............................ 132 4.5.1 Analytic Hierarchy Process .......................................................... 133 4.5.2 Evaluation of Software Development Process.............................. 133 4.6 Software Verification and Validation .................................................... 134 4.7 Data Analysis......................................................................................... 135 4.8 Failure Data Sets.................................................................................... 136 4.9 Further Reading .....................................................................................150 4.10 Problems ................................................................................................150 5 Software Reliability Modeling..................................................................... 153 5.1 Introduction ...........................................................................................153 5.2 Halstead’s Software Metric ................................................................... 154 5.3 McCabe’s Cyclomatic Complexity Metric ............................................ 157 5.4 Error Seeding Models............................................................................ 159 5.5 Failure Rate Models...............................................................................164 5.6 Curve Fitting Models............................................................................. 169 5.7 Reliability Growth Models .................................................................... 171 5.8 Markov Structure Models ...................................................................... 172 5.9 Time Series Models ............................................................................... 174 5.10 Non-homogeneous Poisson Process Models ......................................... 175 5.11 Further Reading .....................................................................................176 5.12 Problems ................................................................................................176 Contents xiii 6 Imperfect-debugging Models....................................................................... 179 6.1 Introduction ...........................................................................................179 6.2 Parameter Estimation.............................................................................180 6.3 Model Selection.....................................................................................181 6.4 NHPP Exponential Models.................................................................... 183 6.5 NHPP S-shaped Models ........................................................................ 188 6.6 NHPP Imperfect Debugging Models.....................................................192 6.7 NHPP Imperfect Debugging S-shaped Models ..................................... 194 6.7.1 A Generalized Imperfect-debugging Fault-detection Model ........ 195 6.8 Applications...........................................................................................203 6.9 Imperfect Debugging vs Perfect Debugging.......................................... 211 6.10 Mean Time Between Failures for NHPP ............................................... 213 6.11 Further Reading .....................................................................................216 6.12 Problems ................................................................................................216 7 Testing Coverage and Removal Models ..................................................... 219 7.1 Introduction ...........................................................................................219 7.2 Testing Coverage Models ...................................................................... 219 7.3 Testing Coverage and Imperfect Debugging ......................................... 222 7.4 Fault Removal Efficiency Model........................................................... 224 7.5 Model Implementations .........................................................................231 7.6 Imperfect Debugging Model with Multiple Failure Types .................... 247 7.6.1 A Constant Fault Detection Rate .................................................. 248 7.6.2 Fault Detection Time-dependent Rate .......................................... 251 7.7 Further Reading .....................................................................................255 7.8 Problems ................................................................................................256 8 Software Reliability Models with Environmental Factors........................ 257 8.1 Introduction ...........................................................................................257 8.2 Data Analysis.........................................................................................257 8.2.1 Survey Analysis ............................................................................ 258 8.2.2 Statistical Methods ....................................................................... 261 8.3 Exploratory Analysis of Environmental Factors.................................... 263 8.4 Further Exploratory Analysis ................................................................266 8.5 A Generalized Model with Environmental Factors ............................... 272 8.6 Environmental Parameter Estimation .................................................... 275 8.7 Enhanced Proportional Hazard Jelinski-Moranda (EPJM) Model......... 276 8.8 Applications...........................................................................................279 8.9 Further Reading .....................................................................................292 8.10 Problems ................................................................................................292 9 Calibrating Software Reliability Models .................................................... 293 9.1 Introduction ...........................................................................................293 9.2 Calibration Factor Approach .................................................................294 9.3 Model Application.................................................................................295 9.4 Calibrating Models with Random Field Environments ......................... 296 9.4.1 A Generalized Random Field Environmental Model ................... 298 xiv Contents 9.4.2 RFE Reliability Models ................................................................ 301 9.4.3 Applications.................................................................................. 303 9.5 Further Reading .....................................................................................313 9.6 Problems ................................................................................................313 10 Optimal Release Policies .............................................................................. 315 10.1 Introduction ...........................................................................................315 10.2 A Software Cost Model with Risk Factor.............................................. 316 10.3 Cost Model with Testing Coverage ....................................................... 319 10.4 A Generalized Software Cost Model..................................................... 323 10.5 Cost Model with Multiple Failure Errors .............................................. 326 10.6 Gain Model with Random Field Environments ..................................... 332 10.6.1 Model Formulation ..................................................................... 334 10.6.2 Applications................................................................................ 338 10.7 Other Cost Models................................................................................. 342 10.8 Further Reading .....................................................................................343 10.9 Problems ................................................................................................343 11 Complex Fault-tolerant System Reliability Modeling............................... 347 11.1 Introduction ...........................................................................................347 11.2 Basic Fault-tolerant Software Techniques ............................................. 348 11.2.1 Recovery Block Scheme............................................................. 349 11.2.2 N-version Programming ............................................................. 350 11.3 Other Advanced Techniques..................................................................352 11.3.1 Self-checking Duplex Scheme.................................................... 352 11.3.2 Hybrid Fault-tolerant Scheme..................................................... 353 11.3.3 Reduction of Common-cause Failures........................................ 355 11.4 Triple-version Programming Model with Common Failures ................ 357 11.4.1 Modeling Assumptions ............................................................... 360 11.4.2 TVP Reliability Function............................................................ 364 11.4.3 Numerical Example .................................................................... 366 11.5 Complex-system Reliability Modeling..................................................375 11.5.1 System Considerations................................................................ 375 11.5.2 Reliability Modeling................................................................... 377 11.6 Application Example .............................................................................383 11.7 Further Reading .....................................................................................385 11.8 Problems ................................................................................................386 Appendix 1: Distribution Tables....................................................................... 389 Appendix 2: Laplace Transform....................................................................... 395 Appendix 3: Survey of Factors that Affect Software Reliability................... 399 References ........................................................................................................... 407 Glossary............................................................................................................... 423 Solutions to Selected Problems.......................................................................... 429 Index ...............................................................................................................437