Systematic Capability: The Third Leg of a SIL Claim
A SIL claim has three legs. The third isn’t always proven.
1. Two legs shown, one asserted
Not long ago I was the functional safety assessor on an IEC 61508 FSA for a simple safety instrumented system (SIS) with a SIL 2 target. A pressure transmitter (PT) detected the hazardous condition, and the safety function closed an HVAC damper to the safe state.
The PT was not certified to IEC 61508, but the SIL verification report showed the PFD was within target and the architectural constraints were met for SIL 2. When it came to the systematic capability of the PT, the argument went something like this:
- “The PT is a Type A device with no software, so the SC assessment is easier.” True, only the hardware needs assessing, but there is nothing easy about demonstrating SC.
- “The PT will be installed and operated within the manufacturer’s instructions and limits.” This IS genuinely important to SC.
- “The manufacturer is a global market leader and operates an ISO 9001 quality management system, so the likelihood of systematic faults is low.”
- “Therefore, the PT has been assessed as SC 2.”
This quantum leap was too much for me.
Under IEC 61508, an uncertified device leaves two ways to show SC: the manufacturer’s evidence that it was developed to the standard (Route 1S), or proven in use (Route 2S), with documented operating experience and failure data. The project offered neither.
I marked the requirement as non-compliant and recommended that SC be demonstrated properly.
That FSA is the point of this article. Two legs of the SIL were shown; the third was asserted. And it happens more often than it should.
2. A SIL claim has three legs
An FSA has to confirm the SIL in three areas:
- Random hardware failure (PFD/PFH)
- Architectural constraints (HFT and SFF, or minimum HFT)
- Systematic safety integrity, shown through the systematic capability (SC) of each element
IEC 61508-2 7.4.2.2 requires all three to be demonstrated. I theorise that because engineers are comfortable with calculations and tables, the quantitative leg, random hardware failure, gets done. Architectural constraints are semi-quantitative but not too difficult: the standard gives us an easy table that can usually be checked against the manufacturer’s data. Systematic safety integrity is the weak leg of the stool (Figure 1), because it needs a qualitative assessment that is both difficult and expensive to demonstrate.
“If you can’t show the third leg, you haven’t shown the SIL.”
3. Systematic faults are real, not a theory
Systematic faults are built in rather than worn out. They affect hardware and software, and they recur under the same conditions: same conditions, same failure. Identical redundant channels don’t help, because the fault is now common cause.
So why worry about them? Three sources show the size of the problem.
Systems. HSE’s “Out of Control” analysed 34 control system incidents and traced the primary cause of each to a phase of the lifecycle (Figure 2).
HSE concluded that 60% of the failures were built in before the system went into service. The sample is small, but every one of those causes is systematic.
Hardware. SINTEF PDS field data from Norwegian offshore safety systems, as reported by Dearden (2015), indicates that random hardware failures account for only 30% to 50% of dangerous undetected failures in field sensors and final elements. Where do the other half come from? We can guess they are mostly systematic.
Software. Software faults are systematic by nature: software doesn’t wear out. Lutz analysed the safety-related software faults found in testing on the Voyager and Galileo space probes. About three-quarters began in requirements or in communication between teams; only about a quarter in design and implementation.
The lesson is that systematic faults are a large share of real failures, and avoiding them means managing the work of the teams building safety systems. Failing to demonstrate systematic safety integrity is a serious gap in any SIL claim.
4. Three routes, none of them a calculation
IEC 61508-2 allows an element’s systematic capability to be demonstrated by one of three routes:
Route 1S: compliant development. The element is developed to the standard’s requirements for avoiding and controlling systematic faults, under functional safety management. This is the normal route for new products and what most certificates rest on.
Route 2S: proven in use. A restricted, clearly specified function with documented operating experience in the same or similar conditions, evidence that the dangerous failure rate hasn’t been exceeded, and an analysis of any differences. It needs an effective failure reporting system and significant operating hours.
Route 3S: assessment of non-compliant development. This is for pre-existing software only, which is brought up to the standard required for the SC claimed, sometimes by reverse engineering.
The process sector adds a fourth path: IEC 61511-1 lets the end user justify a device by prior use, instead of an SC claim.
Each route produces evidence and an argument, not a figure. IEC 61508-4 says so plainly: “systematic safety integrity cannot usually be quantified”.
For Route 1S, IEC 61508-2 lists techniques and measures for each SIL. Measures to avoid systematic faults (Tables B.1–B.5) are applied through the safety lifecycle. Measures to control them (Tables A.15–A.17) are features built into the design, so that the system tolerates residual design faults, environmental stress and operator mistakes. Each entry is graded by SIL, from ‘mandatory’ to ‘not recommended’ (see the extracts in Figure 4, at the end). IEC 61508-3 does the same for software (Annexes A and B). As IEC 61508-2 itself warns, following the tables “does not guarantee by itself the required safety integrity”: judgement is still needed.
5. What a proper SC case looks like: a worked example
What should the argument for that pressure transmitter have looked like? Assume the design is settled and nobody wants to redesign the SIF. Figure 3 shows the order I’d try.
Start with the manufacturer. Ask whether the transmitter was developed under an assessed functional safety management system, and whether they can provide the IEC 61508 assessment behind an SC claim for your exact device revision. A bare statement isn’t enough: the assessment has to show how SC was judged, by someone suitably independent. If they can, that is by far the cheapest fix.
If not, does IEC 61511 apply? Under IEC 61508 alone, the end user’s route is proven in use (Route 2S), which is statistical and rarely practical. Under IEC 61511, the end user has a further option: a prior-use case, which justifies the device from operating experience instead of an SC claim. That means fixing the exact model and device revision, defining how it is used, gathering evidence from the same model in similar service (numbers installed, years in service, proof test results, and every failure identified as random or systematic), and assessing the manufacturer’s quality and configuration management. The ISO 9001 certificate belongs here, but as one input, not the conclusion. A competent and independent assessor then judges whether the evidence is enough.
In practice the evidence may not exist, or the manufacturer may be unwilling to provide it. To sell one or two transmitters, weeks of administrative effort may not be worth their while.
If the evidence isn’t there, change the device. Swapping to an equivalent transmitter with a certified SC is a device change, not a complete redesign. The SIL itself isn’t yours to change: it is the target handed to you from the risk assessment. If no suitable device exists, return to the owner of that assessment, who can decide whether other risk reduction changes the target. What you can’t do is use the PFD margin to make up for missing SC. Field data puts some systematic failures into the PFD; manufacturers’ data largely leaves them out. Neither is a substitute for the SC case.
Remember: a certificate without an SC number doesn’t always mean no systematic assessment. Certificates issued under the 2000 edition of IEC 61508 predate the term, yet some state a “full assessment”. Read the report to the certificate and the safety manual, not just the certificate.
The effort isn’t comparable. A PFD calculation for a SIF like this takes an hour or two. A proper SC case, from manufacturer evidence or prior use, can take days of work and, where records are poor, weeks of chasing them.
6. Why the third leg gets skipped
Scale the example up to a whole SIS with many devices and four things get in the way.
The evidence is scattered. One engineer can put a PFD calculation together at a desk. Evidence of systematic safety integrity sits with the manufacturers, integrator, application programmer, installer and operator, and builds up across the whole lifecycle. Nobody holds all of it, and the assessor usually arrives late.
It needs judgement, not a tool. There is no SC calculator. Someone competent has to decide whether the evidence is enough, and that judgement is harder to buy, and harder to defend, than a number from a spreadsheet.
The application program is a separate story. A safety PLC certificate covers the platform, not the application program written on it for each project, and that project-specific layer is where Lutz found most of the dangerous faults.
Time, money and missing records. On legacy and brownfield systems the records are often gone, and as we’ve seen, manufacturers may have little economic incentive to help.
The result is predictable. Projects present the legs they can show. Where there is no certificate, but the PFD is within target and the HFT meets the architectural constraints, the SC assessment gets glossed over, as if two legs were enough to stand on. My opening example is a typical shortcoming: a sound calculation, a sound architecture, but an assertion where the SC argument should be.
7. Five common misreadings
Even where SC is addressed, a few points are regularly misread.
- SC depends on how a device is used. The SC is valid only when the element is applied in line with its safety manual (IEC 61508-4 3.5.9).
- Software has no SIL of its own. Demonstration of software systematic integrity depends on the language: configuration of a fixed program language (FPL) device, a safety PLC programmed in a limited variability language (LVL) under IEC 61511 clause 12, or full variability language (FVL) software under IEC 61508-3.
- Redundancy has limits. Combining elements can raise SC by one step at most, once only, and only where a common cause failure analysis justifies their independence (IEC 61508-2 7.4.3). Two identical devices configured from the same wrong datasheet share the same fault, and safety manuals often rule out devices with software in redundant channels.
- Prior use is mainly about systematic faults. Its main intent is evidence that dangerous systematic faults are low enough (IEC 61511-1 11.5.3.1). An approved device list can support it, but only if the list is actively maintained, with devices added on evidence and removed on poor performance.
- Systematic capability is not systematic safety integrity. SC belongs to an element (device) and usually comes from the manufacturer’s assessment. Systematic safety integrity relates to the system, and depends on how the elements are combined, programmed, installed and managed. They are different assessments.
8. Make the third leg a deliverable
All of this is achievable for a well-managed project, provided the third leg is treated as a deliverable from the start, not a question for the assessor at the end. My checklist:
- Use SC-certified devices wherever you can. In my view this is often the best course of action, because the hard work has already been done for you. Before ordering, check that the certificate, assessment report and safety manual cover your function and device revision.
- Treat the application program as a deliverable in its own right, traceable from the SRS through to the logic and its testing and developed with its own lifecycle.
- Write a short SC argument for each SIF: which route, what evidence, and who judged it.
- Plan independent review early enough for findings to avoid significant rework.
- Build real prior-use cases for legacy systems (or, under IEC 61508, proven-in-use cases) from site failure records; or replace the devices that can’t be justified.
9. No third leg, no SIL
A SIL claim stands on three legs. Two are shown by calculation and a table. The third needs evidence and judgement, so it’s the one that sometimes gets skipped. Yet systematic faults are where many real failures start.
If you can’t show the third leg, you haven’t shown the SIL.
Where does your SC evidence usually go wrong? I’d be interested to hear in the comments.
This is the first in a short series. Coming soon: how the application program achieves systematic safety integrity, how it is shown for software by language type (FPL, LVL and FVL), and how device and system assessments differ.
Mike Ireland is an independent functional safety assessor and safety-PLC software verifier at 1oo2 Limited, and a member of BSI GEL/65/1 and the IEC 61511 application programming team. More at www.1outof2.co.uk.
Appendix: what the IEC 61508 tables look like
Two of the techniques and measures tables referred to in section 4, as examples of how each entry is graded by SIL.
References
- HSE, Out of Control (HSG238), 2nd ed., 2003. www.hse.gov.uk/pubns/books/hsg238.htm
- Lutz, R. R. (1993), "Analyzing software requirements errors in safety-critical, embedded systems", Proc. IEEE International Symposium on Requirements Engineering. NASA JPL.
- Dearden, H. T. (2015), "Systematic Failures, Systematic Capability and Prior-Use", Measurement and Control. doi.org/10.1177/0020294015572665
- IEC 61508:2010, parts 1 to 7.
- IEC 61511-1:2016.
Acronyms: SIL safety integrity level · PFD/PFH probability of failure on demand / average frequency of dangerous failure per hour · SC systematic capability · FSA functional safety assessment · HFT hardware fault tolerance · SFF safe failure fraction · SIS safety instrumented system · SIF safety instrumented function · SRS safety requirements specification · PT pressure transmitter · HVAC heating, ventilation and air conditioning · PLC programmable logic controller · FPL/LVL/FVL fixed program / limited variability / full variability language · HSE Health and Safety Executive · HSG238 HSE "Out of Control" · PDS SINTEF's SIS reliability method.