Computerised systems IT operations have concentrated time and energy in seeking to assure data integrity and security. This paper seeks to shift some of that focus onto the software component of computerised systems.
Data must be safe from a range of threats. Users accidentally enter erroneous data; staff seek to change data to fabricate results; cybercriminals seek Personal Identifiable Information (PII) to steal patient identities, and hackers seek data to share secrets. All of these threats exploit vulnerabilities latent within computerised systems and in the various activities used to operate and manage them. The need for trustworthy and reliable computerised systems is apparent given the regulations and guidance focus on integrity and security (1,2,3,4,5).
Organizations define policies, procedures and technical controls to prevent unauthorized access, alteration, and theft of data. Examples include software monitoring, hardware redundancy, data backup, storage encryption, and administrative procedures (6).
Cyberattacks arguably pose the biggest threat to data integrity and are a daily occurrence. As of June 15, 2020, there were 8878 new Common Vulnerability and Exposures received by the NIST(7). Cyberattacks are non-discriminate, affecting technologies across business verticals including healthcare; examples include:
- Ransomware attacks accounted for 50% of all reported security breaches by healthcare organisations between October 2015 and September 2016(8).
- Bizmatics Inc’s Electronic Health Record (EHR) software was compromised in 2015 by a malware attack; 264,208 of their customer patients were notified of a breach(9).
- UK NHS attack in 2017 caused operations to be cancelled, ambulances diverted, and patient records made unavailable (10).
- In 2013, doctor disabled WiFi in former US Vice President’s pacemaker to reduce the risk of hacking (11).
- In June 2017, Merck was targeted by a ransom ware hack (“NotPetya”) that shut down computer systems which led to concerns in the US government (12).
- In August 2019, biometric company Suprema, (used by banks and UK police) left 1 million user fingerprints, facial recognition, and passwords on an unencrypted database. 44
- In January 2020, medical testing company Lifelabs data breach affected an estimated 15 million users 45
The increase in data issues is reflected in the attention of the regulators. WHO has reported that the number of data related observations have been increasing during inspections(13). Data related observations are evident in FDA 483 Warning Letters; for example, failure to prevent unauthorised data access (14); failure to provide adequate controls to prevent data omission (15); and partially testing that network ports would not open with an unauthorised interface (16).
Given the maturity of IT activities that exist to “harden” computerized system and the regulatory focus, why is there an increase in data issues? The answer resides in software.
Research opinion is not kind to software: 75% of security breaches occur at the software layer (17), and over 92% of reported vulnerabilities exist in software and not in networks (18). Despite the increased spending on network defenses, attacks continue. Outer network defenses are “cracked” to access the “soft” inner software (code); for example: to change clinical trial data; switch off audit trails; deny access to records for exploitation; access to PII, and for identity theft; birth certificates, credit cards, and social security cards cost $1 on the dark web (19,20)
The reason for vulnerable software may be due to computerised systems being used in a way that were never considered. For example, connecting a simple temperature probe with limited code to the network as part of a smart hospital bed system - a weakness in one connected node of the system can easily compromise other parts of a system. US hospital beds are estimated to have between 10 and 15 interconnected devices (21). Examples of such a compromise include a 11-year boy accessed conference attendees mobile phone contact information using a blue tooth enabled Teddy Bear and $35 computer (22); a robust casino system was compromised by accessing a networked fish tank(23).
Vulnerabilities may not be adequately considered during the building of computerised systems. Much of the complex tasks involved in building software have been hidden in 3rd party code libraries which are “copied and pasted” with developers not afforded time to consider their integrity(24).
Given that the average length of time that cyberattacks can go undetected before containment is 279 days (25), the approach to building software demands an alternative look.
A CHANGE OF MINDSET
People who write software focus on solutions and how things should work. This positive outlook is necessary, but more is required when seeking to reduce latent vulnerabilities in software. The use of critical thinking is a cornerstone for building robust software. Critical thinking is to adopt an attitude that is open-minded, skeptical, and seeks to question all underlying assumptions (26). For software, this involves expecting anomalies and seeking to mitigate them, seeking to understand varying perspectives, challenging the status quo, and identifying alternative approaches to problems (27). This is a defensive mindset.
The Software Lifecycle (SLC) is the first line of defense as it denotes a framework of human and technical processes to plan, prioritise, design, develop, verify, deploy, accept, maintain, and retire computerised systems.
The defense practitioner understands that a gap or lack of rigor in one SLC process will detrimentally impact the success of subsequent activities. Efficient verification is key. Inspections and Reviews are simple concepts, but when performed correctly, they are the most efficient at removing defects in software by exceeding 60% efficiency in defect removals (28).
The defensive approach focuses on how the data must be complete, consistent, and accurate(29) by thinking about issue origins within the following key SLC areas:
- Quality approach origin of data problems arise from poor project management, insufficient requirements reviews, inadequate testing, and insufficient IT administration processes.
- Functional requirements origins of data problems are evident in design assumptions, software defects, implementation language deficiencies, and operational misuse.
- Non-functional requirements origins of data vulnerabilities materialize in: electro-magnetic faults, material fatigue, power outages, storage retrieval, performance bottlenecks, or malicious system access.
The defensive approach seeks to prevent defects from being designed and built into the computerized system and starts with requirements. Requirements elicitation is the process of studying user needs to arrive at a system definition and quantify the needs of a system. The defensive mentality seeks to eliminate errors at the requirements stage where it is cheaper to correct and where it reduces the scope of false assumptions being built into the software (28).
Checklists (30) are useful aides to help focus on requirements completeness. For example, when faced with data storage, consider questions such as:
- How will the data be stored (non-functionality)?
- What will the size of the data store be?
- What will happen during processing if the data store size limit is reached?
- What happens if the data store connection is broken?
- How will the user or IT administrators know?
- How will the data in transit not become lost or corrupt?
- Where will the encryption keys be stored?
Rudyard Kipling’s 5 W’s: What, Who, Why, What, Where and How (31) is a root cause analysis approach to be applied to each requirement during inspection that will help prevent defects from being designed and built into the solution. Using the 5 W’s from a negative viewpoint (e.g. what happens if the data store is unavailable) can help reduce the risk of data integrity and security issues.
Each requirement should undergo robust risk analysis, and this should be an iterative process throughout the SLC (30, 32). An example approach is summarized as follows:
- Assess the business process and associated data according to risk:
- Risks and vulnerabilities present in each requirement
- Risk and vulnerabilities between components of the system
- Business consequence of failure
- Probability of risks materializing.
- Devise technical and/or procedural mitigation controls to reduce or eliminate the risks.
- Establish the tasks to verify that the mitigation controls are implemented; e.g., design reviews, source code reviews, unit testing, creation of automated test harnesses to verify component interfaces, security testing of encrypted traffic between components, stress testing of system memory, and so on.
- Quantify the cost of the mitigation controls in terms of time, effort, and business criticality in order to secure funding for verification activities.
- Review the requirements, risks, and mitigations; concatenate activities together where feasible such as test scenarios.
The defensive approach embraces designing acceptance test scenarios. The task of designing a test is a mental activity that will raise additional questions on the feasibility, reduce assumptions, and gaps of requirements. The inclusion of independent testers provides an alternative perspective of the system.
Design is an iterative, multi-layered approach that describes how the system will be built. FDA recognises that design is required to satisfy the process and to prevent data loss (33). It is an essential element of a defensive approach as it seeks to reduce complexity (34). Reducing complexity is an important feature of safety and mission-critical software (35). The defensive practitioner will apply a range of techniques in order to reduce the risk of error.
The simplest design methodology is “Divide and Conquer” (35). The system is iteratively decomposed into subsystems and components until distinct functions are identified to contain the business logic. The data control flow and defined data are then aligned to components. At each stage, the defensive mentality is applied: what should happen if data are changed at this juncture or what roles cannot access the system at this point? This approach helps to enhance maintenance, reduce complexity, and help ensure highly cohesive – loosely coupled code (reduce the connections within code) and support code reuse (35).
Key activities that a defensive practitioner considers include:
- Creating an architectural overview of the proposed system to illustrate how components will relate to each other -- what they can and cannot do. A good overview will include alternatives with reasons for selecting the finalized approach (35).
- Leveraging suitable aspects from design methodologies, such as Unified Modelling Language (36), help define the different states that system components can assume, the data that can be used, and communication between components.
- Identifying non-permitted use cases and how the system behaves should an unexpected scenario occur are useful tools.
- Performing data flow analysis to focus on data ingestion, processing, and storage within system; who and what can access the data; data transfer, and how data relate to other data. Focus centers on defining the data and how it is to be managed within the code such as data type (integer, floating point value, string or array of integers), data file descriptions, and database tables. It is useful to define checklists to verify data from the terminal, other computerized systems, or from data stores.
- Conducting system boundary analysis to examine the interaction between software components and the software with its environment to try and identify vulnerabilities between entities and so help elicit problem conditions. For example
- Designing a State Transition Machine to ensure that a data item cannot be created or amended until a certain sequence of events have occurred successfully
- Identifying verification routines to check the processing logic
- How to communicate data between systems such as the specific content of data message, message sequencing, or failed message delivery
- Designing the network communications bandwidth to be able to process essential data transfer during high peak usage.
- Threat modelling (27) is the specific risk management practice of thinking about threats to data integrity and security (Vulnerability Risk Management). Items that may arise from a threat model include:
- Security protocols to ensure that external data is coming from the expected source. For example, using trusted handshakes (digital signatures, certificates) between 2 systems
- Connecting the computerized system to the managed company IT network to benefit from the shared supporting processes provided by the IT team; for example, disk replication.
The implementation stage translates the rules and constraints defined in the Design Stage into executable machine code. This is performed by writing commands in a software programming language (code) that is in turn compiled into machine code. The defensive mentality seeks to enforce design and reduce the scope of potential vulnerability issues.
There were over 200 programming languages in active use in 2008 (28), each with their own specific range of functions. It is important to understand the specific limitations of the programming language in use. Defensive programming will devise software engineering constructs to make up for any language deficits (35).
The defensive developer will trust other code but will seek to verify that it performs as intended. The adoption of a protective mindset safeguards that other code does not harm the data or the underlying business process flow. The defensive approach assumes nothing and checks everything. Examples of this mindset in action includes:
- Pre-function checks, for example, data value range is in the correct sequence, use of white, grey or black data check lists, and so forth
- Initializing variables to prevent reading of previously stored-in-memory data
- Setting all return values to negative before processing
- Implementing self-diagnosing code, such as trace statements that write to logs to show what code was executed and when
- Hiding information such as algorithms within modules to minimize the impact of ripple effects to other code when logic needs to be changed (37)
- Adding robust error handling to log an error occurrence that facilitates root cause analysis, such as pass the error statement and value back to a calling component or to gracefully shut down the system (35)
- Post-function checks; for example, return memory to the processor when it is no longer needed, or freeing up connections to a database at the end of processing.
Defensive coding standards include items such as the programming language aspects related to security requirements, software language weaknesses, standard encryption algorithms, interface definitions, how to perform debugging, performance considerations, and how to perform robust and effective unit testing (35).
Defensive Unit testing seeks to examine that the design constraints and defensive coding standards have been followed such as reconciliation routines or record count iterations. Unit tests should also challenge vulnerability constraints such as exception handling, writing to trace files, audit trails, malformed data handling and incorrect loop counters.
The defensive developer embraces code reviews as they are very effective at identifying vulnerabilities and errors such as missing validation checks, inaccurate logic, and concurrency issues. Code reviews are excellent at training junior coders in the practice of defensive programming (26).
The defensive practitioner will make use of auxiliary tools in order to assess confidence in the capability of the software code:
- Static analysis tools to check for code violation, duplicated code, concurrency problems, exposed data definitions, buffer overflows (38)
- Use of code coverage tools to discover which code has not been exercised during unit testing and which test cases are redundant
- Automated compiler checks to check for memory constraints and leaks.
There are a wide range of techniques and tools available for the defensive approach. The simplest is also the most important: All input data should be untrusted; all data must be verified (39).
NIST estimated that a better testing infrastructure would save more than $22 billion per annum (40). An objective of the defensive approach is to reduce the scope of latent vulnerabilities in production that lead to data integrity and security issues. A defensive testing framework involves testing throughout the SLC, from requirements inspections and repeatedly through unit testing and on to specialised non-functional testing (Test Early - Test Often). Regular and repetitive testing seeks to identify defects as early as possible to reduce the risk of fixing ever increasing connected logic at later phases of development.
Defensive testers need to understand what the software will do and how the software should behave. They will be involved from the outset of the project and engage in the requirements elicitation process.
A defensive test strategy will confirm that the system does what it is supposed to do:
- Test for each aspect of the design to ensure that it has been correctly implemented
- Test for the data flow within the system
- Check the capability of the logic
- Ensure that the correct data is stored in the expected location in the data store. The data store should be checked to categorically demonstrate.
The defensive strategy will then focus on challenging the system so that it does not do what it is not supposed to do (not work as intended (41)).
It is when things that were not expected to happen that vulnerabilities will surface. Testing needs to reflect this. The defensive approach is to make the system behave in unpredictable ways and watch how it behaves. For example, challenge the system to ascertain if it can:
- Accept invalid data
- Perform the incorrect sequence of actions
- Deal with missing data values
- Change the state labels in the storage repository to force an error event from the data store into the business logic
- Maximize the connection points to the data repository
- Change the data input file type and file
- Apply logically incorrect data
- Amend the database integrity management constraints to force errors into the logic.
Audit trails would be reviewed during these test conditions to ensure audit trail integrity.
The defensive approach places a strong focus on nonfunctional testing. Tools are typically built or bought to force unintended changes in the system. Examples include:
- Maximizing processing capacity, system memory, and hard disk capacity to try and see how the system copes with hardware stress. Does the system deal with these scenarios correctly, or are the data corrupted?
- Performance testing will check how the system constraints can deal with ever increasing processing demands
- Role-based testing will check how the processing and data will cater for changes in user permissions. It will also need to challenge system permissions and underlying system processing. It may influence the hardening of the computerized system architecture and thereby switch off unneeded system accounts, processes, and communication ports.
- Penetration testing challenges running software remotely (as users) to find security vulnerabilities and exploit them. This approach has been shown to be beneficial for network security but less so for software applications (27).
- Security testing checklist (27) to provide simple test cases to be used by testers to challenge exposure to common vulnerabilities such as buffer overflows, SQL injections, XML and SOAP issues (27).
The defensive practitioner will want to know how effective the approach is and whether there are any trends in errors and associated SCL processes to which action is needed. Metric collection is necessary for facilitation.
By capturing metrics at each phase (requirement inspections metrics, design review metrics, various testing metrics), the defensive practitioner will be able to assess the adequacy of the approach and establish if quality is being built in. Effective organizations will be able to predict the level of quality in operational use (28).
“The problem of insecure software is perhaps the most important technical challenge of our time” (27).
The focus of IT ecosystems protection and monitoring reflects the discussions with IT departments on data integrity and security. The same discussion may have been neglected when it came to software.
This paper discussed the defensive approach to building software for computerised systems and briefly discussed some of the techniques and processes available for the defensive approach to building integrity and security into computerized systems. No single technique will offer full immunization to data integrity ills. It takes an amalgamation of appropriate SLC techniques to build data quality in and limit the window of data integrity risk.
A defensive approach incurs more upfront costs but will yield:
- Enhanced compliance to regulatory expectation with greater SLC process control (34)
- Reduced defects in software, e.g. 45% reduction in XP/Vista vulnerabilities after 1 year (42)
- Hardened software against IT system breaches
- Reduced costs from enhanced quality, for example HP’s average time to fix fell from 2 weeks to 2 hours (43)
- Reduced software vulnerabilities with a reduction in the breadth and depth of issues (43).
- 21CFR Part 11 (11.10), FDA, April 2019
- MHRA Data Integrity Guidance, MHRA, March 2018
- 21CFR Part 211 Sub-‐part D, J, FDA, April 2019
- 21CFR Part 820 (72.2) (180) , FDA, April 2019
- FDA Guidance for Industry, Computerized Systems used in Clinical Trials, FDA, April 1999
- Essential Management Information Systems, p231, Laudon & Laudon, 8th Edition, Pearson Prentice Hall, 2009
- National Vulnerability Database, NIST, 15-June-2020
- Healthcare Ransomware Attacks Accounted for 50% of All Security Incidents, spamtitan.com, April 2017
- 264000 and counting: Hack of HER/EMR vendor leaves clients scrambling , Databreaches.net, Jun 2016
- NHS seeks to recover from global cyber-attack as security concerns resurface, The Guardian, May 2017
- Yes, terrorists could have hacked Dick Cheney’s heart, Washington Post, Oct 2013
- Merck & Co. Becomes Victim of Massive Ransomware Cyber Attack, BiosSpace.com, June 2017
- WHO, Guidance on Good Data and Record Management Practices, draft, WHO, September, 2015
- FDA Warning Letter 320-17-36, FDA, Apr-2017
- FDA Warning Letter 320-16-12 FDA,May 2016
- FDA Warning Letter, FDA,Apr 2017
- Gartner Says More than 75 Percent of Mobile Applications will Fail Basic Security Tests Through 2015, Gartner Newsroom, 14 September 2014
- 92% of External Web Apps Have Exploitable Security Flaws or Weaknesses: Report, Security Week, 30 October 2018
- Cyber-Attacks to Devices Threaten Data and Patients, IEEE PULSE, Jan 2018
- Cybercrime and Other Threats Faced by the Healthcare Industry, Trend Micro, 2017
- Medical Devices Are the Next Security Nightmare, Wired, March 2017
- Boy, 11, hacks cyber-security audience to give lesson on 'weaponisation' of toys, Guardian, May 2017
- How a fish tank helped hack a casino, Washington Post, Jul 2017
- How to get rich in Silicon Valley, Guardian, 17-Apr-2018.
- Cost of a Security Breach Report 2019, IBM Security, 2019
- Tools of Critical Thinking, Levey, Waveland Press, 2003
- OWASP Testing project 4.0, chapter 3, 2013
- Applied Software Measurement, Capers Jones, 3rd edition, McGraw Hill, 2008
- ‘GXP’ Data Integrity Definitions and Expectations, MHRA, 2018
- ISPE GAMP Records and Data Integrity GPG Data Integrity - Key Concepts, Appendix 6 Requirements Planning, ISPE, 2018
- I keep Six Honest Serving Men, Rudyard Kipling, circa 1900
- Guide for Conducting Risk Assessment, NIST, Sep-2012.
- Guidance for Industry, Computerized Systems Used in Clinical Investigations FDA, May 2007
- General Principles of Software Validation, FDA, 2002
- Code Complete 2, A Practical Handbook for Software Construction, 2004, Steve McConnell, Microsoft Press.
- OMG® Unified Modelling Language® (OMG UML®) Version 2.5.1, Object Management Group, 2017
- Improving Software Productivity, in Computer, vol. 20, 1987.
- Static Code Analysis, OWASP, (Website in transition with date not published)
- Writing Secure Code, Howard, LeBlanc, Microsoft Press, 2002.
- The economic impacts of inadequate infrastructure for software testing, NIST, MAY 2002
- “Negative Testing” definition, Boris Beizer, The Testing Standards, version 6.3 BCS SIGIST
- Windows Vista One Year Vulnerability Report, Microsoft Security Blog 23, Jan 2008, referenced in: Introduction to the Microsoft Security Development Lifecycle SDL, Summary PowerPoint.
- Does Application Security Pay? Measuring the Business Impact of Software Security Assurance Solutions, Mainstay Partners, 2020
- Major breach found in biometrics system used by banks, UK police and defence firms, Guardian, Aug 2019
- Lifelabs Data Breach, the Largest Ever in Canada, May Cost the Company Over $1 Billion in Class-Action Lawsuit, Scott Ikeda, CPO Magazine, January 2020