Prev Up Next
Go backward to 6 Approaches for Overcoming Deficiencies 1
Go up to Top
Go forward to 8 Architectures for Survivability 1

7 Approaches for Overcoming Deficiencies 2

ENPM 808s
Information Systems Survivability:
7. Overcoming These Deficiencies 2

- - - - - - - - - - - - - - - - - - -
Role of security and fault tolerance, software engineering, good development practice, principles, good experience, selective use of formal methods

This lecture is in memory of John Gannon of the Maryland Computer Science Department.

NOTE: Proposals for your final projects are due by e-mail the day after the NEXT class (in ASCII, PostScript, pdf, html, or other format that I can read easily on a Unix-based systems).

Reminder of Serious Obstacles
- - - - - - - - - - - - - - - - - - -
Existing system development practice is generally terrible.

The tendency toward COTS products is a self-defeating prophecy unless dramatic improvements occur.

Development Tools Can Help
- - - - - - - - - - - - - - - - - - -
Record design decisions
Enforce desired design structure
Check syntax of specifications
Record/control development status
Provide programming environment (compilers, optimizers, debuggers, test data generators, editors, symbolic executors, simulators, etc.)
Prove design fulfills requirements
Prove code consistent with specs
Perform refining transformations
Generate and configure systems

Good software engineering practice (e.g.,)
- - - - - - - - - - - - - - - - - - -
Modular design
Functional abstraction
Well-specified functionality
Reusable and interoperable interfaces
Information hiding to mask implementation detail
Constructive use of analysis techniques and tools, selective use of formal methods
Higher-level programming languages less susceptible to characteristic errors (missing bounds checks, mismatched types, mismatched pointers, off-by-one errors, missing exception conditions)
Good documentation

    Desired Characteristics 
of Survivable Systems and Networks
secure           reliable
available        responsive
flexible         monitorable
evolvable        reusable
portable         convenient
understandable   simple/realistic
uniform          forgiving
efficient        cost-effective
compatible       usefully documented    
interoperable    rapidly prototypeable  
Experience and Principles

- - - - - - - - - - - - - - - - - - -
.. our heads are full of general ideas that we are now trying to turn to some use, but that we hardly ever apply rightly. This is the result of acting in direct opposition to the natural development of the mind by obtaining general ideas first, and particular observations last; it is putting the cart before the horse. ...
The mistaken views ... that spring from a false application of general ideas have afterwards to be corrected by long years of experience; and it is seldom that they are wholly corrected. That is why so few men of learning are possessed of common sense, such as is often to be met within people who have had no instruction at all.

Arthur Schopenhauer, Parerga
and Paralipomena,

Einstein: Thinking and Judgement
- - - - - - - - - - - - - - - - - - -
The development of general ability for independent thinking and judgment should always be placed foremost, not the acquisition of special knowledge. If a person masters the fundamentals of his subject and has learned to think and work independently, he will surely find his way and besides will better be able to adapt himself to progress and changes than the person whose training principally consists in the acquiring of detailed knowledge.

Albert Einstein, Out of My Later Years, 1950.

Security Design Principles 
(after Schroeder-Saltzer 1972)
   0.  Methodological basis    
   1.  Simplicity
   2.  Fail-safe defaults
   3.  Complete mediation
   4.  Open design
   5.  Separation of privilege
   6.  Principle of least privilege
   7.  Principle of least common mechanism
   8.  Psychological acceptability
   9.  Sufficiently high work factor
  10.  Noncompromisible auditing
  11.  Principle of least surprise
       (especially in interfaces)
       0. Unifying Methodological View
Development    |        Analysis stages
Stages         | Functional     Performance
Requirements   | Consistency,   Consistency,
 (functional+  | adequacy,      adequacy,
  performance) | realizability  realizability
Design:        | Design proofs  Design feasibility
Hierarchy,     | (spec-req      (spec-req
Specifications | consistency)   consistency)
 (func+perf)   | Design reviews
Implemen-      | Code proofs    Performance test
tation:        | (code-spec     and measurement
Code, linkage  | consistency,   Formal analysis
               | portability      as appropriate
               | checks),         
               | Tiger teams
Development    |        Analysis stages
Stages         | Functional     Performance
Maintenance    | Incremental    Check for adequacy
               | closure after  of performance of
               | change         revised system
Documentation  | Consistency of Consistency of
               | documentation  documentation
               | with design,   with design,code
               | code 
    Everything should be made as 
    simple as possible, but no simpler.
     * Conceptual simplicity,
       economy of mechanism:
       Keep it simple, Keep it small.
       (But complex systems are not simple.)
     * Comprehensibility of design 
     * Controllability of implementation
     * Comprehensibility of validation
     * Constrained code development 
     * In-house expertise
    All of these suggest the use of high-level 
    design languages and programming languages.
     * Protection scheme permits access unless 
       denied (typically uncontrollably dangerous)
     * Protection permits no access unless 
       specifically authorized (typically too 
       restrictive and causes many error messages)
     * Multilevel security with preassigned levels
       (conceptually simple, but full of pitfalls)
     * EVERY access must be checked
     * Must be system-wide
     * ``Foolproof'' identification
     * Audit changes in authority
     * Reference Monitors
       The Reference Monitor Concept
             (Anderson, 1972)
    Reference monitor mediates security
    on every access by subject to object.
    May use hardware and software.
    Small enough be analyzed thoroughly
    (formal verification?)
    Protected from tampering, 
     * Design should not be a secret.
     * Protection mechanisms separate 
       from protection keys
       - Pins and tumblers vs. combinations
       - Security design vs. passwords
     * Provision for open review of
        protection mechanism

     * Offer bounty for penetrations
     * Enforce distinct roles,
       Firewalls of protection
     * Require multiple authorizations,
       not all in one (avoid superuser)
     * Require need-to-know
     * Allocate just what is needed
     * Minimize amount of mechanism 
       common to more than one user.
     * Minimize pervasiveness of 
       shared variables.
     * Easy to use and understand,
       no incentive to circumvent.
     * Related to degree of difficulty 
        of cracking
     * Increased possibility of exposure
     * Need to record all interesting events, 
       particularly potential compromises
     * Thresholds depending on context
     * Must be tamperproof: nonbypassable,

     * Relevant especially to systems 
       in the large

     * Vital in human interfaces
       (user interface coherence), and
       also in accessible internal 
       program interfaces
    Implementation Principles
    Superb design can be compromised
      by shoddy implementation.
      Good implementation is vital.
    Implementation structure should 
      relate to design.
    Use of appropriate higher-level 
    Use of appropriate hardware 
    Defer binding until essential.
    Defer optimization until needed.
    Anticipate later problems.
    Good Programming Language Features
    Compatibility with methodology
    Ease of programming, understanding
    Efficient code compilable
    Support for separate compilation 
    Encapsulation of data types
    Strong typing and type safety
    Symmetry (built-in, extensions)
    Dynamically declarable attributes
    Dynamic creation and deletion 
    Ease of expressing concurrency
    Sound synchronization primitives
    Clean handling of exceptions 
    Initialization and finalization
    Controlled argument passing 
    No aliases 
    Riskful Programming Language Problems
    Nonisolated dynamic memory management 
    Multitasking (synchronization, process 
    Dependence on untrustworthy runtime 
    Do-it-yourself exception handling
    GoTos and nonnested calling semantics
    Unchecked arguments, bounds, arrays,
      buffer overflows
    Lack of strong typing
    Unchecked escapes into machine code
    Leaky garbage collection
    Undetectable Trojan horses

Roles of Formal Methods in Survivability

Formal methods can play an important role in the attainment of systems and networks that must achieve generalized survivability, in specification, design, and execution. Great improvements in system behavior can be realized when the requirements (for survivability, security, reliability, etc.) have a formal basis. Similarly, there are enormous benefits whenever design specifications have a formal basis, especially if they are derived from well specified requirements rather than the common practice of being established after the fact to represent an ad hoc assembly of already developed software (sometimes referred to as putting the cart before the horse). Formal design verification then involves formal demonstrations that the specifications are consistent with their requirements, providing no less than required, and to the extent that the absence of Trojan horses can be demonstrated, nothing unexpected that might be harmful. Verification of designs is difficult for systems that were not designed to be readily analyzed, but can nevertheless be valuable in legacy systems (as in analyses of the risks associated with the Year-2000 problem). Finally, although it is less commonly practiced in software, formal code verification can demonstrate that a given implementation is consistent with its specifications. Formal hardware verification is being used increasingly, and demonstrates the potential effectiveness of formal methods where there are considerable risks (financial or otherwise) of improper design and implementation.

Various formal methods can be valuable in specifying and analyzing requirements, designs, and implementations. Of particular importance in connection with survivability are techniques that can provide formal relationships between different layers of abstraction, with respect to requirements and specifications alike. The use of formal methods is recommended in particularly critical applications, and can help move the current highly unpredictable ad hoc development process into a much more predictable formal development process. In the long run, use of such techniques can dramatically decrease the risks of system failure. Contrary to popular myth, judicious use of formal methods can also decrease the overall development and operating costs, especially when the costs of aborted developments (e.g, cancellations of IRS, FAA, FBI systems) are considered, along with the costs of overruns, delays in delivery, and subsequent maintenance.

Judicious use of formal methods can have a very high payoff, particularly in requirements, specifications, algorithms, and programs concerned with especially critical functionality, such as concurrency, synchronization, avoidance of deadlocks and race conditions in the small, and perhaps even network stability and survivability in a larger context, derived on the basis of more detailed analyses of components. There is no substitute for using demonstrably sound algorithms.

Of particular importance is the formal analysis of requirements, for example, determining whether a given set of requirements at a particular layer of abstraction is consistent within itself, whether the different sets of requirements at the lower layer are fundamentally incompatible with one another, and whether the requirements at a lower layer are consistent with the requirements at the upper layer. Once such an analysis is done, then it is also beneficial to determine whether system specifications and implementations are consistent with the relevant requirements.

It must be emphasized that the most valuable uses of formal methods are in finding flaws and inconsistencies, not in attempting to prove that something is completely correct. However, formal methods approaches are not absolute guarantees, because problems that can exist outside of their scope of analysis. For example, suppose that a given analysis does not detect any flaws or inconsistencies in a specification or implementation. It is still possible that the requirements are inadequate (for example, the specifications could fail to prevent a problem not covered by the requirements), or that the analysis methods themselves could be flawed. For these reasons, extensive testing of developed systems is also important - albeit inherently limited. Unfortunately, testing is itself inherently incomplete and incapable of discovering many types of problems, for example, stemming from distributed system interactions and concurrency failures, subtle timing problems, unanticipated hardware failures, and environmental effects. Exhaustive testing over all possible scenarios is basically impossible in any complex system.

Past Uses of Formal Methods
- - - - - - - - - - - - - - - - - - -
Fault tolerance and Byzantine algorithms; SRI's 1970s Software Implemented Fault Tolerance (SIFT) and Provably Secure Operating System (PSOS); security and cryptographic protocols; network protocol development; hardware verification and computer-aided design; compiler correctness; and human safety; but not yet for survivability, or for security and fault tolerance in combination.

The European notion of dependability attempts to encompass various requirements.

Belief logics for cryptographic protocols (e.g., Burrows-Abadi-Needham BAN logic)

Formal methods can also be used in execution: Proof-carrying code, providing evidence that a component has not been tampered. See references to George Necula and others.

Directions for the Future
- - - - - - - - - - - - - - - - - - -
Establishment of survivable heterogeneous open-system and network architectures, designed for the full set of requirements, with robust information infrastructure, exploiting robust mobile code; focus on selected trustworthy critical components; minimizing trustedness in dumb terminals and hand-held wireless communicators; real-time detection of anomalies; robust configuration management

Dramatically improved system development practice and procurement practice

Better use of research and development, sensible fault tolerance, security; formal methods for specification, selected proofs of critical properties of critical modules, proof-carrying code, safe reconfiguration, ...

Establishment of explicit requirements and understanding of the interdependencies

Better system/software engineering practice: disciplined use of structural organizing principles for survivable systems, such as abstraction, composition, encapsulation, reusability, separation, least privilege; domains; baseline survivable architectures; enriched IPSec/IPv6 for realistic survivability requirements; adherence to the Generally Accepted System Security Principles (GASSP)

Practical development and use of real-time analysis and response subsystems coupled with dynamic configuration control

Find ways to develop the missing pieces of the computer-communication infrastructure, and ruggedize the best R&D available. We need sound operating systems for boundary controllers, crypto, trustworthy servers, pervasive authentication, and architectures that localize critical trustworthiness needs.

Provide better architectural and implementation guidance to avoid characteristic weaknesses.

Improve the evaluation criteria framework (Common Criteria!) and establish specific requirements for evaluation of survivability

General References
- - - - - - - - - - - - - - - - - - -
Computer-Related Risks and the arl-one report provide an enormous number of references. For example:

Software engineering: Many references to works by Dijkstra beginning in 1969s, Parnas in 1970s, FPBrooks, ...

Robust systems: Multics (1965 and on,, T.H.E. system (Dijkstra 1968), domains of protection (Schroeder 1972), confined execution (Lampson 1973), PSOS (1975, 1977, 1980), Rushby's isolation kernels (Rushby 1984).

Early efforts linking security and reliability: Lampson 1974, Dobson and Randell 1986, Randell and Dobson 1986.

Reading for the Class Periods on Architectures
- - - - - - - - - - - - - - - - - - -
Read Chapter 7 of arl-one:

Look at the Feiertag-Neumann PSOS paper for its hierarchical layering and object-oriented architecture:

Scan the Proctor-Neumann paper for its architectural content, not for the covert channel analysis:

Prev Up Next