Additional Readings

Additional - Quality Factors for the World Wide Web - Ronan Fitzpatrick

Introduces 5 quality factors for websites, to develop web dev into a professional discipline:

• Visibility: easy to find, download, or access
  • Tracibility: find/refind a website
  • Retrievability: Download time
  • Ease of Access: ease to gain entry to the site and be welcomed by the welcoming philosophy
• Intelligibility: Understanding Content
  • Legibility (presentation)
  • Audibility (sound)
  • Comprehensibility (understanding all of it)
• Credibility: Confidence in Content
  • Integrity: confidence in owner's motications, qualifications and trustworthiness
  • Accuracy: correctness and currency of content presented
• Engagibility: Engaging the visitor
  • Navigability: access any part of the site or link to other sites
  • Interactivity: engagement of visitors, enabling them to somplete whatever process/experience is offered by the site
  • Appeal: vizually high quality, ease of use, promotion of efficiency, (onsite other media), expertices, ability ot evoke emotion
• Differentiation: Corporate Superiority
  • Specialty
  • Image/Identity

Future research should suggest software tools to achieve quality factors and establish metrics. Domain specific quality factors may need to be identified.

Developing operational profile for ERP software module reliability prediction - Frane Urem and Želimir Mikulić

Operation Profiles for reliability.

Sofware reliability growth = Measures, Predictions, Improvements though testing.

PR profiles ~~ test cases.

Test time reduced, reliability with no effect.

Test cases need to be as similar to real usage for good reliability estimation. Operational profile tells us how the system will be used in REAL conditions, and helps create test cases & choose test runs. Operational profile = operations & probability of occurrence.

• Customer types --> user types
• Operations identified & enumerated with occurrence rates and probabilities
• Test scripts designed according to operational profile for the specific customer group.

Framework for Quality Metrics in Mobile-Wireless Information Systems - Ruti Gafni

List of Qs --> empirically vaildated quality metrics, extending the ISO/IEC 9126 quality standard --> compare systems

Benefits:

• Productivity enhancement
• Flexibility
• Service improvements
• Information accuracy

These can be ruined by insufficient quality...

What er need to measure

• functionality: suitability, accuracy, interoperability security
  • Does the application use location aware functions where applicable?
  • Is the location aware functions output updated according to the user’s position change during the transaction operation?
  • Does the application apply user profile in order to adapt the output to user and device?
  • Does the application use standard protocols and interfaces?
  • Does the application include mechanisms of authentication, encryption, authorization, and confidence?
• Reliability: maturity, fault tolerance, recoverability
  • To what degree does the mobility cause interruptions?
  • Does the application use cache memory efficiently to avoid loss of data?
  • Does the transaction interruption damage data?
• Usability: understandability, learnability, operability, attractiveness
  • To what degree is the screen over-loaded and diminishes the application understandability?
  • Are there specific menus for each possible operation?
  • Are the buttons which operate each option clear enough?
  • Is the help function for tasks easy to find?
  • Is the application configurable according to user and device?
  • Do the input fields have default values or choices instead of textual input in order to minimize errors?
  • Does the system use location aware functions in order to minimize inputs?
  • Is the length and format of the outputs optimized to screen size?
• Efficiency: Time and behaviour and resource utilization sub characteristics
  • Is the transaction execution time minimal?
  • Does the application utilize the cache memory?
  • Is the size of the application stored in the device proportional to the memory size?
  • Is the size of the help system stored in the device proportional to the memory size and the application size?
• Maintainability: Analyzability, Changability, stability, testability
• Portability: adaptability, installability, co-existance, replacability
  • Does the application adapt to different devices?
  • Does the application utilize middleware to adapt outputs to users and devices according to the user profile?
  • Is the application easy to install in all required devices?
  • Does the application interfere with other applications/services installed in the same device?
• Quality in Use: capability of the software product to enable specified users to achieve specified goals with effectiveness, productivity, safety and satisfaction in specified contexts of use
  • Does the application prevent unnecessary paging for input or output?
  • Does input operation require minimum typing?

OPERATIONAL PROFILES

• Software performance is dependent on environment on which it operates
• Hardware and it's changes may cause faulty behavior
• Operational Profiles are a quantitative characterization of how the software will be used, essential to Software Reliability Engineering (SRE):
  1. Customer prof: who's buying it?
  2. User prof: who's using it?
  3. System-mode prof: the way the system can operate
  4. Functional prof: each function occurring during each mode. Implicit (each distinct value per parameter) or explicit(each value for each param in combination with the values of the other parameters) dependent on key input vars
     1. Generate an initial function list
     2. Determine environmental variables
     3. Create final function list
     4. Assign occurrence probabilities
  5. Operational prof: A function may comprise several operations, operations may have many run types. It is a user-oriented view of system capabilities. These are the focus of testing.
     1. Divide execution into runs (discrete task performed by a system in response to external vars)
     2. Identify input space(input states and probabilities)
     3. Partition input space into operations(should be limited to several hundred elements)
        1. reduce # operations

Reducing Operations

By Reducing # Run types by reducing # input Vars

1. Reduce functionality.
2. Reduce the number of possible hardware configurations.
3. Restrict the environment the program must operate in.
4. Reduce the number of fault types.
5. Reduce unnecessary interactions between successive runs.

By Reducing run interactions:

1. Minimize the input variables that application programs can access at any one time.
2. Reinitialize variables between runs.
3. Use synchronous, as opposed to asynchronous, design.

Reliability Optimization testing using OOA/D Steps

1. Develop a complete object model, including all use cases
2. Identify key operational variables
3. Establish operational relationships for each use case
4. Develop an operational profile for all use cases
5. Estimate the frequencies of uses cases, then operations
6. Estimate testing productivity
7. Evaluate system test effort budget
8. Evaluate coverage, add more tests, if necessary

Understanding and Controlling Software Costs - Barry W. Boehm

Understanding costs --> controlling costs

Requires us to understand and control costs, means understanding and controlling quality.

Most attractive strategies to improve software productivity:

• Writing less code (reuse, highlevel languages, avoid sofware gold plating
• Better management, staffing, incentives, and work environments
• Avoiding Rework - risk management, protyping, incremental development, CAD, modern practices (information hiding)
• Developing and using integrated project support environments

Controlling Budgets, schedules, and work

• There are great techniques, but they need more...
  • Management by Objectives (MBO)
  • Optimizing around Software Predictability and Control

2 ways to understanding software costs:

Black-box/Influence function

Experimental and observational insights on relative software productivity and quality leverage of various management, technical, environmental, and personnel options

Comparative analyses on overall results of a number of entire software projects (team objectives, approaches, hardware constraints, turnaround time, personnel experience/capability) and their effect on costs

Experimentation earliest experiment is effects of batch vs. time-sharing computer operation on programming productivity

• Specification vs/ prototyping:
  • Equivalent productivity (code/hour)
  • Equivalent performance, but prototyping has less delivered source instruction/man-hour
  • Better debugging/integration in specifying projects (b/c good interface specs!)
• More research & experimentation needed to find more relevant metrics than DSI/MH

Observational Analysis

• Most significant cost influence is SLOC (source lines of code)
  • Fourth generation langs or reusable components
• Reduce unnecessary complexity
  • Focus on cost-quality trade-offs
• Use Incremental development
  • Reduces schedule change and requirements volatility
• Productivity can be used to assess impact of proposed software strategies

Glass-box/Cost Distribution

Identify strategies (i.e. value chain and software productivity opportunity tree) for integrated software productivity and quality improvement programs

Comparative analyses on one or more projects to characterize internal costs (i.e. labour/capital costs, code/documentation, development/maintenance.

Insights gained by analyzing distribution of:

Development vs. Rework Costs

• Rework is used to compensate for innapropriately defined req's or fix errors in them
• Use spiral for risky things because it constantly reevaluates requirements

Code and Documentation Costs

Most projects have their efforts spent on documentation rather than code!

Labor and Capital Costs

More investment in workers have shown as more returns in software productivity

Software costs by phase and activity

Lots of effort goes to integration and test - test planning, support, and interface specs ate WORTH IT. Most costs come from maintenance.

The Software Product Value Chain

Software Quality and Software Economics - Capers Jones

To become an engineering discipline, software needs quality control, quality measures, and economic analysis than current norms.

• With troubled software, there too many serious defects
• Quality control can greatly reduce defects in reach success
• As the number of function points increases:
  • The probability of a project being delivered late or being canceled exponentially increases

Methods to reduce Req's & Design Defects:

• A joint client/development change control board or designated domain experts
• Use of Quality Function Deployment (QFD)
• Use of Six-Sigma for Software and/or Lean Six-Sigma
• Use of formal requirements and design inspections
• Use of joint application design (JAD) to minimize downstream changes
• Use of formal prototypes to minimize downstream changes
• Formal review of all change requests
• Revised cost and schedule estimates for all changes > 10 function points
• Prioritization of change requests in terms of business impact
• Formal assignment of change requests to specific releases
• Use of automated change control tools with cross-reference capabilities

Software Quality Models and Philosophies

Views on Quality

Fixed/Qualitative view on quality. More to the point and capture the idea of "Quality"

• Transcendental
  • subjective, quality is recognized but not defined/quantifiable. Usually exceeds customer expectations
• User
  • fitness of purpose - meet the user's needs
  • Reliability (failure rate, MTBF), Performace/Efficiency (time to complete tasks), Maintainaility, Usability
• Manufacturing
  • conformance to specification/process and organization's capacity to produce based on that
  • Waste reduction / Zero Defect / Right The First Time (defect count, fault rates, staff rework costs)
• Product
  • product is defined by it's sub parts (size, complexity, test coverage, module complexity, design, code)
• Value
  • for money by balancing requirements, budget/time, cost/price, deliver dates, productivity, etc.

Quality Models

Rigid models to measure quality and/or increase quality in a software product

McCall

Depends on correctness to specification, reliability against failure, efficiency of execution and storage, integrity of security, and usability.

Boehm

Addresses the contemporary shortcomings of models that automatically and quantitatively evaluate the quality of software. Wider range of Quality Characteristics than McCall. Asks these three questions:

• As-is utility: How well (easily, reliably, efficiently) can I use it as-is?
• Maintainability: How easy is it to understand, modify and retest?
• Portability: Can I still use it if I change my environment?

FURPS/FURPS+

• Basically the same as McCall + Boehm !
• 2 Different types: Functional (F) & Non-Funtional (URPS)
• Can be used as product req's and assessment of quality
  • Functionality
  • Usability
  • Reliability
  • Performance
  • Supportability

Dromey's Quality model

• More recent, still basically the same as Mc Call, Boehm, and FRUPS/+ ...
• Basically says that each product should be evaluated differently for quality so we need a more dynamic model
• Focuses on relationship between quality attributes and sub-attributes, attempting to connect product properties with quality attributes

ISO

ISO 9000: International Quality Management System

Proposes designing, documenting, implementing, supporting, monitoring, controlling and improving (more or less) each of the following processes:

• Quality Management Process
• Resource Management Process
• Regulatory Research Process
• Market Research Process
• Product Design Process
• Purchasing Process
• Production Process
• Service Provision Process
• Product Protection Process
• Customer Needs Assessment Process
• Customer Communications Process
• Internal Communications Process
• Document Control Process
• Record Keeping Process
• Planning Process
• Training Process
• Internal Audit Process
• Management Review Process
• Monitoring and Measuring Process
• Non-conformance Management Process
• Continual Improvement Process

ISO 9126: Software Product Evaluation: Quality Characteristics & Guidelines for Use-Standard

6 quality factors of importance.. like McCall, Boehm, FURPS/+, and Dromey....

ISO/IEC 15504 (SPICE): Information Technology Software Process Assessment

Large international standard framework for process assessment to address all processes involved in:

• Software acquisition
• Development
• Operation
• Supply
• Maintenance
• Support

9 Components:

1) ISO/IEC 15504-1 Part 1: Concepts and Introductory Guide 2) ISO/IEC 15504-2 Part 2: A Reference Model for Processes and Process Capability 3) ISO/IEC 15504-3 Part 3: Performing an Assessment 4) ISO/IEC 15504-4 Part 4: Guide to Performing Assessments 5) ISO/IEC 15504-5 Part 5: An Assessment Model and Indicator Guidance 6) ISO/IEC 15504-6 Part 6: Guide to Competency of Assessors 7) ISO/IEC 15504-7 Part 7: Guide for Use in Process Improvement 8) ISO/IEC 15504-8 Part 8: Guide for Use in Determining Supplier Process Capability 9) ISO/IEC 15504-9 Part 9: Vocabulary

IEEE

Has similar characteristics to ISOs, and can be seen as a replacement

Capability Maturity Model(s)

Like ISO & IEEE, about software quality on a process perspective

Six Sigma

Management philosophy using customer-focused measurement & goal-setting to create results. It strongly advocates listening to the voice of the customer and converting customer needs into measurable requirements.