CSQA

Become a Certified Software Quality Analyst (CSQA)

2006-12-23T11:19:00.000-08:00

CBOK 2006 material is available now!
Click here...

Link to March 2006 Questions

There has been a tremendous response to CSQA blog - meant for the aspirants of CSQA certification. This has been set up to make sure every CSQA aspirant finds all the necessary material at one place.

I have given few details of CSQA below. For those who want to goto the contents directly, use this link - CONTENTS. Those who would like to goto the Questions section can directly go there using this link - Q&A. I am also updating the CSQA FAQs with all of the queries I got from many people. Please go through this FAQ page before raising any query.

For all of the CSQA aspirants, apart from this Introduction page, I also suggest you to go through the following pages before moving on to the CBOK-2006 material page:

CSQA - An Overview

Certified Software Quality Analyst (CSQA) is a widely recognized certification in the software industry now. As a CSQA certified professional, the QA executives will be able to put the QA prinicples into practice in the right manner at the right time and hence will be able to play a huge role in building the organisations. Getting certified always helps in three ways:

Indication of a level of knowledge and competence
Certification can be a deciding factor in hiring, promotion, and salary action decisions
Recertification requirements are a spur to continuing education

CSQA - Statistics

The number of active CSQA certified professionals according to the CSQA Active Roster is 7396 as of 24th January, 2006. The earliest person who was CSQA certified and is still active according to the Active Roster is William Perry. This Roster is available at this link: http://www.softwarecertifications.org/static/RosterCSQA.pdf

Exam format

The four and a half-hour exam is made up of four parts; two parts multiple-choice and two parts
essay and short answer. You will have forty-fi ve minutes to complete each of the multiplechoice
sections and one hour and fi fteen minutes to complete each of the essay and short answer sections. There is a ten-minute break between each part.

Application, Recertification & Others

Just to brief you, inorder to be able to sit for the examination, we have to satisfy the pre - requisites, Code of Ethics, Submit a Character Reference along with the application. This information is available in the SoftwareCertifications website in the page - "Applying for CSQA/CSTE"
You can view the complete CBOK contents at this link:
http://www.softwarecertifications.org/csqabok/csqabok.htm

CSQA certified candidates must be recertified 3 years after passing CSQA to stay current in the field. This can be demonstrated by 1) submitting a Recertification Journal of qualified Continuing Professional Education activities, or 2) by taking an Examination for Recertification to demonstrate the maintenance of the certification.

There are individual seven categories eligible for Continuing Professional Education (CPE) reporting which are given at this link:
http://www.softwarecertifications.org/qai_recertification.htm

There are Advanced Levels of Certifications too - Do check them out @ SoftwareCertifications

CSTE/CSQA ->
ASTP/ASQP ->
MSQA/MSTE

More info is available at: http://www.softwarecertifications.org/qai_levels.htm

For further details on the Schedule of the exam, Application procedures, Recertification, Obtaining & Using Credentials etc can go to www.softwarecertifications.org/qai_cqa.htm

PS: Though the old version is still available, there may be problem with few links as changes were made directly to the pages which are useful for the new CBOK. You can refer the CBOK-2006 Contents page for those Knowledge Categories.

The old version of the material is still available.
Click here to go there

CBOK 2006 - CONTENTS

2006-12-22T22:01:00.000-08:00

KC1 - Quality Principles and Concepts - CONTENTS

KC 2 - Quality Leadership - CONTENTS

KC 3 - Quality Models - CONTENTS

KC 4 - Quality Assurance - CONTENTS

KC 5 - Quality Planning - CONTENTS

KC 7 - Quality Control Practices - CONTENTS

KC 8 - Metrics and Measurement - CONTENTS

KC 9 - Internal Control and Security - CONTENTS

KC 10 - Outsourcing, COTS and Contracting Quality - CONTENTS

KC1 - Quality Principles and Concepts - CONTENTS

2006-12-19T23:52:00.000-08:00

2006 CSQA Body of Knowledge
Knowledge Category 1

Quality Principles and Concepts

Before an organization can begin to assess the quality of its products and services, and identify opportunities for improvement, it first must have a working knowledge of quality principles and basic concepts. This category will test the CSQA candidate’s ability to understand and apply these principles, which include the quality vocabulary, various ways of defining quality, key concepts, distinguishing between quality control and quality assurance, and the contributions of quality pioneers.

Vocabulary of Quality

Understand the terms used to explain and implement quality in an IT organization.

The Different Views of Quality

The Two Quality Gaps
Quality Attributes for an Information System

Quality Concepts and Practices

PDCA Cycle (Plan-Do-Check-Act)
Cost of Quality
Six Sigma Quality
Baselining and Benchmarking
Earned Value

Quality Control and Quality Assurance

Quality Control
Understanding and Using the Just-In-Time (JIT) Technique
Quality Assurance
Differentiating Between Quality Control and Quality Assurance

Quality Pioneers Approach to Quality

Dr. W. Edwards Deming
Philip Crosby
Dr. Joseph Juran
Total Quality Management

KC 1.1 - Vocabulary of Quality

2006-12-19T00:24:00.000-08:00

Coming Soon........

KC-1.2.1-The Different Views of Quality

2006-12-18T01:11:00.000-08:00

The Two Quality Gaps

The definitions in summary are given below. For those who love to read more, further info is given.

From the customer's perspective, satisfaction after the delivering of the product is the ultimate validation of the product quality. It is clear that the concept of quality must involve customers or, simply put, quallity is conformance to customers' expectations and requirements. Crosby's "conformance to requirements" and Juran's "fitness for use" both implied the customers' perspective.

From the producer's perspective, developing and producing the product in accordance with the specifications is the path to achieving quality.

Therefore the three ideal metrics any organization should have are product quality, process quality, and customer satisfaction (referred to as the big capital “Q”).

From a high-level definition of a concept, to a product being operationally defined, many steps are involved, each of which may be exposed to possible shortcomings. For example, to achieve the state of conformance to requirements, customers' requirements must first be gathered and analyzed, then specifications from those requirements must be developed, and the product must be developed and manufactured appropriately. In each phase of the process, errors may have occurred that will negatively affect the quality of the finished product. The requirements may be erroneous (especially in the case of software development), the development and manufacturing process may be subject to variable that induce defects, and so forth. Form the customer's perspective, satisfaction after the purchase of the product is the ultimate validation that the product conforms to requirements and is fit to use. From the producer's perspective, once requirements are specified, developing and producing the product in accordance with the specifications is the basic step to achieving quality. Usually, for product quality, lack of functional defeats and good reliability are the most basic measures. In order to be "fit for use," the product first has to be reliably functional.

Because of the two perspectives on quality (i.e., customer satisfaction as the ultimate validation of quality, alnd the producer's adherence to requirements to achieve quality), the de facto definition of quality consists of two levels. The first is the intrinsic product quality, often operationally limited to product quality,often operationally limited to product defect rate and reliability; this narrow definition is referred to as the "small" q (q for quality). The broader level of the definition of quality includes both product quality and customer satisfaction; it is referred to as the "big" Q.

In software, the norrowest sense of product quality is comonly recongnized as lack of "bugs" in the product. This definition is usually expressed in two ways: defect rate (e.g., number of defects per million lines of source code, or per function point), and reliability (e.g., number of failure per n hours of operation, mean time to failure, or the probability of failure-free operation in a specified time). Customer satisfaction is usually measured by the percentage of those satisfied or nonsatisfied (neutral and dissatisfied) on customer satisfaction surveys. To reduce bias, usually techniques such as double-blind surveys (the interviewer not knowing who the customer is, and the customer not knowing what company the interviewer represents) are used. In addition to overall customer satisfaction with the software product, satisfaction toward specific attributes is CUPRIMDSO satisfaction levels of its software products (i.e., capability [functionality], usability, performance, reliability, instalability, maintainability, documentation/information, service, and overall satisfaction). The Hewlett-Packard Co. focuses on FURPS (functionalilty, usability, relability, performance, and supportability)[1].

For people with much more interest, here is the link from HP's Center for Quality Management Journal. I believe the differences between Customer's & Producer's view of qualities cant be better enumerated. The copyrights do not allow to reproduce any part of it. You can go to this link to find this quite a long paper: http://cqmextra.cqm.org/cqmjournal.nsf/reprints/rp08300
The PDF form of this article can be downloaded from this link: DOWNLOAD Article(PDF)

References:

Software quality: an overview from the perspective of total quality management - Tutorial - IBM Systems Journal, March, 1994 by Stephen H. Kan, Victor R. Basili , L arry N. Shapiro

KC 1.2.1 - Quality Attributes for an Information System

2006-12-17T09:29:00.000-08:00

Previous Topic <<

Next Topic >>

Correctness

The software system’s fulfillment of the specifications underlying its development. The correctness of a software system refers to:

Agreement of program code with specifications
Independence of the actual application of the software system.

The correctness of a program becomes especially critical when it is embedded in a complex software system.

Example 1.1 Let p be the probability that an individual program is correct; then the probability P of correctness of the software system consisting of n programs is computed as P = pⁿ.

If n is very large, then p must be nearly 1 in order to allow P to deviate significantly from 0 [Dijkstra 1972b].

Reliability

Reliability of a software system derives from

Correctness, and
Availability.

The behavior over time for the fulfillment of a given specification depends on the reliability of the software system.

Reliability of a software system is defined as the probability that this system fulfills a function (determined by the specifications) for a specified number of input trials under specified input conditions in a specified time interval (assuming that hardware and input are free of errors).

A software system can be seen as reliable if this test produces a low error rate (i.e., the probability that an error will occur in a specified time interval.)

The error rate depends on the frequency of inputs and on the probability that an individual input will lead to an error.

User friendliness:

Adequacy
Learnability
Robustness

Adequacy

Factors for the requirement of Adequacy:

The input required of the user should be limited to only what is necessary. The software system should expect information only if it is necessary for the functions that the user wishes to carry out. The software system should enable flexible data input on the part of the user and should carry out plausibility checks on the input. In dialog-driven software systems, we vest particular importance in the uniformity, clarity and simplicity of the dialogs.
The performance offered by the software system should be adapted to the wishes of the user with the consideration given to extensibility; i.e., the functions should be limited to these in the specification.
The results produced by the software system:
The results that a software system delivers should be output in a clear and well-structured form and be easy to interpret. The software system should afford the user flexibility with respect to the scope, the degree of detail, and the form of presentation of the results.Error messages must be provided in a form that is comprehensible for the user.

Learnability

Learnability of a software system depends on:

The design of user interfaces
The clarity and the simplicity of the user instructions (tutorial or user manual).

The user interface should present information as close to reality as possible and permit efficient utilization of the software’s failures.

The user manual should be structured clearly and simply and be free of all dead weight. It should explain to the user what the software system should do, how the individual functions are activated, what relationships exist between functions, and which exceptions might arise and how they can be corrected. In addition, the user manual should serve as a reference that supports the user in quickly and comfortably finding the correct answers to questions.

Robustness

Robustness reduces the impact of operational mistakes, erroneous input data, and hardware errors.

A software system is robust if the consequences of an error in its operation, in the input, or in the hardware, in relation to a given application, are inversely proportional to the probability of the occurrence of this error in the given application.

Frequent errors (e.g. erroneous commands, typing errors) must be handled with particular care
Less frequent errors (e.g. power failure) can be handled more laxly, but still must not lead to irreversible consequences.

Maintainability

Maintainability = suitability for debugging (localization and correction of errors) and for modification and extension of functionality.

The maintainability of a software system depends on its:

Readability
Extensibility
Testability

Readability

Readability of a software system depends on its:

Form of representation
Programming style
Consistency
Readability of the implementation programming languages
Structuredness of the system
Quality of the documentation
Tools available for inspection

Extensibility

Extensibility allows required modifications at the appropriate locations to be made without undesirable side effects.

Extensibility of a software system depends on its:

Structuredness (modularity) of the software system
Possibilities that the implementation language provides for this purpose
Readability (to find the appropriate location) of the code
Availability of comprehensible program documentation

Testability

Testability: suitability for allowing the programmer to follow program execution (run-time behavior under given conditions) and for debugging.

The testability of a software system depends on its:

Modularity
Structuredness

Modular, well-structured programs prove more suitable for systematic, stepwise testing than monolithic, unstructured programs.

Testing tools and the possibility of formulating consistency conditions (assertions) in the source code reduce the testing effort and provide important prerequisites for the extensive, systematic testing of all system components.

Efficiency

Efficiency: ability of a software system to fulfill its purpose with the best possible utilization of all necessary resources (time, storage, transmission channels, and peripherals).

Portability

Portability: the ease with which a software system can be adapted to run on computers other than the one for which it was designed.

The portability of a software system depends on:

Degree of hardware independence
Implementation language
Extent of exploitation of specialized system functions
Hardware properties
Structuredness: System-dependent elements are collected in easily interchangeable program components.

A software system can be said to be portable if the effort required for porting it proves significantly less than the effort necessary for a new implementation.

Find out more info on Quality Attributes in the Technical Report of Carnegie Mellon University on QUALITY ATTRIBUTES. You can also download this paper from: THIS LINK of SEI.

Reference: SOFTWARE ENGINEERING By Nguyen Xuan Huy

Previous Topic <<

Next Topic >>

KC 1.3.1 - The Deming's PDCA Cycle

2006-12-15T10:49:00.000-08:00

I am giving simple definitions of P-D-C-A here. There are very good resources available for PDCA cycle which is also known as Deming's Cycle or Schewart's Cycle. Please go through the sources below for better info on PDCA cycle.

PLAN: Design or revise business process components to improve results
DO: Implement the plan and measure its performance
CHECK or STUDY: Assess the measurements and report the results to decision makers
ACT: Decide on changes needed to improve the process

Benefits of the PDCA cycle:

- Daily routine management-for the individual and/or the team
- Problem-solving process
- Project management
- Continuous development
- Vendor development
- Human resources development
- New product development
- Process trials
Note: The PDCA cycle was in fact originally developed by Walter A, Shewhart, a Bell Laboratories scientist who was Deming's friend and mentor, and the developer of Statistical Process Control

Kaoru Ishikawa has expanded Deming's four steps into Six

Determine goals and targets.
Determine methods of reaching goals.
Engage in education and training.
Implement work.
Check the effects of implementation.
Take appropriate action.

PDCA Resources:

KC 1.3.2.1 - Cost of Quality & it's Components

2006-12-14T11:29:00.000-08:00

Cost of Quality (COQ) is one of the very important sub domain of the Knowledge Domain-1 in the CSQA BOK. Getting this concept clarified is very important for any QA Professional and more importantly for CSQA aspirants. I am starting here with the definitions and the best possible sources here on COQ.

Cost Of Quality

The cost associated with the quality of a work product.

As defined by Crosby ("Quality Is Free"), Cost Of Quality (COQ) has two main components: *Cost Of Conformance* and *Cost Of Non-Conformance* (see respective definitions).

Cost of quality is the amount of money a business loses because its product or service was not done right in the first place. From fixing a warped piece on the assembly line to having to deal with a lawsuit because of a malfunctioning machine or a badly performed service, businesses lose money every day due to poor quality. For most businesses, this can run from 15 to 30 percent of their total costs.

Cost of Poor Quality - COPQ

COPQ consists of those costs which are generated as a result of producing defective material.

This cost includes the cost involved in fulfilling the gap between the desired and actual product/service quality. It also includes the cost of lost opportunity due to the loss of resources used in rectifying the defect. This cost includes all the labor cost, rework cost, disposition costs, and material costs that have been added to the unit up to the point of rejection. COPQ does not include detection and prevention cost.

There usually goes confusion between these two definitions COQ and COPQ.

Cost of quality (COQ) is actually a phrase coined by Philip Crosby, noted quality expert and author and originator of the “zero defects” concept, to refer to the costs associated with providing poor-quality products or services. Many quality practitioners thus prefer the term cost of poor quality (COPQ).

To illustrate the impact this can have on the workforce can not be underestimated. With a little imagination you could imagine the impact of storing all scrap for one year then presenting it as an "in-your-face" visual presentation tactic at a business improvement initiative.

Cost Of Conformance

(COC) A component of the *Cost Of Quality* for a work product. Cost of conformance is the total cost of ensuring that a product is of good *Quality*. It includes costs of *Quality Assurance* activities such as standards, training, and processes; and costs of *Quality Control* activities such as reviews, audits, inspections, and testing.

COC represents an organisation's investment in the quality of its products.

Cost Of Non-Conformance

(CONC.) The element of the *Cost Of Quality* representing the total cost to the organisation of failure to achieve a good *Quality* product.

CONC includes both in-process costs generated by quality failures, particularly the cost of *Rework*; and post-delivery costs including further *Rework*, re-performance of lost work (for products used internally), possible loss of business, possible legal redress, and other potential costs.

Cost of Quality (COQ )

1. Cost of Control (Conformance)

Prevention Cost
Appraisal Cost

2. Cost of not controlling (Non-conformance)

Internal Failure Cost
External Failure Cost

Wanna know the definitons more clearly? Here they are:

Four categories of costs contribute to an organization’s overall COQ:

Internal failure costs - costs associated with defects found before the customer receives the product or service
External failure costs - costs associated with defects found after the customer receives the product or service
Appraisal costs - costs incurred to determine the degree of conformance to quality requirements
Prevention Costs - Costs incurred to keep failure and appraisal costs to minimum.

Table - Examples of Quality Costs Associated with Software Products

*Prevention*	*Appraisal*
Statistical Process Control Capability/feasibility studies Improvement programmes Preventive actions Consultancies Training Procedures/Work instructions Communications Calibration systems	Inspection Design review Code inspection Audit (internal & external) Testing Glass box testing Black box testing Training testers Beta testing Test equipment & automation Usability testing Pre-release out-of-box testing by customer service staff
*Internal Failure*	*External Failure*
Rectification Scrap Rework Concessionary work Investigations Corrective Actions	Rectifying returned products Replacements Warranty claims Complaints Site repairs Lost custom/reputation Legal ramifications

The minimum total cost,for example is shown below as being achieved at 98% perfection. This percentage is also known as best practice. That is, the cost of achieving an improvement outweighs the benefits of that improvement.

In my previous post, similar cost curve created doubts to few as the X-axis was labelled "Defects" there. In fact it is the Degree of perfection achieved towards the individual curves like Appraisal, Prevention etc. In few instances, this axis is taken with increasing Quality of Design.

Sources referred:
http://www.isixsigma.com/dictionary/glossary
http://www.asq.org/topics/coq.html
http://www.educesoft.com/quality/costofquality.htm
http://www.hq.nasa.gov/office/hqlibrary/ppm/ppm35.htm

KC 1.3.2.2 - Cost of Quality (Contd..)

2006-12-13T22:06:00.000-08:00

Just came across this S/W Quality Icerberg - showing the importance of internal Quality maintenance which is invisible to the external agent. Looking at this iceberg, the amount by which the Orgnanisation spends towards reducing the Cost of Poor Quality should idealy be more compared to the cost incurred in meeting the external defects identified.

Read more on this Quality Icerberg here:
http://www.asq.org/pub/qualityprogress/past/0501/qp0501defeo.pdf

The Total Quality View is given in the figure below. Realize from the figure that as the emphasis on Prevention & Appraisal Cost is increasing, the failure cost is reducing at a linear slope. Also see the slope of the Prevention & Appraisal Cost is less than that of the Cost of Failure which clearly indicates the cost reduction because of the difference in slope gives you the net profit. Also shown below is the ideal divisions among the different Costs of Quality.

Interested ones can go to this to find out why more organisations cannot use it effectively.
http://www.asq.org/ed/conferences/aqc/public_proceedings/57_2003/19384.pdf

KC 1.3.3 -Six Sigma, Benchmarking, Continuous Improvement & Best Practices

2006-12-12T23:58:00.000-08:00

I dont feel there is need of long notes here as many resources are available on the web. So I am just briefly giving their definitions with the related resources.

Six Sigma
The goal of Six Sigma is to increase profits by eliminating variability, defects and waste that undermine customer loyalty.

Six Sigma can be understood/perceived at three levels:

Metric: 3.4 Defects Per Million Opportunities. DPMO allows you to take complexity of product/process into account. Rule of thumb is to consider at least three opportunities for a physical part/component - one for form, one for fit and one for function, in absence of better considerations. Also you want to be Six Sigma in the Critical to Quality characteristics and not the whole unit/characteristics.
Methodology: DMAIC/DFSS structured problem solving roadmap and tools.
Philosophy: Reduce variation in your business and take customer-focused, data driven decisions.

Six Sigma is a methodology that provides businesses with the tools to improve the capability of their business processes. This increase in performance and decrease in process variation leads to defect reduction and vast improvement in profits, employee morale and quality of product.

Resources:

Benchmarking
The concept of discovering what is the best performance being achieved, whether in your company, by a competitor, or by an entirely different industry.

Benchmarking is an improvement tool whereby a company measures its performance or process against other companies' best practices, determines how those companies achieved their performance levels, and uses the information to improve its own performance.

Benchmarking is a continuous process whereby an enterprise measures and compares all its functions, systems and practices against strong competitors, identifying quality gaps in the organization, and striving to achieve competitive advantage locally and globally.
Resources:

Best Practice
A way or method of accomplishing a business function or process that is considered to be superior to all other known methods.

A lesson learned from one area of a business that can be passed on to another area of the business or between businesses.

Continuous Improvement (CI)
Continuous Improvement (CI): Adopting new activities and eliminating those which are found to add little or no value. The goal is to increase effectiveness by reducing inefficiencies, frustrations, and waste (rework, time, effort, material, etc). The Japanese term is Kaizen, which is taken from the words "Kai" means change and "zen" means good.
Resources:

KC 1.3.4 - Earned Value Management - Basics

2006-12-11T22:51:00.000-08:00

Earned Value Management is a methodology used to measure and communicate the real physical progress of a project taking into account the work complete, the time taken and the costs incurred to complete that work.

Earned Value helps evaluate and control project risk by measuring project progress in monetary terms.

We spend time and materials in completing a task. If we are efficient we complete the task with time to spare and with minimum wasted materials. If we are inefficient we take longer and waste materials.

We also plan how we will accomplish the task. How long it will take, the resources we need and the estimated costs.

By taking a snap-shot of the project and calculating the Earned Value metrics we can compare the planned with the actual and make a subjective assessment of the project progress.

By extrapolating the curves and further calculation we can also estimate the costs to project completion and the probable completion date.

The basics of Earned Value can best be shown on the ubiquitous 'S-Curve'.

The S-curve in its simplest form is a graph showing how project budget is planned to be spent over time.
We can complicate the graph by showing the actual costs of doing the work over the same period.
And also on the same graph we can show how the value of the product of the project increases over the same period.

The three curves on the graph represent:

Budgeted Cost for Work Scheduled (BCWS) - the budgets for all activities planned to be completed.
Actual Cost of Work Performed (ACWP) - the real costs of the work charged against the completed activities.
Budgeted Cost of Work Performed (BCWP) - the planned costs of the work allocated to the completed activities. This is the Earned Value.

The BCWS curve is derived from the Work Breakdown Structure, the project budget and the Project Master Schedule. The cost of each Work Package is calculated and the cumulative cost of completed Works Packages is shown based on the planned completion dates shown in the Master Schedule.

The ACWP curve is found by actual measurement of the work completed. Actual costs recorded from invoices and workmen's time sheets. This appears a daunting task but it can be very simple with sufficient planning and organising.

The BCWP is calculated from the measured work complete and the budgeted costs for that work.
Earned Value = Percentage project complete X Project Budget

Variances

Schedule and cost variances can both be calculated in monetary terms from the data needed to produce the S-curves.

Schedule variance is the difference between the Earned Value and the planned budget.
SV = BCWP - BCWS

Cost Variance is the difference between the Earned Value and the actual costs of the works.
CV = BCWP - ACWP

Performance Indices

Schedule Performance Index and Cost Performance Index give indications of the health of the project. Is the project on time, in budget or what?

Schedule Performance Index is a ratio of Earned Value and the planned value of completed works. A SPI < spi =" BCWP">

Cost Performance Index is a ratio of Earned Value and the actual costs of completed works. A CPI < cpi =" BCWP">

Estimate At Completion

The EAC gives an idea of the final costs of a project. It takes into account the original budget (BAC), the Earned Value and the Cost Performance Index of the already completed works.
EAC = ACWP + ((BAC - BCWP)/CPI)

Other Resources:

KC 1.3.4.2 - Earned Value Management - Example

2006-12-10T22:56:00.000-08:00

Earned value provides an objective measurement of how much work has been accomplished on a project. Using the earned value process, the management team can readily compare how much work has actually been completed against the amount of work planned to be accomplished. All work is planned, budgeted, and scheduled in time-phased "planned value" increments constituting a Performance Measurement Baseline (PMB).

Let's look at a simplified example:

(Fig) Baseline plan
The baseline plan shown in the above graph illustrates a task with a total budget amount of $240k, which is planned for accomplishment over 24-month time frame. The "time-now" line shows that $100k of the project resources was planned to be completed at this point in the project. Another way to look at this is that the project was planned to be 41.6% complete ($100k / $240k) at this point in time.

As work is performed, it is "earned" on the same basis as it was planned, in dollars or other quantifiable units such as labor hours. Comparing earned value with the planned value measures the dollar value of work accomplished versus the dollar value of work planned. Any difference is called a schedule variance.

Earned Value - Planned Costs = Schedule Variance (SV)

(Fig) Schedule variance chart

In our example the task was planned to have accomplished $100k worth of work in twelve months, but the real accomplishment was only $60k. The graph shows a "behind schedule" condition. The schedule variance in dollars would be a negative $40k, the difference between the earned value accomplished ($60k), and the value of the planned work ($100k) to date. According to our formula then:

$60k - $100k = ($40k)

The value earned for the work performed compared with the actual cost incurred for the work performed (taken directly from the contractor's accounting systems), provides an objective measure of cost efficiency. Any difference is called a cost variance.

Earned Value - Actual Costs = Cost Variance (CV)

A negative variance means more money was spent for the work accomplished than was planned. Conversely, a positive variance means less money was spent for the work accomplished than was planned to be spent.

(Fig) Cost variance chart
From the performing organization's own accounting system, we determine the actual costs for performing the $60K work was $110K.

When the actual costs are compared with the earned value of $60k, the difference is the cost variance. The earned value of $60k less the actual cost of $110k, is a negative cost variance of $50k. In this example, the task is in an overrun condition by $50k.

$60k - $110k = ($50k)

Analysis of these variances should reveal the factors causing the deviation from plan.

The Task Manager uses this information in conjunction with his knowledge of the task, to project an estimate to complete for this task. The Task Managers analyzes variances resulting from comparisons of these five basic data elements; planned work, work accomplished, actual costs, total budget at completion and the estimate at completion. The work breakdown structure provides a useful framework for summarizing this performance information for all levels of management.

Earned value improves on the "normally used" spend plan concept (budget versus actual incurred cost) by requiring the work in process to be quantified. The planned value, earned value, and actual cost data provides an objective, quantifiable measurement of performance, enabling trend analysis and evaluation of any cost estimate at completion within multiple levels of the project.

EVM is a valuable tool in the Project Manager's "toolbox" for gaining valuable insight into project performance and is the tool that integrates technical, cost, schedule and risk management. In addition, EVM provides valuable quantifiable performance metrics for forecasting at-completion cost and schedule for their project.

KC 1.4.1 - Quality Assurance & Quality Control

2006-12-09T19:49:00.000-08:00

Quality Assurance

A planned and systematic set of activities to ensure that variances in processes are clearly identified, assessed and improving defined processes for fullfilling the requirements of customers and product or service makers.

A planned and systematic pattern of all actions necessary to provide adequate confidence that the product optimally fulfils customer's expectations.

A planned and systematic set of activities to ensure that requirements are clearly established and the defined process complies to these requirements.

"Work done to ensure that Quality is built into work products, rather than Defects." This is by (a) identifying what "quality" means in context; (b) specifying methods by which its presence can be ensured; and (c) specifying ways in which it can be measured to ensure conformance.

Quality Control

Also called statistical quality control. The managerial process during which actual process performance is evaluated and actions are taken on unusual performance.It is a process to ensure whether a product meets predefined standards and requisite action taken if the standards are not met.

Quality Control measures both products and processes for conformance to quality requirements (including both the specific requirements prescribed by the product specification, and the more general requirements prescribed by *Quality Assurance*); identifies acceptable limits for significant *Quality Attributes*; identifies whether products and processes fall within those limits (conform to requirements) or fall outside them (exhibit defects); and reports accordingly. Correction of product failures generally lies outside the ambit of Quality Control; correction of process failures may or may not be included.

KC 1.4.2 - Just in Time (JIT) technique

2006-12-09T02:19:00.000-08:00

Coming Soon.......

KC 1.5.1 - Quality Pioneers - Edwards Deming

2006-12-08T05:41:00.000-08:00

Dr. W. Edwards Deming

If you can't describe what you are doing as a process, you don't know what you're doing.
W. Edwards Deming

Dr. W. Edwards Deming is known as the father of the Japanese post-war industrial revival and was regarded by many as the leading quality guru in the

United States

. He passed on in 1993.

Trained as a statistician, his expertise was used during World War II to assist the United States in its effort to improve the quality of war materials.

He was invited to Japan at the end of World War II by Japanese industrial leaders and engineers. They asked Dr. Deming how long it would take to shift the perception of the world from the existing paradigm that Japan produced cheap, shoddy imitations to one of producing innovative quality products.

Dr. Deming told the group that if they would follow his directions, they could achieve the desired outcome in five years. Few of the leaders believed him. But they were ashamed to say so and would be embarrassed if they failed to follow his suggestions.

As Dr. Deming told it, "They surprised me and did it in four years."

He was invited back to Japan time after time where he became a revered counselor. For his efforts he was awarded the Second Order of the Sacred Treasure by the former Emperor Hirohito.

Japanese scientists and engineers named the famed Deming Prize after him. It is bestowed on organizations that apply and achieve stringent quality-performance criteria.

Deming's business philosophy is summarized in his famous "14 Points," listed in my previous post. These points have inspired significant changes among a number of leading US companies striving to compete in the world's increasingly competitive environment.

But the 14 Points pose a challenge for many firms to figure out how to apply them in a meaningful way that will result in continual improvement. Leadership Institute has developed powerful processes for coaching executive teams, and eventually their entire organizations, to begin accomplishing what Deming referred to as "the transformation."

His work is outlined in two books: Out of the Crisis and The New Economics, in which he spells out his System of Profound Knowledge.

Courtesy: www.lii.net

Other Very good resources on Edwards Deming:

KC 1.5.1.1 - Deming's 14 Principles

2006-12-06T10:45:00.000-08:00

Deming's 14 Points

Create constancy of purpose for improvement of product and service. Dr. Deming suggests a radical new definition of a company's role: A better way to make money is to stay in business and provide jobs through innovation, research, constant improvement and maintenance.
Adopt the new philosophy. For the new economic age, management need to take leadership for change into a 'learning organisation'. Furthermore, we need a new belief in which mistakes and negativism are unacceptable.
Cease dependence on mass inspection. Eliminate the need for mass inspection by building quality into the product.
End awarding business on price. Instead, aim at minimum total cost and move towards single suppliers.
Improve constantly and forever the system of production and service. Improvement is not a one-time effort. Management is obligated to continually look for ways to reduce waste and improve quality.
Institute training. Too often, workers have learned their job from other workers who have never been trained properly. They are forced to follow unintelligible instructions. They can't do their jobs well because no one tells them how to do so.
Institute leadership. The job of a supervisor is not to tell people what to do nor to punish them, but to lead. Leading consists of helping people to do a better job and to learn by objective methods.
Drive out fear. Many employees are afraid to ask questions or to take a position, even when they do not understand what their job is or what is right or wrong. They will continue to do things the wrong way, or not do them at all. The economic losses from fear are appalling. To assure better quality and productivity, it is necessary that people feel secure. "The only stupid question is the one that is not asked."
Break down barriers between departments. Often a company's departments or units are competing with each other or have goals that conflict. They do not work as a team, therefore they cannot solve or foresee problems. Even worse, one department's goal may cause trouble for another.
Eliminate slogans, exhortations and numerical targets for the workforce. These never help anybody do a good job. Let workers formulate their own slogans. Then they will be committed to the contents.
Eliminate numerical quotas or work standards. Quotas take into account only numbers, not quality or methods. They are usually a guarantee of inefficiency and high cost. A person, in order to hold a job, will try to meet a quota at any cost, including doing damage to his company.
Remove barriers to taking pride in workmanship. People are eager to do a good job and distressed when they cannot. Too often, misguided supervisors, faulty equipment and defective materials stand in the way of good performance. These barriers must be removed.
Institute a vigorous programme of education. Both management and the work force will have to be educated in the new knowledge and understanding, including teamwork and statistical techniques.
Take action to accomplish the transformation. It will require a special top management team with a plan of action to carry out the quality mission. Workers cannot do it on their own, nor can managers. A critical mass of people in the company must understand the 14 points.

KC 1.5.2 - Quality Pioneers - Philip Crosby

2006-12-06T06:20:00.000-08:00

Philip Crosby

"Do It Right the First Time"

Dr. Deming and Dr. Juran were the great brains of the quality revolution. Where Phil Crosby excelled was in finding a terminology for quality that mere mortals could understand. His books, "Quality Without Tears" and "Quality is Free" were easy to read, so people read them. He popularized the idea of the "cost of poor quality", that is, figuring out how much it really costs to do things badly.

Like Frederick Taylor, Philip Crosby's ideas came from his experience on an assembly line. He focused on zero defects, not unlike the focus of the modern Six Sigma Quality movement. Mr. Crosby was quick to point out, however, that zero defects is not something that originates on the assembly line. To create a manufacturing process that has zero defects management must set the tone and atmosphere for employees to follow. If management does not create a system by which zero defects are clearly the objective then employees are not to blame when things go astray and defects occur. The benefit for companies of such a system is a dramatic decrease in wasted resources and time spent producing goods that consumer's do not want.

Mr. Crosby defined quality as a conformity to certain specifications set forth by management and not some vague concept of "goodness." These specifications are not arbitrary either; they must be set according to customer needs and wants.

Four Absolutes of Quality Management

Quality is defined as conformance to requirements, not as 'goodness' or 'elegance'.
The system for causing quality is prevention, not appraisal.
The performance standard must be Zero Defects, not "that's close enough".
The measurement of quality is the Price of Nonconformance, not indices.

Courtesy: www.skymark.com

More Resources:

KC 1.5.3 - Quality Pioneers - Joseph M. Juran

2006-12-05T06:04:00.000-08:00

Joseph M. Juran

Joseph M. Juran made many contributions to the field of quality management in his 70+ active working years. His book, the Quality Control Handbook, is a classic reference for quality engineers. He revolutionized the Japanese philosophy on quality management and in no small way worked to help shape their economy into the industrial leader it is today. Dr. Juran was the first to incorporate the human aspect of quality management which is referred to as Total Quality Management.

The process of developing ideas was a gradual one for Dr. Juran. Top management involvement, the Pareto principle, the need for widespread training in quality, the definition of quality as fitness for use, the project-by-project approach to quality improvement--these are the ideas for which Juran is best known, and all emerged gradually.

Juran is well known for his Quality Trilogy.
The following table outlines the major points of Dr. Juran's quality management ideas:

Quality Trilogy:

Quality Planning	Identify who are the customers. Determine the needs of those customers. Translate those needs into our language. Develop a product that can respond to those needs. Optimise the product features so as to meet our needs and customer needs.
Quality Improvement	Develop a process which is able to produce the product. Optimise the process.
Quality Control	Prove that the process can produce the product under operating conditions with minimal inspection. Transfer the process to Operations.

You can find out more at the links given below:

KC 1.5.4 - TQM (Total Quality Management)

2006-12-04T05:38:00.000-08:00

TQM used to be mentioned as one of the parts of the Quality Concepts Sub-domain in CBOK Skill Category-1 of CBOK. But now, though it is not explicitly mentioned, it is still a buzz word among QA professionals. So, I am putting it as a part of Quality Vocabulary & am giving the necessary resources for it.

TQM - Introduction

Total Quality Management (TQM), a buzzword phrase of the 1980's, has been killed and resurrected on a number of occasions. The concept and principles, though simple seem to be creeping back into existence by "bits and pieces" through the evolution of the ISO9001 Management Quality System standard.

Companies who have implemented TQM include Ford Motor Company, Phillips Semiconductor, SGL Carbon, Motorola and Toyota Motor Company.

The latest changes coming up for the ISO 9001:2000 standard’s "Process Model" seem to complete the embodiment. TQM is the concept that quality can be managed and that it is a process. The following information is provided to give an understanding of the key elements of this process.

Total Quality Management (TQM)

Total = Quality involves everyone and all activities in the company.

Quality = Conformance to Requirements (Meeting Customer Requirements).

Management = Quality can and must be managed.

TQM = A process for managing quality; it must be a continuous way of life; a philosophy of perpetual improvement in everything we do.

TQM Compared to ISO 9001

ISO 9000 is a Quality System Management Standard. TQM is a philosophy of perpetual improvement. The ISO Quality Standard sets in place a system to deploy policy and verifiable objectives. An ISO implementation is a basis for a Total Quality Management implementation. Where there is an ISO system, about 75 percent of the steps are in place for TQM. The requirements for TQM can be considered ISO plus. Another aspect relating to the ISO Standard is that the proposed changes for the next revision (1999) will contain customer satisfaction and measurement requirements. In short, implementing TQM is being proactive concerning quality rather than reactive.

TQM Process Improvement and Problem Solving Sequence
PLAN (PLAN A CHANGE)			DO (IMPLEMENT THE CHANGE)	CHECK (OBSERVE THE EFFECTS)	ACTION (EMBED THE FIX INTO THE PROCESS FOR GOOD)
DEFINE THE PROBLEM	IDENTIFY POSSIBLE CAUSES	EVALUATE POSSIBLE CAUSES	MAKE A CHANGE	TEST THE CHANGE	TAKE PERMANENT ACTION
1. Recognize that what you are doing is a "PROCESS" 2. Identify the commodity being processed. - Process Inference 3. Define some measurable characteristics of value to the commodity. 4. Describe the "PROCESS" o Process Flow Analysis's o Flow charts o List of steps 5. Identify the "Big" problem o Brainstorming o Checklists o Pareto analysis	6. "BRAINSTORM" what is causing the problem. 7. Determine what past data shows. o Frequency distribution o Pareto charts o Control charts - sampling	8. Determine the relationship between cause and effect o Scatter diagrams o Regression analysis 9. Determine what the process is doing now o Control charts - sampling	10. Determine what change would help Your knowledge of the process Scatter diagrams Control Charts - sampling Pareto analysis ****Then make the change.	11. Determine what change worked (confirmation). Histograms Control charts - sampling Scatter diagrams	12. Ensure the fix is embedded in the process and that the resulting process is used. Continue to monitor the process to ensure: A. The problem is fixed for good. and B. The process is good enough o Control charts - sampling ****To ensure continuous improvement, return to step 5.

Courtesy:

http://home.att.net/~iso9k1/tqm/tqm.html#Introduction

Resources:

KC 2 - Quality Leadership - CONTENTS

2006-11-30T06:24:00.000-08:00

2006 CSQA Body of Knowledge
Knowledge Category 2

Quality Leadership

The most important prerequisites for successful implementation of any major quality initiative are leadership and commitment from executive management. Management must create a work environment supportive of quality initiatives. It is management’s responsibility to establish strategic objectives and build an infrastructure that is strategically aligned to those objectives. This category will cover the management processes used to establish the foundation of a quality-managed environment, as well as commitment, new behaviors, building the infrastructure, techniques, approaches and communications.

Leadership Concepts

Executive and Middle Management Commitment
Quality Champion
New Behaviors for Management

Traditional Management versus Quality Management (differences in philosophy and cultural changes)

Leadership (modeling, coaching, reinforcing)

The Importance of Establishing Mentoring Relationships

Establishing Trust

Empowerment of Employees

Quality Management Infrastructure

Quality Council
Management Committees
Teams and Work Groups
Process Improvement Teams

Quality Environment

The Six Attributes of an Effective Quality Environment
Setting the Proper “Tone” at the Top
Code of Ethics and Conduct
Open Communications
Implementing a Mission, Vision, Goals, Values, and a Quality Policy
Monitoring Compliance to Organizational Policies and Procedures
Enforcement of Organizational Policies and Procedures

KC 2.1.1 - Executive and Middle Management Commitment

2006-11-25T05:53:00.000-08:00

Quality management is a way of doing business that allows an organization to design products and services that meet or exceed customer needs and build operational processes that achieve high levels of performance and quality.

Working with managers and teams, Quality Management helps clients apply quantitative and qualitative methods to assess organizational performance and identify opportunities for improvement. It develops leadership skills in senior management so that they can establish and communicate a vision and goals for quality and guide staff to accomplish those goals. It enables program staff to set quality assurance objectives and achieve measurable results through team-based problem solving and other performance improvement approaches. To support organizational change and develop leadership for quality, customized and off-the-shelf training in quality management, as well as nontraditional methods, such as interactive self-instructional systems and "just-in-time" training is very important.

Executive management: Executive Management has to realize the importance of quality in an organisation and the drive towards building better quality has to come from the top. It is quite evident that in all the successful organisations, the drive and commitment from the Executive Management reaped huge benefits. Best example is GE and Motorola in leading the way through their emphasis on quality, through Six Sigma as of late.Hence the Executive Management also need to be trained on how well an organisation can use the quality techniques,approaches and models to reap high ROIs, how to drive and the tecniques to sustain the results for a longer period etc.
Middle management:Once the drive and the initiation on quality comes from the top, it is upto the middle management on how well they can see through these initiatives for successful implementation through the work force. The training for the middle managers emphasizes more on driving the practices of quality models and approaches. This link of middle managers unless handle carefully can create chaos in the organisation.
Work force: Once the commitment is shown from the top as well as the middle management, the wok force which has to implement the processes and procedures set has to be trained on these aspects. Everyone in the organisation has to be trained in such a manner that every person works with a common vision.

KC 2.1.2 - Quality Champion

2006-11-24T10:11:00.000-08:00

Success in today’s quality deployment is almost always expressed in financial terms. Because of the recent economic conditions in the software industry, any significant cost saving metric is almost certain to cause an immediate and "eye-catching" phenomenon amongst most managers and executives. Frustrated by the lack of progress and efficiency, despite significant spending on tools, training and processes, management is always on the hunt for a new fix or an answer to their concerns. This in itself is part of the problem. Software is still lagging in terms of efficiency, quality, cost and customer satisfaction. However, uneducated decisions by these very managers may have been the cause the problem in software to begin with. Another uneducated decision about how to implement quality management will only compound the problem. In response to this we need to be diligent in how we educate interested managers and executives relative to the truths and myths of quality management.

Here comes the role of a true champion.

Defn. Business leaders and senior managers who ensure that resources are available for training and projects, and who are involved in project tollgate reviews.
As given by the CBOK itself, the quality champion is the spokesperson for quality within the IT organization; shares responsibility with IT management to market and deploy quality programs.
This role can have provocative names including Executive Champion, Deployment Champion and so forth in larger organisations which clearly differentiates the functions.

Implementation of quality management needs a culture change. A Quality Champion should be such a person who can handle things dynamically, effectively and efficiently. The need for a quality champion may last for 2 to 3 years.

KC 2.1.3 - New Behaviors for Management

2006-11-23T08:06:00.000-08:00

Controlling Organizational climate

Organizational climate is the employees’ attitude towards their organization. The climate is crucial in creating & maintaining an effective organization.

Steps to assess Organizational climate

• Look at the organization

• Examine jobs within the organization

• Assess Employee’s performance

• Evaluate how the employees feel about their leader

• Create a dialog with the members of the group

• Rate organizational climate (1-5)

Suggestions on how to change the Organizational climate (if required)

• Develop within the group a vision of what needs to be changed

• Change group procedures

• Develop a plan to accomplish the group mission

• Reassign tasks

• Develop new methods of rewarding people

Don’t re-invent the wheel

• Identify Effective quality practices through visits to other companies

– Specific information

– General information

Visit other companies

• Identify discussion areas

• Identify target companies

• Schedule the visit

• Conduct the visit

• Put new practices into use

KC 2.2 - Quality Management Infrastructure

2006-11-22T05:47:00.000-08:00

Building an environment that emphasizes activity throughout the organization is very important. Any Organization’s Quality/Process Focus involves developing and maintaining an understanding of the organization's and projects' software processes and coordinating the activities to assess, develop, maintain, and improve these processes.

The organization provides the long-term commitments and resources to coordinate the development and maintenance of the software processes across current and future software projects via a group such as a Software Engineering Process Group or Process Development Group or Core Process group on the top. Depending on the size of the organisation, there may be other strategic groups to steer the processes through out the organisation. The other groups may be the Steering committes, Process Action Teams etc.

Administering the processes forms very important for the process group. As mentioned int he responsibilities of the Process Group, maintenance of the Process Data Base is very important for the organisation. The requests for changes in the existing processes may frequently be occuring if the processes are not defined or if the people are not trained on them properly. So, the enterprise has to take care of all the intricacies of the administration aspects of the Processes.

KC 2.3 - Quality Environment

2006-11-20T21:41:00.000-08:00

2.3.1 - Attributes of an Effective Quality Environment

Coming soon......

2.3.2 - Setting the Proper “Tone” at the Top

Coming soon......

2.3.3 - Code of Ethics and Conduct

Coming soon......

KC 2.3.4 - Open Communication

2006-11-20T05:54:00.000-08:00

Communication is very important in the quality drive. All the three kinds of the communication – Oral, Written & through Effective Presentations play a major role.

The best example of oral communication is the emphasis that the management gives orally on quality through the face to face sessions with the employees, meetings, work outs etc. Through this, the entire work force understands the commitment of higher management towards quality.

Written communication can be from intimating the work force on who the Management Representative or the Quality Champion is to the extent of educating the people on the happenings on the quality front.

The presentations are very effective tools to drive any initiative. The same thing can have a brand new and attractive look when presented in an effective manner. It all depends on the innovation that the presenter can do.

Lastly, but most importantly, listening – it helps in understanding the people/problems effectively. Most of the successful managers are very good listeners.

So communication, unless planned properly can creat road blocks though the other things are planned very well. This is the reason why many successful project plans inlcude communication plan as well.

KC 2.3.5 - Mission, Vision, Goals, Values, and Quality Policy

2006-11-19T05:44:00.000-08:00

• Mission

1. Official statement of the reason for an organization’s existence, its basic goals and purpose (Horton et al. 2000).

2. Brief summary, approximately one or two sentences, that sums up the background, purposes, and benefits of the project or organization (based on Welcom Glossary).

• Vision

– Is a clear definition of the result you are trying to achieve.

– Established by Senior management

• Goals

– Explain how the vision will be achieved.

– Established by Operational management

• Principles

– Can be defined as procedures

– Established by Senior/Operational Management

• Values

– Can be defined as standards

– Established by Senior Management

KC 2.6 - Compliance to Organizational Policies and Procedures

2006-11-17T21:54:00.000-08:00

Monitoring & Enforcement of Compliance to Organizational Policies and Procedures

Coming soon.........

Quick Guide to Skill Category - 1

2006-10-30T23:01:00.000-08:00

“Quality Principles”

CBOK says: Before an organization can begin to assess the quality of its products and services and identify opportunities for improvement, it first must have a working knowledge of quality principles. This category will test the CSQA candidate’s comprehension and ability to apply these principles.

Vocabulary of Quality:

Defect: Any requirement/functionality that is not met; Any issue causing customer dissatisfaction
Policy: A guiding principle that is typically established by senior management related to processes or products that is adopted by an organization to influence and determine decisions
Procedure: Step by Step activities or tasks performed to achieve a given purpose or standard
Process: CBOK says Process is the work effort required to produce a product. But to understand the main difference between Procedure & Process, understand this – Procedure tells you just the step by step activities or tasks; where as a Process gives the entire picture i.e. Process is a combination of the following: Objective & Scope, Entry Criteria & Exit Criteria, Inputs & Outputs, Responsibility/Ownership, Procedure (Steps)
Productivity: Amount of work done in a specified amount of time (e.g. Productivity of Coding Process can be measured in LINES OF CODE PER HOUR) or in general, ratio of output to input
Quality: “Conformance to Requirements” & “Fit for Use”
Standard: Any type of requirement which should be satisfied by Process or Product

The Different Views of Quality

The Two Quality Gaps: It is the difference between the two Quality Definitions stated above in the section “Vocabulary of Quality”. The organization developing the product (referred as Producer by CBOK) usually thinks they are doing a good job if they are satisfying the documented requirements specifications. But, for the Customer, irrespective of what is documented in specifications, the end product should satisfy their needs. This is where a Quality Gap exists. Closing this gap can be done through better Customer involvement throughout the product development and through Joint Application Development sessions (JAD).

Quality Attributes for an Information System

Correctness: Level of compliance with the requirements specifications and customer needs.

Metrics for correctness: No. of Defects/Bugs, No. of Missed Requirements, No. of Customer Complaints

Reliability: Period for which a product works with required accuracy; simply saying, it is the extent to which we can depend on the product

Metrics for Reliability: Mean Time Between Failures, Mean Time To Repair

Efficiency: Many of the engineers might have remembered that any efficiency is the ratio of output to the input. In case of software, amount of software resources or code required to build the product

Here some people ask me what is the difference between efficiency and effectiveness. If efficiency is bothered about the inputs you are taking to get the required output, effectiveness is bothered only about the output or goals or objectives. That means if you achieve 100% of the required functionality of a product, then your team is said to be very effective whether you have taken 100 days or 1000days to develop the product. Whereas efficiency is bothered about the input effort also while considering the overall efficiency.

Metrics for efficiency: No. of Computer resources required for the product; No. of Source Lines of Code required to achieve a particular functionality

Integrity: It talks about security; this is the level of security which can prevent unauthorized access to the data or software

Metrics for Integrity: No. of virus attacks vs. No. of times firewall is broken

Usability: There are many cases where excellent functionality is built in the products, but are not used by the users just because they cannot understand how to use the product in the easiest possible manner. Hence Usability is defined as the effort required to learn and operate a product or program.

Metrics: Time taken to learn and operate a new product

Maintainability: Many of the production support engineers might have seen that poorly documented code is difficult to maintain. That means, Maintainability is the time required to find and fix a bug in the product

Metrics: Time to locate and fix a bug in an operational program

Testability: Time required to test a product

Metrics: Testing Time

Flexibility: Ease with which a program can be modified to suit the requirements

Metrics: Time to modify a program

Reusability: Extent to which a program or a product component can be reused in other product

Metrics: No. of times a program is reused

Interoperability: Effort needed to merge two systems or products

Metrics: Time to couple two or more products or systems

Quality Concepts and Practices

PDCA Cycle: PDCA stands for Plan à Do à Check à Act; it is one of the basic and simple guidelines for a quality product or process. It expects you to Plan your activity by describing the goals or objectives, Do it by training people and then doing as per the plan, Checking whether it is going as per the plan or not and then Acting on it if it is not going as per the plan
Cost of Quality (COQ): If a team can achieve the entire product functionality right the first time, then they need not spend cost on training, reviews, testing etc. Hence, COQ is defined as the additional cost spent on the project or product to ensure that the final product is defect free. It includes three components:

Preventive Cost: Costs of proactively preventing defects like training cost
Appraisal Cost: Cost for checking the product through reviews and testing
Failure Cost (also called Cost of Poor Quality or COPQ): i.e. the cost of rework as well as cost of lost business

The Three Key Principles of Quality:

Management is primarily responsible for quality (repeated many time in CBOK)
Quality is a journey, not a destination
Use effective Quality Control

Six Sigma Quality: This is a statistical technique to achieve better quality. If a product has only 3.4 defects per million opportunities, is to be said a six sigma product. For example, if a software product is used by 1 million users across the world, it is said to be a six sigma product if not more than 3 users have complained against the product. If you want to see the efficiency of a six sigma product, here it is: 99.99966%
Base lining and Benchmarking: Baseline is the current level of process performance or capability. This is also called Process Performance Baseline (PPB) or Process Capability Baseline (PCB). Base lining can be done for any process or activity. Benchmarking is the process of comparing an organization’s or department’s process or capability with that of another external or internal organization or department
Earned Value: This term is from the famous project management methodology called “Earned Value Methodology”. This technique helps in better project monitoring and control. The term “Earned Value” refers to the amount you have earned by doing a task

Quality Control and Quality Assurance

· To achieve quality, you can proactively take some measures in the project. Those activities which are taken proactively are said to be Quality Assurance (QA). Advantage is you can try to get things right first time and hence you can reduce rework and save time and cost. Training is an example of QA; QA is often related to the processes
· Whereas Quality Control (QC) is the process of verification or checking after a task is done. Therefore, if you find any defects you will have some additional work to do. Therefore QC is reactive. Reviews and Testing are examples of QC; QC is related to the products

Total Quality Management

· All the definitions of Total Quality Management emphasizes one important point : This is a Quality Management method which emphasizes on Quality from everyone in the organization which leads to an Organizational Quality Improvement à hence the name “TOTAL” Quality Management
· Typically in many organizations, quality is confined to a small group known as “Quality Assurance Group” or “Process Group” – this is called small “Q”; whereas TQM focuses on quality from everyone – this is called Big “Q”
· TQM is accomplished using a team organization; both management and the employees are members of "quality teams" (also called process improvement or process action teams), that focus on continuous process improvement
· Success of TQM relies on three Cs – Commitment (from everyone in the organization including management), Competence (of everyone in the organization) & Communication (among all)

Just-In-Time (JIT) Technique

· The basic principles of Just In Time are analogous to those of Agile Software Development Methodologies in Software
· JIT technique helped manufacturing organizations overcome the usual inventory/storage problems and inter departmental bottle necks. JIT came up with revolutionary techniques of easy reproduction through quick job shift from one work to another (known as SMED [single minute exchange of dies]). Similarly this technique can be used in software for better use of reusable components, preparing teams ready for any requirements or requirements changes, which are some of the principles of Agile Software Methodologies

Quality Pioneers Approach to Quality

· Dr. W. Edwards Deming :: Deming’s 14 Principles

Create Consistency of Purpose in the Company (Leadership)
Learn the New Philosophy (Training)
Require Statistical Evidence of Information Technology Quality (Statistical Management)
Reduce the Number of Vendors (Coordination)
Use Statistical Methods to Find Sources of Trouble (Statistical Management)
Institute Modern Aids to Training on the Job(Training)
Improve Supervision (Leadership)
Drive Out Fear (Coordination)
Break Down Barriers Between Departments (Coordination)
Eliminate Numerical Goals, Slogans, Pictures, Posters Urging People to Increase (Leadership)
Look Carefully at Work Standards (Coordination)
Institute a Massive Training Program for Employees in Simple but Powerful Statistical (Statistical Management)
Institute a Vigorous Program for Retraining People in New Skills (Training)
Create a Structure in Top Management that Will Push Every Day on the Above 13 (Leadership)

[No. of Points: Leadership-4; Coordination-4; Training-3; Measurement & Statistical Management-3]

· Philip Crosby :: 14 Steps to achieve Quality

Management Commitment (Leadership)
The Quality Improvement Team (Coordination)
Quality Measurement (Statistical Management)
The Cost of Quality (Statistical Management)
Quality Awareness (Training)
Corrective Action (Statistical Management)
Zero Defects Planning (Statistical Management)
Supervisor Training (Training)
ZD Day (Coordination)
Goal Setting (Statistical Management)
Error-Cause Removal (Statistical Management)
Recognition (Leadership)
Quality Councils (Coordination)
Do it Over Again (Leadership)

[No. of Points: Leadership-3; Coordination-3; Training-2; Measurement & Statistical Management-6]

· Dr. Joseph Juran :: Quality Trilogy

Quality Planning [PLAN for Quality]
Quality Control [DO & CHECK]
Quality Improvement [ACT & Improve]

KC 3 - Quality Models - CONTENTS

2006-10-29T22:15:00.000-08:00

2006 CSQA Body of Knowledge
Knowledge Category 3

Quality Baselines (Assessments and Models)

Organizations need to establish baselines of performance for quality, productivity and customer satisfaction. These baselines are used to document current performance and document improvements by showing changes from a baseline. In order to establish a baseline, a model and/or goal must be established for use in measuring against to determine the baseline.

Quality Baseline Concepts

Baselines Defined
Types of Baselines
Conducting Baseline Studies

Methods Used for Establishing Baselines

Customer Surveys
Benchmarking to Establish a Baseline Goal
Assessments against Management Established Criteria (e.g. software requirements and user acceptance criteria)Assessments against Industry Models

Model and Assessment Fundamentals

Purpose of a Model
Types of Models (Staged and Continuous)
Model Selection Process
Using Models for Assessment and Baselines

Industry Quality Models

Software Engineering Institute Capability Maturity Model Integration/CMMI (CMMI®)
Malcolm Baldrige National Quality Award (MBNQA)
ISO 9001:2000
ISO/IEC 12207: Information Technology – Software Life Cycle Processes
ISO/IEC 15504: Process Assessment
Post-Implementation Audits

KC 3.1 - Quality Baseline Concepts

2006-10-24T23:19:00.000-07:00

Quality Baseline Concepts

Baselines Defined

Types of Baselines

Conducting Baseline Studies

Coming soon............

KC 3.2 - Methods Used for Establishing Baselines

2006-10-23T23:22:00.000-07:00

Methods Used for Establishing Baselines

Customer Surveys

Benchmarking to Establish a Baseline Goal

Assessments against Management Established Criteria (e.g. software requirements and user acceptance criteria)Assessments against Industry Models

Coming Soon..........

KC 3.3.1 - Purpose of a Quality Model

2006-10-23T09:07:00.000-07:00

At a broader level, the two views of quality in terms of ground level expectations are looked in terms of the following parameters:

External view

Market shares, customer satisfaction -
Customers expect that state of the art processes and technologies are employed.

Internal view

Profit and loss, people satisfaction, growth - Internal needs exist: Management of Skills & Resources, Risks Management

To wrap in few sentences, the purpose of a Quality Model can be listed down in the following points:

To satisfy business goals and objectives
Requirements are imposed by a customer
For competitive reasons

to expand market share
to extend market penetration
to better support the customers (real need)
to be more profitable

As a guide (roadmap) to continuous improvement

to improve competitiveness (productivity)

KC 3.3.3- Model Selection Process

2006-10-21T09:14:00.000-07:00

There are different models each designed for a specific purpose. Some of the models are listed below:

ISO/IEC 9000
ISO/IEC 12207
ISO/IEC 15504
ISO/IEC 15288
CMMI Source Models

IPD-CMM v0.98
EIA/IS 731 (SECM)
SW-CMM v 2, draft C

CMMI A-Spec
IEEE
SW-CMM v1.1
EIA 632
SA-CMM
FAA-CMM
P-CMM

Now coming to the point, how an IT organization selects a model depends on many factors which include:

Applicability of model to the IT organization’s goals and objectives. For example, CMMI model alone offers Software Engg./ Systems Engg./ Integrated Process & Product Development/ Supplier Systems for Process Improvement & higher maturities. Likewise the other models offer different features the suitability of which depends on the nature of products/services that the organisation delivers.
Management commitment to include needed:

i. Budget – One of the important factors which decides the model that is going to be chosen. There may be similar models which suits the organisation processes, but the practices that need to be followed and the training required for the staff may differ. Based on all these constraints, the model can be chosen.

ii. Schedule – The time frame in which the goal has to be achieved plays an important role. Based on the schedule, the organisation can choose the model.

iii. Resources – Availability & other factors related to resources too may affect the model that is going to be chosen.

Need for Baseline Assessments – This is discussed in detail in the next topic “Models for Assessment & Baseline”
Need for measurable goals and objectives – This too is one need that can play a big role in the selection of model. This too can be found out in detail in the next topic “Models for Assessment & Baseline”

KC 3.3.4.1 - Using Models for Assessment & Baseline-1

2006-10-19T09:24:00.000-07:00

This section explains the concepts and importance of baselines and how baselines can be used to improve IT capabilities.

First it is important o understand the concept of Baselines first.

Process Capability is the capability of a process is some measure of the proportion of in-specification items the process produces when it is in a state of statistical control. When this mesure for a process is baselined as a standard across the organisation, then it is known as Process capability Baseline.

There is another commonly used term – Process Performance Baseline. There is very thin line between these two.

Process Performance Baseline is a documented characterization of the actual results achieved by following a process, which is used as a benchmark for comparing actual process performance against expected process performance.

Note that a Performance baseline is only an indication of the previous results, the Capability Baseline is the measure which is set to measure future parameters against this set standard. The Process Performance Baseline may be used to decide the Capability Baseline.

This PCB can be established using the historical data in the PDB.

Usually, when the project is completed, a thorough analysis of its actual performance is done with respect to the goals set and deviations are analysed. This is stored in the PDB and used for the following

· Assessing current organization levels of performance in quality, productivity & cycle time

· Assessing current levels of process performance

· Establishing process capability baselines

· Setting process performance improvement goals

Process Variance

At a gross level, major steps involved in delivering IT solution look similar. These are requirements gathering & analysis, design, building and testing the application, verification, validation and installation. However, the processes followed in each project vary significantly owing to a range of factors like nature of service offering, business domain, criticality and complexity of application, technology and so on. Consequently, quality and process performance parameters pertaining to these processes also differ considerably.

Business Impact

Improving the capability of software processes is imperative for achieving organizational business goals. Therefore, the entire gamut of process improvement throws up two major challenges to the practitioners:

· Firstly, how to measure the capability of existing processes right so that it reflects the true picture of different process variants and

· Secondly, how to improve them based on measurement results.

KC 3.3.4.2 - Using Models for Assessment & Baseline-2

2006-10-18T09:17:00.000-07:00

Assessment Principles

Start with an appraisal reference model.
Use a formalized appraisal process.
Involve senior management as appraisal sponsor.
Focus the appraisal on the sponsors business objectives
Observe strict confidentiality and non-attribution of data.
Approach the appraisal collaboratively.
Focus on follow-on activities and decision-making based upon the appraisal results.

Assessments can be at different stages of the implementation of the model. There are different kinds of assessments, for example Class-A, Class-B and Class-C. The Assessments help the Organizations in several different ways.

Using the Assessment with a business perspective with whatever model the organisation chose:

The gap between the needs expressed in the business goals and the existing state of practices,
The level of achievement of quality goals;
Previous assessment results,
The appropriateness of the ongoing
improvements, (their suitability to business)
Performance goals, any difficulty or adaptation/modification necessary, possibly some Return on Investment (ROI)
Trends.

The best example for using models for Assessment & Baselining is CMMI. CMMI uses SCAMPI (Class-A) appraisal process for the final assessment, but before that the organisation can go for different classes of appraisals before they go for the final appraisal.

Using the CMMI model with a business perspective

The gap between the needs expressed in the business goals and the existing state of practices,
The level of achievement of CMMI PA goals;
Previous assessment results,
The appropriateness of the ongoing improvements, (their suitability to business)
Performance goals, any difficulty or adaptation/modification necessary, possibly some
Return on Investment (ROI)
Trends.

Using CMMI, the initial appraisal, Class-B for instance is used for the gap-analysis. Then, using the same model, the maturity of the organisation is baselined at certain level (varying from 2 to 5) based on the performance. Similarly, other models can be used for Assessment and Baselining.

KC 3.4.1 - Capability Maturity Model

2006-10-17T02:28:00.000-07:00

CMM is a general application of software and quality improvement concepts to software development and maintenance. The Capability Maturity Model for Software (CMM) is a framework that describes the key elements of an effective software process. There are different kinds of CMMs available like the PCMM (People CMM), SA-CMM (Software Acquisition CMM) etc. The CMM describes an evolutionary improvement path from an ad hoc, immature process to a mature, disciplined process. The CMM covers practices for planning, engineering, and managing software development and maintenance. When followed, these key practices improve the ability of organizations to meet goals for cost, schedule, functionality, and product quality. The CMM can also be used by an organization to plan Software Process Improvement (SPI). It also reflects the needs of individuals performing software process, improvement, software process assessments, or software capability evaluations; is documented; and is publicly available.

The Five Maturity Levels described by the Capability Maturity Model can be characterized as per their primary process changes made at each level as follows:

1) Initial The software process is characterized as ad hoc, and occasionally even chaotic. Few processes are defined, and success depends on individual effort and heroics.
2) Repeatable Basic project management processes are established to track cost, schedule, and functionality. The necessary process discipline is in place to repeat earlier successes on projects with similar applications.
3) Defined The software process for both management and engineering activities is documented, standardized, and integrated into a standard software process for the organization. All projects use an approved, tailored version of the organization's standard software process for developing and maintaining software.
4) Managed Detailed measures of the software process and product quality are collected. Both the software process and products are quantitatively understood and controlled.
5) Optimizing Continuous process improvement is enabled by quantitative feedback from the process and from piloting innovative ideas and technologies.

· SW-CMM has organized the process framework into 5 Maturity Levels which further have few Key Process Areas (KPAs) defined under them.

· The KPAs have goals defined which need to be achieved in order to attain a particular maturity level.

· Under each KPA, the Activities that need to be performed are mentioned.

· For convenience, the key process areas are organized by common features. The common features are attributes that indicate whether the implementation and institutionalization of a key process area is effective, repeatable, and lasting. The five common features are listed below:

· Commitment toPerform

· Commitment to Perform describes the actions the organization must take to ensure that the process is established and will endure. Commitment to Perform typically involves establishing organizational policies and senior management sponsorship.

· Ability to Perform

· Ability to Perform describes the preconditions that must exist in the project or organization to implement the software process competently. Ability to Perform typically involves resources, organizational structures, and training.

· Activities Performed

· Activities Performed describes the roles and procedures necessary to implement a key process area. Activities Performed typically involve establishing plans and procedures, performing the work, tracking it, and taking corrective actions as necessary.

· Measurement and Analysis

· Measurement and Analysis describes the need to measure the process and analyze the measurements. Measurement and Analysis typically includes examples of the measurements that could be taken to determine the status and effectiveness of the Activities Performed.

· Verifying Implementation

· Verifying Implementation describes the steps to ensure that the activities are performed in compliance with the process that has been established. Verification typically encompasses reviews and audits by management and software quality assurance.

The Capability Maturity Model has been criticized in that, that it does not describe how to create an effective software development organization. The traits it measures are in practice very hard to develop in an organization, even though they are very easy to recognize. However, it cannot be denied that the Capability Maturity Model reliably assesses an organization's sophistication about software development.

The following article is a very good criticism of the CMM model.

http://www.satisfice.com/articles/cmm.htm

Goto this link for CMM tutorial -

http://sixsigmatutorial.com/CMM/CMM-Intoduction.aspx

Detailed info on SW-CMM is available at this SW-CMM technical report:

http://www.sei.cmu.edu/pub/documents/93.reports/pdf/tr24.93.pdf

According to SEI, the following are legacy CMMs now:

Capability Maturity Model for Software (SW-CMM)
Systems Engineering Capability Maturity Model (SE-CMM)
Integrated Product Development Capability Maturity Model (IPD-CMM)

Detailed info on legacy CMMs is given at this link:

http://www.sei.cmu.edu/cmm/cmms/transition.html

CMMi & SW-CMM – A comparison

2006-10-16T04:11:00.000-07:00

The first CMM (CMM v1.0) was developed for software and released in August 1990

Based on this success and the demand from other interests CMMs were developed for other disciplines and functions

Systems Engineering
People
Integrated Product Development
Software Acquisition
Software Quality Assurance
Measurement
Others…….

· While organizations found these various CMMs to be useful they also found them to be:

§ ³Overlapping

§ ³Contradicting

§ ³Lacking clean, understandable interfaces

§ ³Lacking standardization

§ ³Displaying different levels of detail

§ In addition, many organizations also had to deal with ISO 9001 Audits or TickIT audits based on ISO 9000-3.uThis resulted in expensive, confusing and conflicting process improvement programs

The CMM Integration Project was formed to:

³Establish a framework to integrate current and future models
³Build an initial set of integrated models

The source models that served as the basis for the CMMI include:

³CMM for Software v2.0 Draft C
³EIA – 731 Systems Engineering
³IPD CMM (IPD) v0.98a

Requirements Management

uBi-Directional Traceability is now explicitly asked for in requirements Management

³Hard to determine if the delivered product matches the requirements and approved requirements change requests and nothing more without requirements traceability
³Always been necessary but not clearly demanded

uRequirements Management is expected to operate in parallel with Requirements Development and offer support as new requirements are discovered and requirements change requests are made

Project Planning

uThere is a heavier emphasis on having a detailed Work Breakdown Structure
uIncludes a focus on the project having the necessary Knowledge and Skills to execute the project according to the estimations and plan
uData Management or the planning and maintaining of project data items and their contents has been added to the list of project management concerns
uData Management Requires administrative control of project data, both deliverable and non-deliverable

³Some large, critical projects demand that even
Engineering Notebooks with daily entries be placed
under control for audit purposes
³Covers all other forms of data such as CD-ROMs,
Disks, Notebooks, etc
³Part of Project Planning process area

uEstimation focuses on size and complexity while effort and cost, and schedule are determined and established respectively based on the size estimation
uEstimate size and/or relative difficulty or complexity
uDetermine the project effort and cost based on the size and complexity estimations
uEstablish and maintain the project schedule based on the size and complexity estimations
u The identification and involvement of stakeholders is

an important evolution of the “all affected groups”

statement that appeared frequently in the SWCMM

u The required plan for stakeholder interaction includes:

³List of all relevant stakeholders
³Rationale for stakeholder involvement
³Expected roles and responsibilities
³Relationships between stakeholders
³Relative importance of stakeholder to project success by
phase
³Resources needed to ensure relevant stakeholder interaction
³Schedule for phasing of stakeholder interaction

o uThe commitment process is now explicitly defined in Specific Practice 3.3

Project Monitoring and Control

uMonitoring Commitments has also been elevated to specific practice level - (SP 1.2)
uMonitoring Risks and Stakeholder Involvement is also more strongly emphasized in the CMMI compared to the SWCMM
uMonitoring Stakeholder Involvement is explicitly brought out and enables the Generic Practice 2.6 – Identify and Involve Relevant Stakeholders

Process and Product Quality Assurance

uStresses the objective evaluation of products

as well as processes!!

uEvaluation criteria must be established based

on business objectives

³What will be evaluated?
³When or how often will a process be evaluated?
³How will the evaluation be conducted?
³Who must be involved in the evaluation?

Configuration Management

uThe idea of “Software Library” has been

replaced by the more encompassing

“Configuration Management System”

uA configuration management system

includes:

³The storage media

³The procedures

³The tools for accessing the configuration system

Supplier Agreement Management

uReplaces the initial ideas found in Subcontract

Management

uNow incorporates the original intent of

Subcontract Management as well as lessons

learned over the past 7 years 

Measurement and Analysis

Provides a description of a measurement

initiative that involves the following:

³Specifying the objectives of measurement and analysis such that they are aligned with established information needs and business objectives
³Defining the measures to be used, the data collection process, the storage mechanisms, the analysis processes, the reporting processes, and the feedback processes
³Implementing the collection, storage, analysis, and presentation of the data
³Providing objective results that can be used in making business judgments and taking appropriate corrective actions

An organization that barely passes the Measurement and Analysis Common Feature requirements of CMM for Software would not pass the measurement requirements of CMMI
Sets up the organization to evolve its measurement program from basic project management measures to those based on the organization’s set of standard processes to statistical control of selected subprocesses according to the organization’s business needs

Requirements Development

uThe concepts presented in Requirements Development are consistent with very modern publications on Requirements Engineering
uClearly defines the need for identification and care of stakeholders
uIncorporates the interface ideas of Systems Engineering and Software Engineering with regards to gathering, analyzing, documenting, and maintaining requirements found in CMM for Software v1.1 and expands on them
u Requirements Development together with Technical Solution truly shows the recursive and iterative nature of developing requirements:

Stakeholder Needs
Customer Requirements
Product and Product
Component Requirements
Requirements Analysis
Derived Requirements
Allocation to Product
Functions and Product
Components including
Objects, People,
and associated Processes
or People

uThe Requirements Development PA includes a description of developing an operational concept and operational scenarios to refine and discover new requirements, needs, and constraints that include the interaction of the product, the end user and the environment
uIt also includes a strong focus on interface requirements
uIt suggests the use of models, simulations, and prototyping to perform risk assessments to reduce the cost and risk of product development
uIt is very tightly coupled to the Technical Solution process area
uIt emphasizes the idea of starting the process of requirements validation very early in the product lifecycle

Technical Solution

Technical Solution practices apply not only to the product and product components but also to services and product-related processes
Technical Solutions are presented as being developed interactively with product or product component requirements definition
Technical Solution stresses the need for developing alternative solutions

(Engineer in Quality)

To engineer in quality means to add quality to software during the engineering process
uTo achieve this, software engineers must be aware of quality requirements at the same time they are building the functional requirements
uQuality requirements thus take on the same relationship to the product as functional requirements do
uIf we engineer quality in, we add quality to the product as we build it
uQuality Factors (e.g., maintainability, expandability, reliability) were discussed in the CMM/SW Level 4 KPA Software Quality Management – “Quality goals for the project’s software products are defined, monitored, and revised throughout the software lifecycle”
uCMMI discusses the quality factors first in Requirements Development and emphasizes their importance in Technical Solution
uDesign criteria are often established to ensure that the product or product component exhibits one or more of the following quality attributes:

³Modularity
³Clarity
³Maintainability
³Expandability
³Portability
³Efficiency
³Reliability
³Security
³Usability
³Scalability

Product Integration

uProduct Integration presents the concepts to achieve complete product integration through progressive assembly of product components, in one stage or in incremental stages, according to a defined integration strategy
uIt stresses the careful analysis and selection of the optimum integration strategy

³The basis for effective product integration is an integration strategy that uses combinations of techniques in an incremental manner

uIt points out the need to establish and maintain the environment required to support the integration of the product components
uIt introduces the concept of product component and product assembly Checkout, to evaluate its performance and suitability
uIt presents the idea of applying (Product Integration, Verification, and Validation) in successive triplets until the product is ready for packaging and delivery
uIt stresses the effective management of all interfaces to ensure that all interfaces will be complete and compatible

³Interface descriptions
³Interface data

uPackaging and Delivery is specifically called out in Specific Practice 3.4 – an improvement over the information provided in the SWCMM
uInspecting Product Elements Upon Receipt is an activity that is not well done in the industry today and deserves the attention that is now defined in the CMMI!

Verification

uVerification is used to assure that selected work products meet their specified requirements
uVerification assures “You built it right”
uExpects a verification strategy that addresses the specific actions, resources, and environments that will be required for work product verification to be developed

³Developed concurrently and iteratively with the

product and product component designs

uCaptures the ideas of using:

³Peer Reviews
³Load, stress and performance testing
³Functional decomposition based testing
³Simulation
³Prototypes
³Observations and demonstrations
³Operational scenario testing

as they apply to ensuring that the requirements are being addressed at each phase of the development lifecycle from a systems, and

software point of view

Validation

uValidation is used to demonstrate that a product or product component fulfills its intended use when placed in its intended

operational environment

uValidation assures “You built the right thing”

Risk Management

uThe concepts inherent to Risk Management finally made it to Process Area status

³Risk Identification
³Risk Assessment
³Risk Analysis
³Risk Prioritization
³Risk Mitigation
³Risk Contingency Planning

uThe ideas behind Risk Contingency Planning and Risk Mitigation have been merged but are definitely clearer

Decision Analysis & Resolution

uDecision and Analysis presents the concepts of identifying alternatives to issues that have a significant impact on meeting objectives, analyzing the alternatives, and selecting one or more alternatives that best support prescribed objectives
uDecision and Analysis is a new concept for the software world whose time has certainly come
u Decision and Analysis helps determine which issues should be examined by formal decision analysis – typical criteria for when to require formal decision making includes:

³When a decision is directly related to topics assessed as being of medium or high risk
³When a decision is related to changing work products under configuration management
³When a decision would cause schedule delays over a certain percentage or specific amount of time
³When a decision has an impact on the ability to achieve project objectives
³When a decision’s costs are reasonable compared to the decision’s impact

uUnderstanding decision making models from Operations Research can help in making full use of this Process Area

³Linear Optimization Models

Simplex Method for executive decision making

³Stochastic Programming Models
³Dynamic Optimization Models
³Unbounded Horizon Models
³Queuing Decision Models

Organizational Process Definition

uThe wording for this process area has changed subtly but significantly from that of the SWCMM

³Establish and maintain a usable set of organizational process assets including the organization’s set of standard processes
³Acknowledges that an organization may utilize more than one standard process to handle its product lines and business needs

uProcess Database evolved into Organizational Measurement Repository

Integrated Project Management

u Integrated Project Management takes on the aspects of Integrated Software Management and Intergroup Coordination that were found in the SWCMM

³The project is conducted using a defined process that is tailored from the organization's set of standard processes

u It also emphasizes the need to proactively integrate the concepts in the Project Plan and all supporting plans such as:

³Quality assurance plans
³Configuration management plans
³Risk management strategy
³Verification strategy
³Validation strategy
³Product integration plans

Organizational Process Performance

uThe Organizational Process Performance process area was developed to help organizations set the stage for quantitative process management:

³Baselines and models that characterize the expected process performance of the organization's set of standard processes are established and maintained

Performance Baselines

uThe organization’s process performance baselines measure performance for the organization’s set of standard processes at various levels including:

³Individual processes (e.g., test case inspection element)
³Sequence of connected processes
³Processes for the complete lifecycle
³Processes for developing individual work products

uThere may be several process performance baselines to characterize performance for subgroups of the organization – Examples include:

³Product Line
³Application domain
³Complexity
³Team size
³Work product size

Quantitative Project Management

uThis Process Area combines the concepts of Quantitative Process Management and Software Quality Management from the SWCMM point of view
uThe concepts of quantitative management and statistical process control are strongly present in this process area.
uQuantitative Project Management is tightly coupled with Organizational Process Performance, taking standard process

measures from it to achieve stability of subprocesses and providing information back to it once the statistical control boundaries are established

Quantitative Management Concepts

uQuantitative Management is tied to the organization’s strategic goals for product quality, service quality, and process performance
uWhen higher degrees of quality and performance are demanded, the organization and projects must determine if they have the ability to improve the necessary processes to satisfy the increased demands
uAchieving the necessary quality and process performance objectives requires stabilizing the processes that contribute most to the achievement of the objectives
uAssuming the technical requirements can be met, the next decision is to determine if it is cost effective
uReducing process variation is an important aspect to quantitative management:

³It is important to focus on subprocesses that can be controlled to achieve a predictable performance

uStatistical process control is often better focused on organizational areas such as Product Lines where there is high similarity of processes, than on the organization’s entire set of products
uSuccessful application of Quantitative

Management concepts must look closely to:

³The business demands
³The capability of existing processes
³The ability of the organization to bring processes and subprocesses under statistical control in a cost effective manner

Organizational Innovation and Deployment

uCombined Process Change Management and Technology Change Management from the SWCMM point of view
uJust Do It! – Or once one has the innovation ideas identified and analyzed against the organization’s business objectives and cost measures, get it tried and expanded wherever possible throughout the organization
uSubpractices are excellent and provide a solid picture of what is required for this process area

Overview

uThe Organizational Innovation and Deployment process area selects and deploys improvements that can improve the organization’s ability to meet its quality and process performance objectives
uQuality and process performance objectives that this process area might address include:

³Improved product quality
³Increased productivity
³Decreased developed cycle time
³Greater customer and end user satisfaction

uProcess and technology improvements that will be deployed are selected from proposals based on the following criteria:

³A quantitative understanding of the organization’s current quality and process performance
³Estimates of the improvement resulting from the deployment
³The resources and funding available for that deployment
³The expected benefits weighed against the cost and impact to the organization

CMMI Framework provides the opportunity to apply the principles of both the staged and continuous representations in a process improvement oriented manner or a manner.

Clear CMM vs CMMI mapping was given at a Practice level by SEI. You can find the PDF at this location:

http://www.sei.cmu.edu/cmmi/adoption/pdf/stsc-mappings.pdf

KC 3.4.2 - Malcolm Baldrige National Quality Award

2006-10-15T10:05:00.000-07:00

The US Congress established the Malcolm Baldrige National Quality Award in 1987 to raise awareness about quality management and to recognize U.S. companies that have successful quality management systems.

The award program focuses on quality as an integral part of today’s business management practices. The award’s criteria are widely accepted as the standard for quality excellence in business performance. They are designed to help companies deliver ever-improving value to customers and improve overall company performance and capabilities
The annual self-evaluation covers the following seven categories of criteria:

· Leadership
· Strategic Planning
· Customer and Market Focus
· Information and Analysis
· Human Resource Focus
· Process Management
· Business Results

The National Institute of Standards and Technology (NIST), a federal agency within the Department of Commerce, is responsible for managing the Malcolm Baldrige National Quality Award. The American Society for Quality (ASQ) administers the Malcolm Baldrige National Quality Award under a contract with NIST.

Resources:

KC 3.4.4 -- Information Technology - Software Life Cycle Processes

2006-10-13T12:17:00.000-07:00

This standard describes the major component processes of a complete software life cycle and the high-level relations that govern their interactions. This standard covers the life cycle of software from conceptualization of ideas through retirement.

ISO/IEC 12207 describes the following life-cycle processes:

Primary Processes: Acquisition, Supply, Development, Operation, and Maintenance.
Supporting Processes: Documentation, Configuration Management, Quality Assurance, Verification, Validation, Joint Review, Audit, and Problem Resolution.
Organization Processes: Management, Infrastructure, Improvement, and Training.

ISO/IEC 12207 also describes how to tailor the standard for a project.

The publication of an interim amendment to ISO/IEC 12207 is expected early in 2002. This Amendment establishes a co-ordinated set of software process information that can be used for process definition and process assessment and improvement. Some of the standards considered in this effort include: ISO/IEC TR 15504 on assessment, ISO/IEC 14598 on product evaluation and ISO/IEC 15939 on measurement.

A major revision of ISO/IEC 12207 and 15288 is beginning in 2002. Presently, the intent is to harmonize ISO/IEC 12207 (software life cycle process), ISO/IEC 15288 (systems life cycle process), the replacements for ISO 9000-3 (application of ISO9000:2000 to software) and ISO/IEC 15504 (assessment). Other standards and documents are considered for the concept exploration phase.

This standard is the international standard for software life-cycle processes. IEEE and EIA have developede the U.S. implementation of ISO/IEC 12207, which is referred to as IEEE/EIA 12207.

Resources:

Official Website for ISO 12207 - http://www.12207.com/
ESSI-SCOPE's description of ISO/IEC 12207 provides a brief overview of the standard.
ISO/IEC 12207 Software Lifecycle Processes is an article from STSC's CrossTalk journal that provides a detailed description of ISO/IEC 12207.
INTERNATIONAL STANDARD ISO/IEC 12207 SOFTWARE LIFE CYCLE PROCESSES (PDF)

KC 3.4.3 - ISO 9001:2000

2006-10-13T12:10:00.000-07:00

The term ISO 9000 refers to a set of quality management standards. ISO 9000 includes three quality standards: ISO 9000:2000, ISO 9001:2000, and ISO 9004:2000. ISO 9001:2000 presents requirements, while ISO 9000:2000 and ISO 9004:2000 present guidelines. All of these are process standards (not product standards).

ISO first published its quality standards in 1987, revised them in 1994, and then republished an updated version in 2000. These new standards are referred to as the "ISO 9000 2000 Standards".

ISO Implementation

When the Organization’s mission is formalized to control or improve the quality of your products and services, to reduce the costs associated with poor quality, or to become more competitive, it needs to develop a quality management system that meets the requirements specified by ISO 9001:2000. ISO 9004:2000 guidelines help guide the organization to achieve the requirements specified by ISO 9000:2000.

Once the QMS has been fully developed and implemented, internal Audit needs to be carried out to evaluate the compliance with the ISO 9000:2000 requirements and when it is fully compliant, an ISO Auditor has to be invited to do the Audit and based on the Audit performance, the certification will be awarded by the Certification body. Go to reference [1] for further info on the principles & guidelines.

ISO Resources – very good ones:
1. ISO 9001 2000
2. ISO 14001 2004
3. ISO 13485 2003 ISO 17799 2000

KC 3.4.5 - ISO/IEC 15504: Process Assessment

2006-10-12T12:20:00.000-07:00

SPICE stands for Software Process Improvement Capability determination. ISO/IEC 15504 provides a framework for assessment methods and does not prescribe a specific assessment method or model.

ISO/IEC TR 15504 is the international standard for Software Process Assessment. It was first published in 1998 as a Technical Report Type 2. ISO/IEC TR 15504 is re-published as ISO/IEC 15504 during 2003/2005.A standard is published as a Technical Report Type 2, when the subject is still under technical development, or where for any other reason there is the future possibility of an agreement on an International Standard. The development of ISO/IEC 15504 has taken place in parallel with empirical studies of its use performed by the SPICE project.

ISO/IEC TR 15504 consists of the following parts, under the general title Information Technology - Software Process Assessment:

Part 1: Concepts and introductory guide
Part 2: A reference model for processes and process capability
Part 3: Performing an assessment
Part 4: Guide to performing assessments
Part 5: An assessment model and indicator guidance
Part 6: Guide to competency of assessors
Part 7: Guide for use in process improvement
Part 8: Guide for use in determining supplier process capability
Part 9: Vocabulary

Please refer to the reference [2] for complete info.

Relation to Other Frameworks:

ISO/IEC 15504 Part 2, Reference Model for Processes and Process Capability, is strongly related to ISO/IEC 12207 standard, Information technology – Software Life-Cycle Processes.
ISO/IEC 15504 Part 6, Guide to Qualification of Assessors, is strongly related to ISO 10011- 2, Guidelines for Auditing Quality Systems – Part 2: Qualification Criteria for Quality Systems Auditors.
ISO/IEC 15504 is intended to be harmonious with ISO 9000.
SE-CMM used the two-axis architecture of the ISO/IEC 15504 process model.

Resources:

Official SPICE Web site
Overview of ISO SPICE
http://isospice.com/standard/tr15504.htm
Hot news on SPICE model updations from TANTARA
SPICE ISO 15504 FAQs (PDF)
The European Software Institute (ESI) provides additional information on SPICE along with training opportunites.

KC 3.4.6 - Post Implementation Reviews/Audits

2006-10-11T11:40:00.000-07:00

A Post Implementation Review (PIR) is a formal review of a programme or project. It is used to answer the question: Did we achieve what we set out to do, in business terms and if not, what should be done?

For a construction, development or procurement project, a PIR is undertaken when there has been time to demonstrate the business benefits of a new service or building. For a major programme of change there may be several PIRs over time.
If Gateway reviews are carried out on the project, the PIR is a major input to Gateway Review

These reviews could relate to:
· Process/Project Kick-Off (E.g. Proposal, ToR)
· Definition of Requirements
· Process/System Design
· System Build
· Testing
· Training
· Implementation
· Rollout
· Communication
· Project Management

Through these review all or few of these may be summarized:

· Lessons Learned
· Successes
· Areas for Improvement
· Cost Benefit Review
· Resources
· Benefits
· Open Issues
· Process Deployment Issues
· Additional Issues

I strongly recommend everyone to go through the following article to know completely about Post Implementation Reviews.

http://www.ogc.gov.uk/sdtoolkit/reference/deliverylifecycle/figd2

KC 4 - Quality Assurance - CONTENTS

2006-09-30T07:18:00.000-07:00

2006 CSQA Body of Knowledge
Knowledge Category 4

Quality Assurance

Quality Assurance is a professional competency whose focus is directed at the critical processes used to build products and services. The profession is charged with the responsibility for tactical process improvement initiatives that are strategically aligned to the goals of the organization. This category will address the understanding and application of quality assurance practices in support of the strategic quality direction of the organization. The quality practitioner should understand the importance of a quality function, how to implement a quality function and how it matures over time, as well as how to create a quality plan, the use of quality tools, process deployment, and differentiating between internal auditing and quality assurance.

Establishing a Function to Promote and Manage Quality

The Challenges of Implementing a Quality Function
How the Quality Function Matures Over Time
Support in Corporate Quality Management Environment
Implementing an IT Quality Function

Quality Tools

Management Tools
Statistical Tools
Presentation Tools

Process Deployment

Getting Buy-in for Change through Marketing
The Formula for Effective Behavior Change (behavior = individual + environment)
The Deployment Process (assessment, strategic, tactical phases)
Critical Success Factors for Deployment

Internal Auditing and Quality Assurance

Types of Internal Audits
Differences in Responsibilities

KC 4.1 - Establishing a function to promote and manage quality

2006-09-25T11:13:00.000-07:00

It is very important to maintain a function in the organisation which supports the quality vision and mission of the senior management. For this purpose, a Support Unit, most widely called as Quality Assurance Group (QAG) will be established with the Quality Champion heading the group.

The QAG helps achieve the Organisation’s mission and vision through successful quality management – from initiation to deployment and redefining processes if necessary. QAG provides feedback to senior management and staff on the quality of Organisation’s efforts to have a much bigger impact on clients in providing quality services.

The Quality Assurance Group provides expertise in all areas relevant to the quality and reliability in the organisation. Support from QAG may include proposal preparation; development of Quality and Reliability program plans and procedures; management and implementation of contract Quality Assurance and Reliability requirements; and control of process activities. In general, some of the areas of support detailed below are implemented at a level that is commensurate with the specific requirements of a given project.

Training and certifications
Technical documentation review
Design review support
Documentation and change control
Evaluation and selection of procurement sources
Procurement document review and approval
Supplier surveillance
Source inspection
Receiving inspection
Reporting and control of discrepant material
Fabrication document review and approval
Process control
Workmanship standards
Equipment calibration control
Quality records control
Test monitoring
Inventory control
Control of government property
Clean Room Operations

KC 4.2 - Quality tools

2006-09-19T11:44:00.000-07:00

Various Quality tools available are:

• Affinity Chart:
Organize facts, opinions, ideas and issues into natural groupings as an aid to diagnosis on a complex problem.
• Brainstorming:
Get ideas from persons who are potential contributors. No criticism or discussion of ideas is allowed until all the ideas are recorded. The ideas are critically reviewed after the brainstorming session.
• Check sheet:
A simple data recording device - this helps delineate important items and characteristics to direct attention to them and verify they are evaluated.
• Control charts:
Based on the statistical variation of any process - control charts help focus on stability of a process.
• Flow chart:
Organize information about a process in a graphical manner thus making it clear who is impacted.
• Focus Group Survey:
Help better understand customer perceptions of a company's products/services. Focus groups consist of 8-10 current or potential customers who meet for a few hours to discuss a product and is usually moderated.

Following are the different tools & methods available for problem characteristic analysis. Note that the links are provided to the respective tools so that you can dig deeper into each one. I am not giving extra notes here.

Pie Charts
Bar Charts
Run Charts
Radar Charts
Scatter Plots
Histograms
Pareto Charts
Normal Test Plots
Control Charts
Relations Diagram
Pathway
Affinity Diagrams
Brainstorms
Cause and Effect Diagrams
Flowcharts
Force Field Diagrams
Tree Diagrams

KC 4.3 - Process Deployment

2006-09-14T11:34:00.000-07:00

A process is a series of steps or actions that lead to a desired result or output.A set of common tasks that creates a product, service, process, or plan that will satisfy a customer or group of customers.

Processes are largely affected by one or more of the following factors:
a) personnel who operate the processes;
b) materials which are used as inputs (including information);
c) machines or equipment being used in the process (in process execution or monitoring/measurement;
d) methods (including criteria and various documentations used along the process);
e) work environmentUnderstanding how these factors interact and affect processes is a key consideration in process identification.

It is a high level activity that the Process Group needs to perform. It is essential to identify the sequence of activities/steps that needs to be performed as a part of the process.

Process analysis & Understanding

Analysis of the gathered data to understand a process and its strengths and weaknesses and ability to watch a process "in motion" is very important so that recommendations can be made to remove flaw-introducing actions and build upon successful flaw-avoidance and detection actions. Variety of tools are available for performing and understanding the process analysis. The tools can be found in the previous section – Quality Tools

BPMA is one of the new tools which is fast becoming popular. BPMA is a category of software tools and analytical activities designed to provide comprehensive capabilities for the design, modeling, analysis, improvement and documentation of business processes.

At a broader level, BPMA performs these activities:

Design the current process flow
Model the process
Analyze the data
Improve the process
Document the process

A detailed examination of each component of BPMA is presented in this article:

www.ultimus.com/ultwhite/BPMA%20White%20Paper.pdf

KC 4.4 - Internal Auditing and Quality Assurance

2006-09-10T07:39:00.000-07:00

The main role of auditing is to identify and report problems, while the role of QA is to find and implement solutions for those problems. QA should be a leadership position, emphasizing the strong interpersonal activities involved in making improvement occur. While QA performs many appraisals, it strives to be independent of the activities being appraised. Auditing, by nature, has a negative role; QA, by practice, should have a positive role. Confusion between the two roles frequently leads to a negative image of QA.

According to the CBOK, some of the skills and activities that an internal auditor has are not applicable to QA analysts.

· Internal auditors must be knowledgeable of the Standards for the Professional Practice of Internal Auditing and are required to comply with those standards in the performance of their work.

· Internal auditors review the means of safeguarding assets and verify the existence of assets.

· Internal auditors verify compliance to corporate policies, plans, procedures, and applicable laws and regulations.

· Internal auditors normally coordinate their activities and work in conjunction with the organization’s firm of external auditors.

· Internal auditors have direct lines of communication to senior corporate officers and frequently to the organization’s board of directors.

KC 5 - Quality Planning - CONTENTS

2006-08-30T07:47:00.000-07:00

2006 CSQA Body of Knowledge
Knowledge Category 5

Quality Planning

Executive management establishes the vision and strategic goals. Planning is the process that describes how those strategic goals will be accomplished. Quality planning should be integrated into the IT plan so that they become a single plan. In simplistic terms, the IT plan represents the producer and the quality plan represents the customer.

Planning Concepts

The Management Cycle
The Planning Cycle

Integrating Business and Quality Planning

The Fallacy of Two Planning Processes
Planning should be a Single IT Activity

Prerequisites to Quality Planning

The Planning Process

Planning Process Overview
The Six Key Basic Planning Questions
The Common Planning Activities in the Planning Process

Maturing IT Work Processes

QAI Model and Approach to Mature IT Work Processes
How to Plan the Sequence for Implementing Process Maturity

KC 5.1 - Planning Concepts

2006-08-26T07:48:00.000-07:00

The Management Cycle
The Planning Cycle

Coming soon........

KC 5.2 - Integrating Business and Quality Planning

2006-08-21T00:48:00.000-07:00

The Fallacy of Two Planning Processes
Planning should be a Single IT Activity

Coming soon......

KC 5.3 - Prerequisites to Quality Planning

2006-08-16T20:48:00.000-07:00

Coming Soon......

KC 5.4 - The Planning Process

2006-08-12T19:49:00.000-07:00

Planning Process Overview
The Six Key Basic Planning Questions
The Common Planning Activities in the Planning Process

Coming soon........

KC 5.5 - Maturing IT Work Processes

2006-08-09T07:49:00.000-07:00

QAI Model and Approach to Mature IT Work Processes
How to Plan the Sequence for Implementing Process Maturity

Coming Soon.........

KC 6 - Define, Build, Implement and Improve Processes - CONTENTS

2006-07-29T04:22:00.000-07:00

2006 CSQA Body of Knowledge
Knowledge Category 6

Define, Build, Implement and Improve Processes

The world is constantly changing. Customers are more knowledgeable and demanding, therefore, quality and speed of delivery are now critical needs. Companies must constantly improve their ability to produce quality products that add value to their customer base. Defining and continuously improving work processes allows the pace of change to be maintained without negatively impacting the quality of products and services. This category addresses process management concepts, including the definition of a process, the workbench concept and components of a process. Additionally, it will address the understanding of definitions and continuous improvement of a process through the process management PDCA cycle.

Process Management Concepts

Definition of a Process
Why Processes are Needed
Process Workbench and Components (standards, input, work and check procedures, output/deliverables)
Process Categories (management processes, work processes, and check processes)
The Process Maturity Continuum (products and services, work and check processes, customer involvement)
How Processes are Managed
Process Template

Process Management Processes

Planning Processes

Process Inventory

Process Mapping

Process Planning

Do Processes

Process Definition

Check Processes

Identify Control Points

Process Measurement

Testing

Act Processes

Process Improvement Teams

Process Improvement Process

KC 7 - Quality Control Practices

2006-06-28T08:04:00.000-07:00

2006 CSQA Body of Knowledge
Knowledge Category 7

Quality Control Practices

Quality control practices should occur during product development, product acquisition, product construction at the end of development/acquisition and throughout product change and operation. During development, the quality control process is frequently called verification and at the conclusion of development, it is called validation. This category will address the various types of controls and when they are best used in the process. The quality practitioner should also be familiar with developing testing methodologies, verification and validation techniques, the framework for developing testing tactics, change control and configuration management.

Testing Concepts

The Testers’ Workbench
Test Stages (Unit, Integration, System, User Acceptance)
Independent Testing
Static versus Dynamic Testing
Verification versus Validation
Stress versus Volume versus Performance
Test Objectives
Reviews and Inspections

Developing Testing Methodologies

Acquire and Study the Test Strategy
Determine the Type of Development Project
Determine the Type of Software System
Determine the Project Scope
Identify the Tactical Risks
Determine When Testing Should Occur
Build the System Test Plan
Build the Unit Test Plans

Verification and Validation Methods

Management of Verification and Validation
Verification Techniques (reviews, code walkthroughs, requirements tracing)
Validation Techniques (white box, black box, incremental, thread, regression)
Structural and Functional Testing

Software Change Control

Software Configuration Management
Change Control Procedures

Defect Management

Defect Management Process
Defect Reporting
Severity versus Priority
Using Defects for Process Improvement

KC 7.1 - Testing Concepts

2006-06-26T08:04:00.000-07:00

Coming Soon..........

The Testers’ Workbench
Test Stages (Unit, Integration, System, User Acceptance)
Independent Testing
Static versus Dynamic Testing
Verification versus Validation
Stress versus Volume versus Performance
Test Objectives
Reviews and Inspections

KC 7.2 - Developing Testing Methodologies

2006-06-17T00:06:00.000-07:00

Coming Soon.........

Acquire and Study the Test Strategy
Determine the Type of Development Project
Determine the Type of Software System
Determine the Project Scope
Identify the Tactical Risks
Determine When Testing Should Occur
Build the System Test Plan
Build the Unit Test Plans

KC 7.3.1 - Validation Methods

2006-06-10T11:48:00.000-07:00

a. Test Concepts

Ø Static Black Box Testing

• Testing the specification

Ø Dynamic Black Box Testing

Behavioral Testing

Testing a running program without any knowledge of how the software behaves.

Test to pass

Test-to-pass approach is where test cases are developed to clarify if the software produced fundamentally works in ordinary circumstances.

Ø Dynamic Black Box Testing

Data Testing

Data testing is performing tests to check the information a user inputs and the results that are received.

Boundary Conditions

Test data by inputting the boundary values.

Ø Dynamic Black Box Testing

• Invalid data

Provide invalid data to the software to check if the software still works.
Example of a Test-to-fail condition.

State Transitions: This is testing a program’s logic flow through its various states.
Regression: The process of re-running test cases to ensure that changes have not propagated unintended side effects.

Ø Static White Box Testing

• Code review and analysis

• Formal Reviews

Detailed rigorous inspection of code to identify problems and decide to follow rules.

• Peer Reviews

An informal meeting between the programmer and other programmers, getting together to discuss the code.

Ø Static White Box Testing

• Walkthroughs

A formal presentation given by the programmer to a group of other programmers & testers, explaining code, its purpose and reason. Key points covered would be to summarize the inputs and how bugs would be addressed.

• Inspection & Coding Standards

The Code would be reviewed by a number of other programmers from a different perspective, such as a user, tester, or a product support person, thus help identify different bugs.Guidelines are adopted for coding standards.

Ø Static White Box Testing

• Generic Code review checklist

Data Reference Errors.

To identify bugs caused by a variable, constant, array, string or record.

2. Data Declaration Errors

To identify improper declaration, usage of variables and constants.

3. Computation Errors.

To identify errors in usage of less than, equal, not equal, true, false comparators.

4. Control Flow Errors

To identify errors in usage of loops.

5. I/O Errors: To identify errors due to improper usage of I/O statements.

Ø Dynamic White Box Testing

• Structural Testing

This testing is using information you gain from seeing what the code does and how it works to determine what to test, what not to test, and how to approach the testing.

• Unit Testing

i) Data Coverage

Divide the code into data and its states.

ii) Data Flow

Track a piece of data completely through the software.

iii) Code coverage

Program and line, Branch , Condition coverage

Ø Other Types of Testing

User Interface Testing

Test the user interface for operability, ease of use.

Performance

Testing the performance of the computations.

Load

Testing the performance of the system under varying load conditions.

Stress

Testing the performance and accuracy of system under resource crunch or any other kind of stress.

Volume: Testing with high volume of data.

Configuration

Testing the system under various hardware/software configurations.

Compatibility

Testing the system for ability to interact with other systems.

8. Installation

Testing the installation of system under one or more configurations.

9. Documentation

Testing the system for documentation clarity.

10. Random- Monkey & Gorillas

Testing for errors with no background on system.

B.2.a.iv. Independent Verification and Validation

Independent Verification and Validation (IV&V) is a process whereby the products of the
software development life cycle phases are independently reviewed, verified, and validated by
an organization that is neither the developer nor the acquirer of the software.  The IV&V agent
should have no stake in the success or failure of the software.  The IV&V agent's only interest
should be to make sure that the software is thoroughly tested against its complete set of
requirements.

The IV&V activities duplicate the V&V activities step-by-step during the life cycle, with the
exception that the IV&V agent does no informal testing.  If there is an IV&V agent, the formal
acceptance testing may be done only once, by the IV&V agent.  In this case, the developer will
do a formal demonstration that the software is ready for formal acceptance.

KC 7.3.2 - Verification Methods

2006-06-09T21:42:00.000-07:00

a. In Process Reviews

Reviews are conducted during and at the end of each phase of the life cycle to determine
whether established requirements, design concepts, and specifications have been met.  Reviews
consist of the presentation of material to a review board or panel.  Reviews are most effective
when conducted by personnel who have not been directly involved in the development of the
software being reviewed.

Informal reviews are conducted on an as-needed basis.  The developer chooses a review panel
and provides and/or presents the material to be reviewed.  The material may be as informal as
a computer listing or hand-written documentation.

i. Walkthroughs.

A walkthrough is a detailed examination of a product on a step-by-step or line-of-code by
line-of-code basis.  The purpose of conducting walkthroughs is to find errors.   The group that
does an walkthrough is composed of peers from development, test, and quality assurance.

ii. Inspections.

Formal reviews (Inspection) are typically conducted at the end of each life cycle phase.  
The acquirer of the software appoints the formal review panel or board, who may make or
affect a go/no-go decision to proceed to the next step of the life cycle. Formal reviews include
the Software Requirements Review, the Software Preliminary Design Review, the Software
Critical Design Review, and the Software Test Readiness Review.

Specific to code inspection, The Code would be reviewed by a number of other
programmers from a different perspective, such as a user, tester, or a product
support person, thus help identify different bugs.Guidelines are adopted for
coding standards.

iii. Requirements Tracing.

Requirements traceability is defined as the ability to describe and follow the life of a requirement, in both a forward and backward direction (i.e., from its origins, through its development and specification, to its subsequent deployment and use, and through periods of ongoing refinement and iteration in any of these phases).

For any given project, a key milestone (or step) is to determine and agree upon requirements traceability details. Initially, three important questions need to be answered before embarking on any particular requirements traceability approach:

What needs to be traceable?
What linkages need to be made?
How, when, and who should establish and maintain the resulting database?

Once the questions are answered, then selection of an approach can be made. One approach could be the structured use of general-purpose tools (e.g., hypertext editors, word processors, and spreadsheets) configured to support cross-referencing between documents. For large software development projects, an alternative approach could be the use of a dedicated workbench centered around a database management system providing tools for documenting, parsing, editing, decomposing, grouping, linking, organizing, partitioning, and managing requirements. Following paragraphs describes the features of each of the approaches.

General purpose tools

· readily available

· flexible

· good for small projects

Workbenches

· fine-grained forward, backward, horizontal, and vertical RT

· RT results may facilitate later development activities (i.e., testing)

· suitable for large projects

Regardless of the approach taken, requirements tracing requires a combination of models (i.e., representation forms), methods (i.e., step by step processes), and/or languages (i.e., semiformal and formal) that incorporate the above techniques.

b. Phase-End Reviews

A formal review held at the end of each project phase to gain consensus that the phase is complete.

The objectives of the review are:

Verify that phase objectives have been met
Verify phase exit criteria
Evaluate phase performance in terms of:
Has the schedule been met?
Has the required scope been delivered?
Do planned costs equal actual costs?
Determine the effectiveness of software development processes used
Identify process improvements

The required outcome from the review is to receive authorisation to continue to the next phase or close out the project if it is in its final phase.

KC 7.4 - Software Change Control

2006-06-06T12:20:00.000-07:00

Software Change Control - The process by which a software change is proposed, evaluated, approved or rejected, scheduled, implemented, closed, and tracked

It is formally defined (IEEE610.12-90) as “A discipline applying technical and administrative direction and surveillance to: identify and document the functional and physical characteristics of a configuration item, control changes to those characteristics, record and report change processing and implementation status, and verify compliance with specified requirements.”

a & b. Project & Environment.

An effective software change request (SCR) process requires the use of supporting tools and procedures ranging from paper forms and a documented procedure to an electronic tool for originating change requests, enforcing the flow of the change process, capturing CCB decisions, and reporting change process information. A link between this tool capability and the problem-reporting system can facilitate the tracking of solutions for reported problems. Approved SCRs are implemented using the defined software procedures in accordance with the applicable schedule requirements. Since a number of approved SCRs might be implemented simultaneously, it is necessary to provide a means for tracking which SCRs are incorporated into particular software versions and baselines. As part of the closure of the change process, completed changes may undergo configuration audits and software quality verification. This includes ensuring that only approved changes have been made. The change request process described above will typically document the SCM (and other) approval information for the change.

c. Version Control.

Software items evolve as a software project proceeds. A version of a software item is a particular identified and specified item. It can be thought of as a state of an evolving item. A revision is a new version of an item that is intended to replace the old version of the item. A variant is a new version of an item that will be added to the configuration without replacing the old version.

KC 7.5 - Defect Management

2006-06-03T12:02:00.000-07:00

Identification of the most common sources of information and the different methods, frequency, and types of reporting is a very important activity in an organisation. Through the defect reports provided, further plans on defect prevention or process improvement can happen. Presently, many defect tracking software are available both open source as well as commercial.

Defect Prevention involves analyzing defects that were encountered in the past and taking specific actions to prevent the occurrence of those types of defects in the future. The defects may have been identified on other projects as well as in earlier stages or tasks of the current project. Defect prevention activities are also one mechanism for spreading lessonslearned between projects.

Trends are analyzed to track the types of defects that have been encountered and to identify defects that are likely to recur. Based on an understanding of the project's defined software process and how it is implemented (as described in the Integrated Software Management and
Software Product Engineering key process areas), the root causes of the defects and the implications of the defects for future activities are determined.

Other resources:

Defect tracking tools - http://testingfaqs.org/t-track.html
Gauging software readiness with defect tracking

KC 8 - Metrics and Measurement - CONTENTS

2006-05-30T08:15:00.000-07:00

2006 CSQA Body of Knowledge
Knowledge Category 8

Metrics and Measurement

A properly established measurement system is used to help achieve missions, visions, goals, and objectives. Measurement data is most reliable when it is generated as a by-product of producing a product or service. The QA analyst must ensure that quantitative data is valued and reliable, and presented to management in a timely and easy-to-use manner. Measurement can be used to gauge the status, effectiveness and efficiency of processes, customer satisfaction, product quality, and as a tool for management to use in their decision-making processes. This category addresses measurement concepts, the use of measurement in a software development environment, variation, process capability, risk management, the ways measurement can be used and how to implement an effective measurement program.

Measurement Concepts

Standard Units of Measure
Metrics
Objective and Subjective Measurement
Types of Measurement Data (nominal, ordinal, interval, ratio)
Measures of Central Tendency (mean, medium, mode etc.)
Attributes of Good Measurement
Using Quantitative Data to Manage an IT Function
Key Indicators

Measurement in Software

Product Measurement (size, complexity, quality and customer perception)
Process Measurement

Variation and Process Capability

The Measurement Program
Installing the Measurement Program
Common and Special Causes of Variation
Variation and Process Improvement
Process Capability

Risk Management

Defining Risk
Characterizing Risk (situational, time-based, interdependent, magnitude dependent, value-based)
Managing Risks
Software Risk Management
Risks of Integrating New Technology

Implementing a Measurement Program

The Need for Measurement
Prerequisites

KC 8.1 - Measurement Concepts

2006-05-28T08:19:00.000-07:00

Coming soon........

Standard Units of Measure
Metrics
Objective and Subjective Measurement
Types of Measurement Data (nominal, ordinal, interval, ratio)
Measures of Central Tendency (mean, medium, mode etc.)
Attributes of Good Measurement
Using Quantitative Data to Manage an IT Function
Key Indicators

KC 8.2.1 – Product measurements - Size

2006-05-24T10:15:00.000-07:00

Lines of code (LOC) & Function/feature points are the most commonly used size measures for software.

Lines of code

A size-oriented metric
Easy to measure
Easy to compare
Easy to differentiate wrt time, cost, etc.
Programming language dependent (e.g. 1 OO-LOC = 3 3GL-LOC = 9 assembler-LOC)
Meaningless in isolation
Penalize efficient design and coding
"Measure what is easy to measure, not what is important to measure"

Function/feature points

An empirical formula based on...
...the number of inputs to a (sub-)system
...the number of outputs it produces
...the number of files (data repositories) used
...the number of interfaces to other systems
...the number of algorithms encapsulated
Numerical measure the "functionality" or "utility" of a system.
But is empirically derived and does not correspond to any directly perceivable quantity (i.e. just a number)

KC 8.2.2 – Product Measures - Complexity

2006-05-23T10:12:00.000-07:00

In general, Cyclomatic complexity and the number of software knots are used as the complexity measures.

McCabe's Cyclomatic Complexity

This metric is an indication of the number of 'linear' segments in a method (i.e.sections of code with no branches) and therefore can be used to determine the number of tests required to obtain complete coverage. It can also be used to indicate the psychological complexity of a method.

A method with no branches has a Cyclomatic Complexity of 1 since there is 1 arc. This number is incremented whenever a branch is encountered. In this implementation, statements that represent branching are defined as: 'for', 'while', 'do', 'if', 'case' (optional), 'catch' (optional) and the ternary operator (optional). The sum of Cyclomatic Complexities for methods in local classes is also included in the total for a method.

Cyclomatic Complexity is a procedural rather than an OO metric. However, it still has meaning for OO programs at the method level.

Example:

Cyclomatic complexity is a software metric that provides a quantitative measure of the logical complexity of a program. When used in the context of basis path testing method, the value computed for Cyclomatic complexity defines the number of independent paths in the basis set of a program and provides us with an upper bound for the number of tests that must be conducted to ensure that all statements have been executed once. An independent path is any path through the program that introduces at least one new set of processing statements or a new condition. When stated in terms of a flow graph, an independent path must move along at least one edge that has not been traversed before the path is defined. For example, a set of independent paths for the flow graph given below

Path 1: 1-11

Path 2: 1-2-3-4-5-10-1-11

Path 3: 1-2-3-6-8-9-10-1-11

Path 4: 1-2-3-6-7-9-10-1-11

Each new path introduces a new edge. The path 1-2-3-4-5-10-1-2-3-6-8-9-10-1-11 is not considered to be an independent path because it is simply a combination of already specified paths and does not traverse any new edges. Paths 1, 2, 3 and 4 constitute a basis set for the for the above flow graph. That is, if tests can be designed to force execution of these paths(a basis set), every statement in the program will have been guaranteed to be executed at least one time and every condition will have been executed on its true and false sides. This basis set is not unique. In fact a number of different basis sets can be derived for a given procedural design.

Cyclomatic complexity has a foundation in graph theory and provides us with an extremely useful software metric. Complexity is computed in one of the three ways.

1. The number of regions of the flow graph corresponds to the Cyclomatic complexity.
2. Cyclomatic complexity, V(G), for a flow graph, G is defined as

V(G) = E – N + 2

Where E is the number of flow graph edges, N is the number of flow graph nodes. For those having doubt on how to find what is a node and what is edge - consider the following definition: "In graph theory, a graph consists of a set of connections between objects. Each object is a vertex or node. Each connection, between two vertices, forms an edge, or arc"

3. Cyclomatic complexity, V(G) for a flow graph G is also defined as

V(G) = P + 1

Where P is the number of predicate nodes contained in the flow graph G.

Referring to the above figure, the Cyclomatic complexity can be computed using each of the algorithms.

1. the flow graph has four regions.
2. V(G) = 11 edges – 9 nodes + 2 = 4
3. V(G) = 3 predicate nodes + 1 = 4

Therefore the Cyclomatic complexity of the flow graph is 4. The value of V(G) provides us with an upper bound for the number of independent paths that form the basis set and by implication, an upper bound on the number of tests that must be designed and executed to guarantee coverage of all program statements.

You can go through the reference [1] for very good info on software complexity.

Further References:

Software Complexity by Thomas J. McCabe and Arthur H. Watson, McCabe and Associates, Inc.

KC 8.2.3 – Product Measurements - Defect Metrics

2006-05-20T10:16:00.000-07:00

Defects Per Unit (DPU)

A "Unit" simply refers to the bounded work-product that is delivered to a customer. In a software product, Units usually exist in a hierarchy - just as the software in a 'unit test' (a Unit at that level) may be integrated into subsystems (each one a Unit at the next higher level) and finally a system (a high level Unit).

DPU, the simplest Six Sigma measure, is calculated as Total Defects/Total Units. If 100 units are inspected and 100 defects are found, then DPU = 100/100 = 1.

Defects (%)

Once you have determined the operational definition of what constitues a defect:

The total number of defects counted on the population in question divided by the total population count.

Defect Containment Effectiveness (DCE)

Go through this article to find out what exactly DCE is:

http://www.isixsigma.com/library/content/c031001a.asp

Defects per million opportunities (DPMO)

Defects per million opportunities (DPMO) is the average number of defects per unit observed during an average production run divided by the number of opportunities to make a defect on the product under study during that run normalized to one million.

Defects Per Million Opportunities. Synonymous with PPM.

To convert DPU to DPMO, the calculation step is actually DPU/(opportunities/unit) * 1,000,000.

Defects / Function Point or Defects per KLOC

No. of defects/Size in FPs or KLOC

KC 8.2.4 – Customer Satisfaction

2006-05-18T10:17:00.000-07:00

Determination of the level of service perceived by the customer including the ability to meet requirements and overall expectation is very important to measure the customer satisfaction and to improve it to stay in today’s competetive world. Here the customer may rate the quality, usefulness, and effectiveness of the delivered product/service.

Customer satisfaction is typically measures through one or more of the following:

Surveys
Interviews
Feedback forms …………

Following is a good article on Customer Satisfaction measures:

http://www.stickyminds.com/getfile.asp?ot=XML&id=3415&fn=XDD3415filelistfilename1%2Epdf

Other References

KC 8.2.5 – Service Level Agreements (SLA)

2006-05-16T10:18:00.000-07:00

These are agreements between suppliers and customers, or between operating units of a company, to provide business services under specific terms. Whether explicit or implied, SLAs form the basis of all operating aspects of a company. Failure to deliver on known or unknown SLA commitments cause chaos within companies. Key elements include provisioning time, performance, security, and availability.

In most of the organisations, adherence to SLA is set upon as one of the top metrics that both management as well as the client reviews.SLA adherence is typically measured int erms of per cent.

KC 9 - Internal Control and Security - CONTENTS

2006-04-30T06:04:00.000-07:00

2006 CSQA Body of Knowledge
Knowledge Category 9

Internal Control and Security

Privacy laws and increased accessibility to data have necessitated increased security. Accounting scandals and governmental regulation such as the Sarbanes-Oxley Act have placed increased importance on building and maintaining adequate systems of internal control. The quality assurance function can contribute to meeting those objectives by assuring that IT has adequate processes governing internal control and security.

Principles and Concepts of Internal Control

Internal Control and Security Vocabulary and Concepts
Preventive, Detective and Corrective Controls

Risk and Internal Control Models

COSO Enterprise Risk Management (ERM) Model
COSO Internal Control Framework Model
CobiT Model (Control Objectives for Information and related Technology Model)

Building Internal Controls

Perform Risk Assessment

Building Adequate Security

Where Vulnerabilities in Security Occur
Establishing a Security Baseline
Security Awareness Training
Security Practices

March 2006 Exam Questions

2006-04-22T09:10:00.000-07:00

Hey friends,

I'm posting here a post as it is from "CQAfolks" yahoo group for your reference. Please note that I cant guarantee the correctness of this.

------------------------------------------------------------------------------------

Following is the list of subjective questions from the March 2006 exam. Each question carries 10 marks.

Paper 1:
1. Explain testing process with the help of PDCA.
2. Why do companies select quality models (ISO, CMMi, etc)
3. A question on PDCA and the concept of continuous improvement.
4. Explain: a) Environment planning. b) Assumption planning
5. A question on Control chart
6. Define the following with 2 examples each. a) Preventive controls. b) Detective controls c) Corrective controls.
7. Differentiate verification and validation. Explain 2 techniques for each and the output from each technique.
8. Describe reliability and validity.
9. Demonstrate software in operation - desired aspects to be available.
10. A question on Quality Management Infrastructure.

Paper 2:
1. Metrics for reliability, usability, maintainability, security.
2. Design a mentoring program to transfer knowledge from seniors to juniors.
3. Cause-&-effect diagram for customer satisfaction.
4. Benchmarking - definition. what should be done before that?
5. Sample Control charts of 2 projects were given - one with defect rates within control limit - other outside control limits. Explain.
6. Software contracted outside. Failure reasons and mitigation plans.
7. People Management Capability Model.
8. Steps for security baseline.
9. Describe a customer satisfaction program.
10. Process Improvement Team - a) who are the members. b) what are their responsibilities.

KC 9.2.1 - COSO Internal Control Framework Model

2006-04-20T04:47:00.000-07:00

Committee of Sponsoring Organizations of the Treadway Commission (COSO), is a U.S. private-sector initiative, formed in 1985. Its major objective is to identify the factors that cause fraudulent financial reporting and to make recommendations to reduce its incidence. COSO has established a common definition of internal controls, standards, and criteria against which companies and organizations can assess their control systems.

COSO definition of internal control

Internal control is a process, effected by an entity’s board of directors, management and other personnel, designed to provide reasonable assurance regarding the achievement of objectives in the following categories:

Effectiveness and efficiency of operations
Reliability of financial reporting
Compliance with applicable laws and regulations

COSO framework: the five components

Internal control consists of five interrelated components. These components provide an effective framework for describing and analyzing the internal control system implemented in an organization. The five components are the following:

Control environment: The control environment sets the tone of an organization, influencing the control consciousness of its people. It is the foundation for all other components of internal control, providing discipline and structure. Control environment factors include the integrity, ethical values, management's operating style, delegation of authority systems, as well as the processes for managing and developing people in the organization.

Risk assessment: Every entity faces a variety of risks from external and internal sources that must be assessed. A precondition to risk assessment is establishment of objectives and thus risk assessment is the identification and analysis of relevant risks to achievement of assigned objectives. Risk assessment is a prerequisite for determining how the risks should be managed.

Control activities: Control activities are the policies and procedures that help ensure management directives are carried out. They help ensure that necessary actions are taken to address risks to achievement of the entity's objectives. Control activities occur throughout the organization, at all levels and in all functions. They include a range of activities as diverse as approvals, authorizations, verifications, reconciliations, reviews of operating performance, security of assets and segregation of duties.

Information and communication: Information systems play a key role in internal control systems as they produce reports, including operational, financial and compliance-related information, that make it possible to run and control the business. In a broader sense, effective communication must ensure information flows down, across and up the organization. Effective communication should also be ensured with external parties, such as customers, suppliers, regulators and shareholders.

Monitoring: Internal control systems need to be monitored--a process that assesses the quality of the system's performance over time. This is accomplished through ongoing monitoring activities or separate evaluations. Internal control deficiencies detected through these monitoring activities should be reported upstream and corrective actions should be taken to ensure continuous improvement of the system.

The role of Internal Audit

Internal auditors play an important role in evaluating the effectiveness of control systems. As an independent function reporting to the top management, Internal Audit is able to assess the internal control systems implemented by the organisation and contribute to ongoing effectiveness. As such internal audit often plays a significant monitoring role. In order to preserve its indepedence of judgement Internal Audit should not take any direct responsibility in designing, establishing or running the controls it is supposed to evaluate. It may only advise on potential improvement to be made.

KC 9.2.2 - COSO Enterprise Risk Management (ERM) Model

2006-04-18T10:08:00.000-07:00

Internal Controls & ERM

"Management" owns I/C & ERM
Internal auditors & others provide information
"Internal Control" is broadly defined & includes ISO, TQM, process improvement, balanced scorecards, six sigma etc.
Enterprise Risk Management is broader than, and encompasses, I/C
One definition...

One Definition of IC & ERM
"COSO" stands for the Committee of Sponsoring Organizations of the Treadway Commission. The sponsoring organizations are:

Institute of Internal Auditors (IIA)
American Institute of Certified Public Accountants (AICPA)
American Accounting Association (AAA)
Institute of Management Accountants (IMA)
Financial Executives Institute (FEI)

Later, COSO is also endorsed by GAO, Federal agencies & SEC.

The Internal Control (I/C) - Integrated framework was introduced in the year 1992 where as the Enterprise Risk Management (ERM) - Integrated framework was introduced as recently as 2004.

ERM Defintion
"Enterprise Risk Management is a process, enected by an entity's board od directors, management & other personnel, applied in strategy setting & across the enterprise, designed to identify potential events that may affect the entity, and manage risk to be within its risk appetite, to provide reasonable assurance regarding the achievement of entity objectives."

Objective Categories are:

Strategic - high level goals aligned with & supporting its mission
Operations - effective & efficient use of its resources
Reporting - reliablity of reporting
Compliance - Compliance with applicable laws & regulations

Definition of Internal Control
Internal Control is a process, affected by an entity's board of directors, management and other personnel, designed to provide reasonable assurance regarding the achievement of objectives in the following categories:

Effectiveness & efficiency of operations
Reliablity of financial reporting (SOX focus)
Compliance with applicable laws & regulations

Components of Internal Control
Control Environment - The core of any business is it peoplr - their individual attributes, including integrity, ethical values & competence - and the environement in which they operate. They are the engine that drives the entity and the foundation on which everything rests.
Risk Assessment - The entity must be aware of and deal with the risks it faces. It must set objectives, integrated with the sales, production, marketing, financial and other activities so that the organization is operating in concert. It alsomust establish mechanisms to identify, analyze & manage related risks.
Control Activities - Control policies & procedures must be established and executed to help ensre that the actiuons identified by management as necessary to address risks to achievement of the entity's objectives are effectively carried out.
Information & Communication - Surrounding these activities are information & communication systems. These enable the entity's people to capture & exchange the information needed to condct, manage and control its operations.
Monitoring - The entire process must be monitored, and modifications made as necessary. In this way, the system can react dynamically, changing as conditions warrant.

Soft Controls vs Hard Controls
Hard controls tend to be

formal
objective
quantitatively measurable
the "map"

Soft controls tend to be

informal
subjective
intangible
the real terrain

COSO Internal Control - 3 dimensional representation

COSO Internal Control: ERM interaction

COSO Internal Control: ERM interaction

Effective I/C or ERM means

That management has a "flow of reliable information" about each component of control for all the objectives, from all areas of the organization.
COSO does not specify who shold provide what information, just that mangement should be receiving & acting on the info.
Many different sources, or flows of info exist in the organization
"Soft Controls" relate to ehe people doing the work to meet the objectives of the organization; "hard control" related the processes and activiteis those people do.

Limitations

Reasonable, but not absolute, assurance
Different levels of assurance for different objectives
The future is uncertain
Other limiting factors......

Judgement, breakdowns
Collusion, management override
Cost vs benefits

Not part of IC or ERM

The objectives selected to be achieved
The responses taken to the risks

Other thoughts on I/C & ERM

Controls for reliablity of financial reporting are mainly in finance areas (Financial)
Controls over effective and eficient opeations (operational) & compliance with laws & regulations (compliance) are mainly in operational areas
Discussing objectives, risks and responses is the most valuable part of ERM
Anyone can put together a list of risks and controls, but true ERM can only be done by those directly responsible for those objectives
The same "Soft Controls" in the COSO I/C framework also apply to the ERM framework. I/C is fully incorporated into ERM
ERM doesnt replace good management practices, doesnt replace setting the right objectives, & doesnt replace the business experience needed to have the right vision of where an organization should be heading.
The discussion about the risks "are the controls" - it's all about readiness for the unknown

Other Resources

CSQA

Become a Certified Software Quality Analyst (CSQA)

CBOK 2006 - CONTENTS

KC1 - Quality Principles and Concepts - CONTENTS

KC 1.1 - Vocabulary of Quality

KC-1.2.1-The Different Views of Quality

KC 1.2.1 - Quality Attributes for an Information System

KC 1.3.1 - The Deming's PDCA Cycle

KC 1.3.2.1 - Cost of Quality & it's Components

KC 1.3.2.2 - Cost of Quality (Contd..)

KC 1.3.3 -Six Sigma, Benchmarking, Continuous Improvement & Best Practices

KC 1.3.4 - Earned Value Management - Basics

KC 1.3.4.2 - Earned Value Management - Example

KC 1.4.1 - Quality Assurance & Quality Control

KC 1.4.2 - Just in Time (JIT) technique

KC 1.5.1 - Quality Pioneers - Edwards Deming

KC 1.5.1.1 - Deming's 14 Principles

KC 1.5.2 - Quality Pioneers - Philip Crosby

Philip Crosby

"Do It Right the First Time"

KC 1.5.3 - Quality Pioneers - Joseph M. Juran

KC 1.5.4 - TQM (Total Quality Management)

KC 2 - Quality Leadership - CONTENTS

KC 2.1.1 - Executive and Middle Management Commitment

KC 2.1.2 - Quality Champion

KC 2.1.3 - New Behaviors for Management

KC 2.2 - Quality Management Infrastructure

KC 2.3 - Quality Environment

KC 2.3.4 - Open Communication

KC 2.3.5 - Mission, Vision, Goals, Values, and Quality Policy

KC 2.6 - Compliance to Organizational Policies and Procedures

Quick Guide to Skill Category - 1

KC 3 - Quality Models - CONTENTS

KC 3.1 - Quality Baseline Concepts

KC 3.2 - Methods Used for Establishing Baselines

KC 3.3.1 - Purpose of a Quality Model

KC 3.3.3- Model Selection Process

KC 3.3.4.1 - Using Models for Assessment & Baseline-1

There is another commonly used term – Process Performance Baseline. There is very thin line between these two.

Note that a Performance baseline is only an indication of the previous results, the Capability Baseline is the measure which is set to measure future parameters against this set standard. The Process Performance Baseline may be used to decide the Capability Baseline.

Process Variance

Business Impact

KC 3.3.4.2 - Using Models for Assessment & Baseline-2

KC 3.4.1 - Capability Maturity Model

CMMi & SW-CMM – A comparison

KC 3.4.2 - Malcolm Baldrige National Quality Award

KC 3.4.4 -- Information Technology - Software Life Cycle Processes

KC 3.4.3 - ISO 9001:2000

KC 3.4.5 - ISO/IEC 15504: Process Assessment

KC 3.4.6 - Post Implementation Reviews/Audits

KC 4 - Quality Assurance - CONTENTS

KC 4.1 - Establishing a function to promote and manage quality

KC 4.2 - Quality tools

KC 4.3 - Process Deployment

KC 4.4 - Internal Auditing and Quality Assurance

KC 5 - Quality Planning - CONTENTS

KC 5.1 - Planning Concepts

KC 5.2 - Integrating Business and Quality Planning

KC 5.3 - Prerequisites to Quality Planning

KC 5.4 - The Planning Process

KC 5.5 - Maturing IT Work Processes

KC 6 - Define, Build, Implement and Improve Processes - CONTENTS

KC 7 - Quality Control Practices

KC 7.1 - Testing Concepts

KC 7.2 - Developing Testing Methodologies

KC 7.3.1 - Validation Methods

KC 7.3.2 - Verification Methods

KC 7.4 - Software Change Control

KC 7.5 - Defect Management

KC 8 - Metrics and Measurement - CONTENTS

KC 8.1 - Measurement Concepts

KC 8.2.1 – Product measurements - Size

Lines of code

Function/feature points

KC 8.2.2 – Product Measures - Complexity

McCabe's Cyclomatic Complexity

Software Complexity by Thomas J. McCabe and Arthur H. Watson, McCabe and Associates, Inc.

KC 8.2.3 – Product Measurements - Defect Metrics

KC 8.2.4 – Customer Satisfaction

KC 8.2.5 – Service Level Agreements (SLA)