Y2k-001.txt
2 OBJECTIVES
=============
The objectives of this document are:
· Define key terms with regard to Year-2000 Testing.
· Highlight normal process concerns as applied to Year-2000.
· Additionally, add any concerns brought about by Year-2000.
· Help to retain our sanity in this "just-another-crisis" challenge.
2.1 SCOPE
==========
This document presents normal process issues with the focus on
Year-2000 testing issues. It discusses both the actual testing
process, as well as test and facility planning. Year-2000 issues
essentially place one more barrier in the over-all process of
performing the usual QA process.
Additionally, for completeness, many design-related, as well as
customer-support and documentation-related issues are mentioned.
The intended audience is the test department, program management,
and customer support groups.
2.2 REFERENCE DOCUMENTS
========================
Beiser, Software Testing
Brooks, the Mythical Man Month.
Kaner, Falk, and Nugent, Software Testing.
2.3 DEFINITIONS
SME Subject Matter Expert; permanent full-time staff
member well versed in the operation of equipment,
systems processes in use at THE company, etc.
SUT System Under Test; ie, the specific product and other
hardware, as well as specific software that is to be
delivered as part of this feature.
SWDLC Software Development Life Cycle (also SWLC); a process
model used commonly to describe the process of specifying,
developing, programming, debugging, testing, releasing,
and then supporting software-based products.
SWQA Software Quality Assurance (do not confuse with SWQC);
The over-all process of managing the development/testing/
field-support of software products. Q/A is a MACRO process,
Q/C is a MICRO process. Q/A usually monitors the Q/C sub
processes within each portion of the over-all product
development/support process.
SWQC Software Quality Control (do not confuse with SWQA);
the micro-process of controlling the software release
process.
In general Q/C is the specific part of the development/support
process within each group. Thus, the micro-code (lowest
level of software) will have its own Q/C process that
will radically different from the Q/C process that is
carried out in (eg) Sales and Service Support.
Testing The process whereby the degree of quality is assessed
in a product. The usual progression (once design is
finished) is: Unit test, Function Test, Local
Integration Test, Systems Test, Full Integration Test,
Regression Test, Customer Acceptance Test. And then
release, and then repeat as required.
TC Test Case; a set of instructions written out in clear
and un-ambiguous terms that detail the steps to be
followed in testing a specific part of the operation
of the SUT. A test case, when run, produces a result
that is to be verified if the expected results are
"as expected" then the TC is said to have "passed"
otherwise, it is said to have failed.
TP Test Plan; a document that is made up of background
information and a list of the test cases. Note: Test
Planning is a generic term for that part of the over-all
SWQA process that deals with the testing portions of the
QA process.
Y2K Year-2000; the programming problem brought about by
storing of just the last two digits of the year number
- eg, "69" for "1969" or "01" for "2001".
3 INTRODUCTION
==============
This document details aspects of the normal SWDLC (Software
Development Life Cycle) as they pertain to the Year-2000
problems (Y2K for short). The so-called Y2K crisis is simply
one more aspect in the normal maintenance cycle of software
products. Admittedly there is much hype concerning this issue,
and the problems are real. However, if one simply takes the
Y2k as "just another headache" to be solved in the usual way,
then ones perspective is less prone to the panic that is much
hyped in the media.
Also note: The wording of this document carefully avoids the
use of the word "compliant" as in the mysterious
phrase "Y2K-compliant".
Compliance is an arbitrary concept - compliance is
with respect to some "standard". The ANSI standard
not with standing - almost in every case, each
company will develop its own compliance policy).
Now as long as it is understood that the compliance
refers to a standard that has been agreed upon -
and not some sort of "guarantee" that a system that
is in compliance is now immune to problems ---
then well and good. Otherwise, compliance is a
totally meaningless concept: There are no "actual"
standards for Y2K compliance; the phrase is totally
meaningless.
This is especially true the interactions with other
companies - many of whom have a less-than-thorough
approach to their own "compliance" policy. In such
cases, we will find ourselves dependent on implementing
special input-filters to protects us from THEIR mistakes.
(Following the year 2000, there will probably be a
standards committee meeting that will produce an
actual meaningful standard - at this point, it as
much anybody's guess as anyone else's - so let's
just make those "guesses" as good as possible).
Therefore, the phrase that will be used is
"Y2K-compatible"
If you want to imagine some mystical, silver bullet,
magical elixir of "compliance" that will assure
absolutely zero defects, then please feel free to do
so. The main guarantee of success is:
"dogged, un-dying determination
alone is omnipotent"
(attributed to Calvin Coolidge).
Y2K panic aside, the key issues are:
· Routine and already in-place QC procedures (as well as
the over-all QA function) can be used to treat the
"Y2K problem" just like any other maintenance issues.
· The "crisis" will help to bring most programs into the
lime-light so that an assessment of their utility,
maintainability, and necessity can be focused upon.
· Y2K is simply another aspect of "problem awareness";
as with any product, there are product-specific issues
and concerns that must be addressed in the normal
maintenance cycle.
· The primary problem is in the fact that the date on the
computer must be changed several times (usually something
that isn't even considered); this causes the test setup
to be rather slow, since many such systems must be re-booted.
· Additional problems occur in the creation of test data to
verify proper operation in an un-controlled environment;
many systems interact without side systems that may not have
been thoroughly tested. (Again, these issues are nothing new).
· The Key "hot" dates must be identified and their impact
assessed; again, just one more headache to worry about.
4 ROAD MAP -- What's going to be covered.
=========================================
First off, to establish a common point of reference, we will look
at a process model known as the "IPAFT-R" model. This outlines a
simple, adaptable model for the deployment of hardware and software
products. The idea here is to be able to discuss the standard sorts
of issues in a common frame-work.
Then, we then trace through each step of the IPAFT model, looking
at the relevant Y2K issues. We examine these from the points of
view of process-related as well as procedure-related problems,
with additional detail paid to the documentation and training issues.
Note too that the "R" (Release") in the IPAFT model, is really a
combination of the review and support steps in many of the
software development models.
5 YEAR 2000
============
5.1 The IPAFT-R Model
There are many "process" models for the development and release
of hardware and software systems. The following is a fairly
detailed summary of the so-called IPAFT (Identify, Prioritize,
Assess, Fix, Test - Release) model. This should cover all of
the bases and concepts presented here will of course be
compatible with those processes already in place.
IDENTIFY
========
The identify step consists of the "planning" or "getting
ready to do" stage of development. This includes the
standard sorts of things:
· Proposals; eg, "Request for Change", "Work Orders", etc.
· Feasiblity Study; eg, "Proof of Concept", "Prototyping", etc.
· Meetings between Staff & Customer.
If a new system is to be created the "feasibility" work will be
extensive, as well as much of the development & testing.
In many cases, no new programs are actually written, only
modifications to existing "legacy" systems. In this case,
the concern is that of introducing side effects to the
existing functionality. As such, the emphasis will be on
regression and negative-impact testing (both in the design
and test groups).
Finally, the main problem encountered in the identify stage
is to simply locate the source code, documentation, schematics,
etc. Often they were created without version or source-code
control. Many times, the only documentation is the expert
knowledge of the original designer. Getting that information
down on paper will obviously impact their on-going work.
(And Y2K does not make this any easier; especially if a large
number of components are found to be Y2K-incompatible).
PRIORITIZE
==========
Obviously the resources for on-going efforts will have to be
diverted (in full or in part) to the remediation work to be
done. It is during the "prioritization" step that the
action plans are drawn up. The usual functions are:
· Impact/Risk Assessment; "delaying vs. deploying",
"Actual Cost to Fix", etc.
· Business Impact Analysis; "market presence", "competitive
edge", etc.
· Criticality to Normal Operations; Critical, Major, High,
Low, Future Enhancement.
ASSESS
======
During this phase, the goal is to answer the question: "How bad
and how much?". The key is not to start fixing everything until
you have a clear idea of the interactions between the components
and a clear idea as to WHAT needs to be fixed.
The usual "process tools" are applicable at this point.
· Code Reviews & Walk Throughs
· Test Case Reviews; New vs Regression test cases.
· Tools Review/Assessment.
· Review of Documentation
· Release Planning; timing, press releases, damage control, &
the usual field-support issues, training of
customer-support staff, etc.
· Configuration Management Reviews; specifically, new-release
planning.
· Data and Program Flow Analysis (detailed view of the systems)
· Functional and Systems Analysis (more global, less detailed
view of the systems)
· Prototyping / Modeling
· Performance Analysis
· Dependency Charts, Flow Charts, Timing Diagrams
· Other diagrams: Warnier/Orr, HIPO, Burr, etc.
· Debuggers, Data Drivers, Test Equipment, Protocol Analyzers,
etc, etc, etc.
FIX
===
Finally, we can start "fixing" things. The key issue here is that
resources are limited. As such, there will be the need to share
these resources. The key concepts are:
· Optimizing Resources; schedualers, e-mail, working groups,
COMMUNICATE.
· Lab-time planning; New Installation, Upgrades, Routine
Maintenance.
· Project Management; Assign/Do/Report/Release.
· Configuration Management; Check-in/Check-out,
"Development Process"
· Establish the Exit Criteria; Number of open trouble reports,
make/break tests, etc.
· Unit Testing, White Box Testing, Systems Testing,
Local Integration Testing.
· Tools Development.
· Documentation Reviews; Technical doc's, Field support docs,
customer support procedures are updated,
new product literature, etc, etc, etc.
TEST
====
As the finished products become available from the design group the
independent test and validation process begins in earnest.
(Prior to this, the test cases, and the test equipment have been
under development). The usual stuff here:
· Establish Entrance Criteria; Open room vs Clean Room,
Trouble Report Review,
fit-for-use criteria,
· Make/Brake Testing; Ready to test assessment.
· Multi-function testing, Black Box Testing, Ease-of-use Assessment.
· System Integration & Full Integration Testing.
· Re-testing & Regression Testing.
· Fit for Use assessment.
· Customer Acceptance Testing.
· Beta Testing.
· Tools Development.
RELEASE
=======
At this point, the documentation is reviewed, the field-testing
(and often customer acceptance testing) occurs, and the field-support
and customer service people have one more feature to support.
A final "post-mortem" review should be done as well as the final
sigh of relief. Until the next project...
· Post Mortem; What worked? What didn't work?
How can we work smarter?
· New Documentation is shipped.
· New Product is shipped.
· New Support Documentation ready for field-support/customer
service.
· Pagers are issued.
· On to the next!
5.2 Identify
==============
For each software and hardware unit, the identify step accomplishes
three things:
· Separate the various components into identifiable units
· Locate the source code, documentation
· Note any dependencies between the components
During the identification phase, it is easy to over-look components
that we take for granted in normal day-to-day operations;
for example:
· The Operating System in use on both the Workstations and SUT's
· Databases, Editors,
· E-mail,
· Voice-mail (PBX),
One very definite problem that MUST be addressed is that of the
so-called "software licenses". These are almost always
time-dependent, and when the date is rolled forward, they will
report that your license has expired --- And just imagine what
it will be like actually in 2000! Some of the issues are:
· When performing the tests, you will necessarily need to
roll the calendar forward - possibly causing the license
to expire. A call will need to be made to the software vendor.
· It is possible that the licensing algorithm itself is NOT
Y2K-compatible.
· When the calendars do click over on December 31st, the
software vendors are probably going to be VERY slow in
responding to requests for license extensions -- imagine
all of the customers calling AT once on January 1, 2000 !!
We now examine the two aspects (process & procedure) as they apply
to the Identification step.
xxxxxxxxxxxxxxx
