Key Issues in Software Testing
Software testing - Activities performed to assess and improve software quality. This activity, in general, based on the detection of defects and problems in software systems.
Testing of
software systems consists of a dynamic verification of program behaviour
on a finite (limited) set of tests (set of test cases), selected as appropriate
from the commonly performed operations applied field and provide verification
of compliance with expected behaviour of the system.
In accordance with
IEEE Std 829-1983 Testing - a process analysis aimed at identifying differences
between the real and the desired properties (defect) and to evaluate the
properties of the software.
According to GOST R
ISO IEC 12207-99 in the life cycle of software defined, among other supporting
processes of verification, validation, joint review and audit.
The verification
process is the process of determining what software products operate in full
compliance with the requirements or conditions that are implemented in previous
works. This process may include analysis, inspection and test.
The process of
certification is the process of determining the completeness of compliance with
established requirements established by the system or software product of their
functional testing purpose.
Joint review process
is a process of assessment and, where appropriate, results of products for the
project.
The audit process is
a process for determining compliance with the requirements, plans and contract
terms.
In sum, these
processes constitute what is commonly referred to testing.
QA Outsourced Testing is based on the test
procedures with specific input data, initial conditions and expected results
developed for a specific purpose, such as checking a separate program or
verification of compliance for a certain requirement. Test procedures can test
different aspects of the program - a separate function to work properly to
adequately fulfil the business requirements.
Software Testing
services is usually done in cycles, each of which has a specific list of goals
and objectives. Testing cycle may coincide with or correspond to an iteration
of a certain part. Typically, the cycle test is conducted for a particular
build system.
Theoretical and practical limitations of testing
Theory Test against
unreasonable level of confidence in the successful series of tests passed.
Unfortunately, most of the established results of the theory test - negative,
meaning, according to Dijkstra, that "the testing program can be used to
demonstrate the presence of defects, but never show their absence." The
main reason for this is that a comprehensive testing beyond the reach of real
software.
Levels of testing
Testing is usually
performed throughout the development and maintenance at various levels. The
level of testing determines the "over what" made tests: on a separate
module, a group of modules or system as a whole. However, none of the levels of
testing cannot be considered a priority. All levels of testing are important,
regardless of the used models and methodologies.
·
Unit Testing. This level of testing makes it possible to test the
individual elements of the system. What is considered an element - a module of
the system is determined by context. The most complete form of this test is
described in the standard IEEE 1008-87 «Standard for Software Unit Testing»,
giving the integrated concept of a systematic and documented approach to unit
testing.
·
Integration Testing. This level of testing is the process of
verification of the interaction between software components / modules.
·
System Testing. System testing covers their entire system. Most
functional failures should be identified even at the level of unit and integration
tests. In turn, system testing, typically focusing on non-functional
requirements - security, performance, accuracy, reliability, etc. At this
level, and tested interface to external applications, hardware, operating
environment, etc.
Objectivies of Testing
Testing is conducted
in accordance with specific goals (which may be explicitly or implicitly) and
different levels of accuracy. Defining the purpose accurately, expressed
quantitatively, enables control of test results. Test scenarios can be
developed for testing the functional requirements (known as functional tests)
and to evaluate non-functional requirements. In this case, there are tests,
when quantified and test results can only speak indirectly to satisfy the
objectives of testing (eg, «usability» - lightness, ease of use, in most cases cannot
be explicitly described quantitative characteristics).
There are the
following, the most widespread and reasonable goals (and, accordingly, views)
Test:
·
Acceptance / qualification testing. Examines the behaviour
of the system for customer satisfaction. This is possible if the customer takes
the responsibility associated with such works as the party "host"
software system, or routine tasks are specified, the successful validation
(testing) which allows you to talk about customer satisfaction. Such tests can
be conducted with the involvement of both system developers, and without them.
·
Installation testing. From the title
implies that the tests are conducted to verify the installation procedure in
the target system environment.
·
Alpha and beta testing. Before the software
is available as a minimum, it must pass an alpha (internal test use) and beta
(a test involving the use of selected external users) versions. Bug Reports received
from users of these versions of the product processed in accordance with
certain procedures, including confirmatory tests (any level) that are conducted
by the team. This type of testing cannot be pre-planned.
·
Conformance testing / Functional testing /
Correctness testing. These tests may be called differently, but their essence is simple -
check the system meets the requirements for her requirements described in the
specification level of behavioural characteristics.
·
Reliability achievement and evaluation. Helping to identify
the causes of failures, the testing means and improving the reliability of
software systems. Randomly generated test cases can be used for statistical
evaluation of reliability. Both goals - increasing reliability and evaluation -
can be achieved by using models of reliability.
·
Regression testing. Determining the
success of regression tests (IEEE 610-90 «Standard Glossary of Software
Engineering Terminology») states: "The re-sample testing of the system or
component to verify the modifications made should not lead to unintended
effects." In practice this means that if the system has successfully
passed tests before making any modifications, it should take them and after
making such.
·
Performance Testing. Specialized tests
for meeting the unique requirements of performance parameters. There is a
separate subspecies of these tests when an attempt is made to achieve the
quantitative limits due to characteristics of the system and its operating
environment.
·
Stress testing. Need to understand the difference
between performance testing discussed above in order to achieve its real
(achievable) possibilities of performance and execution of a software system
increases the load c, up to the planned performance, and further enhanced behaviour
throughout the system load increase.
·
Back-to-back testing. A single set of
tests to compare the two versions of the system.
·
Recovery testing. The purpose - to check the restart
capabilities of the system in case of unforeseen disasters that affect the functioning
of the operating environment in which to execute the system.
·
Configuration testing. In cases where
software is created for use by various users, this type of testing aims to
verify the behaviour and performance of the system in various configurations.
·
Usability testing. Purpose - to test how easily the end
user can master it, including not only the functional component - the system
itself, but its documentation, how well you can perform tasks that automation
is performed using this system, and finally, how well insured (with in terms of
potential failures) of user errors.
·
Test-driven development. In fact, it is not
so much the technique of testing, how much style organization of the design
process life cycle, when the tests are an integral part of the requirements
(and related specifications), rather than considered independent verification
activities meet the requirements of the software system.
Comments