6. Testing

6.1 Test Plan

6.1.1 Introduction

This section presents a plan for the testing of the Abstract Database Viewer. In using this plan, any inadequacies in the package will be found and reported on in the Test Report presented in section 6.2.

6.1.2 Test Coverage

The general areas that need to be covered in the test plan are as follows:

6.1.3 Format of Test Plan

The two example tables below show the general layout of the tables used in the test plan.

The first shows all the tests to be carried out, ordered by test number and showing the corresponding test script numbers and the function that is being examined in each case.

No. Script No. Function What it is testing for
EG1 1 Function of test Purpose of test

The second table shows actual test scripts to be carried out, ordered by test script number. A description is given of how the test should be performed, followed by the criteria to be met in order for the test to pass.

No.

What to do

Pass Criteria

1

How to perform the test

Requirements for the test to pass

6.1.4 Program Execution

No. Script No. Function What it is testing for
PE1 1, 2 Start up Tests that all programs start up successfully
PE2 3 Exit Tests that all programs exit successfully

6.1.5 Tool

These tests assume that the tool program has been started

No. Script No. Function What it is testing for
T1 4 ® 7 Connect Connecting to a template database successfully, ignoring bad parameters
T2 8, 36, 37 Disconnect Closing connection to a template database successfully
T3

9 ® 11, 55

Viewers Opening of Viewer Editor window in a default state
T7 12 ® 14

 
Updating viewer information
T5 15 ® 21

 

Adding a viewer
T6 22, 23

 

Deleting a viewer
T7 24, 55

 

Updating a viewer application
T8 25

 

Checking viewer has been saved successfully
T9 32, 33

 

Removing a viewer application
T10 26 Templates Opening of Template Editor window in a default state
T11 4 ® 6, 27

 

Connecting to a data database successfully, ignoring bad parameters
T12 28

 

Adding a template
T13 29

 

Deleting a template
T14 30

 

Connecting to a new data database
T15 31

 

Updating template information
T16 33

 

Updating of viewer application list successfully
T17 34, 58 Options Opening of options window in a default state
T18 35, 58

 

Editing of information

6.1.6 Server

These tests assume that the server program has been started

No. Script No. Function What it is testing for
S1 4 ® 6, 38 Connecting Connecting to a template database successfully, ignoring bad parameters
S2 39, 56, 57 Disconnect Closing connection to a template database successfully
S3 40, 43 Starting Starting a process to listen for, and connect clients
S4 41, 57 Stopping Stopping the listener process, making sure that there are no open connections

6.1.7 Client

These tests assume that all programs have been started

No. Script No. Function What it is testing for
C1 42 ® 45 Logon Logging on to a server successfully, ignoring bad parameters
C2 57 Logoff Logging off from a server successfully
C3 46, 50, 52, 55, 58 Viewing data Viewing data through loaded viewers
C4 47

 

Viewing data through default viewer
C5 48 ® 52

 

Windows are tidied up automatically
C6 50, 53 ® 55, 58

 

Cache is functioning correctly
C7 53

 

Clearing the cache
C8 58 Auto Startup Auto startup functionality works successfully

6.1.8 System

No. Script No. Function What it is testing for
SYS1 60 Execution Common look and feel
SYS2 61 Errors Are all errors handled by the program

6.1.9 Test Scripts

No.

What to do

Pass Criteria

1

Execute each program in turn with no parameters

Should start up without error

2

Execute each program in turn with some parameters

Should start up without error, ignoring the parameters

3

Select the ‘Exit’ button in each program once they are running

Should exit without error

4

Enter invalid URL. Select ‘Connect’ button

Should inform the user that the URL was invalid

5

Enter invalid Driver. Select ‘Connect’ button

Should inform the user that the Driver was invalid

6

Enter valid URL and Driver with invalid User and Password. Select ‘Connect’ button

Should inform the user that the user name was invalid

7

Clear User and Password. Set URL and Driver to the test template database. Select ‘Connect’ button

Should connect to database and enable ‘Viewers’, ‘Templates’ and ‘Options’ buttons

8

Select ‘Close’ button

Should disconnect from database and disable ‘Viewers’, ‘Templates’ and ‘Options’ buttons

9

Select ‘Connect’ button. Select ‘Viewers’ button.

Should open Viewer Editor window and disable ‘Viewers’ button

10

Try to edit fields

They should not be editable

11

Select ‘Update’ button

Nothing should happen

12

Select ‘ConsTK’ viewer from ‘Current Viewers’ list

Details should update to reflect this selection

13

Set version number to ‘2’. Select ‘Update’ button

Details should be updated successfully

14

Set version number to ‘blah’. Select ‘Update’ button

Should be informed of invalid version number, and version number should be set back to 2

15

Select ‘Add’ button

Should be presented with file dialog box

16

Enter invalid filename. Select ‘OK’ button

Should be told that this file name is not valid

17

Select ‘Add’ button. Select ‘Cancel’ button

Should return to original state

18 Select ‘Add’ button. Select file without ‘.class’ extension

Should ask to make sure that this file is to be imported.

19

Select ‘No’ button.

File should not be imported

20

Select ‘Add’ button. Select file without ‘.class’ extension. Select ‘Yes’ button

File should be imported, and details be updated to reflect new file

21

Select ‘Add’ button. Select file with ‘.class’ extension

File should be imported, and details be updated to reflect new class

22

Select second viewer in list. Select ‘Delete’ button

List should be updated to reflect deletion

23

Select last viewer in list. Select ‘Delete’ button

List should be updated to reflect deletion

24

Select ‘Add’ button. Select file with ‘.class’ extension. Select ‘OK’ button. Select ‘Update Class’ button. Select new file with ‘.class’ extension. Select ‘OK’ button

Should update viewer information accordingly

25

Close Viewer Editor window. Select ‘Viewers’ button. Select each viewer in turn

Window should close successfully, and then on reopening, the new details should be set up as they were before the window was closed.

26

Select ‘Templates’ button on main window

Template Editor window should open successfully, with New Connection window on top.

27

Clear User and Password. Enter correct Driver and URL for test database. Select ‘Connect’ button

Should connect successfully and show objects in ‘Selectable Objects’ list which are not already in ‘Current Templates’ list. ‘Add’ and ‘Delete’ buttons should be disabled

28

Select an object from the ‘Selectable Objects’ list. Select ‘Add’ button

Should remove object name from lower panel and add to ‘Current Templates’ list, setting up details and displaying them

29

Select ‘Delete’ button

Template should be removed from ‘Current Templates’ list and added to ‘Selectable Objects’ list, refreshing screen as necessary

30

Select ‘Connect’ button. Connect to test template database

Should clear ‘Selectable Objects’ list, as there are no usable object in the template database

31

Select ‘Q_PROJECTS’ object from ‘Current Templates’ list. Change Description to ‘Projects’. Select ‘Update’ button

Details should be saved successfully

32

On Viewer Editor window, select second viewer in list. Select ‘Clear’ button. Select ‘No’

Class should not be cleared

33

Select ‘Clear’ button. Select ‘Yes’

Class should be cleared, and viewer list on Templates Editor window should be updated to reflect the fact that no class is loaded into this viewer.

34

On main window, select ‘Options’ button

Options window should open with default view

35

Change Port number to 8305. Select ‘OK’ button. Select ‘Options’ button on main window. Select ‘Cancel’ button

Port number should have been updated to 8305

36

Select ‘Close’ button on main window

Template Editor and Viewer Editor windows should close, and connections to databases be closed as well

37

Select ‘OK’ on New Connection window. Select ‘Viewers’ button on main window. Select ‘Exit’ button on main window.

Template Editor and Viewer Editor windows should close, and connections to databases be closed as well. Tool should exit without error.

38

Clear User and Password. Set URL and Driver to the test template database. Select ‘Connect’ button

Should connect to database, enable ‘Start’ button and set caption on ‘Connect’ button to ‘Close’

39

Select ‘Close’ button

Should disconnect from database, disable ‘Start’ button and set caption on ‘Close’ button to ‘Connect’

40

Select ‘Connect’ button. Select ‘Start’ button when enabled

Should start up a listener process and show that the process was started successfully and that clients can now connect, along with setting the caption on the ‘Start’ button to ‘Stop’, and disabling the ‘Close’ button

41

Select ‘Stop’ button

Should stop listener process and show that the process has been stopped, and that no more clients can connect, along with setting the caption on the ‘Stop’ button to ‘Start’ and enabling the ‘Close’ button

42

Select ‘Logon’ button on Client.

Logon should fail and the user should be told

43

Select ‘Start’ button on Server. Select ‘Logon’ button on Client

Logon should fail (as the port number is invalid) and the user should be told

44

Enter invalid Host name. Select ‘Logon’ button on Client

Logon should fail and the user should be told

45

Enter valid Host name. Set Port to 8305. Select ‘Logon button

Logon should succeed. Tables window should appear with a list of all the tables referenced in the template database.

46

Select ‘Projects’ table from list. Select ‘Show’ button

Construction Toolkit program should be loaded successfully and shown to the user

47

Select ‘Organisations’ table from list. Select ‘Show’ button

Organisations table should be loaded into default viewer and shown to the user

48

Close "ORGS - Default Viewer" window. Close "Construction Toolkit" window. Close "ADBV Client: Tenplates" window

‘Templates’ and ‘Clear Cache’ buttons should be enabled

49

Select ‘Templates’ button.

"ADBV Client: Templates" window should reopen and ‘Templates’ and ‘Clear Cache’ buttons should be disabled

50

Select ‘Projects’ table from list. Select ‘Show’ button

Construction Toolkit should be reloaded, but this time, from disk instead of being downloaded from the server

51

Close "ADBV Client: Templates" window

Construction Toolkit windows should be closed

52

Select ‘Templates’ button. Select ‘Projects’ table from list. Select ‘Show’ button twice. Select ‘Organisations’ table from list. Select ‘Show’ button. Close "ADBV Client: Templates" window

Two copies of Construction Toolkit should be loaded, along with one Default Viewer showing the ORGS table. On close of "ADBV Client: Templates" window, all loaded viewers should also be closed.

53

Select ‘Clear Cache’ button

Cache should be cleared, and ‘Clear Cache’ button should be disabled

54

Select ‘Templates’ button. Select ‘Projects’ table from list. Select ‘Show’

Viewer should be downloaded from server again, rather than loaded from local disk

55

Select ‘Connect’ on Tool. Select ‘Viewers’ on Tool. Select ‘ConsTK’ class from list. Update version number to one that is higher than the currently one. Select ‘Close’ on main Tool window. Select ‘Show’ button on Client "ADBV Client: Templates" window.

Viewer should be downloaded from server again, as the version number has been updated

56

Select ‘Stop’ on Server.

Server should not be able to stop, as there is still a client connected

57

Select ‘Logoff’ on Client. Select ‘Stop’ on Server. Select ‘Close’ on Server.

Listener process should stop successfully and connection to database should be closed successfully. All Client windows should be closed

58

Select ‘Connect’ on Tool. Select ‘Options’ on Tool. Set Auto Startup option to on. Select ‘OK’ button on "Options" window. Select ‘Close’ on Tool. Select ‘Connect’ on Server. Select ‘Start’ on Server. Select ‘Logon’ on Client

On logon by Client, ‘Templates’ button should not be shown on main window this time, and the "ADBV Client: Templates" window should not automatically load, but instead, just the Construction Toolkit window should be loaded from disk immediately.

59

Select ‘Exit’ on Server.

Server should not be able to stop immediately, as there is still a client connected, but it should be possible to exit if necessary.

60

Run all executable programs and examine their appearance and the way they interact with the user.

There should be a common look and feel throughout

61

Put in junk data and long text strings (longer that 255 characters), along with pressing any buttons possible, and generally trying to make the program error

All errors should be trapped and, if necessary, the user should be notified through the program. All messages should appear within the environment of each program, and not on the command line

6.2 Test Report

6.2.1 Introduction

This section presents a report of the testing of the Abstract Database Viewer as described in the plan of section 6.1.

6.2.2 Format of Test Report

The two example tables below show the general layout of the tables used in the test report.

The first table references the tests (as described above) by test number and script number and indicates whether this test passed or failed. If the test failed, then a report number is given, which links this table to the second table.

Test No. Script No. Pass / Fail Report No.
EG1 1 Pass

 

EG2 2 Fail 1

The second table shows what went wrong and gives an indication as to the importance of fixing the problem. The ‘Report No.’ column links back to the ‘Report No.’ column in the first table.

Report No. Report Description Fix Priority
1 Why the test failed 1

The fix priority values can be 0, 1, 2 or 3, where each priority has the following meaning:

0. A fault which has been fixed (original priority given in brackets)
1. A major fault that needs to be fixed
2. A less major fault, but still a problem which needs to be fixed
3. A minor annoyance, which it would be nice to fix.

6.2.3 Test Report

Test No. Script No. Pass / Fail Report No.
PE1 1 Pass

 

 

2 Pass

 

PE2 3 Pass

 

T1 4 Pass

 

 

5 Pass

 

 

6 Pass

 

 

7 Pass

 

T2 8 Pass

 

 

36 Pass

 

 

37 Pass

 

T3 9 Pass

 

 

10 Fail 1

 

11 Pass

 

 

55 Pass

 

T4 12 Pass

 

 

13 Pass

 

 

14 Fail 2
T5 15 Pass

 

 

16 Fail 3

 

17 Pass

 

 

18 Pass

 

 

19 Pass

 

 

20 Pass

 

 

21 Pass

 

T6 22 Pass

 

 

23 Pass

 

T7 24 Fail 4

 

55 Pass

 

T8 25 Pass

 

T9 32 Pass

 

 

33 Pass

 

T10 26 Pass

 

T11 4 Pass

 

 

5 Pass

 

 

6 Pass

 

 

27 Pass

 

T12 28 Pass

 

T13 29 Pass

 

T14 30 Pass

 

T15 31 Pass

 

T16 33 Pass

 

T17 34 Pass

 

 

58 Pass

 

T18 35 Pass

 

 

58 Pass

 

S2 4 Pass

 

 

5 Pass

 

 

6 Pass

 

 

38 Pass

 

S2 39 Pass

 

 

56 Pass

 

 

57 Pass

 

S3 40 Pass

 

 

43 Pass

 

S4 41 Pass

 

 

57 Pass

 

C1 42 Pass

 

 

43 Pass

 

 

44 Pass

 

 

45 Pass

 

C2 57 Pass

 

C3 46 Pass / Fail 5

 

50 Pass / Fail 5

 

52 Pass / Fail 5

 

55 Pass / Fail 5

 

58 Pass / Fail 5
C4 47 Pass

 

C5 48 Pass

 

 

49 Pass

 

 

50 Pass

 

 

51 Pass

 

 

52 Fail 6
C6 50 Pass

 

 

53 Pass

 

 

54 Pass

 

 

55 Pass

 

 

58 Pass

 

C7 53 Pass

 

C8 58 Pass

 

SYS1 60 Pass

 

SYS2 61 Pass

 

Report No. Report Description Fix Priority
1 Fields were editable, even though no viewer was selected 0 (2)
2 The viewer information was not saved, but no message was given as to why, and ‘blah’ was left as the version number 0 (2)
3 The filename was not reported as being invalid until after the program had tried to load the file 0 (2)
4 Viewer details not updated 0 (1)
5

Under IBM Visual Age for Java, which uses a version of JDK 1.1.2, the test passed without error.

Under JDK 1.1.5, the Java application would not load properly, due to an Illegal Access Exception when the program tried to create a new instance of it.

0 (1)
6 All windows, except Default Viewer ones are closed. 0 (1)