This is a presentation I gave at a local test meet up group called VanQ
PDF version of slides: Test Coverage Outlines
Intro
Test Coverage Outlines are just elaborate spreadsheets. They were originally inspired by Jonathan Kohl, as part of exploratory testing training at Sophos. A colleague, (Jose Artiga), and I built on the initial inspiration and came up with the first version of the test coverage outline. This was then adopted by the teams at Sophos, and of course refined through use and practice.
(slide 8) Simple spreadsheets to help;
- Structure & simplify your test planning
- Inspire & structure your test design
- Be test professionals and not just test execution machines
- Collaborate on testing
- Show clear and concise status at any point in your testing
- Discuss risk in terms of planned vs actual coverage
(slide 9) An example coverage outline
- Each white background row is a test idea – a simple sentence or few words to convey an idea you want to cover by testing
- Try to keep these simple like a heuristic that acts as a guide rather than a step by step procedure
(slide 10) Divide your test ideas up into sections
- Example shows this as the black lines with white text
- You can do this by functional area, test focus, heuristic etc (whatever makes sense for you in your context)
(slide 11) Divide your test ideas up into sheets
- Again divide into functional area per sheet or perhaps copy your sheet to provide same/similar coverage against different builds or versions of your software
(slide 12) Configurations – show your coverage against different configurations
- Example given is a common one – browsers and OS, but could equally be versions of HW, SW, Mobile device etc
(slide 13) Test types/levels – show your coverage at different test levels or different testing types
- Matrix your test ideas against different test levels, for example unit tests, integration tests, automated UI tests, manual tests, exploratory tests
- Even if you are not familiar with the unit tests you can suggest the developers gain some coverage of the test ideas with unit tests and fill out the column with you
(slide 14) Prioritise your test ideas
- I use simple (high, medium, low) priority levels to organise, sort and filter test ideas by priority
- Make sure you are executing/covering in priority order
(slide 15) Prioritise your configurations
- I use a simple left to right prioritisation for configurations, i.e. the highest priority configuration to cover is the left most column
(slides 16 & 17) Use colour and conditional formatting
- I use simple colours and conditional formatting to make updates easy and to show status and priority clearly
- Make sure you also use a colour (I use x and grey) to indicate coverage that you are consciously not planning to cover
(slide 18) I use mind maps as a visual test design/planning aid
(slide 19) Start by outlining your test idea areas, use the outline to inspire your test ideas
- I sometimes use heuristics to structure and inspire my testing, for example using a quality criteria heuristic like usability or performance I can think about test ideas that come under each of those ares
- Creating templates (see examples) for common types of testing focus can help inspire testers as well as providing some base or consistent tests
(slide 20) Using test ideas rather than detailed test cases
- enables and encourages variation, exploration and more ‘brain engaged’ testing
- thus avoiding the pesticide paradox [Boris Bezier]
(slide 21) Collaboration
- Using google docs or similar enables you to easily collaborate in real time with other testers/colleagues
- Some of the easiest ways in which you can organise and facilitate collaboration is to allocate a config column, test area or even tabs
- You can see who is accessing/updating the sheet and can see status of tests others are covering in real time too
(slide 22) Use colour to provide very easy and quick to read status
- You can just look at a tab and immediately see if you have any fails or blocked tests
(slide 23) Colour coding also makes it easy to see coverage in terms of planned vs actual
Example Coverage Outlines
Feel free to copy and adapt these for you own purposes, hopefully these will inspire you to refine the outlines and share your ideas back with me and others;
