The people of “The Ministry of Testing” put together a conference on the theme “A Glimpse into the future”. They have just published the videos of most of the talks.
Asserting test effectiveness with mutation testing
I noticed a trend lately were my managers wanted to be able to measure the time | capacity of the test teams working under them. I was at first puzzled by this behavior. I was not really sure what kind of value this would bring to my managers to know that it takes me 8 hours or 8 days to test a component. Is that really the question you want to ask you team?
Videos from Google Test Automation Conference 2011
In October last year, I shared the videos of Google GTAC 2010. Now that finally the video from the 2011 conference are available (actually they have been for months now but I just noticed today) I wanted to share them on this blog as some of the contents is really nice. At this time, the buzz on the software testing was about its death. The good news, test is not dead, yet.
Opening Keynote Address – Test is Dead
PNSQC 2011 – Videos, slides and abstracts
The incredible people behind PNSQC updated the website with videos of the keynotes of invited speakers! I haven’t attended personally to all keynotes but I can strongly recommend Julian Harty presentation on the future of quality and the value sync presentation from Robert Sabourin.
You can also enjoy the slides from most of the presentations. All available for free on the 2011 papers and presentation page from PNSQC.
You can still access the past conferences, up to 1983…
Google’s Approach to Test Engineering – Talk by Dr James Whittaker
Yesterday at QASIG (Quality Assurance Special Interest Group), Dr
James Whittaker made a talk about Google’s approach to test engineering.
The presentation was really interesting; I was hugely impressed by
the ACC test plan patterns. Even if there were a lot of arguments about
software testing being dead, he agreed that some part of testing will probably
never die because they require special skills, like:
- Privacy
- Security
- Internationalization
- Accessibility
Unfortunately the slides are not readable on this video but, probably, most of the ideas are shared in the ‘to be soon published’ book: How Google Tests Software (currently rough cuts only).
Update: here is the link to the slides.
Videos from Google Test Automation Conference 2010
While GTAC 2011 is taking place, I believe that it could be interesting to watch what happened during the 2010 conference.
Continue reading
Test design pattern template
While preparing the future post of this blog, I realized that maybe some other people would be interested to share their patterns.
I know that most people are already busy doing work for them or blogging on their side of the Internet but let me clear, if you want to publish a pattern on this blog, you are more than welcome. As you can see there is no advertisement on the blog so that would be a free effort you would offer to the community! There will definitely be a link back to you and some promotion possible (because there is no such things as a free lunch!).
For incoming test design pattern I will follow the template from the book Testing Object-Oriented Systems: Models, Patterns, and Tools by R. Binder (page 330). Below is a reproduction of the template with an explanation of the terms:
| Test design pattern template | |
| Name Identify the patterns, should suggest its general approach. |
|
| Intent Should be a succinct answer of “what kind of test suite does this pattern produce”. |
|
| Context What will the pattern solve, when it applies and a what scope? |
|
| Fault Model What kind of fault does it target? Why it will be useful to find those faults? It must explain how it will reach the fault, how it can trigger the fault and how will it be observable? |
|
| Strategy How is the test suite designed and implemented? |
Test Model Define the responsibility and the focus of a test design. |
| Test Procedure Define an algorithm, technique or heuristic to produce the test cases |
|
| Oracle Define an algorithm, technique or heuristic to evaluate the result of the test cases |
|
| Automation
Approaches the test suite generation run execution and run evaluation. Manual tests are discussed here. |
|
| Entry Criteria List the precondition for using the pattern |
|
| Exit Criteria Define the results necessary to achieve an adequate test. Explicit goal for this pattern. |
|
| Consequences List the advantages and disadvantages of using the pattern. |
|
| Known Uses Known uses of the pattern, know test models and strategies? What is the efficiency and effectiveness of the pattern? |
|
| Related patterns What other test design pattern are similar or complimentary? |
|
A blog about testing patterns?
Let me start this blog by explaining, a bit, why I hope this blog can be useful to more than just me. As a fresh tester (i.e. without years of experience) I am often confronted to the problem of knowing how things were tested (or not) before?
As you might have notice, there is quite a lot of documentation about software testing let it be in books or any format on the Internet.
The first comupter bug report
Often, what we are looking for is a pattern to solve our current problem. I currently haven’t found a good place to find test patterns and this is exactly what I am going to try doing here.
Now I am not going to pretend that I will only write post about testing patterns nor that I will write regularly. I will try to stay focused around software testing and to be as regular as possible.
I really hope you will enjoy this blog and hope, that, at some point you will also comment!
The too short story of a bug life
It all started when a developer gave him life. He probably have some brothers and sisters as the odds are high that if a developer make a mistake in one place, he would have done this mistake in other places.
In this post, I want to focus on what happens after the bug / defect is closed. In most flowcharts I have seen, the life cycle of the bug ends once it is verified. This is a huge mistake!
Once a bug is closed, it can be for several reasons, some questions needs to be asked and answered.
Bug lifecycle with some additions
Bug was closed because it was invalid, this is taking away time and resources. It is important that we can answer why the bug was opened.
- Was it opened because the software UX was leading the user to wrong assumption?
- Was it because the documentation was not clear?
- Was it because the developer made assumption on how the user should use the software?
- Was it because the user was simply wrong or lacking the knowledge he needed to use the software?
If the bug is closed because it is fixed, it is time to ask how to prevent the bug to be checked-in again and to find out if some similar bugs already exist in the software under test.
Some steps I try to follow whenever I close a bug are:
- Add automation check / test to validate the function / scenario that was fixed
- Look for similar code defect. It is very probable that the developer used the same patterns to write another bug of the same type (loop setting error, reading in a collection and so on and so forth).
If for resources or time problem, the team decide not to fix the bug, then it is important to have a system that the team to track the defect on a longer term.