Tag Archives: software testing

Progressive vs. Non-Progressive Bugs

In the past year or so, I have used this comparison when talking with our product and development teams. I highlight the differences in what I term Progressive and Non-Progressive Bugs.  While they are both in the bug category, one is more helpful in gaining insight into the state of the system as well as moving us along the path to even more insight.

Others may have varying terms/definitions for the same thing, but I have found this to be helpful in getting my point across when discussing perhaps a lack of Unit and Integration testing before a build, a late arriving change in the release process, etc.

Non-Progressive Bug:

I refer to a Non-Progressive Bug as those that most likely should have been caught in Unit testing and block additional testing. Yes they are a bug, but they aren’t a bug in the type that provides additional information to the tester on the actual functionality being delivered.

An example would be:

  • Development is working on a Create User module.
  • They build & deploy the code for you to test.
  • When you receive the build and open the Create User module, the Create button is not functional – either disabled or generates an error.

This grinds additional testing to a halt and this bug does not need any special type of skill in testing to come across.  It is a bug that must be logged and fixed to move forward, but doesn’t move the process forward to getting this module ready for release and provide the stakeholder with additional information as to its readiness.

Progressive Bug:

A progressive bug is one where you are able to dig your teeth into the code and really explore the boundaries and the bug detected provides you important information to the readiness, stability, etc. of the code under test. It also provides you perhaps a new pathway to go in your round of testing that you may have overlooked otherwise. It continues to allow you to provide stakeholders relevant data to make the informed decision on state of the feature, the risk associated and additional steps needed to get it to a state where you all feel comfortable releasing it to the public.

An example would be:

  • Development is working on a Create User module.
  • They build & deploy the code for you to test.
  • You begin testing the new Create User module and in testing decide to test it for XSS (Cross Site Scripting) vulnerabilities.
  • Your test uncovers that the module does in fact have vulnerabilities in the code.

This is critical information to have and especially in this day and age of so many security exploits occurring across the globe. This is data you can take to the stakeholders and they then have critical information to make the decision on what to do next. This could spark you to try additional security vulnerabilities and/or allow you to move on to other areas of the module to do more testing to gain more insight/information into the feature under test.


In conclusion, both of these types of bugs need to be resolved. The difference is Non-Progressive bugs need to be fixed in order for the tester to do the actual testing and begin finding Progressive Bugs that will allow them to provide relevant and accurate information to the stakeholders which in turn allows them to make better formed decisions on direction, readiness for release, etc.


Code Lochness & Code Frost

These terms I came up with as a joke over the last couple of years in regards to adherence to code lock down and code freeze.  We have all been there and will be there again where something (or many) slip into the release post-lock down.

Back in the day I used to lament and complain and use this as a crutch to state I could not be job successfully.  Now I have to come to see it differently and plan for it and adjust our approaches to accommodate late changes as well as educating others what testing can/cannot do for them. I will to get to more on that later, but first the origins of the names.

Code Lochness:

This I feel is a more appropriate name for what we think of “Code Lock down” in our daily lives. This idea came out during many conversations/venting sessions with a developer friend of mine.

The term Lochness is in reference to the mythical Loch Ness Monster.  Like Code Lock Down, the Loch Ness Monster is claimed to exist by many yet has never been seen. They are both mythical creatures.

Code Frost:

This is a more recent addition to my vernacular. The term “Code Freeze” is misleading and is more of a Frost. Code is not frozen, but generally (hopefully) the stream of new code/features coming in to a release under test slows down to a trickle after Code “Frost” hits.

Lessons Learned in Software Testing – Lesson #162 “There are always late changes”

I recently re-read this lesson and jumped out now as have been thinking about this a lot.

This lesson talks about how often times new software is being built and there hasn’t been something like this built and/or used before. There are many reasons listed that drive late changes.

  • All requirements not known when project kicked off and planning was done.
  • Requirements change through design sessions, UAT, etc. and as the team gains more info/insight into the target and intended use.
  • Differing needs of the various stakeholders.
  • Sometimes the work involved takes much longer than expected it will take.

I have seen this time and time again as well as other needs that drive late changes:

  • A customer escalation that is found in production and needs to be fixed right away. The risk of impacting that customer relationship by doing nothing is often times higher than that of coding the fix and deploying to release for testing.
  • A customer commitment with a high $$$ value and/or good publicity can weigh in and be willing to accept a certain level of risk to meet a deadline and satisfy the customer.
  • Similar to first bullet point, the risk of NOT making the fix/enhancement is far greater than that of getting the code in place, tested and out the door to production.

How to Respond:

  • Expect and plan for late changes. Understand why these changes are coming in late and develop a context driven approach on how you will test these changes in the limited time frame and what tools/techniques you can use to aid in your testing. If you have an hour or two, figure out what are the best tests and approaches to explore and provide the most information possible on the state of the fix and its impact on the feature/release.
  • Educate development and leadership on what they can actually expect testing to be able to cover given time & resource availability. Make sure they understand the risks associated with implementing a late change post lock down/near release.
  • Don’t use it as a crutch to NOT do your very best work & effort.

My job is to explore and provide relevant information to all relative stakeholders and continue to educate them on software testing’s role in the overall quality of a product and clarify what folks can/cannot expect from the testing group. We can assure quality, we cannot guarantee issue releases.  It does not mean we stop trying for that goal and in that hopefully our understanding of the issue is accurate, our strategy to attack this issue is sound and that we give nothing less than 100% to the task at hand.

The long story short is we will most likely see the The Loch Ness Monster before we ever see a true, full code lock down. And you know what? I am okay with that (now).

My Internal Keynote Address

Any conference you attend there is generally a keynote (or two) from an industry expert and/or celebrity that ties into the general theme of said conference. Generally, the attendees leave the keynote both educated as well as enthused to learn more on the topic.

This enthusiasm can be short-lived and dropped by the time you have arrived home from the conference or it can have a long-lasting impact in which the attendee is awakened and continues forward post conference to learn and try to implement the lessons they learned.

This got me to thinking, what if I gave myself my own internal keynote address each day/week. The goal would be to find a way to frame the upcoming day/week to both energize myself to conquer the task at hand as well as to continue my pursuits (both professionally and personally). Whether it be an upcoming meeting with a co-worker, a current project I am responsible for we all have those days that are taxing and hard to generate the same level of excitement.

A lot of how your day goes is heavily dependent on your attitude. Do you want that day to be a good day or a bad day? Are you going to see challenges as a road block or as an opportunity to learn something new? These are just a few examples of conversations or keynote addresses on could give themselves.

I will blog more on various testing ideas, concepts and/or strategies, but wanted to start (or resume) on this topic as well as make the following plan over the upcoming calendar year and thus hope then helps others hold me accountable (and for me to be accountable to myself as well).


  1. Blog on a routine basis. My initial goal is 1 per week and have set a weekly reminder to myself asking “Have you blogged yet this week?” Hopefully more than 1/week and realize that not every blog post has to be a novel. They can be short and to the point as well.
  2. Engage in testing related conversations on social media (Twitter, LinkedIn, etc.) on a weekly basis.
  3. Continue to read and experiment with new techniques/procedures and work to train my team on these.
  4. Actively participate in the IWST’s (Indianapolis Workshops for Software Testing) and present information as well.
  5. At CAST 2014, develop and sign up to do a Lightning Talk. Also to introduce myself to 10 new people that I don’t know and engage in discussion.

After previous conferences I have come back rolling like a ball on fire, but eventually this fades a bit and fall back to the wayside as real life kicks in and let myself get defeated (or delayed). My goal is to continue on and try to make each week as enlightening and motivating as a week attending a CAST conference does for me. I also strive to help those that work on my team gain the same excitement for their craft as well.

Thanks for reading. I will be working on my takeaways from CAST 2013 in Madison, Wisconsin over the next week.