Raising Babies, Testing Software – Much the same thing!

Shortly after becoming a Father for the second time, I thought I was about to become a slave to ‘routine’. ‘Sure the first one had his bottle at this time, so we’ll do this with the newborn, ‘sure when the first one cried during the night, we did this, let’s do this with the newborn’, ‘sure the first one slept during these hours, so will the newborn’ and so on.
Now as anyone who has had more than one child will testify, it’s not as simple as this. Things that worked the first time around can be pretty much obsolete this time, they were outdated, irrelevant and just didn’t work. Essentially, routine went not only out the window, so much as straight through the glass.
So what do you do? Do you persevere with the tried and tested even though it may be outdated and doesn’t fit anymore or do you invent different ways?

After all necessity is the Mother of innovation

The answer is of course the latter. You adapt your approach to the situation at hand, try things, they may work, they may not, toss them out (the idea, not the baby), tinker the approach, reinvent it. You end up finding a rhythm that suits and an approach that works. You end up with the same result (all being well) a healthy growing baby, but raised a different way than his older sibling.
Now what if we were testing software and not raising children?
Consider the following. Test Manager Seamie is given the responsibility for testing the initial release which is a new product going to market, after weeks of coming up with an approach, reviewing, refactoring and essentially agonizing, the test approach that best suits the product (and the company’s testing ethos) is in place. Given the company’s commitment to agile development, Seamie has put in place a mostly automated approach based on end user scenarios. He has agreed that the Test team will work approximately ¼ of a sprint behind development and this gives the automation time to catch up and test the stories and functionality in an automated fashion.
Now, there is nothing whatsoever wrong in Seamie’s approach. It is what I like to term ‘agile lite’ in that it takes whatever facets of agile that best suit the company’s make-up and utilizes them. Not fully blown agile (not any company claiming to be agile that I have ever come in contact with actually is) but that’s ok. It’s ok to adopt an agile approach to being agile
As it turns out, Seamie’s approach is very successful. The product is delivered on or around time, the testing effort found a number of issues (mainly early), the test pack is 95% automated and quick to run, customer beta testing was a success and straightforward and the testing effort suited the Test team and the Development team who were both equally comfortable with the style. – GOLDEN CHILD

Due to its success, another Product Owner in the same company but responsible for a different area has come to Seamie asking him to lead the testing for a similar type product. However, this product offers different functionality and isn’t so open to automation. In addition, the development team here are ‘pure agilists’, they want their Test team to work with them in getting a story ‘done done’ once development is complete. Seamie tries to adopt the same approach to project 2 as he did in project 1. This leads to confusion over where the overall effort is and what is required, friction between the Test and Development teams and to the eventual breakdown of communication resulting in long delays to project 2 and damaged reputations – PROBLEM CHILD

A short story, straight to video and a little general maybe. But this is a common problem today in the Testing world. I have seen this on a number of occasions throughout the years. In people terms it is what we would deem as ‘set in our ways’, it’s the way it’s’ always been so it is the way it is. What could and should Jamie have done for project 2?

Well, for starters the best way to approach any project, irrelevant of how ‘similar’ it may seem to a previous one is to treat it as standalone initially. Ask the question, ‘if I had a blank sheet in front of me with ANYTHING or ANY OPTION open to me, how would I test this?’ Re-invent testing each time! You may come out at the same destination each time (hopefully not or you should maybe re-consider your career as you are most likely getting stale in Testing) but each time new ideas should come out, mistakes from last time can be called out and made priorities this time, tools, ‘what ifs’, etc. should all come out.

After that, ask this question ‘what makes this project different (no matter how small) from previous projects?’ each project, even subsequent releases, have something new or different than before. Make sure you know, inside out, what those are and their usage. How will they be used, how could they be used, and what will a customer think of doing here? All these will help you build a picture as to what ‘extra’ you have to bring to your testing effort.

Finally and only finally, what facets of a previous project can I re-use? This has to be the question asked last. Why? Because, it keeps your mind fresh, sharp and doesn’t allow you to become lazy. If you ask this question up top, you probably won’t get to questions 1 and 2.
Don’t be a slave to routine because ‘it’s the way I did it before’. Things that worked previously may work this time but may not, new things may work but may not. But ask the above questions first – don’t assume and worse still, don’t shoe horn them in as Testing, as is raising babies, is not a ‘one size fits all’ business.
What you will find though is that after a few projects, you will start to notice patterns and this will eventually equate to different ways of doing similar things. Different approaches, tools, mindsets that all lead us to releasable software and a tested (as best we can) product. As mentioned earlier, we end up with the common result but their journeys may take different paths.

A healthy growing baby but raised a different way than his older sibling!

Negative Testing or Negative About Testing

Just a few quick thoughts regards something I have been noticing and thinking about recently.

It seems to me, more and more, we read or hear nothing but negativity about testing. That is to say, when testing comes up in many people's blogs, it is only to criticize the lack of ability in the testing craft, highlight the deficiencies of checking, complain about what Testers think, what Testers are, etc. As Testers, I think we do a very bad PR job on what testing is and brings.

I am a big admirer of some of James Bach's ideas and principles. However, I am not a fan of his abrupt approach to Testers, who don’t sign up to his ideas and principles. To be a Tester, you need to possess an open mind, you need to see all sides of everything and then make your own judgment call. Sometimes, your call won't tally with others, that’s fine, but it doesn’t give anyone the right to belittle you, because your way of seeing testing is different. There is still a way to be a person, sometimes; I think James fails the test here.

Testing is a specialized industry, you need to be a different type of thinker than a regular programmer to be a Tester, be more of a questioner. As opposed to damning Testers constantly and surrounding testing with an air of negativity, we should be trumpeting committed people involved, embracing the committed who are in the Testing domain (even if their thoughts on Testing differ from ours).
Even at interviews, negativity reigns: -

I want to be a Tester because I like breaking things

is an answer that is so stereotypical, it gives clichés a bad name. However, it is so limited. If I wanted to hire someone to break things, I would hire my 2 year old son! But that's the aura that has been created about testing, to break things, to prove things wrong, to dispel assumptions - Negativity!

The purpose of testing is surely to provide information on the product. That's it, pure and simple. Whether that be via the 'frowned upon' checking approach, manual testing, 'super cool' exploratory testing, automated regression suites or whatever individual approach; It is all testing and it is all to provide information - good and bad. However, the message that seems to filter to the top is testing is to provide bad news.

Think of it this way:-
The message about Testing needs to change. We need to promote testing as a valuable industry, a valuable asset to projects, a place where you can advance and hone your skills, create new trends, create new ways of testing, become the difference, gain respect, gain enjoyment, make a career.

Now, read the above paragraph again, but change the spin and think of it positively!

The Principled Tester

What type of Tester are you?
Are you the ‘process driven’ tester? Are you the ‘checks pass so we are ok’ driven tester? Or are you the ‘fly by the seat of my pants, full on exploratory son of a gun’ tester?

All have their roles in the Test industry, there will always be companies who need each of the types (and more!) mentioned above.

The type of Tester you are really comes from within. It depends on a number of things; such as attitude, intelligence and experience. Overall though, every Tester worth their salt will have their principles. That is to say, they hold dear various things that are important to them when it comes to Testing. Some don’t, some are very much 9-5 type Testers with no real genuine interest or passion in what they do or the consequences of their actions. In my eyes, those people shouldn’t even be classified as Testers, they are hired hands doing (more than likely) a below average job. Eventually those people are, or will be, weeded out.

But back to the true Testers, yes, they have principles. I would hazard to guess if we pulled into a room, some of the world’s best Testers, their principles would differ…but probably only by a hairs breadth!
Their language explaining it may vary, their rating of their principles may differ, and some may even see them as not being principles but just part of their make-up. However, hair splitting aside, they will be similar. Core principles are common among good Testers.

Now, whilst I wouldn’t attempt to place myself in such exulted company as among the best Testers in the world (whomever they may be), I believe I have ‘some game’ and see myself very much so, as a Principled Tester.

I place my Testing principles at the very forefront of what I do for a living and indeed, influence me in my job every day. I wish not to impose these on anyone, but for the record they are below. I call it my T.E.S.T.S. approach.

Test the right things
Preparation
Knowledge of AUT with a Test Journey in Mind

End the day never satisfied
Must do better mantra
How can I improve that?

Strength of Purpose
Willing to admit you may be wrong – Humility
Stand by your theories and thoughts – don’t be swayed because others have
Stay true to your testing compass
Take ownership for testing the product/application

Thoroughness
Methodical mindset
Attention to detail
Going the extra mile, run the extra Test, answer the nagging ‘what if’

Speak the truth to power
Your job is to provide information not ‘the sugar coat message’
No Testing is better than poor Testing
Keep others honest


They are in no particular order, but to get the T.E.S.T.S. handle in there, I had to play with the order ever so slightly

Whether we like it or not, Testing is a results driven business. We have project delivery time constraints and managerial pressure and expectation for the right message.
One would think this type of environment wouldn’t lend itself to allowing for principles? It can!

You can still deliver on time, give the right message and be principled.
How?
Project delivery on time can be achieved as long as you provide the evidence which details what you have found as a result of your Testing, on time. Testers are not in the project delivery business, that is not their job. If someone else wants to deliver based on your evidence, with or without your recommendation, that decision is theirs.

Managerial pressure for the right message will always be there, but if the message you provide is that you have found issues/defects that are detrimental to what the project states it should do, then you have delivered the right message because it has provided information related to the health of the project.

Again, what Managers do with that message is not the Testers job.
So you have delivered on time, gave the right message AND stuck to your principles.

First, Fast, Now Testing

Testing in today's world has to be done yesterday. Such are customer demands and in turn, project demands. Good Project Managers and Companies at least attempt to ensure that testing is not squeezed as a result and that the due time and attention is given to ensure at least adequate testing, is carried out. But a lot of the time, their hands are tied.

So how do we ensure adequate testing in a world that is deadline driven?
Any UK and Ireland Sports fans among us will be familiar with 'Sky Sports News'. Essentially this is an around the clock news channel dedicated to sport, almost sycophantically at times reveling in the 'exclusives'. Their philosophy is 'First, Fast, Now'.

I liken agile testing to this, or any testing in today's world. Many companies carry out 'sky sports testing'. The need to do everything immediately, quickly quickly, 'best we can'.

In some instances, that may be fine, as long as you can be confident that you have covered all you think you should and provided the stakeholders with as much information they need to make their decisions on whether to go live. You have done as much as you can. This is a skill that the better testers have. Usually not through design but neccessity.

However, sometimes as Testers, we need to push back, sometimes we need to incorporate a 'steady as she goes' mentality. Take the time to do the testing, investigate, explore, provide more information that merely the confirmatory type.

Hand on heart, how many times do we ever do this?

Admittedly, we are testers and under the direct management of others who are under the direct management of those higher, who want the project done without delay. 'Sure, we can get the customers to test it'. So we might not be able to take our time and perform the due diligence.

Again, that's fine. As Testers, our primary role is to provide information to the health and status of a project. The time we are given to do this is usually someone else's call. As long as that someone is aware that there is the chance that 'Sky Sports Testing' might not lead to a 'Sky Sports Product'.

Wouldn't it be great though, just a little more often, to be asked by PM, 'how long would you like to test' as opposed to 'here is what is left to test, what can you do?'

I’m a tester, I can’t waste time doing testing activities?

I was involved in a discussion one day about the regression testing activities and more implicitly, the post execution activities. Someone was making a point that validating the results and outcomes of the Tests was ‘taking too much my time’.
This point intrigued me and my initial thoughts were ‘ok, so what did these validations prevent you from doing that was more important, than a key aspect of your actual job?’ I simply don’t get this.
As a tester, you have many aspects to your job but the general thesis is this, you create and execute Test Cases (be it exploratory or scripted checks) with the intention of finding out information to provide feedback on the health of a project. As part of this execution, there is an inevitable ‘wash up’ activity that involves verifying the results. That is, to say, did the outcomes match what was expected or is there a potential issue? There is no way around this, the results need to be verified (do you ever hear TV talent shows say before a result is announced, ‘the results have been counted but we don’t need to VERIFY them’ – NO).
If you don’t verify the results, how can you:-

• Provide meaningful information?
• Find out if there is a problem?
• Carry out further testing?
• Improve your testing?

How you do this verification is a different discussion. If it is manual, it will take longer than if you have automation that can spit out results and potentially examine them as well.
It is my belief that automation has been the best and the worst thing that has happened to testing in the past number of years. It has helped improve monotonous checking, increase the speed of provision of information through quicker execution and when used properly, extremely powerful to various areas of testing such as reporting and data entry. However, I believe automation has made testers and testing LAZY. Some Testers (as well as Managers) now see automation = testing and anything that cannot be automated is not worth testing. Further to that, some testers are seeing executing automated Tests as their entire testing activities…period. In other words, the automation runs when it is finished, I am done and nothing else is required. This is equivalent to saying it NEEDS to provide Passes and the Green bar so as to not mean I spend time verifying the results.
This mindset is damaging to the core aspects of testing and actually quite insulting to the good intentions of the keepers of the testing industry.
All Testers want to be providing value, being involved in the creation of the next big Testing thing, creating solutions. However, all Testers need to be true to the core aspects of their job; the creation of Test Cases, Test Case execution and Test Case execution FOLLOW UP.
There are no workarounds here, no quick fixes, its simply part of ‘doing your job’. If you are a tester who doesn’t believe in these facets of your job, my advice is quite simple ‘Testing is a waste of your time, time for a career move’.

Testers get lost too

A little experience I had the other night that you may (or may not) find interesting.

I was walking to my Sister-in-law's from work and came across a schools soccer match at the local park. Being a soccer fan, I naturally stopped and watched along. Being a true fan of the fine arts of football; the skill, flair, sublime passing, vision and will to win, I was immediately drawn back to my own youth when we raced out to the local park and played for hours. In all this time, we attempted to do things that you wouldn’t see in a modern day top class league. Things like trying the unexpected, dribbling forever, shooting from unique angles, everything you wouldn’t see nowadays. In a way, it was the innocence and impotence of youth; a carefree and unabandoned attitude. I liken this to the modern day true tester who sees testing as a journey, a learning experience and not a job.

What I saw that evening was everything that wasn’t the wild freedom of youth but a tightly constrained, follow the rules, process driven, shackled attitude that prevented these young fellas from expressing themselves. There were constant kicking without looking, long balls, no flair or imagination, nothing that could be deemed enjoyable or 'easy on the eye'. In essence, they were ‘over coached’, if you like, they were given ‘certifications’ in todays testing parlance. ‘Follow the rules and you will be accepted and go further’.

I walked on home and immediately drew comparisons with the scene I had just witnessed and the testing community today. Those kids were like testers who start out with, or have, the correct attitude to testing and see it as fun, exploring, looking for things not thought of, improving. However, somewhere along the line, the kids, like the testers, fell in with 'coaches' who coached the carefree and ‘want to explore’ out of them. In the testing world, the 'coaches' are the managers or the 'ISEB clones' who have kidnapped or brain washed these testers with ability and skill and mis-directed them with tales of certification, industry acceptance and make your cv/resume look good!

Good and conscientous testers, testers who see their job as a continual learning experience, who are never satisfied, who are looking for better, are not constrained, are not shackled, do not need certifications and are essentially like the carefree kids playing football back in my day. Those testers need salvation and need to read blogs from testing lights to see there are other ways than the ‘official coaching’.

To become a great footballer, you do not need a badge or certificate. You need ability, desire, the want. All the great footballers of today and yesteryear had these traits in abundance.

Testers, look at the great testers in the world today, the likes of Kaner, Bach, Weinberg, Bolton. Then ask yourself 2 things:-

• Does the above paragraph sound familiar??
• What road do you wish to go down?