October 17, 2007

Green Means Go?

Posted by Ben Simo

Traffic control devices are used on roads to help regulate the flow of traffic. When I taught defensive driving classes, I ensured that each class included a discussion about these devices. It is imperative that all drivers understand what each device means. My first Driver License test (in Germany) required that I properly identify 94 of 100 different signs to pass. There was a time that each local governing authority created its own traffic control devices. However, it was not long after the automobile became common that governments began working together to standardize these safety-critical devices. While there is no universal standard that is really followed (the USA being one of the countries that differs from most), standardization within each country (and some continents) has led to safer streets and highways.

These devices include signs, signals, pavement markings, barricades, and policemen. These devices are tools used to control the flow of traffic. However, these devices cannot really control traffic -- except for massive barricades and armed policemen. They provide information to drivers but they cannot force drivers to be safe or legal. As can be seen here, sometimes drivers ignore the signals.

Traffic lights are mostly standardized around the globe. Green means go. Yellow means caution. Red means stop. ... except for the extraterrestrial visitor in the movie Starman who learned by watching a bad example.

Red means stop;
Green means go;
and Yellow means go very very fast!

- Starman

The red, yellow, and green traffic light colors have become common in software development, testing, and production monitoring. Traffic light colors are regularly used to report the status of projects, systems, and individual tests.

I like simple status indicators -- in context. One of the first test execution tools I helped create used smiley faces to indicate passed tests and fulfilled requirements. I currently use color coding to indicate status in the test automation I develop. Colors help me quickly find test results that need attention. Simple color coding helps communicate test results at a high level.

I like to see green. I don't like to see red. However, I am not a member of the Church of the Green Bar*. I do not worship the green light. I do not trust the green light. I find green lights and bars useful but wrong. Green lights remove the story from the status.

Essentially, all models are wrong, but some are useful.
- George E. P. Box

A green traffic light may tell us that it should be safe to go but it does not guarantee that it is safe to go. A green light does not indicate that the intersection is clear. A green light does not indicate that it is safe to drive through the intersection. Drivers need to wait for the green light but they still need to check the intersection, identify potential risks, and decide if they believe it is safe to drive through the intersection. A green light means go if it has been determined that it is safe to go.

In the same way, a passed automated test does not mean that the software is good. Green simply means that the coded criteria was met.

Every time we test software -- whether with human eyes and mind, or with automation -- we only monitor the things that we choose to monitor. A human tester may notice things about the quality of a product that are not scripted. Automated test execution will only notice things that are coded in the script -- no matter how many time we run the test. If a factor that might indicate a problem is not part of the automation's green/red (pass/fail) criteria, it will not turn the light (or bar, or text) green.

I also find that there is often a misunderstanding of what automation does and does not do. The coder of the automation may know what it does when they code it. But will they really understand what a green light does and does not mean six months later? How about a year later? Five years later? Do other testers understand what the automation does? Does management understand what the automation does do and what it does not do?

Software systems can easily become complex. Computers allow us mortals to create complex systems that are beyond our ability to fully understand. We testers seek out software problems based on what we understand. We cannot completely test the software. We use a variety of tools and approaches to learn as much as possible. However, we are unlikely to completely understand a complex software system.
- Ben Simo^

We mere mortals and the automation we create are unable to monitor every thing that might matter. Therefore I believe it is dangerous to conclude that green means good.

The same goes for project management and system monitoring systems that use quantifiable metrics to set status.

Traffic lights based on the judgment of the people involved in the project are better indicators of status. If a light is green because a person set it to green, that person should be able to tell me the story behind the decision to make the light green.

Beware automated traffic lights.


Automated traffic light indicators in testing tools are only badometers+. A badometer tells us when something is suspected to be bad cannot tell us if it is good. Better traffic light indicators are set by people that consider the risks associated with the information reported by our tools.

So, Does green mean go? Yes, but only after a human being has judged it safe to go.

* I don't know who first coined the term, but I first heard it from Brian Marick.
^ Since I regularly quote other people, I think I am entitled to quote myself. :)

+ A term I think I first heard used by Gary McGraw. Or what it Kim Possible?



October 18, 2007  
Pradeep Soundararajan wrote:

^ Since I regularly quote other people, I think I am entitled to quote myself. :)

You certainly can quote as long as you owe the credits to Ben Simo.

The video that you have taken as an example actually has nothing wrong in it - and I can say that sitting in India but probably I might have not said that if I were a North American or a European.

The Indian swallows are allowed to carry a coconut as long as they disturb the traffic by dropping it on someone who pretends to follow the traffic rules.

I learned from Rapid Testing training that it is important to break the rules and I was glad that I had been doing it all the while though not as conscious as I do now after the training.

The video clearly shows the need of having a police officer ( you might want to say someone who does a manual activity of automated traffic light )

An automated traffic light system cannot control traffic in India neither can an unskilled traffic police.

Here is an amazing co-incidence: While typing this comment there was a big SCREEECH sound and I looked outside the window - a common sight - a scooter had crashed into a car.

So, to control the traffic not only a skilled traffic police is required but skilled drivers who have had and practice "discipline" from a long time.

October 19, 2007  
Ben Simo wrote:


Especially in the normal-for-India example in the video, drivers need to look before they go. When the light turns green, there is a high chance that there will still be opposing traffic in the intersection. Only when the light is green and the intersection is clear is it safe to go -- although you may need to kind of push your way through or get run over.

This kind of reminds me of waiting in line (or queue). In the US, we usually respect other's place in line. In other countries, I've discovered that waiting in line is an active activity. Each person needs to actively defend their place in line or they will lose it.

I digress. Now back to the green lights in software tools...

As we learn what things the traffic lights do not tell us, we can create heuristics to help us judge the situation. We can also enhance our tools and add tests (test cases, charters, examples -- whatever you call them and however you implement them) to check for things not presently covered by our traffic light setting algorithms.