SUBSCRIBE TO MY CHANNEL

Five Test Automation Mistakes to Avoid - Must watch for testers!

How to be successful QA leader? - Future leaders, QA Leads and Managers

LATEST THINKING

Monday, November 30, 2015

IS TEST AUTOMATION A SILVER BULLET?

1:43:00 AM Posted by Prashant Hegde 8 comments

QA plays a crucial part in product quality and product delivery. In agile development methodologies like scrum, QA needs to work in highly compressed test execution cycles. One of the challenges the QA engineer's face is that the code that worked in previous sprints often gets churned by the new features and bug fixes in each subsequent sprint, increasing the risk of regression. Without test automation it might be hard to achieve sufficient test coverage and provide rapid feedback.  
So is test automation a bullet?
Before we jump to the conclusion...we need to understand the following:

WHEN TO AUTOMATE?

Check if your application is automation ready? Below are the questions you need to ask yourself before you rush to automate the tests.
1. Is your application ready for automation?
We need to careful when automating GUI. Never start automating at the start of the project or you might end up re-writing the automation scripts that you have written.
To start automating, you need to have a set of stable and functional features ready. It is wise to start automating when you are sure that these features and the GUI will not change. If you proceed and automate the features that are undergoing continuous changes in GUI then the cost of maintaining the script will be too high. Predicting what will change and what will remain stable is tough to determine when working in an agile environment as we need to embrace and welcome the changes.
Another important thing is not to overlook is the maintenance that is required to keep all your scripts up to date and working. Even after spending hours on programming the automation scripts, your automation scripts might fail when you get a new version of the product. You need to design the scripts in such a way that it will require the smallest amount of maintenance in the future. We should have a plan of how we are going to accomplish that in our 'automation strategy'.
2. Do you have skilled testers in your team?
Your QA team need testers with the skillet to utilize the capabilities of the tool. No tool can help you unless you have skilled resources to implement automation. Your test team should capable of handling challenges that might arise in automating the test suite. Most of the tools demand testers to code. Most of the automation tool vendors provide training and support to the resources else you should consider bringing in an expert from outside. Test Automation is a long term investment. It will take time and expertise in developing and maintaining test automation frameworks and automated scripts. 
 3. Have you selected the right tool?
Choosing the right tool is crucial to succeed in test automation. There are several factors that need to be considered when selecting a test automation tool. Some of the tools are free, some are expensive.
Several times after few days of initial excitement and after automating few tests the tool often becomes "shelf-ware" and testing returns to Manual.
 Here are some factors that you might need to consider before selecting the tool:
  • Does the tool fit into current process and infrastructure?
  • Does the tool vendor support training and provide good services,
  • Do you have the budget for the tool?
  • Product features: How does the tool identify the objects? Image based recognition? Object-based recognition?
  • Which programming languages does it support? Does the QA team have the knowledge on these languages?
  • If using an open source tool: Does the tool have good community support and enough documentation?
  • The list goes on…
There are risks of the vendor going out of stock or the tool being acquired or retired. It's a good practice to explore the tool on a small scale with a pilot project and determine if the tool can accomplish what is needed.
We should keep in mind that no tool is perfect, be it an open-source or commercial.we need to consider the above factors and select the tool that serves our purpose and provides good ROI.

WHAT TO AUTOMATE??

Did you say "Everything"?
 Then it is an unrealistic and impractical expectation!! 
 1. AUTOMATE REPETITIVE STUFF!!
Testing software involves a lot of repetitive work which is tedious to do manually. People become bored and make mistakes when doing the same task over and over. People tend to do the same task differently when they are repeating it again and again. A tool will exactly reproduce what it did before, so each time it is run the result is consistent. The regression suite is the ideal candidate for automation. 

Repetitive work is excellent job for a computer to do.

2. AUTOMATE THE TESTS THAT ARE HARD TO BE PERFORMED MANUALLY.
Assume you are testing an app which has a feature called ‘People’ that sync's all your contacts from various social media and email accounts. You are testing if the ‘contact count’ displayed is accurate. The test account has 5000+ contacts. It is not feasible count the contacts manually and check if the count displayed is accurate. 
Assume your testing an email app and the app crashes frequently when you 'archive' around 30-35 emails in succession(one after another). When developers ask you to replicate this issue, think how tiresome it would be to do it manual. Let's be positive and think that you replicated the issue after few attempts..and the developer fixes it. But that's not the end of the story.... you need to confirmation test if the issue is really fixed..Good luck!
Automation tool eases the life of a QA engineer in such situations and can save a lot of time and effort by automating these annoying tests.
3. AUTOMATE BVT/SMOKE TESTS
BVT(Build Verification tests) are cursory tests that are helpful to determine the stability and testability of the build. These tests need to be run every time a build is deployed. The BVT’s are ideally quick tests that concentrate on core functionality.
Automating BVT's and integrating them to the continuous integration server will ensure that the changes made have not led to regressions. One thing to we need to keep in mind while designing these tests is that the test should be short and should the test should execute in few minutes. There may be multiple commits a day it does not make sense if the automated BVT's run for hours.
Also consider the following while automating:
  • Automation is a great option when we have to validate the same feature with large set of different inputs and data (Data-driven testing).
  • Load and performance testing should be ideally automated where we test the system’s ability to handle load by creating virtual users. 

WHAT NOT TO AUTOMATE?

Do not try to automate the tests that would require human intelligence and intuition.

  • Tests which are not executed frequently or would be too cumbersome to automate.
  • Tests like exploratory testing which based on intuition and expertise of the tester.
  • Tests that need human intervention. An automated test should be able to run and verify the outcome entirely unattended.
  • Do not automate the volatile features which will have frequent changes.
Prioritize your automation using the 20-80-20 rule. Automate 20 percent of the scripts that take 80 percent of the execution time and 20 percent of the resource skills.
CAN TEST AUTOMATION REPLACE HUMAN TESTERS?
One of the developer who watched me automate was fascinated and said “This is cool stuff! Now you need not have to do anything, the computer will do everything for you. I feel that computers will replace the testers soon”.
I replied “The day when testers will be replaced by automated tester's is the day when developer's will be replaced by auto-developer or a CEO will be replaced by an auto-CEO ...”
QA experts do not use the word 'test automation'. They rather call it ‘automation checks’ as they feel testing cannot be automated, but checking can be completely automated. Humans have the ability to do things, notice things, and analyze things that computers cannot. Our humanity as testers plays a big role in identifying defects while testing. A computer cannot experience or cannot express emotions.
A computer cannot 'determine if the game is fun or not?' ..... 'if an app has good user experience or a pleasant interface'..... Or to 'take decisions using heuristics' etc. Automated test can only find defects that you already thought of or predicted might happen.
You may ask if the test automation can run the same test more quickly, more accurately and with same consistency every time, is it not a better than a human tester?
No! Because running a test quickly with more precision and consistency does not produce better software but it would definitely reduce the time required to complete testing. Test Automation is probably one of the most useful weapon in tester's inventory. It has its own place in the inventory. The weapon is useless without its wielder. Automated tests require scripts to be written and maintained, this in turn requires a HUMAN.

 

Automation tools are here to assist the testers not to replace them.


A right mix of manual tests and automated tests are essential to test a software effectively. Automated testing requires a higher initial investment but can yield a higher ROI. Test Automation increases the test team’s efficiency by allowing releases to get to production on schedule. Planning, training the resources, selecting the right tool, understanding the limitations and setting realistic expectations is important for the success of test automation. 
Test Automation is certainly not a "Silver bullet".
Please share your thoughts on this!

Friday, August 7, 2015

ARE YOU STILL USING OLD-SCHOOL BUG LOGGING PROCESS?

7:21:00 AM Posted by Prashant Hegde 4 comments


REPORTING BUGS CAN BE REALLY PAINFUL!

A good bug report is the one that describes the problem well enough that anyone related to the project will easily understand the report without the need to speak to the person who wrote it. A visual representation of the bug can help the developer understand the problem even more quickly. So testers always attach a screenshot of the bugs they encounter, for more complex issues they attach a video.

It’s not that simple!

Mobile app testers get frustrated with taking screenshots and transferring it to their systems and later annotating and highlighting the area of the bug. At last upload it to the ticket in the defect tracker. Think how tedious this task is if a separate ticket has to be raised for every issue.
Recording a video can be even more time consuming. More time is spent in transferring the videos from their device to the system. You might need to compress the videos in case it’s size too large. Even after all this effort the developer might not have enough details to debug the issue?

ARE YOU FED-UP OF RAISING BUGS?

WAIT! THERE IS THE SOLUTION.

YOU NEED AN “IN-APP BUG REPORTING TOOL”

WHAT IS AN IN-APP BUG REPORTING TOOL?

An in-app bug reporting tool allows you to take screenshots, add drawings over them, capture video screen recordings and send them from inside your own app directly to your issue tracker. You can log a bug without the need to close the app you are testing.

HOW TO INTEGRATE IT IN MY APP?

To use the in-app bug reporting tool you need to integrate the SDK given by the tool to your app. The developers need to add few lines of code that’s all. It takes just a few minutes!

HOW DOES IT WORK?

You can trigger the in-app feedback from a “button” or by a gesture like “Shake” within the app which will invoke the SDK. You get options to ‘Take screenshot’ and ‘Record a video’. You can take a screenshot of the issue and annotate it within the app.
You can record the video of the bug as well. Once you’re done taking a screenshot or a video you can describe the bug in a form that is similar to your bug tracker. That’s all you need to do! The tool will create a ticket for you in your test management tool.

Some In-app bug reporting tools let you add voice memo for super-fast bug recording. Cool isn’t it? 

HOW CAN IT HELP ME?

1.      Increase productivity and save more than 50% of your testing time!

Testers spend more than half of their time in reporting bugs they encounter. As mentioned earlier the in-app bug reporter helps to log bugs quickly which helps you to concentrate on testing rather than on reporting bugs. The in-app feedback helps to overcome the lack of communications between developers and testers. When there is a lack of detail or confusion in the bug reports, developers need to request more information in order to understand and replicate the bug. So the tester might have to invest more time to make the developers understand the bugs. The tools provide every technical information that developer wants. Thus eliminating the time needed to explaining the bugs. The tool highly increases the efficiency.

2.      Create comprehensive bug reports instantly.

An in-app bug reporter automates many of the most error-prone manual tasks. The tool auto-populates details like device model, operating system and the version, screen size, app version, language, time zone, and other environment details. They also capture the crash logs for you. The bug reports that are logged from the tool give you deep insights and let you pinpoint the exact issue. They help you to create a ticket quickly and without missing out the essential details.

3.      Integrate with your favorite tools.


Do not worry if you are already using an issue tracker. Most of the tools have the ability to integrate to your issue trackers.  You can add an issue right from your device to your project’s defect tracker.
Several  in-app bug reporting tools support JIRA, Bugzilla, GitHub, Bitbucket, Asana, Trello, Slack, Sprintly, Zendesk, Basecamp, Pivotal tracker, Unfuddle, Redmine and much more.

4.      Collect feedback from Beta testers and customers easily.

  

When you are beta testing your app you might have a hard time to collect and organize the feedback from your users.  Clients may communicate their feedback verbally or via emails while the beta testers raise issues in a defect tracker.

An in-app bug reporting tool makes your life easier by capturing all the issues, feedback in the same place so that you don’t miss anything. The customers and beta testers can report the issues directly from within the app to your bug tracker. It’s easy for a non-technical person to report issues using this tool as it automatically adds all the technical stuff. It saves the costumer’s time in reporting the issues they face by writing long emails. I assure that they will be in love with this tool.

5.       Awesome Dashboards - Get Statistics, graphs, and trends.


Most of these tools provide with a dashboard where you can track the progress, monitor the issues.

Ø  Which OS report the most issues? 
Ø  Which networks are your beta testers using?  
Ø  How many bugs were reported last week?
Ø  Which devices are reporting the most issues?
Ø  How many major bugs are still open?
Ø  ….and much more

You will get all these insights aggregated in one page with various filtering options. 
It let's you identify trends of your app by providing various statistics and graphical reports.

TOOLS

There are several in-app reporting tools in the market. These tools are available for web-apps and mobile platforms like iOS, Android and Windows:

Commercial

·         Bug-Clipper – (Free for 50 issues/month and 5 team members)
·         Instabug - (Free for 2 team members and 2 Apps).
·         Bugdigger
·         Snaffu- (Free for 2 team members and 2 Apps)
·         Applause (The in-app reporter comes a part of the package)

Free

·         Testfairy
·         Jira-Mobile Connect - This add-on is free for JIRA Server instances
·         Blitz Feedback –   is free when registered in beta phase.
·         App-Assist-   You need an invite to use this.

Visual bug reporting  tools for your website

·         Lean Testing(Free for limited features)
·         Usersnap
·         Bugherd

Not many testers are aware of these tools. Every tester needs to come out of the traditional bug reporting technique and utilize the in-app reporting tools to create quality bug reports and speed up the bug logging process.

When will you start using an in-app bug reporting tool?


Thursday, July 23, 2015

MINDMAP: A KILLER WAY TO INCREASE YOUR TEST COVERAGE.

6:44:00 AM Posted by Prashant Hegde No comments

What is a mindmap?

A mindmap is a diagram used to visually organize information. It can be called as a visual thinking tool. A Mindmap allows complex information to be presented in a simplified visual form. A mind map is created around a single concept. The concept is represented as an image in the center to which the associated ideas are added. Major ideas are connected directly to the central concept, and other ideas branch out from those.

Mindmap is a great for note taking, planning, studying, brainstorming etc. The term 'mind map' was first used by Tony Buzan in 1974. I drew my first mindmap when I was in school. I preferred mind-mapping over text notes and it proved to be a great aid to revise and recall the concepts quickly. This is because the information in mindmap is structured in a way that mirrors exactly how the brain functions - in a radiant rather than linear manner. A Mind Map literally ‘maps’ out your thoughts, using associations, connections and triggers to stimulate further ideas.

How to draw a mind map?
Tony Buzan the "Father of Mindmaps" suggests the steps below:
1. Start in the center of a page.
Why? Because starting in the center, gives your Brain freedom to spread out in all directions and to express itself more freely and naturally.
2. Use an IMAGE or PICTURE for your central idea.
 Why? Because an image is worth a thousand words and helps you use your Imagination. A central image is more interesting, keeps you focused, helps you concentrate, and gives your Brain more of a buzz!
3. Use COLORS throughout.
 Why? Because colors are as exciting to your Brain as are images. Colour adds extra vibrancy and life to your Mind Map, adds tremendous energy to your Creative Thinking, and is fun!
4. CONNECT your MAIN BRANCHES to the central image and connect your second and third level branches to the first and second levels, etc.
Why? Because your Brain works by association. It likes to link two (or three, or four) things together. If you connect the branches, you will understand and remember a lot more easily.
5. Make your branches CURVED rather than straight-lined.
Why? Because having nothing but straight lines is boring to your Brain.
6. Use ONE KEY WORD PER LINE.
Why? Because single keywords give your Mind Map more power and flexibility.
7. Use IMAGES throughout.
Why? Because each image, like the central image, is also worth a thousand words. So if you have only 10 images in your Mind Map, it’s already the equal of 10,000 words of notes!         

PRO TIPS

  • If a mind map is getting too big or complicated try splitting it.
  • Do not use long detailed sentences in mind maps. Using one word per line improves clarity and understanding. Using single keywords will make your mind maps more powerful and flexible.
  • Develop your own personal style of mind mapping.

HOW TO USE MINDMAPPING TECHNIQUES IN SOFTWARE TESTING?

Mind maps can be used in all the test stages from test planning to test case execution.Mindmaps can be used for:
  • Test Planning
  • Requirement analysis
  • Impact analysis
  • Task allocation
  • Test case design
  • Traceability
  • Test reporting -Quick test reports 

Test Planning

While test planning, you can draw an initial mindmap keeping in mind the list of tasks, schedules, tools, roles, responsibilities, milestones etc. Present the mindmap and discuss it with your stakeholders. Modify the mindmap if any changes are required. One thing you will love about mind maps is its flexibility to adapt to changes. All you might have to do is to add or remove a node/branch. This flexibility might not happen when you draw on a paper, but a mind mapping software assists any changes easily.The final mind map shows you the scope of testing in one glance. This mindmap can be used as a blueprint and later converted into a plan. This ensures that no test activity is missed.


Test case design

Mindmaps are an efficient way of creating lean test cases. It reduces the time required for creating test cases yielding better results. Mindmaps are very easy to maintain and are flexible to changing requirements.Draw branches from every user story/epic and associate all its functionalities as sub-nodes.Start adding test ideas/test case for each functionality.
I created a mindmap covering test ideas for the major functionality. My team started to expand the mindmap by branching out more and more test ideas. We kept adding new nodes when we found unique scenarios that uncovered the bugs during our test sessions. This drastically increased our test coverage.The final mindmap can be as the basis for test case document or it's cool if it's used as it is. The best part of mind mapping is that you generate more ideas when drawing them. Collaborative mind mapping with the team gives you best results.

Traceability mindmap

A traceability matrix is an essential tool for every tester to analyze and improvise the test coverage. You can use a mindmap instead of a tabular traceability matrix.

To create a traceability mindmap - add nodes of all the Epics. Draw branches from every module and associate all its user stories as subsequent nodes.  Now link the test cases for every functionality.You can link the requirement number of the test management tool.
This ensures that you have not missed out writing test cases for any user story. This mindmap gives you the birds-eye view of your test coverage.You can identify the areas where you need to strengthen your coverage.

You can use mind maps anywhere and everywhere!

Below is a mindmap for "App store compliance-iOS":

Tools

There are tons of commercial and open-source tools that lets you visualize your ideas as a mindmap. I prefer the following tools:
  • Xmind(Windows/Mac/Linux)- Probably the most popular and free mindmapping tool. 
  • Coogle(Web app) - Coogle is a web app that lets your team collaborate and work on a single mindmap.

The use of mindmaps are getting popular with agile testers and lean test practitioners.

Do you use mindmaps for testing? What do you use them for? I would love to hear your thoughts about this! 

Friday, June 12, 2015

7 AMAZING THINGS THAT BIRDS TEACH US ABOUT TEAMWORK

1:04:00 PM Posted by Prashant Hegde No comments
Migratory birds like Geese, Ducks, Swans, Pelicans move towards southern hemisphere during the winter when the weather becomes too cold for them to survive.

You might have seen a flock of birds fly in a V formation. Have you ever wondered why they do so?
Here is the secret:The V formation highly boosts the efficiency of the migrating birds. They take advantage of the aerodynamic effect. When a bird in the front flaps its wings it reduces the air friction for the birds behind it giving them an additional lift. A bird can fly 71 times more efficiently in a V formation than flying alone. Thus they can reach their destination faster and with minimum energy expended.

What pilots learnt from these birds?

The "V", or "Vic" formation is a basic flight formation for military aircraft in many air forces.The Vic formation is also common in ceremonial flyovers and airshow flights.V formation is said to improve fuel efficiency of aircraft.

The military aircraft can save 20% of fuel when in a V formation. Allied bomber pilots of the Second World War are rumored to have noticed this first.

What can we learn?

1. One Goal- One purpose

The flock has a common goal - to reach the destination and reach quickly because “WINTER IS COMING”.

Working towards a common goal is most essential when working as a team. If every individual works to achieve their own objectives then it becomes impossible to win as a team. There might be 100 directions to reach the destination but only when everyone agrees on general direction it will be more effective.

2. Stay Coordinated 

Just staying in the V formation won’t help the birds it’s about flapping at the right time that matters.

Having a team won’t help if you do not coordinate. You need to in sync with the team to get things done.  So every member of the team needs to co-ordinate and collaborate effectively to win.

3. Give everyone a chance to lead

When the lead (the bird at the front of the formation) is tired it flies backwards to rear of the formation (where the resistance is least) and lets another bird to take the leadership. In this way the birds keep taking turns when flying in a V formation. Rotating through various positions in V maximizes the utilization of every bird’s energy.

Everyone have their own strengths and capabilities. A leader should empower others to become leaders by letting them take the lead in the areas they excel.

Leaders don’t create more followers, they create more leaders.” – Tom Peters

 4. Increase the Visibility

Flying in a V offers each bird unimpaired line of sight when they fly. Every bird has the clear view of what’s in the front. It makes easier to keep track of each member in the flock.
An organization should have visibility in two ways. Having top-down “V”isibility enables leaders to stay connected to make better informed decisions. On the other hand, bottom-up “V”isibility enables employees to see the bigger picture, which in turns helps in their engagement, empowering them for better alignment with the organizational objectives.

5. Recognize and appreciate hard work

The birds at the rear of the V formation honk to encourage and appreciate the leaders in the front. This motivates the lead to keep up the speed. 

Everyone likes to be recognized for their work. Appreciation confirms that their work is valued. Appreciation is one of the top motivators for employees to work harder and to be more productive. Studies show that appreciation is directly linked to happiness in the workplace. A study by online career site Glassdoor revealed that more than 80 percent of employees say they are motivated to work harder when their Boss shows appreciation to their work.

6. Stop being selfish.

Being selfish few birds may try to fly solo but they soon realize the advantage of being with in the formation with other birds and join them.

All beings are born selfish. We think of ourselves first before anyone else. Selfishness makes success harder to achieve and our growth gets limited. Overcoming selfishness is hard in the beginning but will be easier over time.  For the team to grow it is really important that the team members stay selfless and act for benefit of everyone.

7. Help each other in challenging situations

If a bird in the flock is injured or shot then two other birds stay with it and protect it until it’s cured or is dead. They later rejoin their group.

A person’s true identity is revealed during tough times. A team needs to stand by each other in difficult times.  

It’s amazing how these birds have the natural ability to collaborate, co-ordinate .Birds have realized the benefits of working together as a team, Have you?
 ‘I’ CANNOT DO IT. ’ V’ CAN DO IT.

Popular Posts

Total Pageviews