RSS

Tag Archives: Devops

Steps to Introduce Test Automation

Today, software testing is an integral part of the development process. To reduce the time spent on testing, many companies opt for test automation. However, automation capabilities move beyond speed increase and include test coverage enhancement and overall QA costs optimization in the long run.

Assume, You’ve just started a brand new role at a company and the very first task you’ve got in front of you, is to implement test automation.

Now, this company has never done any kind of test automation and right now is relying on a hodgepodge of methods to test and measure quality.

Where do you start and how do you begin such a momentous task? I’m glad you asked.

We’re going to be talking about the general, fundamental concepts surrounding test automation – not specific tools or frameworks you need to use because those will vary based on what your organizations unique needs.

shutterstock_727680178

Analyzing applications to determine which can be automated

If your organization is working on 3 applications, it is not necessary that each should be automated. We have to see the various factors while selecting any application to automate.

The application which should be automated must have these factors:

The application should not be in the early stages of its development. (The application should have all or some modules which are stable and tested by manual testers)
The UI of the application must be stable. (The UI must not change frequently)
The manual test cases of this application should be in written form.
Basic application flow should have been already tested and working

The main goal of automation is to make sure that if the application is bug-free in one build, it should remain bug-free in the next build. The manual tester should not waste their time in finding regression issues, these issues should be identified in automation.

Educate on myths & misconceptions

Educating management, and other groups, about these particular myths and misconceptions surrounding test automation, gives you a shot at success. It provides the understanding and opens lines of communication that will benefit you and your work.

The myths & misconceptions include things such as:

You can automate 100% of your tests.
Automation allows Management to reduce test staff.
One tool is all you need.
Automation will speed up the testing process.
Anyone (untrained/unskilled) can do automation.
Open-source tools are “free.”
“Codeless” and “Scriptless” tools can do all the work.
Automation can be created quickly.
Only hire a Software Design/Development Engineer in Test (SDET).
Automation provides an immediate return on investment (ROI).

Determine the scope

Any process starts with the definition. Therefore, before implementing test automation, you should determine the automation scope. When starting tests development, a QA engineer should first define the order according to the tests’ priority rate.

Prepare to automate

Having quite profound experience in test automation, I can say the following: automated tests should cover the most stable part of the functionality and the one that is tested for about 3-4 times per week.

As a rule, smoke tests (or other regression tests) are chosen for that very purpose.

Select tools for automation

As soon as the scope is defined, a QA engineer should select test automation tools. The tested interfaces define the package of applied tools. Different types of interfaces presuppose different tools’ range.

There is no any one-size-fits-all solution. The choice of a test automation tool will depend on the technology the software is built on. For example, QTP does not support Informatica. That means the tool cannot be used for the software. We prefer the most reliable and proven solutions: Selenium WebDriver, Coded UI, Ranorex, TestStack.White, Appium, Xamarin, and many more.

Having decided upon the tools, testers get to framework implementation.

Training the Team

After tool selection and resource hiring, the next step is logically the training of the resources. If manual testers are converted into automation engineers, they have to be trained on automation terminologies and concepts. If automation architect is hired from outside, he must get knowledge about the product to test, the manual testing process and what management is expecting.

Give resources some time to try different things until they finally come up with a winning automation strategy. Train them on the tools which organization is already using bug tracking software and requirements management software.

Good training and strong communication between manual testers, developers and automation team is really necessary.

Develop the framework

The framework is the basis for further automated tests’ development. It provides an opportunity to optimize test development efforts by re-using the code. You can utilize any of the ready-to-use frameworks presented on the market, like the Robot framework for Selenium.

The usefulness of frameworks is hard to underestimate. Thanks to frameworks, it’s possible to maintain consistency of testing and improve re-usability. You can also count on minimizing code usage and improving the structure of tests, which is a perfect scenario for long-term projects.

Configure the environment

All the tests run in the environment, which is to be well-configured. Upon this step, you should create and support the environment to successfully run automated tests and store the results.

Test automation will require test data, which means you are to prepare the set of files and test accounts beforehand. Otherwise, you tackle risks that may damage the process and provide you with irrelevant test results.

Test automation process

Developing an Execution Plan

The execution plan includes selecting which environments the scripts will be executed. The environment includes OS, Browser and different hardware configurations.

For example, if the test case demands that it should check the website in 3 browsers, namely, Chrome, Firefox and IE, then the automation team will write the script in such a manner that it will be able to execute in each browser.

Start to automate

Eventually, when all the preliminary preparations are done, testers can get down to automated test development. A regular process of providing new automated tests includes the following points:

Selection of the manual test case according to the stated priorities
Code writing for the automated test
Adding the automated test to the debug test execution
Adding the automated test to the test execution for newly created builds.

Writing Scripts

When the framework is designed, the execution plan is known and resources are trained on the new tool, now it’s the right time to start writing scripts.

Scripts should be written in an organized manner with proper naming convention. The source code should be maintained in a source control to avoid code loss. Version control and history should be maintained. Test automation is just like software development. All best programming practices should be taken care while writing the scripts.

Reporting

The reporting feature is usually provided by the tool. But we can create custom reporting mechanisms like auto-emailing the results to management.

We can create reports at the end of each execution in the form of charts and tables if management needs it. The management should always be informed about the test case coverage, that means which manual test cases are covered in automation and which of them are remaining.

Maintenance of Scripts

If best programming practices are followed and framework is good, then maintenance will not be a problem.

Maintenance usually occurs when there is a change request an application. The scripts should immediately be updated to cope with that change to ensure flawless execution.

Monitoring

When the tests are launched, they should be monitored. You cannot let them go along without tracking the process. While monitoring the automated test, remember to take into consideration the following aspects:

Automated test coverage, cost per test
Useful vs. irrelevant results after test execution
Cost per test
The scope of support in comparison with the number of executed tests
Economic effect (ROI, return on investment).

Tracking the process, keep in mind, that test automation is much more than computers launching test programs. It is also delivering information about quality.

Ultimately, mastering automation can significantly increase your business value. Automation includes many factors that need to be understood and addressed before the start of the process. Follow the main points to avoid the risks and get your benefits.

The Final Word:

Following the steps discussed above will help you in successfully implementing automation testing seamlessly in your organization. Once the process is in place, you need to follow the best practices. Whenever there is a change/update in the application, the scripts should be checked and updated as required. Have you tried introducing automation to your testing process? Share your experience in the comments below.

What is DevOps? Role of QA in DevOps

21 Nov

Each minute of our life is a lesson but most of us fail to read it. I thought I would just add my daily lessons & the lessons that I learned by seeing the people around here. So it may be useful for you and as memories for me.

Article by V Rana & Vaibhav Choksi..

In most software development organizations, QA is considered a function separate from development units. Indeed, testers and developers have their own roles and responsibilities to take care of. However, QA and development teams should sync very closely to achieve common objectives of delivering a single, robust application that functions and behaves as per agreed standards.

When one examines the present drift toward DevOps, it’s quite evident that more and more organizations are implementing this software release model with the intention of enabling automation in software deployment. DevOps affords the framework that allows teams to move forward software on a more frequent basis. The conventional idea of a “software release” dissolves into a uninterrupted cycle of service improvement over time.

What is DevOps?

DevOps-Tools-DevOps-Tutorial-Edureka-1

DevOps is projected as the next higher level of agile software development practice that enables coding and testing to take place simultaneously with timely collaboration and communication amongst all concerned functions/ departments. It provides a framework that enables teams to speed-up software releases and gain faster time-to-market, even while ensuring quality at the same time. Traditional role of QA is to detect bugs, but in DevOps, the key responsibility of QA is to avoid the bugs from occurring in the first place

DevOps encourages development and operations teams to function as one single cohesive unit, focused on delivering business values across the IT value chain. It also helps in release management, delivery and deployment with continuous automation. Here we will discuss how DevOps can be helpful to deliver continuous integration & continuous delivery in QA..

Test automation tools are capable of performing tests, reporting outcomes and comparing results with earlier test runs. Tests carried out with these tools can be run repetitively, at any time of day. Test automation framework is a set of guidelines, coding standards, concepts, processes, practices and more, which provides an execution environment for automation test scripts. It allows the user to develop, execute and report on the automation test scripts in an effective and efficient manner.

Build Verification Test (BVT) is a set of tests run on every new build to verify that build is testable, before it is released to a test team for further testing. These test cases are fundamental functionality test cases that ensure application is stable and can be tested systematically. BVT saves time, cost, resources and efforts for test teams to setup and test a build. Test environment virtualization enables optimization of various simultaneously started processes. It can make testing much easier and efficient by eliminating certain dependencies and bring down the cost of setting one or multiple test environments. It considerably increases IT infrastructure flexibility and the efficiency of hardware usage.

How implementing DevOps can be useful in QA?

The role of QA in DevOps becomes more of an enabler or gatekeeper that links developers and the operations teams. DevOps is an approach to highlight swift, small, iterative development and deployment of applications. DevOps flawlessly integrates all the three functions – Dev, Ops and QA as a single IT entity that deliver value.

14Tools2 Take an example of an ecommerce portal. In its release environment, hourly releases are very common. In such cases, load, stress, and security testing become important. With DevOps approach, faster timely releases can be delivered by eliminating the opportunity for people to make errors. The continuous performance evaluation of the portal helps in improving customer experience.

DevOps helps in creating rapid, reliable and repeatable processes. It also benefits in achieving process improvements throughout the software development life cycle by identifying issues and recommending necessary improvements. With the help of DevOps, testers can leverage best practices in agile testing, continuous integration and test-driven development to accelerate QA processes and reduce cycle time.

What is the role of QA in DevOps?

QA has always been seen as a “gatekeeper” for software deliveries. However, in a DevOps environment, QA is part of the process itself since there are no boundaries between development and operations leading to Agile results unlike a traditional environment where development and operations are separated functions and operations team fixed issues raised by development teams.
In DevOps organizations, QA pushes out new development code to a production environment and DevOps teams provide infrastructure and tools to make the deployment process easier while simultaneously verifying that the changes are functioning as per the requirement.

In a traditional QA focused organizations, quality assurance teams focus on finding bugs while in DevOps organizations, the responsibility of quality assurance is not limited to that but to prevent them from arising in the first place. This approach helps in an environment where there is a need for newer releases every few minutes or hours. Here, manual testing is simply ruled out as an option and test automation has to be deployed.

Hourly releases are very common if we consider the release environment of an eCommerce portal. In this case, load, security and stress testing become critical and DevOps practice can help manage better quality releases since quality is an integral part of DevOps. This helps to identify and fix issues before new code goes live on the portal while checking for performance when it’s live.

DevOps helps achieve continuous improvement throughout the development cycle of a software product as QA teams can simultaneously identify issues in processes and recommend improvements wherever necessary.

Please feel free to share your story and any lessons you learned, you experienced, you came across in your life in the comments below.

Lessons from Devops Experience – 1

25 Oct

WHAT’S DEVOPS?

Wikipedia defines DevOps as:

DevOps (a clipped compound of “development” and “operations”) is a software development method that stresses communication, collaboration, integration, automation, and measurement of cooperation between software developers and other information-technology (IT) professionals.

I like to simplify this definition by saying that DevOps is when you’re not only responsible for developing the application but you’re also responsible for running and supporting the application in your testing and production environments. As opposed to the traditional way of developing where you have the luxury of developing the application then throw it over the wall to the Ops team.

DevOps came as a response to shortcomings of the traditional way of separating development and operations activities. If we look at Damon Edwards’ article What is DevOps?, we will find a description of the traditional way, the way it worked, and what shortcomings it has. Also Mike Loukides, in one of his articles, does a great job in describing the history of the relationship between development and operations, and why we need DevOps in its modern form. I strongly recommend you read both of these articles.

devops-hero-1-87966cfbc9c5713ae047551c7b22985c

KEY COMPONENTS OF DEVOPS

From my experience, there are several key components or principles when implemented then we are set for a successful DevOps experience. Those components and principles are:

Automated Delivery Pipeline
Configuration Management
Regular Integration
Automated Monitoring & Health Checks
Firefighter Role
Infrastructure as Code

When you first look at this list you may say: “We’re already following those principles without doing DevOps”. That might be true but DevOps adds emphasis to the fact that the development team needs to be responsible for defining, understanding and implementing these components. In other words, DevOps encourages developers to dive into aspects of software delivery that Ops would traditionally own. It’s worth mentioning that from a project management point of view, DevOps tasks and stories should be treated the same way we treat those that solve direct business problems like generating a report, or providing a log-in page. That is, DevOps work needs to be well-defined, prioritized, planned, executed and tested just like other user stories or features because the same team of developers will be working on them. Unfortunately, I won’t be able to go into details about this aspect of DevOps management in this post. In the following sections, I will explain a bit what each component is all about.

1. AUTOMATED DELIVERY PIPELINE

First, let’s talk about the “pipeline” part of this terminology. We want to create a process that defines what needs to happen in what order from the moment new code is published to source control to the last step of making that code available to customers in production. That is, the pipeline is an ordered list of well-defined and stable steps that start with pushing new code and ends with that code being in production. The second aspect of an Automated Delivery Pipeline is automation. Once our steps or recipe for delivery is defined, we need to script and automate it.

2. CONFIGURATION MANAGEMENT

When we deploy the same artifact like a web application WAR file to different environments, how do we ensure that the application in Development environment is only using the Development database and that in Production it’s only using the Production database? Or more generally, how do we ensure that the application is only talking to resources specific to the environment in which it’s running? Configuration management is the answer to this problem. There are two approaches to determining what resources an application should use; we can either determine those resources at run time or at deploy time. Both approaches rely on externalizing those concerns from source code into a configuration file. Hence, it’s called configuration management.

3. REGULAR INTEGRATION

Integration in this context is simply to deploy our application to an environment where it will interact with other applications and components in the ecosystem. Agile software development puts strong emphasis on having a feedback cycle in order to validate the solution we’re implementing is indeed solving customer problems and to also detect issues early on. Having a regular schedule of integrating our application is essential in creating a feedback cycle. When our integration cycle is tighter, so would our feedback cycle.

4. AUTOMATED MONITORING & HEALTH CHECKS

Since DevOps involves operational duties, we want to know about problems before they are reported or noticed by users so we can solve them before they impact our users. We need to develop the ability to periodically and automatically check the health of our application and react appropriately when it’s in a bad state in a timely manner.

5. THE FIREFIGHTER ROLE

When the development team takes on the “ops” duties as part of implementing DevOps, it means that the team will have to deal with issues not only related to the business functionality they implemented but also with infrastructure issues, monitoring alerts going off, impaired servers and services, etc. These issues are interruptions to the team’s development activities which can result in reduced velocity and loss of concentration. An approach that I like to employ is to define a Firefighter role and rotate it among the team. All issues that crop up are directed to the Firefighter and she is tasked with resolving the issues.

Dealing with issues that crop up are opportunities to get first-hand experience at how the application actually works in real life and those insights will certainly help the application to evolve and mature.

6. INFRASTRUCTURE AS CODE

This could be the most recognizable part of DevOps but I intentionally list it at the end to emphasize that it’s not the only aspect. Over the years, many good practices came out of developing software for business problems that are widely adopted but many of those practices are not employed to infrastructure as commonly. Treating infrastructure as code does not only mean to write code for infrastructure but it also mean to apply the aforementioned best practices to infrastructure code.

CONCLUSION

All of the previous DevOps components discussed in this post are not to be treated in a “all or nothing” manner. Instead, it’s better to take them one by one. We can pick one to understand, design and implement. Once we are satisfied with its benefits, we can move on to the next one.

This list of components is not intended to be a definitive list but they are what I personally experienced to be most influential. If you have other components or principles you believe should be included, please let me know in the comments section. I’m very interested in hearing from fellow “Dev Operators” about their experience. As a follow-up to this post, I will be publishing a second blog post that will shed more light into the aforementioned components.

By Akrem Saed

Thoughts To Inspire

Tag Archives: Devops

Steps to Introduce Test Automation