Test Automation is hard to do well! There are many different things to consider. Not only do you have to think about how to write good code, but you also need to understand how to create good tests. And of all the types of test automation that you might want to write, UI test automation is the hardest kind. It can be tempting to simply write UI tests that mimic what you would do by hand, but any experienced test automation engineer will be able to tell you the pitfalls of doing that.
That might make you think that you shouldn’t write automated UI tests, but that’s not the case either. When you have a powerful tool, you don’t need to avoid it. Rather, you need to learn how to wield it well. TestProject has a powerful test recorder built right into it that makes it easy to create and record tests. This is great for certain kinds of testing, but what do you do when you want to wield this powerful tool with maximum effect? ⚡ In order to do that, you will need to interact with it in more direct ways than you can via the test recorder. This is where the TestProject Software Development Kits (SDKs) come into the forefront.
TestProject has Java, C# and Python open sourced SDKs (and JavaScript is coming right around the corner!). In this article, we will look at how you can get started using the TestProject Java OpenSDK. We will get you from zero to a fully functioning test in just a few steps 🚀
The configuration page should look something like this:
Now that you have a Maven project, you will need to add the TestProject SDK as a dependency in the pom.xml file for your project.
pom.xml file and you can double click to open it in for editing.
<dependencies>
<dependency>
<groupId>io.testproject</groupId>
<artifactId>java-sdk</artifactId>
<version>0.65.3-RELEASE</version>
</dependency>
</dependencies>
Make sure to set the <version> to the name of the latest TestProject SDK version. You can find the list of versions in the Maven repository here.
If you prefer to build with a Gradle project instead, you can do that as well. If you have already installed the SDK with Maven, skip ahead to the next section . As with using Maven, let’s look at how to set up a project in Eclipse. Use the following steps to setup a Grade project in Eclipse:
Once you’ve done this, find the build.gradle file in your project and add the following into the dependencies section.
implementation 'io.testproject:java-sdk:0.63.4-RELEASE'
❗ Note that the Gradle addon usually comes installed with Eclipse. If you are using a different IDE you might need to install Gradle which you can do from
here
. If Eclipse did not install it, you can go the Help>Ecliple Marketplace and search for gradle and install the Buildship Gradle Integration addon.
You can create a new test in Eclipse by going to the src/tests/java folder and then right clicking and selecting Package from the New menu as shown in the following image:
This creates a test that will go to the TestProject example page , login and check that the logout button is there, but you aren’t quite ready to run the test yet. You will need to first configure a development token before you can run this example 👇
In order to run tests, the TestProject Agent on your machine needs to be able to communicate with the TestProject platform. This communication channel needs to be authorized with a development token. You can get a dev token from the
sdk/integrations
page in TestProject. There are a couple of ways to specify the development token, but for the purposes of this tutorial, we will look at how to explicitly provide it to your test. You can do this
when calling the driver constructor. Simply pass in your token as an argument to the constructor. Instead of just setting the ChromeOptions() when creating a new driver, you can also specify the token as a string. If you wanted to take this approach, Line 10 in the example above would be replaced with something like this:
ChromeDriver driver = new ChromeDriver("your token", new ChromeOptions());
Once you have specified your development token in this way, you should be able to run the test! 🏃♂️
Test Passed printed in the console in Eclipse. Don’t worry if there are a few warnings in there. As long as you see Test Passed printed, everything should be ok. If the test does not run, check the Console tab in Eclipse. You may see an exception in there saying that it could not connect to the Agent. You will need to start up the Agent on your local machine before you can run the tests.
In order to run the tests in Eclipse, all you need to do is click the green run button on the toolbar. When you do that, a chrome browser will load and it will open the https://example.testproject.io/web page and do the actions indicated in the test. Once it is done you should see the wordsprinted in the console in Eclipse. Don’t worry if there are a few warnings in there. As long as you see Test Passed printed, everything should be ok. If the test does not run, check the Console tab in Eclipse. You may see an exception in there saying that it could not connect to the Agent. You will need to start up the Agent on your local machine before you can run the tests.
And there you have it! You have created and run a test with the TestProject Java OpenSDK. That was pretty easy, wasn’t it? 😎
If you login to the
TestProject App
and mouse over the Reports menu item, you should see the SDK run that you just did listed there.
By default, the TestProject OpenSDK automatically reports all driver commands and their results to the TestProject Hybrid Cloud. This allows you to see beautifully designed reports and statistics in the TestProject app dashboards.
🎁 Bonus Material: Free Test Plan Template
In 1962, NASA launched the Mariner 1 as their first attempt to send a spacecraft to Venus. However, shortly after it launched, the rocket veered off course and was forced to self destruct.
The cost? $135 million (in today’s dollars).
The issue? A missing hyphen in the code.
Based on our research on best test practices, we compiled a Test Plan Template for you to use for free.
Download Free TemplateYou might think software testing won’t result in as serious consequences for your company. However, a 2017 study found that software failures cost the U.S. economy $1.7 trillion in financial losses (and more than 268 years in downtime) that could’ve been avoided with proper testing.
Before any piece of software or new feature goes out to your users, you need to thoroughly put it through its paces. Test it. Try to break it. And make sure that whatever your users do, it responds as designed.
In short, you need a test plan.
Manage Projects like the Pros. Try Planio.A test plan is one of the most important parts of any software development process. It outlines how you’ll make sure your product or feature will do what it’s supposed to and not break when your users need it most.
But what should your test plan include? How deep do you really need to go to ensure your product holds up and your users get what they expect?
This guide will cover everything you need to know about defining and documenting your test plan and choosing the right test strategies that will ensure your users, development team, and stakeholders are all happy.
Ready to write your own test plan? Download our free test plan template and follow along with the post.
A test plan is a detailed document that outlines the test strategy, objectives, resources needed, schedule, and success criteria for testing a specific new feature or piece of software.
The main goal, of course, is to discover defects, errors, and any other gap that might cause the software to not act as intended or provide a bad experience for your users. More specifically, a test plan ensures your software:
While these sound like pretty straight-forward criteria, in practice, they rarely are. The problem is that a “test” means you’re naturally testing against something. In most cases, this will mean your specifications and success criteria you put down in your SOW or planning doc. (But could also include things like comparable products, past versions, user expectations, standards or laws).
Free your time from busy work and get organized.
Use PlanioIt’s nearly impossible to test every scenario, environment, or use case your software will encounter during its lifecycle. Instead, software errors, bugs, and defects could pop up from:
This makes writing a clear, yet comprehensive test plan a difficult balance. You want to include as much detail as possible to make sure you’re not missing any glaring errors. But you also don’t want to drown your team in testing tasks, delay your release, or add in new errors from your “fixes.”
So what should go in your test plan then? Well, that depends.
Each product and feature will have its own specific testing criteria, strategies, and needs. Also, the goal of your test will change how you approach it. For example, User Acceptance Testing (UAT) is completely different from stress and load testing and your plan will need to be tailored to your ultimate goal.
However, this doesn’t mean you want to start from scratch each time you’re testing a new piece of software. Creating different test plan templates for different products is a great way to quickly guide your approach to testing new product releases, updates, and features.
So what should (or could) you include in your test plan? Broadly speaking, there are a few main areas you’ll want to include in your test plan that will act as the foundation of your test plan document:
As we said before, creating a test plan is all about balance. You want to be comprehensive, but not overwhelming, which means getting specific about what will (and won’t) be included in the test plan.
Get Everyone on the Same Page. With Planio.After a brief introduction highlighting your test plan objectives, high-level scope, and schedule you need to define what you will or won’t be testing.
This is your test scope and it can quickly get out of hand if you don’t take time to be specific with it and answer both what you will test and why you’re going to test it.
Everyone needs to be on the same page with the test criteria and scope. As a best practice, make sure you use industry-standard or at least agreed-upon standards and terminology to describe your tests and why they were (or weren’t) completed. This way, there’s no grey area or confusion about what you tested.
Next, you need to clearly explain what your test strategy is. Go into as much detail as possible.
You also need to know when your test has been successful. In other words, what are the pass/fail criteria for each test?
This isn’t the only criteria you need to be aware of. There are a few other common situations you need to outline in your test plan, including:
It’s also a good idea at this point to list your assumptions and risks. In other words, what are you assuming is going to happen and what are some of the risks you’re going to face during the test?
Lastly, you need to outline your test project’s resource needs and schedule. Who is in charge of testing and what resources do they need (both technical and human)? When is the testing going to take place and for how long?
What are your required test deliverables? This means the data you want to collect, how you’re going to compile them in reports, and the issues and tasks that will be passed back to the development team.
Ditch the Spreadsheet and Get Real with Project Management.
Start using PlanioTo make sure nothing gets missed, each test deliverable should be assigned to a specific person on your team in a section on roles and responsibilities.
It’s important to remember that this is just a basic framework of what to include in a test plan. Over time, you’ll create your own library of test plan templates that will serve as guides for new product releases, updates, and features.
Download our free Test Plan Template to get started.
Now that we have a high-level idea of what to include in our test plan template, let’s dig into the specifics. To make sure your testing scope doesn’t get way out of proportion, it’s important to have a step-by-step process for creating your test plan and executing on it properly.
Here’s where you should start:
You need to have a deep understanding of the product or feature before you can start creating a test plan for it. For example, let’s say you’ve just gone through a website redesign and want to test it before launch. What information do you need?
This step is what gives you the context to write your test plan introduction and objectives and start to plan out the resources you’ll need to complete it.
Next, it’s time to decide the scope of your test plan. What’s included in the scope of your testing will depend on a number of factors beyond just the product or feature. You need to dig in and think about:
For our website redesign example, we might want to say that functionality, UX, and checkout flow are in scope. While stress, performance, and database testing are out of scope.
Finish Projects on Time and Budget. With Planio.You might also want to think of this in terms of commonly used testing approaches, such as:
Deciding what to test and documenting your test strategy are the most critical parts of your test plan. Don’t rush through it. Take the time to really understand your goals and needs and balance them against the resources you have for testing.
As you define each different test you’re going to run, you need to know when your test is “done.” This means defining the pass and fail criteria for each specific test, as well as some of the things we mentioned above, such as exit and suspension criteria.
To do this, you’ll want to identify individual system metrics that you’re checking and decide what success means for each one. For example, if you were doing a performance test you might look at metrics such as:
Stop the distractions, get your projects in line.
Sign up for PlanioRemember, you can continue testing and iterating forever. So you need to decide what’s “good enough” to get your software out and in the hands of users.
The results of your test plan will depend as much on the feature you’re testing as the environment you’re testing it in. As part of the scope, you need to determine what hardware, software, operating system, and device combinations you’re going to test.
This is a situation where it pays to be specific. For example, if you’re going to specify an operating system to be used during the test plan, mention the OS edition/version as well, not just the name.
Once your test plan is in place, there’s a specific process you need to follow. Think of this as the Software Testing Life Cycle (STLC). Similar to the Software Development Life Cycle, the STLC follows each phase of testing and usually looks something like this:
This is roughly the path that we’ve described so far. But what about actually executing on your test plan and tracking/reporting the results?
Using a tool like Planio, its easy to set up and track any number of testing scenarios.
Planio’s customizable trackers and workflows can be used to track and create issues or repeatable tasks related to each test. Each issue associated with a tracker has a fixed selection of statuses for the test phases to work through.
Never Miss Another Deadline. Try Planio.
Using a workflow like this, you can identify when a test has failed, decide what to do next and prevent the issue being closed. Even better, once you’ve created these workflows they can be repeated in Planio every time you need to test a new feature or piece of software.
For example, let’s say you’re going to test the checkout functionality of your new e-commerce software. Using Planio, you could:
Want to learn more about Planio’s powerful workflows? Check out our post on How to create a project workflow in Planio that will save your team hours a week.
If you’ve read our Guide on Software Development Processes, you know that when you test features, products, and specific code is dependent on what style you’re using. The biggest difference probably being between traditional (Waterfall) and Agile development.
In traditional software development (aka Waterfall), your test plan starts once you’ve completed the project. Whereas in Agile, requirements, development, and testing are all done simultaneously as you develop new pieces of usable software.
So which is right? And when should you start testing?
While there are debates on both sides, it’s probably safe to say that the earlier you start testing the better.
Project management software everyone on your team will love.
Try PlanioIn fact, IBM commissioned a report almost a decade ago that discovered that the further into the Software Development Life Cycle that a bug or issue is discovered, the more expensive it is to fix.
In other words, if you leave your testing until just before release, you’re adding a huge strain on your resources to deal with it. Even worse, the cost of a bug found after release is nearly 30 times more expensive than if it was found during the design phase.
Testing is an iterative process. When one bug is found and fixed, it can illuminate other, deeper bugs or even create new ones. The earlier you can start dealing with those, the less of an impact testing will have on your launch and go-to-market strategy.
Testing isn’t just another thing to check off your list. It’s an important and potentially project-altering phase that needs to be carefully thought through and planned.
Your test plan will guide you through that process from start to finish, helping you understand the objectives, define the scope of testing, create pass/fail criteria, document your process, and deliver the documents and artifacts you need to make your product or feature the best it can be.