Page History
What is quality?
“Quality is not act. It is a habit.” ~Aristotle
Quality is about making organisations perform for their stakeholders – from improving products, services, systems and processes, to making sure that the whole organisation is fit and effective. Managing quality means constantly pursuing excellence: making sure that what your organisation does is fit for purpose, and not only stays that way, but keeps improving. There's a lot more to quality than just manufacturing widgets without any defects or getting trains to run on time – although those things are certainly part of the picture. What quality means for your organisation is ultimately a question for your stakeholders. And by stakeholders we mean anyone who has an interest in the success of what your organisation does.
Customers will be the most important group of stakeholders for the majority of businesses, but investors, employees, suppliers and members of our wider society are stakeholders too. Delivering quality in your organisation means knowing who your stakeholders are, understanding what their needs are and meeting those needs (or even better, exceeding expectations), both now and in the future.
Testing Pyramid
“Quality is more important than quantity. One home run is much better than two doubles.” – Steve Jobs
The testing pyramid is often used in agile enviroments, for more information see this video: http://www.agilenutshell.com/episodes/41-testing-pyramid
The testing pyramid and blueriq
What we still see lately is that the traditional testing pyramid is being used, but it is used inverted. So what we see is that a lot of projects create a lot of tests on their user interface by using selenium (or any other automated framework). This should be changed so that the most tests are created as low as possible in the pyramid so that the feedback loop is as short as possible. This results in less scope change, less related code, less people involved etc. And this causes that bugs which are being found are cheaper to solve than when you will find them in the end of the lifecycle. This can be seen in the pictures below.
i.
Agile test quadrants
“The definition of insanity is doing the same thing over and over again and expecting different outcomes.” – Einstein
Testing type | Possible Assignees | Targeting | Reasoning | Tool which can be used |
|
Model testing (Q1) | Business engineers and testers | All The new models | Ensuring that the Model is correctly developed. According to standards | •Blueriq Model Insight •Blueriq Model Analyzer •Studio Unit tests | |
API/ Logic/ Page models (Q2) | Business engineers and testers | Functionalities implemented in new stories/ past issues or bugs with high recurrence chances. Testing on the page modelling and exposed services | Checking to see if the Runtime is working correctly on the developed models | •SoapUI NG •Cucumber •Jmeter •Telerik •Selenium •Ranorex •Etc | |
GUI testing (Q2) | Testers | The graphic interface and it’s logic, For example the view controller | Making sure no GUI related bugs are introduced when committing new code | •Selenium web driver •Testcafé •Nightwatch •Capure and playback tool | |
Performance (Q4) | Development team/ External expertise (Testters) | All the Blueriq components (Studio, Runtime, Publisher) | Verifying how Blueriq behaves when it comes to processing time and reliability | •Jmeter •Load UI | |
Security (Q4) | Development team/ External expertise (Testers) | The Runtime and its relation to other third parties’ components. | Keeping and improving security standards for our application | •OWASP ZAP •Burp suite |
- How?
- Unit tests
- BMA (/ BMI)
- ReadyAPI
- Front-End full stack (?)
- Performance
- Link to performance test page
i. https://my.blueriq.com/display/DOC/Performance+Reports
- Explanation that it is possible to test the landscape with our test project.
i. By using the complete test project that Blueriq supplies, it is possible to test the landscape that your application will run in. If there are significant differences in the results from Blueriq and on location, then there is a big chance there are some hardware issues involved (server / network / client etc.). After doing this initial check it is always wise to create a performance test yourself with, for example, jmeter. This way you will know that the performance of your application is not decreasing a lot by publishing the new code.
- Security
- OWASP Dependency Checker
i. The dependency checker by OWASP is a utility that identifies project dependencies and checks if there are any known, publicly disclosed, vulnerabilities. The dependency checker should run every night to be sure that no libraries with known vulnerabilities are being used by your (web-)application.
- OWASP ZAP
i. The OWASP Zed Attack Proxy (ZAP) is one of the world’s most popular free security tools and is actively maintained by hundreds of international volunteers. It is used as a proxy to run selenium tests through and then ZAP can spider further throughout the complete application. ZAP will attack the application with the most popular (OWASP top10) attacks like injections, clickjacking, xss, csrf etc. ZAP should also run every night so you are sure that no important vulnerabilities are being introduced by new code.
- Integration with Jenkins
- Automated Regression
- Automated ZAP runs
- Automated Dependency Checker
- Examples
- R9 ReadyAPI example
- R10 ReadyAPI example
- Unit test example project?
- (Link to) Performance project + jmeter file
Overview
Content Tools