Quality assurance with Blueriq [Editor]

You can also view the webinar on my.blueriq.com: Webinar Blueriq Quality Assurance

What is quality?

“Quality is not an act. It is a habit.” ~Aristotle

Quality is about making organisations perform for their stakeholders – from improving products, services, systems and processes, to making sure that the whole organisation is fit and effective. Managing quality means constantly pursuing excellence: making sure that what your organisation does is fit for purpose, and not only stays that way, but keeps improving. There's a lot more to quality than just manufacturing widgets without any defects or getting trains to run on time – although those things are certainly part of the picture. What quality means for your organisation is ultimately a question for your stakeholders. And by stakeholders we mean anyone who has an interest in the success of what your organisation does.

Customers will be the most important group of stakeholders for the majority of businesses, but investors, employees, suppliers and members of our wider society are stakeholders too. Delivering quality in your organisation means knowing who your stakeholders are, understanding what their needs are and meeting those needs (or even better, exceeding expectations), both now and in the future.

Testing Pyramid

“Quality is more important than quantity. One home run is much better than two doubles.” – Steve Jobs

The testing pyramid is often used in agile enviroments, for more information see this video: http://www.agilenutshell.com/episodes/41-testing-pyramid

The video and the use of the pyramid gives an indication about what to test on which level. This to benifit the most of automated test on the correct level.

The testing pyramid and blueriq

In most organization the traditional testing pyramid is used, but it is used inverted (See picture below). This happens because most projects create a lot of tests on their user interface by using selenium for example (or any other automated framework). This should be changed so that the most tests are created as low as possible in the pyramid so that the feedback loop is as short as possible. This results in less scope change, less related code, less people involved etc. And this causes that bugs which are being found are cheaper to solve than when you will find them in the end of the lifecycle.  

This is described in the graph of Boehm, where the cost to fix will be greatly increased from testing and further in the proces. This supports the integration of early testing in the agile sprint teams and getting the most tests within development.

If we map the test pyramid on a standard blueriq setup,it would give this picture:

    In this model most of the test are on the lowest level, the benifits which this will give are:

• Cost to develop and maintain automated test is up-to-par
• Execution times on automated tests  will be optimal
• Possibillity of false negatives is decreased
• Increase in coverage closest to the team
  
  

Agile test quadrants

“The definition of insanity is doing the same thing over and over again and expecting different outcomes.” – Einstein

For customers our advice is to use the agile testing quadrant to track which test you want to execute. In this paragraph we give an example about how this can be filled in from a customer point of view for Blueriq. Keep in mind that every enviroment on a customer side has specific challenges. 

Worked out the agile testing quadrants:

Performance

“Just as athletes can’t win without a sophisticated mixture of strategy, form, attitude, tactics, and speed, performance engineering requires a good collection of metrics and tools to deliver the desired business results.”— Todd DeCapua

With the introduction of Blueriq 9.9 performance test are introduced with Blueriq, for more information on this topic, see: Performance reports 

Blueriq's flexibillity gives us the option to run the performance test on your enviroment, to use as an insight of your own enviroment setup. By using the complete test project that Blueriq supplies, it is possible to test the landscape that your application will run in. If there are significant differences in the results from Blueriq and yor online installation, then there is a big chance there are some hardware issues involved (server / network / client etc.). After doing this initial check it is always wise to create a performance test yourself with, for example, jmeter. This way you will know that the performance of your application is not decreasing with the introduction of new modelling.

Please be advised that we also have a page about modelling and performance, see also:Performance Tuning for more information about modelling in combination with performance.

For more information about these services please contact our support desk.

Security

“Testing is an infinite process of comparing the invisible to the ambiguous in order to avoid the unthinkable happening to the anonymous.”— James Bach

Developing secure software is critical to a company’s reputation and bottom line. Blueriq customers feel the legal and financial pressure to assure the security of their (mission-critical) applications and have strong regulatory and privacy requirements to protect private data. Therefore security at Everest is treated at highest priority in the development of Blueriq. For are customers we have the following advice to introduce in your agile development proces. 

Keep in mind that it's advisable that pentration test should always be executed before going live on your own enviroment.

OWASP Dependency Checker

Dependency-Check is a utility that identifies project dependencies and checks if there are any known, publicly disclosed, vulnerabilities in 3rd party components. This dependency checker is integrated in our Jenkins CI server. Currently Java and .NET are supported. The OWASP Dependency-check is used to scan our dependent libraries to identify any known vulnerable components. The core engine contains a series of analysers that inspect the dependencies, collect pieces of information about the dependencies (referred to as evidence within the tool). The evidence is then used to identify the Common Platform Enumeration (CPE) for the given dependency. If a CPE is identified, a listing of associated Common Vulnerability and Exposure (CVE) entries are listed in a report.

Dependency-check updates using the NVD Data Feeds. For more information about the dependency checker see: OWASP dependency plugin (Jenkins). The standard libraries with Blueriq are checked this way. If you use extra libraries or replace our libraries, it would be advisable to run this on your enviroment.

OWASP ZAP

The OWASP Zed Attack Proxy (ZAP) is one of the world’s most popular free security tools and is actively maintained by hundreds of international volunteers. It is used as a proxy to run selenium tests through and then ZAP can spider further throughout the complete application. ZAP will attack the application with the most popular (OWASP top10) attacks like injections, clickjacking, xss, csrf etc. ZAP should also run every night so you are sure that no important vulnerabilities are being introduced by new code. For more information

Why did we choose READY API as our test automation tool ?

• bla bal

Ready API exampels

Under construction....