Using non-functional testing to guide user interface, backend services, voice interface, and media development

How do you prevent your app from getting slower with every new feature? Use non-functional testing to build a data-driven safety net against performance regressions.

#1about 3 minutes

Understanding non-functional vs functional testing

Non-functional testing measures performance, quality, and usability against baselines, moving beyond simple pass/fail criteria to create a five-star user experience.

#2about 7 minutes

Building a data-driven regression safety net

Establish a framework to catch non-functional regressions by collecting data, identifying hotspots, measuring against baselines, and fixing performance drops.

#3about 7 minutes

Methods for collecting raw performance data

Collect raw data for analysis by either instrumenting the application with code bookends or by instrumenting the test environment to capture video, network, and system stats.

#4about 9 minutes

Analyzing raw data to create useful metrics

Transform raw data from instrumented environments into actionable metrics like page load time by defining a region of interest and applying visual analytics.

#5about 3 minutes

Structuring a cross-functional testing team

Integrate non-functional testing into project teams by combining the skills of product engineering, data analysis, and testing to drive improvements.

#6about 4 minutes

Applying non-functional testing to gaming performance

Use non-functional testing in game development to set and track baselines for CPU usage and frames per second (FPS) across different devices.

#7about 3 minutes

Testing the responsiveness of voice interfaces

Measure and improve the time-to-respond for voice inputs by setting a baseline, such as 500 milliseconds, and testing across various locales and devices.

#8about 4 minutes

Ensuring quality in streaming media applications

Monitor streaming media performance by tracking loading animations, buffering events, resolution switches, and perceptual quality factors like blockiness and blurriness.

#9about 9 minutes

Q&A on tools, frameworks, and implementation

Discussion covers recommended tools like Appium, integrating tests into CI/CD pipelines, and the benefits of instrumenting the test environment over the application.