For the past six months, I’ve been leading a new initiative at Instrument: creating an internal Quality Engineering discipline within our larger Technology discipline.

Instrument has historically partnered with external agencies for quality assurance, using a model that brings in a separate QA team during the closing sprints of a project. Instrument has launched beautiful and innovative products under this model - but now I want to take it even further, and set a foundation that will allow us to solve bigger and better challenges together.

Quality Vision

My vision for Instrument is that we’re known and celebrated for the quality that goes into our work as much as the design and development of it.

In my first week as Instrument’s first-ever QA Manager, I sent out a survey asking people to tell me what quality meant to them, and I got back some really thoughtful answers:

• Accessible & inclusive
• Functional, reliable, & beautiful
• Performant
• Easy to use and maintain
• Trustworthy
• Work that we can be proud of

I absolutely love these perspectives! Quality isn’t always objective - it’s how we feel when we build it, how clients feel when they see the product we’ve created, and how users feel when they experience it. Owning quality internally allows us to create a consistent level of rigor and a higher level of confidence that we’re delivering exceptional work - our best work - to our clients and their users.

There are two categories of services that we offer within the Quality Engineering discipline - quality engineering (QE) and quality assurance (QA). While these two terms are often used interchangeably, they do mean different things.

Quality Engineering

Quality engineering refers to the overarching discipline group and the broad range of expertise that we bring around a variety of testing and quality topics. Our team acts as subject matter experts for our clients and internal teams. We offer ad-hoc advice when designers or developers have questions on topics like accessibility or performance, and we research testing tools so we can make recommendations to meet project needs.

We create vetted resources for our internal teams. If you look up any quality or testing topic, you’ll find multiple articles with varying levels of accuracy and usefulness. By proactively compiling resources around common themes, we can save everyone else time by making it easier to find the information they need.

Our QE team also works on projects! We review the technical requirements of a project before it kicks off, and partner with project leadership on creating a Technical Design Doc. Our team asks questions to clarify scope and deliverables, gives feedback on complexity for testing, and makes recommendations for types of testing and with what tools. By engaging with our internal teams while the project scope is being formed, we help guide the project to its highest level of quality from the very beginning.

Quality Assurance

So what happens when a project’s development phase kicks off?

I’m so glad you asked! Quality assurance (QA) is the service that we offer for projects once they’ve started. For many people, this is the heart of our discipline because it’s where they see the greatest impact.

QA can take many forms. For each project, QA typically involves a combination of exploratory testing, acceptance testing, regression testing, performance testing, user experience, and accessibility testing. Let’s dig into those to see the impact they have for our clients and the work we do!

Exploratory testing is a form of testing where you don’t have predefined test cases - it’s an approach that allows you to test, explore, and continuously learn about the product. An exploratory testing approach can be used during the various types of testing that we perform. It helps QA engineers maintain their understanding of the product being built, which means they can maintain the integrity of testing and quality considerations throughout the project.

Acceptance testing is validating that the work being done matches the acceptance criteria or stated business requirements. Are we solving the right problem? Are we building the right thing the right way? This is where our client advocacy comes in - making sure that we stay aligned with our clients’ vision and expectations.

Regression testing is part of a model of continuous quality, where QA engineers are embedded and contributing throughout a project’s lifecycle. This type of testing makes sure that new feature work doesn’t affect the existing work’s functionality, design, or user experience.

Performance testing measures a product’s speed, scalability, and stability. How does the app perform under high user loads or increased data processing? What happens as those elevated conditions continue? How long are users waiting for content to load on the page? If there are delays, what’s causing them and how can they be optimized?

QA engineers also act as advocates for user experience - whether it’s the admin experience for a CMS, a design library for a company’s rebrand, or the physical experience of an interactive installation. How intuitive is the UX? Could we simplify something to make it more understandable or approachable? What’s the experience like under heavy load or low cell service? Our goal here is to reduce complexity and lower the barrier of use for as wide an audience as possible.

And that brings us handily to accessibility! Accessibility testing is verifying that our work conforms to the current WCAG standards, from color contrast to screen readers to keyboard navigation. Accessibility should be the default, not an afterthought, and QA engineers help keep those requirements at the forefront during development and planning.

Throughout all of these testing methods, QA engineers consistently support and strengthen the efforts of everyone on our internal team.

A New Model

You might be wondering when all of this QA happens. If the former model at Instrument placed QA at the end of a project, what are we doing differently now?

This is where an embedded model comes into play, and it’s one of the main tenets of internal QA. The goal with an embedded model is to increase productivity, remove silos, and create a shared understanding around the work we’re doing.

When we’re embedded with an internal team, we can keep up with changes in real time. It means we stay current on design, client priority, and development solutions.

And it’s not just the QA reviews! QA engineers are in daily standups, strategy presentations, sprint planning meetings, client demos - these discussions give us insight into the different perspectives of producers, developers, designers, strategists, and client stakeholders. Understanding those perspectives, and the ability to hold them in mind during testing, leads toward building the best product we can.

With a consistent presence, we are more productive and effective in upholding the quality of the project. Being embedded also enables us to build collaborative, trusting relationships with the people we’re working with everyday - and those relationships are also vital to launching work that our company and our clients can be proud of.

The Impact of Internal QE

On paper, it might not seem like much has changed - QA used to happen at the end, and now it happens throughout the project’s lifecycle. Why is that important? What’s the impact of creating an internal Quality Engineering discipline?

An internal QE discipline means we get to truly own the vision and progress of quality at Instrument. Just as having distinct Strategy, Design, Technology, and Production disciplines allows people to focus on the continued improvement, innovation, and well-being of those fields, a Quality Engineering discipline allows us to bring in the same focus for quality and testing.

Our QE team supports other disciplines by acting as subject matter experts - creating solid sources of information for teams, making time to educate and coach people. This allows everyone to stay current on quality topics and bring that knowledge to the products they’re building.

Our embedded model also increases the productivity and efficiency of the teams. In the past six months, Instrument has launched successful projects using the embedded QA model with more efficiency than projects in the past. Embedded QA engineers streamline communication and ensure a quick turnaround to get features shipped. And since the QA reviews are happening on a rolling basis throughout the project, developers are able to minimize context-switching by addressing feedback sooner rather than later, which also reduces the competing priorities that make it harder to focus.

Over the past six months, the feedback we’ve received from internal teams has been unequivocally positive. We’re changing the way people think about quality and QA. We’re making it easier for everyone to do great work, and have confidence that it’s their best work.

We’re making concrete, visible improvements in the way we build and launch projects at Instrument, setting a foundation of quality that will empower us to solve bigger and better challenges together.