The business analyst’s role in ensuring quality, and why it matters

Have you noticed that software apps seem to be getting worse? Every day, my iPhone wants to update apps where the developer is seemingly proud to tell us they’re fixing bugs. Yet how did those bugs get in there in the first place? And as business analysts, what can we do about it?

How should a business analyst define quality?

The management theorist Peter Drucker talked of “Doing the right thing, and doing the thing right”. He originally intended this to refer to the distinction between effectiveness and efficiency, but I think it sums up quality well.

As business analysts, we could define quality as “Fitness for purpose”. This includes:

Our ways of working as business analysts
The outputs we produce
The solutions that are built and put in place based on our outputs

I enjoy thinking about quality as being fitness for purpose, because it gets us away from perfectionism.

With an agile mindset, we have to do just enough to get to the level of quality we decide that we want—but in many cases, the current effort is not enough!

I want to make one thing clear in this article though: Quality is about more than just software testing!

The time has come for business analysts to formally consider quality, because in a lot of cases, nobody else is!

Outside of IT, a lot of organisations follow quality standards such as ISO:9001, but I’ve never really encountered this in the companies I’ve worked with, and it just doesn’t seem to be followed within IT. But we don’t need box-ticking anyway, we need action!

Business analysts must take a holistic view of quality across POPIT

Quality affects all five major aspects of a holistic solution that as business analysts we are trained to investigate: People, Organisation, Processes, Information, and Technology.

Each element will have its own QA approach and its own QC approach: how do we work in a way that ensures quality (QA), then how do we inspect for quality to ensure that we have achieved it (QC).

I don’t have space in this article to go into depth on the measures that would be used in each case, but a commitment to quality means considering QA and QC for all five areas.

What’s the difference between Quality Assurance and Quality Control?

A common way to break down quality is into Quality Assurance (QA) and Quality Control (QC).

Interestingly, I’ve often heard people say “We need to QA” the solution, when they mean testing it, whereas actually testing is a part of QC, not QA. This just shows that when we talk about quality, people literally don’t know what they’re talking about!

So what actually are the two main areas for quality?

Quality Assurance: A proactive focus on process-oriented measures to prevent defects. How are we going to work in a way that prevents defects and gives us high quality?

Quality Control: A reactive focus on product-oriented measures to detect defects. How are we going to find defects once we have an output that we can inspect?

As business analysts, we need to consider QA and QC equally. With QA, we need to consider our own working methods, and those of our teams, vendors, and other stakeholders. With QC, we need to get involved in the test planning and test execution, evaluate our requirements with testers, and not leave it all to them.

Why is quality so neglected by business analysts?

I am a big fan of the approach to business analysis set out by the British Computer Society (BCS). However, their book Business Analyst Techniques: 123 essential tools for success doesn’t even mention the word quality in its index!

The BCS defines a Business Analysis Service Framework (BASF), which again I am a big believer in. It lays out six services that a business analyst can provide:

Situation Investigation and Problem Analysis
Feasibility Assessment and Business Case Development
Business Process Improvement
Requirements Definition
Business Acceptance Testing
Business Change Deployment

Notice that providing a high quality outputs and a high quality solution are critical to the success of every single service that business analysts offer! The most obvious place where business analysts get involved in quality is the Business Acceptance Testing service, since this is explicitly looking at the quality of a solution which has been provided. Yet in every case there is a need to provide high quality in our own work as a business analyst, but also in the solution that gets implemented.

As an example, let’s think about the Requirements Definition service, based on the Requirements Engineering framework. We could produce the most wonderful requirements, doing a perfect job of Elicitation, Analysis, Validation, Documentation, and Management. But if we give these requirements to developers who do a poor job of developing and a poor job of testing their solution, delivering a product full of defects, then our perfect requirements will not lead to a successful outcome or a successful solution.

Even if we use agile techniques like a Three Amigos story review, working with a developer and a tester, is this alone enough? Sadly, the answer is no. Let’s look at why that’s the case.

Root causes preventing business analysts from getting fully involved in quality

I believe there are several root causes that are preventing business analysts from getting properly involved in quality:

Lack of formal quality training. Business analysts have a wealth of training, certification, techniques, and frameworks provided by the BCS, yet training in quality does not form a part of it.

Lack of holistic view of quality across POPIT. There is a dangerous tendency to think of quality only in terms of the software solution. However, as business analysts we take a holistic view of the entire project. When we consider the POPIT model — People, Organisation, Processes, Information, and Technology — each aspect is affected by quality, and the People, Organisation, and Process areas are often especially neglected because not even the software testers are looking at those!

Absence of quality in standard Business Analysis frameworks and tools. Not only do business analysts not receive training, but we have almost no standard techniques and tools that really bring quality to the fore.

Agile methodologies which do not prioritise quality. The most popular agile methodologies, scrum and kanban, do not have a formal focus on quality. There are some techniques in scrum which aim to meet the need for continuous improvement, such as retrospectives, but this is not the same as a quality review.

Lack of discipline and rigour. Agile methods too often lead to sloppiness, to a “Get it done and we’ll fix it later” attitude. Piecemeal attitudes and complacency are our enemies here. If quality gets mentioned, it’s often as an afterthought or an ad hoc comment, often in response to an issue that has already been identified, where it’s starting to become too late to fix things properly! Instead, quality should be formally considered, and quality checklists and quality techniques used.

Too much reliance on the project manager. Lesser-used methodologies such as AgilePM/DSDM and PRINCE2 Agile do put some focus on quality. AgilePM/DSDM even says “Never compromise quality”. But if these methodologies are used at all (and very often they are not), they are used primarily by the project manager, not by the business analyst. This leads to a tendency to see quality as a matter for the project manager and the testers, not for the business analysts.

Too much reliance on testers. There is a tendency to think that the testers look after the entire quality effort. However, in a lot of projects, especially those which are not involved in the actual coding of software and do not have software developers, there are no testers at all. For example, of the six BASF services, only the Requirements Definition service is likely to take place in the context where software testers are a part of the team. The other services, such as Business Acceptance Testing, where we are likely to be checking a SaaS solution, are likely to be delivered in a context where we have no qualified testers and nobody with a formal testing qualification. The International Software Testing Qualifications Board (ISTQB), which is allied to the BCS, has excellent books and certifications on software testing, but these certifications are rarely taken by business analysts. The concept of a T-shaped professional, with broad skills as well as deep skills, rarely extends to formal knowledge of testing. Personally, I think every business analyst should have at least the Foundation level in software testing, and preferably the Intermediate level too. Our work is too important not to!

Business analysts are involved too late. Something I see far too much is that business analysts are brought into projects at far too late a stage. We can attempt to influence quality as soon as we arrive, but sometimes this can be too late. Of course, we often cannot control when we are put onto a project, but where we can influence things to be involved as early as possible, we should. Business Analyst involvement should ideally be as early as the very first meetings, and the establishing of a Project Brief, Project Initiation Document, or Business Case. All of these artefacts should have some consideration of quality across all the POPIT domains, and business analysts are well-placed to provide this level of thinking.

Quality is considered too late. The first place to think about quality is on day 1 of the project. We don’t just wait for something to be delivered before we have ideas about quality, because by then it’s too late. Yet as business analysts, good old BOSCARD, our standard technique for generating a Terms of Reference before we even do any other work at all, does not consider quality at all. I have now changed this technique in my own practice to become BOSQCARD, putting Quality right into the terms of reference: how will we make sure that the work we’re about to do is fit for purpose? (I go into great length on the joys of my technique Super-BOSCARD here, if you’re interested.)

Example: Quality Assurance and Quality Control activities for Business Acceptance Testing of a SaaS solution

Lets consider a situation where, as a business analyst, we are conducting the Business Acceptance Testing service in the BASF. We are evaluating some configuration changes that are being made to a SaaS product that our company has purchased. We have no formal testers on our team, so we’re doing the work ourselves. Let’s suppose that early results have found defects and we are working with the vendor to put in place a performance improvement plan.

Here are some items we could consider in order to improve quality:

Quality Assurance

Accurate requirements: Give detailed requirements, with mock-ups if needed.

Validated requirements: Consultation with stakeholders on requirements to validate requirements.

Non-functional requirements: NFRs should be documented, even if they are quite light.

Walkthrough of requirements: Discuss the requirements on a call with the vendor, to make sure they are clearly understood. Make sure we speak to the developer who is actually doing the work, not just a manager.

Incremental delivery: one thing at a time, and don’t overload the vendor. Could also work in defined sprints.

Dedicated developer: Rather than rely on anyone from the vendor’s technical support, have an assigned individual technical person that will do the work.

Dedicated tester: Ask the vendor for a dedicated tester to work on the end-to-end solution, ensuring consistency, not various people popping in and out of the project among their other work.

Quality Control

Formal test scripts: Create formal test scripts and share these with the vendor, effectively forming the acceptance criteria.

Auditing: Auditing and spot-checking of data and reports, cross-check back to raw data where possible.

SME sanity checks: Checks from SMEs within our organisation, asking “Does this look right?”

Defect tracking: Good defect tracking with ticket management and regular defect/quality reviews.

Resolution times and SLA: Agree defect resolution times with the vendor in advance. Split this into two aspects: configuration defects and underlying platform defects.

Test plan: Ask the vendor to provide a test plan for how they are going to address testing

Quality plan: Ask the vendor for a wider Quality Plan that encompasses QA and QC.

These are just a few suggestions. Depending on the context there could be many more items to include. You see the key point though: discuss quality in detail with the vendor in advance, focusing on the process as well as the outputs, then setting expectations on both ourselves and the vendor.

Conclusion: quality is our responsibility

We have a massive opportunity here to move our profession of business analysis forward, by raising the standards of what we deliver and how we deliver it. This is a huge gap right now, and as business analysts we are doing our organisations a huge service by working to fill that gap.

As business analysts, the quality of our own outputs and the solutions that our projects ultimate deliver is our responsibility, because:

We have the holistic overview of the project
We are the bridge that links business and technology
We are (hopefully) defining and then tracking business benefits
We are uniquely placed, because nobody else has the overview, and the details, and the holistic approach, so nobody else cares in the way that we care!

WE MUST:

Incorporate quality early and often: From the very start of a project, as early as our first Terms of Reference using a tool like BOSCARD, quality must be a foundational component, not an afterthought.

Expand the scope of quality to a full holistic view: Apply Quality Assurance and Quality Control across the entire POPIT model: People, Organisation, Processes, Information, and Technology. A quality solution means more than just IT and more than just testing.

QA and QC are distinct but equally Important: Quality Assurance focuses on preventing defects through effective processes, and Quality Control is about detecting defects in the outputs and the products that our projects produce. Both need equal attention, and business analysts must understand them well.

Education and training in quality is now crucial: As business analysts we must get formal quality training, including certifications like those from the ISTQB, to better understand and implement quality in their projects. This knowledge is essential for both our own personal development and the success of the projects we work so hard to deliver. Education outlets such as the BCS must do more to support business analysts with tailored courses and certifications on quality that go beyond only software testing.

Collaborate with stakeholders on quality and get their buy-in: Engage with all project stakeholders, including project sponsors, project managers, vendors, and developers, to set clear expectations about quality. Get them to see the importance, set the processes, and define the Quality Plan.

Use Agile methodologies appropriately and fill the gaps: While agile methodologies usually emphasise speed and flexibility, they must not compromise on quality, as defined by “fitness for purpose”. We must notice where our project’s methodology falls short or does not encompass quality, and work to fill those gaps, especially when using scrum or kanban.

Business analysts must advocate for quality: We have to champion quality within our projects, because few other people are going to do that! Apart from testers, most people are much more interested in time and cost, probably because they are easier to measure and show on a Powerpoint slide! We have the professionalism and the skills to help our organisations to do better!

Remember: quality is not just about the final product, it’s about the processes and people involved in creating that product — and those people includes us!

About the author

Ian Howlett has formally been a business analyst for 12 years, and was previously a software developer, with 24 years in the IT industry. He has a Computer Science degree, and an MBA with distinction from the University of Oxford. He has worked for some of the largest companies in the UK, for large US-owned companies, and for smaller companies too, in sectors including aviation, telecommunications, healthcare, media, and insurance. Outside of work he has a private pilot’s licence, plays the piano, and enjoys F1 motor racing, snooker, and reading.