Reflections and how I passed Scrum.org’s PAL-EBM certification

The Professional Agile Leadership — Evidence-Based Management (PAL-EBM) certification is the newest offer from Scrum.org and has only been available for two months. I took the exam yesterday and passed it. Because it is so new, there are currently only 318 holders of the certification and little information on passing it. I have shared my experience below, along with some tips to help anyone looking to pass it.

PAL-EBM is an unusual sibling in the Scrum.org family. There isn’t a class currently you can take beforehand. It flows out of the Professional Agile Leadership stream and speaks to both leadership and non-leaders on how management and empiricism should go together. It’s the link between management and Scrum teams, helping to frame what we mean by value and how to generate and measure it. Other certifications cover the EBM principles, particularly the Professional Scrum Product Owner II (PSPO II). If you have taken either of the PSPO classes, you should already be aware of it.

In my view, this is not an easy certification to pass. The material is relatively narrow in scope. You are required to understand the principles of the guide and then apply them to different situations. While some of the questions are textbook answers, many of my exam questions focused on applying that knowledge.

How I prepared

I am a product owner and was aware of some of the concepts already. I became much more familiar with the EBM principles when preparing for my PSPO II certification last year. That gave me some understanding of EBM.

The first and the best thing you can do is read and re-read The Evidence-Based Management Guide. It is foundational to everything covered in the course. It’s like the Scrum Guide for the PSM and PSPO certifications — everything flows from it. You should be able to recite it in your sleep. Make sure you understand the EBM principles: the types of goals, key value areas (KVAs) and when they are used, along with the different metrics and when you should use them.

There are some additional resources on the reading list. These are quite useful and help you to understand what the certification is trying to drive at — particularly as you try to apply the framework to real situations.

I found the open assessment very helpful. I would recommend taking this over and over again until you consistently score 100%.

And that’s about it. There’s not much more you can do to prepare!

Some insights from the other side

Once you have gone through all the materials a few times, you need to start getting into the EBM mindset. Business agility is what we’re chasing. Here are some principles to remember. These are rough notes I made as I went along in my preparation to keep me in the EBM mindset. I’ll use these notes myself as a checklist to apply EBM to my work.

The critical thing to remember is the purpose of EBM. At its heart, it is a framework to help an organisation review and improve its ability to deliver value to its customers. Keep this question at the front of your mind: what should I do to drive value in a complex world?

Empiricism underpins EBM:

• Empiricism is all about gaining knowledge from experience and making the right decisions based on what you see. It helps you to take meaningful steps to deal with complex problems. Those problems can be anything we encounter — different types of risk, new technologies, changing markets and more.
• Remember what you need to apply an empirical approach: it’s about forming a hypothesis, running an experiment, validating that hypothesis and then repeating. You don’t know whether something is true until it is released and validated by your customer.
• A product backlog item (PBI) is then just a hypothesis. It’ll be at the top of a backlog because somewhere in your process you’ve decided it’s valuable. However, you’ll only confirm if a PBI is valuable after you release it in the market.
• So in that spirit, release often! Even in cases where it’s only to improve a product’s performance to release a tiny feature. Or if you have thin development resources and the team needs to release and move onto something else. Releasing often will give you faster insights to make the right decisions on how to drive value.
• If you have to make any choice about when to release, always choose the fastest option — even if that means releasing less each time.

Run experiments:

• Experiment and then experiment again! Running experiments will help you gather insights into how to tap into unrealised value or understand how to close satisfaction gaps. Learn from those insights and plan your next step.
• As discussed already, you can release small parts of a feature rather than the whole to experiment and gain insights if that means you release value to your customers sooner.
• Defining measurements is preferred, but not essential. You can experiment and see where it takes you. Remember empiricism: knowledge comes from experience and making decisions on what is observed.
• If you’re familiar with Kanban ways of working and the PSK accreditation, limiting work in progress and creating a clear focus will increase flow. That will lead to improved cycle time and, hopefully, more value as you will release value to your customers sooner. Though remember, more output doesn’t necessarily equal good outcomes for your customers.
• Therefore focus on outcomes for your customers. Other metrics around activities or outputs are important, though secondary. If you’re not delivering value for our customers first, you’ve missed the purpose of EBM.

There’s no ‘us and them’­ in an organisation — we are all responsible for outcomes:

• The role of management is to set clear goals and measure for the organisation.
• SMART objectives are not smart in EBM — at least the majority of them aren’t! We only want goals which are specific and measurable; otherwise, they’ll be nebulous. We won’t know what is assignable/attainable or realistic in a complex world until we start experimenting. As for timescales, the bigger the goal, the less is known about it. We won’t know when we will achieve that goal, if at all. SMART goals limit an organisation’s strategic horizon and cut against what EBM is trying to achieve.
• Putting in place rigid structures like annual planning or budget cycles will hurt an organisation’s ability to respond quickly. Avoid them.
• The goals of the organisation are the responsibility of everyone — not just management. Don’t break them up for different levels of the organisation. We’re all equal stakeholders in our quest for customer value.

Thoughts and reflections on the EBM

The EBM beautifully explains business agility and what organisations should be focusing on. Simply forcing all your teams to work in Scrum won’t in of itself drive business agility; this has been said before, particularly by people like Klaus Leopold. I finally have a great tool to sit down with a leadership team and start a conversation on real agility.

There are two weaknesses from my perspective, which I hope a future update of the EBM will cover.

1. Goals. In the context of an organisation, a ‘goal’ is not well understood or well used. Sprint goals at a team level is easy to understand. ‘Strategic goals’ and ‘intermediate goals’ in the EBM framework is harder to apply. Everyone is comfortable with vision, mission, strategy and even objectives; particularly with OKRs (objectives and key results). Taking goals and goal setting and then layering in both strategic and intermediate goals isn’t easy to adopt. I wish the writers of EBM had at least included company vision as the primary north star — something everyone already has. Adding in additional processes can make something like this harder to adopt.
2. External measures particularly with ‘Unrealised value’. A bigger weakness of the EBM is with the KVAs and how inward looking they are. Unrealised value as a KVA has the lowest number of example key measures, with just three of them. There is nothing around competitors, industry leaders, new enabling technologies or even the wider economic landscape. As a large enterprise you are probably worried about disrupters. And as a start up, you are probably looking to the big beasts in your industry to disrupt. More example key measures, particularly for Unrealised value, thought across the board for external benchmarking would have really helped. This could have applied to all the KVAs: how much value does your competition capture? How quickly can they innovate? How quickly can they get products to market? All worthy questions and absent.

Despite these two weaknesses, the PAL-EBM and the EBM framework has been a worthwhile learning journey — which I have enjoyed and would definitely recommend. Anyone in any product or leadership role should read the guide. It’ll push you to see the world through an EBM prism and help you to understand what is valuable and not. It’ll also force you to rethink your metrics and to question why you chase particular ones. I’ll be using the EBM to inspect and adapt our dashboards, with the overall aim of driving the right and therefore better conversations. I’m expecting it to change how the product team I work with sees and understands value.