Leave No Stone Unturned: Solving the Problem of Vanity Metrics

Author Date February 25, 2020 Read 5 min
The client ask: “Increase newsletter signups by 10%” If you are a UX designer tasked with tackling this assignment, you will probably want to jump right in and…
Graph showing ebb and flow

The client ask: “Increase newsletter signups by 10%”

If you are a UX designer tasked with tackling this assignment, you will probably want to jump right in and show off your UX expertise by approaching this problem using the proven methodology of design thinking.

You’ll proceed to:

  1. Empathize with end users to determine why they aren’t signing up for the newsletter.
  2. Define the problem that they aren’t seeing the signup form because it’s buried in the footer of their website.
  3. Iterate possible solutions, from raising the form above the fold to throwing up modals on the site.
  4. Build a lightweight prototype and test it with users.
  5. Charge your development team with the task of implementing the takeover modal solution that users said would lead to signup.

Although you successfully deployed a solution and delivered on the ask of achieving a 10% lift in newsletter signups, the client is ultimately left feeling unsatisfied. What happened? We did a fine job establishing empathy for our users, but what about our client? What happened to their original problem?

Although they pointed out what they thought their problem was, we failed to apply design thinking to our stakeholders, and immediately proceeded to empathize with users and define their problem with the newsletter.

The real problem here is that you didn’t take the time to ask yourself: Is the number of newsletter signups just a vanity metric?


The Nielsen Norman Group defines them as “metrics that appear impressive but don’t give insight into the true performance of a digital property.”

Unfortunately, spotting vanity metrics among the meaningful ones is not always easy. The task is especially difficult for design teams that are singularly focused on the end user’s experience or for designers that don’t have experience as UX analysts and strategists. They may believe it’s not their place to question the client ask and may even worry that going down this path will offend their client.

Luckily, the design-thinking process can be applied to everyone involved with a product. The process of design thinking works best when it’s treated as if it’s in front of a mirror.

Always empathize with everyone that interacts with a product, especially your client stakeholders.

Design teams that see the mirror are the ones that go from being an easily replaceable design vendor to becoming long-term strategic partners. This is because successful UX design teams always uncover and marry the needs of users with the needs of the business.

So, let’s go back to the original ask. Are we leaving no stone unturned in resolving the client ask? Did we read the title of this article?


As it turns out, we uncover a few client motivations that led to the original ask. Running through the steps of design thinking — using our client as our central focus — can help us gain a deeper understanding of the real business problem.

So, let’s establish empathy for our client and figure out why they want to increase newsletter signups to begin with.

Motivation #1: The newsletter contains useful information about our industry that you can’t find anywhere else.

Applying design thinking to this client motivation will lead us to a different conclusion about what we should design. Instead of convincing more people to sign up for the newsletter, it could be more impactful for everyone if we recommend putting the useful newsletter content within a resource hub on the company’s website. Of course, there may be other issues to look at and validate here, such as the client’s assumption that their content is useful.

Motivation #2: Customers subscribed to the company’s newsletter convert at a higher rate than customers not subscribed.

Applying design thinking to this client motivation would lead us down another interesting rabbit hole. Maybe we’ll find out that this positive correlation is only true because the newsletter always contains coupons for the product. The new insight to uncover here is that customers with coupons are more apt to convert on this company’s website and the best possible solution might be to put a coupons page right on the website, easily accessible from the main navigation.

Motivation #3: Motivation is unclear; the client is only following their boss’ orders.

Again, when we empathize with our client here, we will probably determine that we need to speak with our client’s boss to get a clear understanding of the real business problem.

At this point, you may be asking yourself how to find these insights — or stones — so you can start to turn them over yourself. As mentioned earlier, we need to stay curious, investigate every lead, and become a UX Sleuth, in addition to a UX Designer.


A helpful strategy for turning over stones and getting to meaningful insights and metrics came from my design mentor. He would review my design presentations and infuriate me by interjecting, “So what?” after every slide. Luckily, after a while, his message started to sink in. When it comes to users and clients, we all need to get to the point so we are aligned and doing meaningful work, and not investing time trying to influence vanity metrics.

For most companies, when we start turning over stones, the real challenge will have something to do with their bottom line. It’s what typically drives clients to seek out the expertise of UX consultants.

So, when I hear “We need more newsletter signups,” what I see is a red herring going by and I try not to let it break my focus.

Remember that design thinking is not a linear process. It should involve everyone from stakeholders to end users. Be constantly empathetic. Remember to look into the mirror so we can ask ourselves and everyone else, “So what?” until we get to the real problems worth solving. That positions us and our teams as irreplaceable strategic partners.

parallax image

Find Your Possible.

Let's Chat