Team evaluations that yield the best results

Conducting client-agency team evaluations is both art and science. Get it right and you’ll see swift improvements.

Why do we need to evaluate agency performance?

If you believe that agencies are hired to do their clients’ bidding, you might ask ‘why do we need to evaluate the agency at all?’ A reasonable question.

Surely if the agency fails, or appears to lapse in this task, one can simply fire them and move to a new, fresh, willing agency?

But there are a million reasons why it’s better to repair team relationships than replace them. Research reveals the average total cost of an agency review in the USA at $1.2 million. For marketers, the key costs for an agency pitch involve hourly costs, staffing changes, disruptions/delays, managing external consultants and compensation to the agencies involved.

Think also of the disruption, the loss of accumulated knowledge and the added stress on all concerned.

The human factor

The fact is that agencies are comprised of individuals. People with dignity, preferences and choices. People who can be stimulated and motivated to work above and beyond the call of duty for things they believe in and clients they enjoy working for. As the saying goes, ‘clients get the agencies they deserve‘ and brands get the teams they deserve.

Simply put, client-agency evaluations are designed to open communication, identify and resolve issues and build trust, allowing teams on both sides to do their best. Good client-agency evaluations unleash intrinsic value of the teams involved.

To prove the hypothesis that better client-agency relationships deliver better work, we conducted a study with WARC, combining two unique and robust sets of data. As illustrated by the concentration of relationships dotted in the top right quadrant of the graph below, the research revealed most of agencies and marketers that won effectiveness awards had relationship scores above global averages.

Stronger relationships produce stronger results
Source: Aprais report: Do stronger relationships produce stronger results?
The managed relationship

Leading companies invest significantly in team evaluations that yield the best results – optimal performance and return on investment. Why? Because evaluations work.

The relationship between a client and its agency is a linchpin for success. A robust relationship is more than a transactional collaboration; it is a strategic alliance that can significantly impact a company’s growth, innovation and overall market standing. Recognising the pivotal role that agencies play in the achievement of business objectives, organisations increasingly acknowledge the need for formal, regular evaluations to ensure the symbiotic growth of both parties.

These evaluations serve as a compass, guiding the evolution of the relationship by providing actionable insights, fostering transparency and laying the groundwork for continuous improvement. In this quest for excellence in client-agency relationships, the method chosen for evaluation becomes paramount.

Various approaches exist, ranging from in-house management to off-the-shelf tools and qualitative-only evaluations. This article delves into the intricacies of agency evaluation and dissects some popular approaches.

Let’s begin with some basic hygiene factors.

The art and the science of the best team evaluations
art and the science of the best agency evaluations

Regardless of the method used to evaluate performance, it’s hard to overstate the importance of getting the approach right in the first place. This is both art and science. It requires an understanding of human nature and statistics. The best agency evaluations should take into account ten critical, deciding factors.

  1. One-way or mutual evaluation?

The importance of mutual (two-way) evaluation cannot be overstated. By way of analogy, could a marriage improve if only one party gave their opinion of the other? Certainly not. If the goal is, as it should be, to strengthen as opposed to critique a relationship, then both parties must have the opportunity to voice an opinion. This helps build understanding of each other’s perceptions and makes it possible to course-correct behaviours as necessary.

  1. Participant selection

Whose opinion matters? This decision is crucial to the evaluation process. We find that relationship issues can arise regardless of seniority or department. Therefore, clients and agencies should agree on the participants who will ensure a balanced view of the relationship. Another key criterion is setting a minimum length of time participants have worked with the team.

  1. Frequency

Let’s face it, most executives experience survey fatigue. Left too long, issues may fester into problems requiring more drastic solutions. Our considerable evidence based on more than 28,000 evaluations, suggests that every six months is the ideal frequency. When evaluations are tied to performance bonuses, this provides teams an interim opportunity to improve any weak points ahead of the ultimate evaluation.

  1. Administration

Convincing participants to complete an evaluation is an often-underestimated challenge. Recognising the pressures of time that executives are under, the process should be made easy and (if possible) enjoyable for participants to complete the surveys and for administrators to manage the system. Leverage technology to streamline the process – particularly the collection and presentation of data.

Leadership buy-in to the process is critical. Encouraging completion often requires a nudge from senior leadership to impress the importance of the evaluations and usefulness of the outputs.

  1. Scoring

Should you use a top-line NPS (net promoter score), a 5-point, 10-point or 100-point scale?

At Aprais, we use a 100-point scale because it provides more accurate results than a 5-point scale. Quite apart from the statistical issues of greater sensitivity, objectivity and linearity (read more here), the 100-point scale is more motivating when teams can see fluctuations with each round of evaluation.

  1. The questions

Whether qualitative discussion guide or a quantitative-based evaluation, the questions must be carefully curated to reflect the responsibilities and scope of work for the various participants at the local, regional and global level.

As with any market research, badly chosen questions can undermine the credibility and value of the entire exercise. As amplified below, we believe the best client-agency evaluation combines both quantitative and qualitative techniques. Scores enable comparison, tracking and benchmarking while qualitative techniques provide context and deeper insights than scores alone.

  1. Disciplines and behaviours

Despite the different scopes of work and required outcomes, client-agency relationships can and should be evaluated and benchmarked. An evaluation that focuses on the disciplines within the client-agency scope of work helps pinpoint any areas where the team need to improve.

Questionnaires are often built around the agency-client scopes of work using disciplines such as account leadership, financial management and creativity etc. However, personal relationships are also based on behaviours and common values. For this reason at Aprais we also consider and compare behaviour traits so we can provide guidance to improve team relationships.

  1. Benchmarks

Relying solely on self-assessment or previous internal evaluations is akin to ‘marking your own homework’. Without comparative benchmarks, it is easy to reach flawed conclusions. External benchmarking in team performance can be a game-changer for businesses aiming to excel.

  1. Reports

Who will read and act on the outputs? This will determine how the reports are structured. Some organisations may need the same report for everyone; others need them separated or combined by brand, agency, region. These details may influence the entire programme and therefore should be decided at the very outset of the programme.

  1. Action planning

Regardless of the method employed, conducting an evaluation and not acting on the results, makes little sense. Participants may question the value of their efforts when then next wave of evaluations come around.

On the other hand, establishing clear action plans following a team evaluation is uplifting, reinforces confidence in leadership and enhances the effectiveness of the entire evaluation programme. Progress against these actions can then be assessed in the next wave.

Choosing the best evaluation approach

As the saying goes, ‘there’s more than one way to skin a cat’ (slightly offensive to we cat owners). The options range from internal management using a simple Excel spreadsheet to off-the-shelf tools and qualitative-only evaluations.

Let’s talk about these two fundamental choices, self-managed versus outsourced and qualitative-only evaluations versus quantitative.

Management of the agency evaluation process internally by the client company is probably the most common practice, as it ‘appears’ to be cheaper and it allows organisations to retain control over the entire process.

Companies can use a range of tools from a simple Excel spreadsheet to more elaborate off-the-shelf (SaaS) tools.

When organisations perform their own agency evaluations, there is an inevitable (real or perceived) bias that undermines the authenticity of the evaluation, essentially turning it into an exercise of ‘marking your own homework’. Such subjectivity can lead to misguided decisions, hindering the organisation’s ability to identify areas for improvement accurately. If, as they should be, the evaluations are mutual, the company must be prepared to manage (and act on) weaknesses revealed on their own side of the relationship.

Furthermore, managing evaluations in-house can be resource-intensive.

Despite the promise of automation, implementing and managing SaaS tools can impose a heavy burden on staff – particularly if there are multiple geographies, brands and agencies involved. The manager assigned may already be heavily burdened with their existing responsibilities, leading to a rushed or incomplete evaluation process. This not only compromises the quality of the assessment but also diminishes the value of the insights gained.

Remember, evaluations can have profound effects on B2B relationships and the lives of those who participate. This responsibility demands qualified, dedicated personnel to operate and interpret the data generated by these tools. These additional costs and resource allocation are seldom figured into the decision to maintain in-house management of the process.

SaaS tools rely primarily on data and algorithms, minimising the role of human interaction. The best agency evaluations extend beyond mere metrics, however; they involve interpersonal dynamics, creativity and collaboration. Off-the-shelf tools often neglect these intangible elements, resulting in an incomplete evaluation that fails to capture the holistic nature of client-agency partnerships.

Qualitative evaluations usually employ structured or semi-structured interviews of a limited number of stakeholders. They focus on subjective aspects of agency performance, providing valuable insights into the softer elements of relationships. However, relying solely on qualitative assessments comes with its own set of limitations.

The obvious major drawback of qualitative-only evaluations is the inherent difficulty in measuring and quantifying results. While qualitative insights offer valuable context, the absence of measurable metrics makes it challenging to track progress and identify areas for improvement objectively.

the best client-agency evaluations are two-way

Subjectivity rules out the possibility of benchmarking performance against previous evaluations or against industry standards. This absence of benchmarks hampers the ability to set realistic goals and expectations for improvement.

Given the high rates of turnover in the industry, qualitative-only evaluations conducted among a group of individuals ‘today’ may provide insights for the short term but lack the longevity needed for sustained team success. Without measurable benchmarks and a focus on long-term performance, organisations risk making decisions based on transient circumstances rather than the enduring capabilities of their agency partners.

The labour-intensive nature of personal interviews also has cost implications and/or places limits on how many stakeholders can be interviewed. This can compromise the credibility and actionability of insights derived in the process.

Improve team performance rapidly

Numerous research studies point to the shortening tenure of a chief marketing officer. According to one such study by Spencer Stuart in the USA, the median tenure for a CMO was 25.5 months, sliding from 30 months in 2019.

With this in mind it is reasonable to ask whether evaluations can impact a business relationship with the agency during the shortened tenure of a CMO? Our analysis shows that when a fair, systematic and objective evaluation methodology is used, business relationships can improve from the first evaluation across both disciplines and behaviours.

Choosing the right approach to evaluating the client-agency relationship is paramount for organisational success.

Kim Walker is Chairman and founder of Aprais.