Volume 2
Literature Reviews
Improving Telephone Service
Through Better Quality Measures
LITERATURE REVIEW: Improving Telephone Service Through Better Quality Measures
LITERATURE REVIEWS — Telephone Service 230
Taxpayer
Attitudes
Penalty
Study
Settlement
Initiatives
Telephone
Service
Taxpayer
Engagement
Use of IRS
Services
OIC
Program
EITC
Letters
Collection
Agencies
INTRODUCTION
The National Taxpayer Advocate has repeatedly expressed concern over the IRS’s increasing reliance
on online channels to communicate with taxpayers while failing to adequately upgrade its telephone
service.
1
The IRS has steadily decreased access to face-to-face assistance available in Taxpayer Assistance
Centers (TACs),
2
leaving taxpayers who are unable or unwilling to access online self-service solutions
with two options to obtain service from the IRS: write a letter or make a phone call.
3
On average, the
IRS receives more than 100 million telephone calls each year on its toll-free lines.
4
Currently, the IRS
evaluates its telephone performance using the Customer Service Representative Level of Service (LOS),
which is the relative success rate of taxpayers who call for live assistance on the IRS’s toll-free telephone
lines in reaching an assistor.
5
In the 2018 Objectives Report to Congress, the National Taxpayer
Advocate acknowledged significant improvement in the IRS’s LOS for the filing season 2017, but found
the IRS’s reported LOS data inaccurately describe the IRS’s ability to properly assist taxpayers over the
telephone.
6
The National Taxpayer Advocate is also concerned that the metric currently used by the IRS
to measure LOS fails to adequately evaluate taxpayers’ actual experiences on telephone calls with the
IRS.
In order to provide telephone service that meets the needs of taxpayers in the twenty-first century, the
IRS should follow the approach taken by other organizations in private industry and government that
treats telephone service as an essential part of an omnichannel service environment.
7
An omnichannel
service environment allows taxpayers to contact the IRS through the channel of their choice and
receive a consistently high quality of service. TAS’s research shows that quality metrics, designed to
assess taxpayer satisfaction, should drive the way the IRS evaluates its telephone service and educates its
telephone assistors.
This literature review analyzes measures and practices most commonly used in the private sector and
other government agencies to evaluate and improve call center performance. The following discussion
explains how quality metrics and employee engagement should be used to optimize taxpayer experience
over the phone.
1
See, e.g., National Taxpayer Advocate 2015 Annual Report to Congress 3-13 (Most Serious Problem: Taxpayer Service: The
IRS Has Developed a Comprehensive “Future State” Plan That Aims to Transform the Way It Interacts With Taxpayers, But Its
Plan May Leave Critical Taxpayer Needs and Preferences Unmet).
2
National Taxpayer Advocate 2017 Annual Report to Congress (Most Serious Problem: Cuts to IRS Walk-in Sites Have Left
the IRS With a Substantially Reduced Community Presence and Have Impaired the Ability of Taxpayers to Receive In-Person
Assistance. vol. 1, supra.
3
National Taxpayer Advocate 2015 Annual Report to Congress 3-13 (Most Serious Problem: Taxpayer Service: The IRS Has
Developed a Comprehensive “Future State” Plan That Aims to Transform the Way It Interacts With Taxpayers, But Its Plan May
Leave Critical Taxpayer Needs and Preferences Unmet).
4
IRS, Joint Operations Center (JOC), Snapshot Reports: Enterprise Snapshot, IRS Enterprise Total (final week of each fiscal year
(FY) for FY 2008 through FY 2016) (showing telephone call volumes exceeding 100 million in every year).
5
Here is the Customer Service Representative (CSR) Level of Service (LOS) formula: (Assistor Calls Answered + Automated
Calls Answered (Info Messages)) divided by (Assistor Calls Answered + Automated Calls Answered (Info Messages) +
Emergency Closed + Secondary Abandons + (Add either Calculated Busy Signal OR Network Incompletes) + (Add either
Calculated Network Disconnects OR Total Disconnects)). See IRS, JOC, Snapshot Reports: Product Line Detail Snapshot
(week ending Sept. 30, 2017). Note that CSR LOS is not a measurement of the total number of callers to the toll-free lines
who speak with a customer service representative.
6
See National Taxpayer Advocate 2018 Objectives Report to Congress (Review of the 2017 Filing Season) (discussing how the
IRS data is misleading because fewer calls were routed to Accounts Management (AM) assistors, and other lines outside of
the AM umbrella saw a decreased LOS.)
7
For a discussion of the limitations of IRS telephone service, see National Taxpayer Advocate 2017 Annual Report to
Congress vol. 1 (Most Serious Problem: Telephones: The IRS Needs to Modernize the Way It Serves Taxpayers Over the
Telephone, Which Should Become an Essential Part of an Omnichannel Customer Service Environment), vol. 1, supra.
Taxpayer Advocate Service — 2017 Annual Report to Congress — Volume Two 231
EITC
Letters
Collection
Agencies
OIC
Program
Taxpayer
Attitudes
Penalty
Study
Settlement
Initiatives
Telephone
Service
Taxpayer
Engagement
Use of IRS
Services
DISCUSSION
Telephone Metrics Should Focus on Evaluating Call Quality and Caller Satisfaction
While the IRS has traditionally used efficiency measures like average speed of answer, such measures can
yield “hollow results” if the quality of a call is below the caller’s expectations.
8
When pressured to reach
targets for number of calls answered, telephone assistors are more likely to handle calls inappropriately
by missing key information given by the caller or failing to effectively complete post-call work.
9
Such
results can create problems and rework in the future, and ultimately drive the caller to other, but less
preferred, communication channels.
Although efficiency metrics should not be completely disregarded, the IRS should follow the approach
of many private sector companies and give more weight to measures gauging the customer’s overall
experience on a call.
10
Using qualitative metrics can help a call center ensure that taxpayers’ questions,
not just calls, are being answered. Customer satisfaction is greatly improved when phone assistors make
proper and accurate determinations and communicate them to the caller properly.
11
The “single biggest
driver of customer satisfaction” is the First Call Resolution (FCR).
12
FCR “measures the percentage
of all calls that are resolved on the first attempt, without the agent needing to refer the customer
to a colleague, their manager, or calling the customer back.”
13
This percentage is most frequently
determined by providing callers the opportunity to submit feedback subsequent to the initial contact.
14
The IRS should solicit feedback, including comments and complaints, through multiple channels, such
as over the phone or online.
15
The global average for FCR is between 70–75 percent.
16
“Achieving 80%
or higher FCR rating typically results in an average of 1.2 calls needed to resolve customer’s inquiry or
problem.”
17
Call centers should solicit feedback, including comments and complaints, through a variety of
channels.
18
The most common ways of measuring FCR include:
Telephone Survey: “customer is surveyed within 1-3 days of the call and asked if their inquiry or
problem was resolved;”
8
George Robinson & Clive Morley, Call Centre Management: Responsibilities and Performance, 17 int’l J. of seRviCe indus. mgm’t.
284, 296 (Dec. 8, 2009).
9
Id.
10
Bank Administration Institute, Evolution of Contact Centers in Banking: Engaging and Empowering Agents in an Omnichannel
Operating Environment, 10 (2015), https://www.avanade.com/~/media/asset/brochure/contact-centers-in-banking-report.
pdf.
11
Terrel Bird, Measuring the Success of a Call Center: Easier than You Think, inteRnational CustomeR management institute (Jul. 13,
2016), http://www.icmi.com/Resources/Metrics/2016/07/Measuring-the-Success-of-a-Call-Center-Easier-than-You-Think
(last visited Dec. 5, 2017).
12
Jeff Rumburg & Eric Zbikowski, MetricNet, The Five Most Important KPI’s for the Call Center, 5 (Feb. 20, 2013), https://ccng.
com/uploads/five_most_important_kpis_for_the_call_center_metricnet.pdf.
13
International Finance Corp., Measuring Call Center Performance: Global Best Practices, 7 (June 2010).
14
Government Accountability Office (GAO), GAO-06-270, Federal Contact Centers: Mechanism for Sharing Metrics and Oversight
Practices along with Improved Data Needed (Feb. 2006), http://www.gao.gov/assets/250/249031.pdf.
15
GAO, GAO-15-84, Managing For Results: Selected Agencies Need to Take Additional Efforts to Improve Customer Service
(Oct. 2014), http://www.gao.gov/assets/670/666652.pdf.
16
International Finance Corp., Measuring Call Center Performance: Global Best Practices, 7 (June 2010).
17
Mike Desmarais, Service Quality Measurement Group, First Call Resolution: The Call Center Measure that Matters Most
(Dec. 2005), http://grandsorganismes.gouv.qc.ca/fileadmin/Fichiers/Veilles%20strat%C3%A9giques/Gestion%20des%20
CRC/2005-first_call_resolution.pdf.
18
GAO, GAO-15-84, Managing For Results: Selected Agencies Need to Take Additional Efforts to Improve Customer Service
(Oct. 2014), http://www.gao.gov/assets/670/666652.pdf.
LITERATURE REVIEWS — Telephone Service 232
Taxpayer
Attitudes
Penalty
Study
Settlement
Initiatives
Telephone
Service
Taxpayer
Engagement
Use of IRS
Services
OIC
Program
EITC
Letters
Collection
Agencies
Script: “[customer service representative] CSR asks the customer at the end of the call if their
inquiry or problem was resolved;
[Interactive Voice Response] IVR Surveys: customer completes an IVR survey at the end of the
call and is asked if their call was resolved;”
Call Backs: “FCR is determined based on whether the customer called back within 3-5 business
days;” and
Case Management Software: “CSR uses their desktop software application to capture whether
the customer’s inquiry or problem was resolved.”
19
Using FCR data would help the IRS to identify how to reduce the amount of calls it receives and also
provide better service to taxpayers on the calls it does receive.
Engaged and Empowered Employees Are Essential to Improving Call Quality
Along with evaluating the experience from the caller’s perspective, the IRS should also implement
practices for accuracy-related oversight and incorporate metrics to evaluate its telephone assistors’ rates
of satisfaction.
20
Keeping telephone assistors engaged in the work they perform is necessary to achieve a
high quality of service.
21
In providing guidance to telephone assistors, the IRS should identify specific
actions a telephone assistor can take to improve his or her performance.
22
More than just correcting
mistakes, supervisors should help telephone assistors understand how their role fits in the larger mission
of the call center by explaining the impact a telephone assistor’s performance had on the taxpayer’s
experience.
23
To further improve satisfaction and engagement, telephone assistors should be empowered
to take affirmative steps to address a caller’s problem themselves.
24
Employees need to be given a sense of
agency over their work, and allowed to think creatively and reach innovative solutions.
LITERATURE REVIEW
1. Accenture, Transforming Customer Services to Support High Performance in New York City
Government, 5, 6 (2013).
“Through [business intelligence] BI reports and dashboards, the city tracks integrated information
from millions of customers, allowing city staff to analyze call patterns and plan for or respond
proactively to issues.”
“Customer satisfaction with NYC 311 now surpasses even the highest-performing private-sector
call centers, according to a survey conducted last year by CFI Group, Inc., an industry leader
in gauging customer satisfaction as measured by the patented American Customer Satisfaction
Index.”
19
Mike Desmarais, Service Quality Measurement Group, First Call Resolution: The Call Center Measure that Matters Most
(Dec. 2005), http://grandsorganismes.gouv.qc.ca/fileadmin/Fichiers/Veilles%20stratégiques/Gestion%20des%20
CRC/2005-first_call_resolution.pdf.
20
Ian Jacobs et al., Forrester Research, How to Measure and Improve the Contract Center Agent Experience, (Apr. 16, 2015)
(“Unhappy contact center agents equal unhappy customers.”).
21
Bernard Marr & Andy Neely, Managing and Measuring for Value: The Case of Call Centre Performance, CRanfield univeRsity staff
publiCations (2004).
22
Ian Jacobs et al., Forrester Research, How to Measure and Improve the Contract Center Agent Experience (Apr. 16, 2015).
23
Rachel Miller, Impact Learning Systems, No Time to Coach, (September 2011).
24
Ian Jacobs et al., Forrester Research, How to Measure and Improve the Contract Center Agent Experience (Apr. 16, 2015).
Taxpayer Advocate Service — 2017 Annual Report to Congress — Volume Two 233
EITC
Letters
Collection
Agencies
OIC
Program
Taxpayer
Attitudes
Penalty
Study
Settlement
Initiatives
Telephone
Service
Taxpayer
Engagement
Use of IRS
Services
2. Annette Reichold, Lutz Kolbe, & Walter Brenner, Performance Measurement of CRM in
Financial Services, 4th International Conference on Performance Measurement and Management,
Edinburgh, 5, 6 (July 28-30, 2004), http://citeseerx.ist.psu.edu/viewdoc/download;jsessionid=4F1
B718C414FB700AC626129550D0EE6?doi=10.1.1.133.196&rep=rep1&type=pdf.
“With the service management process, Service Level Agreements (SLA) are used, especially at call
centers, to measure and manage performance. Examples of such measures are, e.g., ‘the proportion
of calls accepted within defined timeframes’ or ‘the proportion of requests answered within the
first contact’.”
“All the banks studied measure customer satisfaction. The Swiss Regional Bank improved its
customer satisfaction through [Customer Relationship Management] CRM as well as the
willingness of the customers to recommend the bank. Investment Bank specifically evaluates
customers’ satisfaction after having call center contact in order to measure the overall quality and
effect of employee training and the new services offered. The bank benchmarks its results against
similar companies in order to compare its performance achievements.”
3. Bank Administration Institute, Evolution of Contact Centers in Banking: Engaging and
Empowering Agents in an Omnichannel Operating Environment, 10 (2015),
https://www.avanade.com/~/media/asset/brochure/contact-centers-in-banking-report.pdf.
“When polled if existing metrics still apply, roundtable participants generally agreed that while
average handle time may not go away, it could be de-emphasized in measuring performance.
Many of the participating roundtable organizations have started using new customer satisfaction
scores, and while they believe other metrics may be changing, customer experience is still weighted
the most, generally 40 percent across all scoring.”
“[I]n today’s omnichannel banking environment where the typical customer-agent interaction
has moved from transaction to relationship oriented. These types of calls naturally affect average
handle time and first call resolution, and these metrics may no longer be effective gauges of
performance. By recognizing these factors, contact center leadership can converse with agents to
see what’s working and what is not.”
4. Bernard Marr & Andy Neely, Managing and Measuring for Value: The Case of Call Centre
Performance, Cranfield University Staff Publications, 9 (2004).
“The literature provides clear evidence that the current focus on efficiency measures in call centres
can in fact be counterproductive to achieving customer satisfaction. Many call centres seem to
have fallen into the trap of believing that operational measures such as call duration or average
time to answer are indicators of customer satisfaction. The fact is that they are not; they are only
measures of efficiency, which in turn is often seen as a determinant of financial performance.
Most call centres seem to miss the important link between employee satisfaction, service quality,
customer satisfaction, and profitability.”
5. Centers for Medicare and Medicaid Services, Medicare Administrative Contractor (MAC)
Beneficiary and Provider Communications Manual (Oct. 22, 2017).
“The quality of CSR responses to provider telephone calls shall be monitored throughout the
contract year for each [Medicare Administrative Contractor] MAC jurisdiction using Quality
LITERATURE REVIEWS — Telephone Service 234
Taxpayer
Attitudes
Penalty
Study
Settlement
Initiatives
Telephone
Service
Taxpayer
Engagement
Use of IRS
Services
OIC
Program
EITC
Letters
Collection
Agencies
Assurance Monitoring (QAM) (Telephone). The goal of QAM (Telephone) is to ensure the
CSRs provide accurate and complete information, in a courteous and professional manner, while
ensuring that information is released to those authorized to receive it.”
6. Darren Baguley, Contact Centre Benchmarking, Australian Institute of Mgmt.
(June 1, 2008), http://blog.aim.com.au/contact-centre-benchmarking/ (last visited Dec. 5, 2017).
“As contact centres begin to generate more revenue than they cost to run, managers are focusing
on qualitative metrics aimed at improving customer service.”
“Metrics such as average time in queue are still important, but they’re taking a back seat to Key
Performance Indicators (KPIs) that are designed to measure the customer’s interaction: first-call
resolution, conversion ratios and customer satisfaction.”
7. Dep’t of Labor, Unemployment Insurance (UI) Call Center Study Final Report, 33–34, Coffey
Consulting (Jan. 27, 2017), https://ows.doleta.gov/unemploy/docs/CoffeyConsulting_UI_Call_
Center_Study_Final_Report_January272017acc.pdf.
“When establishing [Key Performance Indicators] KPIs, management should concentrate on four
key areas to maximize customer satisfaction and maintain a high performing call center [citation
omitted]:
1. Telephone benchmarks (e.g., average speed of answer)
2. Efficiency benchmarks (e.g., adherence to schedule)
3. Human resource benchmarks (e.g., absenteeism)
4. Service benchmarks (e.g., first call resolution rate).”
“Private call centers focus on first call resolution (i.e., the degree to which calls are resolved in the
first attempt, without the agent needing to refer the customer to a colleague or a manager, or call
the customer back). Recent research suggests that no single KPI has a bigger impact on customer
satisfaction than does first-call resolution (FCR) [citation omitted].
In private industry call centers, customer satisfaction is a critical benchmark. A three-pronged
approach to measuring customer satisfaction is recommended that combines quarterly customer
surveys conducted by team leaders with independent third party analysis, and quality assurance
measurements. The global metric for customer satisfaction is 90 percent [citation omitted].”
8. Donna Fluss, The Real-Time Contact Center: Strategies, Tactics, and Technologies for
Building a Profitable Service and Sales Operation 134-35 (2005).
“The ratios of managers to supervisors and supervisors to agents also vary widely from site to
site and depend on the contact center’s purpose. In shops that handle high-value or technically
sophisticated products, the ratio of supervisors to agents may be as low as 1 to 5. By contrast, in
a high-volume financial services call center where the typical call is ‘What is my balance?’ the
supervisor-to-agent ratio may vary from 1 to 10 to as high as 1 to 20. A ratio of 1 to 10 is optimal,
as this will allow supervisors the time to get to know their staff and offer proper coaching and
motivation.”
Taxpayer Advocate Service — 2017 Annual Report to Congress — Volume Two 235
EITC
Letters
Collection
Agencies
OIC
Program
Taxpayer
Attitudes
Penalty
Study
Settlement
Initiatives
Telephone
Service
Taxpayer
Engagement
Use of IRS
Services
9. Emmy Favilla, 10 Strategies for Customer Service Success, Customer Relationship Mgmt. Mag.
(June 2004), http://www.destinationcrm.com/Articles/Editorial/Magazine-Features/10-Strategies-
for-Customer-Service-Success-43824.aspx (last visited Dec. 5, 2017).
“Using eQuality Balance to record customer interactions, and eQuality Evaluation to measure
associate performance online, Starwood could calibrate performance scoring among agents,
evaluate customer service skills, and maintain important facets of customer connection--from
problem-solving to politeness.”
10. Genesys, Customer Service Strategies for the Retail Banking Industry, 15 (2008),
http://www.genesys.com/resources/brochures/customer-service-strategies-for-the-retail-banking-
industry-strategy-guide-emea.pdf.
“Customer-Centric Routing puts an end to customer frustration by ensuring each interaction is
routed to the ideal resource with the right information — no matter where that resource is located
in your organisation. This helps you increase first-call resolution rates, meet variable call volumes
with limited resources, increase cross-sell and up-sell rates and improve agent satisfaction.”
11. George Robinson & Clive Morley, Call Centre Management: Responsibilities and Performance, 17
Int’l J. of Service Indus. Mgmt. 284, 296 (Dec. 8, 2009).
“Managers need to consider metrics more appropriate to achieving their organisational goals.
Senior management should be asking for more than charts, graphs and a raft of statistics. They
should not allow themselves to be deluded into thinking that favourable productivity statistics
translate into a positive report in terms of a call centre’s performance in delivering value to the
organisation and its customers. More meaningful measures, such as first call resolution, need to
be added to the managers’ KPIs. A greater effort should be made to determine what will satisfy
customers in each organisational setting. Attaining a target such as ‘80 per cent of calls being
answered within 30 seconds’ is a hollow result if the quality of the calls is below the customer’s
expectations. Achieving such a result could mask the problem of all calls potentially being poorly
handled and customers being disenfranchised and eventually defecting. The pressure to meet
the target may result in calls being handled inappropriately, causing call centre agents to miss
vital information or fail to complete post-call work effectively, leading to potential downstream
problems in customer fulfilment or even leaving the organisation exposed in the event of
subsequent legal action (for example, when insufficient information has been recorded by the call
centre agent).”
12. Government Accountability Office, GAO-05-526, Immigration Services: Better Contracting
Practices Needed at Call Centers 9 (June 2005), http://www.gao.gov/assets/250/246932.pdf.
“In order to monitor and evaluate the performance of the four contractor operated Tier 1 call
centers, [U.S. Citizenship and Immigration Services] USCIS planned to use seven performance
measures. These measures were to evaluate the quality of customers’ telephone interactions with
[customer service representatives] CSRs; the accuracy of information provided to callers over the
telephone; the accuracy of callers’ information recorded by CSRs; callers’ levels of satisfaction; how
quickly CSRs handled calls (two measures); and the number of calls abandoned by customers put
on hold. According to USCIS officials, USCIS established the performance measures based on a
review of industry standards for both government and private-sector call center operations.”
LITERATURE REVIEWS — Telephone Service 236
Taxpayer
Attitudes
Penalty
Study
Settlement
Initiatives
Telephone
Service
Taxpayer
Engagement
Use of IRS
Services
OIC
Program
EITC
Letters
Collection
Agencies
13. Government Accountability Office, GAO-06-270, Federal Contact Centers: Mechanism for
Sharing Metrics and Oversight Practices along with Improved Data Needed 4, 11-12 (Feb. 2006),
http://www.gao.gov/assets/250/249031.pdf.
“Although federal policy for disseminating information to the public specifically emphasizes
accuracy, only four of the six agencies include accuracy as a performance metric in their
contracts. With respect to oversight, only two of the six agencies used all four of the accuracy-
related oversight practices we identified — regular knowledge database reviews, regular contact
monitoring, post contact customer satisfaction surveys, and validation of contractor reports.”
“Three of the agencies we reviewed conduct customer satisfaction surveys subsequent to the initial
contact from an individual. [General Services Administration] GSA, [TriCare Management
Activity] TMA, and [U.S. Postal Service] USPS conduct customer satisfaction surveys, which ask,
to limited degrees, questions that address the accuracy of information provided. While providing
some level of insight regarding accuracy, customer surveys may not always provide a valid basis
for oversight of the accuracy of information, since they usually ask the individual’s opinion on the
service provided. If the survey is conducted too closely to the time of the inquiry, the individual
may not have had time to act upon the information to know whether it is accurate or not. [Center
for Disease Control] CDC plans to implement three types of postcontact customer satisfaction
surveys through a third-party contractor beginning in June 2006.”
14. Government Accountability Office, GAO-15-84, Managing for Results: Selected
Agencies Need to Take Additional Efforts to Improve Customer Service 28 (Oct. 2014),
http://www.gao.gov/assets/670/666652.pdf.
“All of the agencies in our review provide customers with opportunities to submit feedback,
including comments and complaints, through a variety of ways such as satisfaction surveys,
comment cards submitted in person or online, e-mails, and call centers.”
15. Help Desk Institute (HDI), Service Desk Metrics – Context is Everything, 2
http://www.barclayrae.com/Knowledge/Service%20Desk%20Metrics%20-%20The%20
basics.pdf.
“First Contact Resolution (FCR)… The things you should consider are:
■
A lack of industry consensus into exactly what constitutes a ‘first call resolution’ makes
comparison of these rates problematic and probably accounts for some of the reported
variation in resolution statistics.
■
What do you consider to be a first contact fix? Some organisations will count a call that is
transferred within the help desk as a first call fix. Some organisations include those instances
where an analyst refers to another staff member, and some organisations count calls that are
routed to 2nd/3rd level as ‘resolved’.
■
Is the time taken to achieve this fix taken into account? — From the perspective of the
customer is a long telephone call which provides successful resolution more or less convenient
than being called back later?
■
What level of complexity are most of the calls to the desk?
■
Good Knowledge Management will play a significant role in improving this metric.”
Taxpayer Advocate Service — 2017 Annual Report to Congress — Volume Two 237
EITC
Letters
Collection
Agencies
OIC
Program
Taxpayer
Attitudes
Penalty
Study
Settlement
Initiatives
Telephone
Service
Taxpayer
Engagement
Use of IRS
Services
16. Hesham A. Baraka, Hoda A. Baraka, & Islam H. EL-Gamily, Assessing Call Centers’ Success: A
Validation of the DeLone and Mclean Model for Information System, 14 Egyptian Informatics J.
99, 100-01 (July 2013).
“The Outsourcing Institute [citation omitted] indicates the growing importance of identification
of qualitative metrics in measuring the performance of the outsourced call center. In [citation
omitted], [North American Quitline Consortium] NAQC Issue Paper presents best practices in
performance measurement and management to maximize call center Efficiency and Quality. The
paper proposes identification of performance evaluation indicators instead of the classical ways
based on customer surveys, customer praise, complaints, and observation of customer interactions.”
“User satisfaction is an important means of measuring customers’ feedback of the call center
system and should cover the entire customer experience cycle based on the services provided from
the call center. Examples of services may include information retrieval, marketing, technical
support, remote diagnosis, remote data entry as in medical transcriptions, mobile, etc. A critical
parameter to be measured for the user satisfaction is the rate of the escalation of calls beyond the
agent representative as the existing modules failed to answer the customer query.”
17. Ian Jacobs et al., Forrester Research, How to Measure and Improve the Contract Center Agent
Experience 1, 6 (Apr. 16, 2015).
“Contact center agents can have a huge impact on customer experience. Unhappy contact center
agents equal unhappy customers. It’s that simple. What is less straightforward is how application
development and delivery (AD&D) professionals can determine how happy and engaged the
agents actually feel. If they can successfully uncover what sort of experiences agents have, AD&D
pros must then create technology scenarios that help improve those experiences.”
“Contact center managers often want to build dashboards of key performance indicators (KPIs),
including hot metrics such as Net Promoter Score and Customer Effort scores to present to
agents. But those metrics usually give agents very little hint of what specific actions to take to
improve the numbers. Contact center pros should skip the Customer Effort score and show agents
an aggregated view of the root-cause analysis done on their interactions rated as high effort by
customers. This gives agents something they can actually bite into and can, with guidance, be
turned into useful improvement plans.”
18. International Finance Corp., Measuring Call Center Performance: Global Best Practices 2, 7
(June 2010).
“Chose metrics that reveal key insights into the customer experience, emphasizing quality and
quantity equally.”
“Customer satisfaction is a critical benchmark. A three pronged approach is recommended that
combines quarterly customer surveys conducted by team leaders with independent third party
analysis, and quality assurance measurements. The global metric for customer satisfaction is
90%…
First call resolution rate measures the percentage of all calls that are resolved on the first attempt,
without the agent needing to refer the customer to a colleague, their manager, or calling the
customer back. This measure should improve as agents become more confident with the product
and systems. The global metric is 70-75%…
LITERATURE REVIEWS — Telephone Service 238
Taxpayer
Attitudes
Penalty
Study
Settlement
Initiatives
Telephone
Service
Taxpayer
Engagement
Use of IRS
Services
OIC
Program
EITC
Letters
Collection
Agencies
Quality assurance measures quality of calls and is based on a set of criteria which the agent must
cover during the call. Including, but not limited to:
■
How the agent answers the call
■
How they navigate the caller to a resolution
■
How they end the call.”
19. Jeff Rumburg, MetricNet, Metric of the Month: First Contact Resolution (2011),
http://www.thinkhdi.com/~/media/HDICorp/Files/Library-Archive/Insider%20Articles/
First%20Contact%20Resolution.pdf.
“FCR is typically measured in one of two ways: (1) the service desk agent checks a box on the
trouble ticket at the conclusion of the call or chat session to indicate that the call was successfully
concluded on the initial contact, or (2) customers are asked in follow-up customer satisfaction
surveys whether their calls were resolved and concluded on the initial contact with the service
desk. The first method requires periodic audits to ensure that agents are accurately reporting
FCR on the tickets they handle. This is done by reviewing a representative sample of tickets each
month to determine whether the tickets designated FCR by an agent are, in fact, being resolved on
the first contact with the customer. Neither method of measuring FCR is perfect, but it is one of
the most important KPIs to track and trend.”
20. Jeff Rumburg & Eric Zbikowski, MetricNet, The Five Most Important KPI’s for the Call Center
(Feb. 20, 2013), https://ccng.com/uploads/five_most_important_kpis_for_the_call_center_
metricnet.pdf.
“The two ‘foundation metrics’ that every call center should track on an ongoing basis are Cost
per Call and Customer Satisfaction. The next two metrics in the top five are the ones that have
the greatest influence on cost and customer satisfaction: Agent Utilization and First Contact
Resolution.” (at 12-13).
“Customer satisfaction is affected by a whole range of other performance variables, including
Average Speed of Answer (ASA), Call Quality, and Handle Time, to name just a few. But the
single biggest driver of customer satisfaction — by far — is FCR.” (at 5).
21. John Bruno & Kate Leggett, Forrester Research, Define the Right Metrics for Tracking CRM
Success 2, 11 (Sept. 20, 2017).
“Voice of the customer (VoC) programs are now widespread. Most execs have undertaken
new strategies to improve their company’s customer experience [citation omitted]. And because
companies need accurate customer insight to provide effective customer experiences, it’s no
surprise that VoC programs are gaining momentum. According to a Forrester survey, 84% of
surveyed companies reported that they ask customers for feedback about past experiences on a
regular basis [citation omitted].”
“Measure what is important to customers. Too many companies spend money on the wrong
things — or actively annoy customers — because they never bother to ask what would help their
customers. Consider using customer surveys or adding more-detailed customer preference options
to let customers specify what they like and don’t like, how they want you to contact them, and
Taxpayer Advocate Service — 2017 Annual Report to Congress — Volume Two 239
EITC
Letters
Collection
Agencies
OIC
Program
Taxpayer
Attitudes
Penalty
Study
Settlement
Initiatives
Telephone
Service
Taxpayer
Engagement
Use of IRS
Services
what is important to them. Start with one-on-one phone surveys or focus groups to home in on
key issues and then automate for broader input and validation…
Use metrics to guide employees. Clear metrics help employees know what ‘success’ looks like.
A telecommunications firm uses five key metrics for service reps: 1) number of repeat calls; 2)
number of topics discussed per contact; 3) hold time; 4) call time; and 5) customer satisfaction.
A customer management technology platform supports the reps, putting them in a position to
answer customer inquiries and respond to customer requests. Clear measures let them know how
well they are performing.”
22. John Horrigan, Pew Research Center, How Americans Get in Touch With Government (May 24,
2004), http://www.pewinternet.org/2004/05/24/how-americans-get-in-touch-with-government/
(last visited Dec. 5, 2017).
“The limits of e-gov have to do with people’s technological assets, preferences, and the wide range
of problems people bring to government. Not everyone has Internet access — about one third of
American adults do not — which means that phone calls, in-person visits, or letter-writing are the
available options to contact government for a large share of the population. Many people simply
do not choose to use the Web or email to get in touch with government. In fact, those who contact
government, including Internet users, are most likely to say they prefer to use the telephone to do
so.”
‘In sum, e-gov is a helpful tool among several options for reaching out to government, but it is by
no means the ‘killer app’ among them. People’s problem-solving capacities matter to successful
outcomes with government, not a specific technology. As policymakers expand the offerings of
e-government, they would be wise also to maintain and upgrade the other channels that citizens
use to contact government, with a focus on how these traditional channels may complement e-gov
applications.’
23. Kate Leggett, Forrester Research, Trends 2016: The Future of Customer Service (Jan. 5, 2016).
“Companies must make it easy for customers to get service over the touchpoint and channel of
their choice at any point in their journey.” (at 3).
“In 2016, organizations … will learn to better route a customer to an agent who can most
effectively answer a question based on skills and behavior data. They will better understand
customer call patterns and preempt future calls.” (at 7).
“In 2016, organizations will begin to equip agents with simpler, more modern, automated, task-
based user interfaces to increase their productivity. These improvements will allow agents to focus
on solving the customer’s issues and delivering differentiated service instead of struggling with
their tool set. Organizations will also empower agents to use their knowledge and experience to
make better decisions for customers by beginning a shift from efficiency success metrics to ones
that highlight customer satisfaction and customer lifetime value.” (at 9).
LITERATURE REVIEWS — Telephone Service 240
Taxpayer
Attitudes
Penalty
Study
Settlement
Initiatives
Telephone
Service
Taxpayer
Engagement
Use of IRS
Services
OIC
Program
EITC
Letters
Collection
Agencies
24. Mike Desmarais, Service Quality Measurement Group, First Call Resolution: The Call Center
Measure that Matters Most (Dec. 2005), http://grandsorganismes.gouv.qc.ca/fileadmin/Fichiers/
Veilles%20strat%C3%A9giques/Gestion%20des%20CRC/2005-first_call_resolution.pdf.
“Based on [Service Quality Measurement Group’s] SQM’s 2005 benchmarking study of over
250 leading North American call centers on 150 traditional operational measures, 55 customer
satisfaction measures and 50 employees satisfaction measures, we have concluded that FCR — is
the measure that matters most for operating an effective and efficient call center.”
“Put simply — just because a call is handled quickly does not mean that it is handled efficiently,
especially if that customer has to call back in because the information was incomplete.”
“Achieving 80% or higher FCR rating typically results in an average of 1.2 calls needed to resolve
customer’s inquiry or problem.”
25. Monica Anderson, Pew Research Center, Digital Divide Persists Even as Lower-Income Americans
Make Gains in Tech Adoption (Mar. 22, 2017), http://www.pewresearch.org/fact-tank/2017/03/22/
digital-divide-persists-even-as-lower-income-americans-make-gains-in-tech-adoption/ (last visited
Dec. 5, 2017).
“Nearly 30 years after the debut of the World Wide Web, internet use, broadband adoption and
smartphone ownership have grown rapidly for all Americans — including those who are less well
off financially. But even as many aspects of the digital divide have narrowed over time, the digital
lives of lower- and higher-income Americans remain markedly different.
Roughly three-in-ten adults with household incomes below $30,000 a year don’t own a
smartphone. Nearly half don’t have home broadband services or a traditional computer. And a
majority of lower-income Americans are not tablet owners. By comparison, many of these devices
are nearly ubiquitous among adults from households earning $100,000 or more a year.”
26. North American Quitline Consortium, Call Center Metrics: Best Practices in Performance
Measurement and Management to Maximize Quitline Efficiency and Quality (2010).
“The service provider should develop clear standards for the call-handling process and train
staff to meet them. Call-handling process data should be regularly reviewed at all levels of the
organization to ensure that call center agents are meeting the standards. The service provider
should adopt an evaluation strategy, such as customer surveys, to obtain qualitative information on
the intake process from the perspective of callers.” (at 14).
“There are many different ways to listen and get customer feedback. The three primary ways are:
■
Customer surveys
■
Customer praise and complaints
■
Observation of customer interactions.” (at 6).
“Regular employee-satisfaction surveys can provide valuable insight into the current perceptions
of the staff. Surveying should be a routine process that allows and encourages each employee to
assess immediate supervisors, the management team, work environment, training, job resources,
career opportunities, and so on. If there is dissatisfaction with some recent change or process, it is
Taxpayer Advocate Service — 2017 Annual Report to Congress — Volume Two 241
EITC
Letters
Collection
Agencies
OIC
Program
Taxpayer
Attitudes
Penalty
Study
Settlement
Initiatives
Telephone
Service
Taxpayer
Engagement
Use of IRS
Services
important to find that out as quickly as possible since such dissatisfaction may affect how the agent
performs on the phone.” (at 7).
FIGURE 1 (at 19)
Performance
Category
Performance Measures for
Frontline Staff
Quantitative
Measure
Qualitative
Measure
Service
Schedule adherence X
Quality
First-call resolution rate X X
Transfer rate X X
Telephone etiquette X
Competency/knowledge X
Error/rework rate X X
Process adherence X
Efficiency
Schedule adherence X
Availability X
Average Handle Time X
After-Call Work time X
On-hold time X
Conversion rate X X
27. The Northridge Group, State of Customer Experience 2017, 2, 14 (2017).
“Just as customers are becoming more digitally savvy, so too must the companies that serve them.
Our data shows that customers want to be met on their channel of choice and have little tolerance
for multiple contacts, long hold times, slow responses and ineffective issue resolution. While
digital self-service channels allow customers to be more self sufficient, consumers still want direct,
personal assistance for more urgent matters.”
“Interactive Voice Response (IVR) systems can procure identifying information from callers
which allows agents to pull up a customer profile. Websites and mobile apps can track historical
information to allow users to see details about their account. By equipping agents with as much
information as possible, they’ll have a more complete understanding of previous interactions and
potential issues, in addition to providing a faster, more accurate solution.”
28. Oracle, Eight Steps to Great Customer Experience for Government Agencies, 6-7 (March 2012).
“Extended subject matter knowledge. Agency employees need to be able to access the same
knowledge that an agency makes available to its customers via online self-service. This ensures
that customers get consistent, accurate, and up-to-date answers regardless of how they contact the
agency. Employees can also be provided with additional knowledge that may not be appropriate for
online posting but is essential for serving customers effectively.”
“Interaction context. Frontline employees can also better serve customers if they’re provided
with knowledge about the context of customer interactions. This is achieved when employees can
view any recent previous interactions with the customer. Thus, for example, if a customer has
LITERATURE REVIEWS — Telephone Service 242
Taxpayer
Attitudes
Penalty
Study
Settlement
Initiatives
Telephone
Service
Taxpayer
Engagement
Use of IRS
Services
OIC
Program
EITC
Letters
Collection
Agencies
already spoken to other agency employees about the issue at hand, the employee can take that
into account—and avoid asking the same questions. This shortens calls and saves customers the
frustration of having to go over the same information again and again.”
29. Peggy Carlaw, Call Center Best Practices, CustomerThink (May 11, 2012),
http://customerthink.com/call_center_best_practices/ (last visited Dec. 5, 2017).
“There are other reasons why giving feedback is so important. It shows your staff that you’re
on top of things, that you’re keeping yourself informed, and that you’re dedicated to a course of
continual improvement. Call center employees who receive ongoing feedback are more engaged
in their job, and more engaged employees create more satisfied customers. What’s more, this call
center best practice shows your staff that you care about them, about their performance, about the
customer, about service levels, and about running a world-class call center.”
“The goal of your call center is to help your organization meet its business goals. Metrics measure
how well you’ve done that. Look at metrics related to quality (call quality, data-entry quality, fix
quality, customer satisfaction and loyalty, etc.) as well as metrics related to quantity (average speed
of answer, number of escalations or transfers, the time it takes to resolve the customer’s issue, etc.).
The goal is to create the highest customer loyalty at the lowest cost.”
30. Rachel Miller, Impact Learning Systems, No Time to Coach, 2 (September 2011).
“[C]orrecting is a very important part of coaching. It is not easy to tell someone that they did
something wrong, but avoiding such issues will affect team productivity and morale. Use the
following acronym as an easy reminder for effective correcting.
B = Behavior. Clearly point out the incorrect behavior.
E = Effect. Explain the effect of the incorrect behavior had on the customer, call, etc.
E = Expectation. Clearly state what you expect employee to do differently next time.
S = Secure Commitment. Secure commitment from the employee to try what you’ve
asked.”
31. Richard A. Feinberg et al., Operational Determinants of Caller Satisfaction in the Call Center, 11
Int’l J. of Service Industry Mgm’t. 131, 140 (2000).
“No amount of measuring speed of answer, abandonment rates, etc. can take the place of knowing,
tracking, and strategically considering how satisfied the callers were with their experience and
outcome in the call center experience. This study shows that resolving the problem and answering
the question the first time the customer calls and making certain that the customer gets through
the first time are the two essential elements of the call center operation.”
32. Scott Keeter et al., Pew Research Center, From the Telephone to the Web: The Challenge of Mode of
Interview Effects in Public Opinion Polls, 2 (May 13, 2015).
“Using its nationally representative American Trends Panel, Pew Research Center conducted a
large-scale experiment that tested the effects of the mode of survey interview — in this case, a
telephone survey with an interviewer vs. a self-administered survey on the Web — on results from
Taxpayer Advocate Service — 2017 Annual Report to Congress — Volume Two 243
EITC
Letters
Collection
Agencies
OIC
Program
Taxpayer
Attitudes
Penalty
Study
Settlement
Initiatives
Telephone
Service
Taxpayer
Engagement
Use of IRS
Services
a set of 60 questions like those commonly asked by the center’s research programs. This report
describes the effort to catalog and evaluate mode effects in public opinion surveys.”
33. Terrel Bird, Measuring the Success of a Call Center: Easier than You Think, Int’l Customer
Mgm’t. Inst. (Jul. 13, 2016), http://www.icmi.com/Resources/Metrics/2016/07/Measuring-the-
Success-of-a-Call-Center-Easier-than-You-Think (last visited Dec. 5, 2017).
“Some of the most common and insightful call center metrics that help us to measure overall
performance include average call duration, call quality, customer satisfaction score, escalation rate,
first call resolution, resolution time and speech analytics.”
25
“Resolution Time measures the amount of time it takes for an agent to successfully resolve a
customer issue. Typically measured in hours and/or days, resolution time is frequently tied to
customer satisfaction because a low resolution time shows that agents are making proper solutions
and communicating them to customers properly.”
25
Escalation rate “measures the frequency that an agent has to send customer calls to higher-ranking agents for resolution.”
Terrel Bird, Measuring the Success of a Call Center: Easier than You Think, int’l CustomeR mgm’t inst. (Jul. 13, 2016),
http://www.icmi.com/Resources/Metrics/2016/07/Measuring-the-Success-of-a-Call-Center-Easier-than-You-Think (last
visited Dec. 5, 2017).
LITERATURE REVIEWS — Telephone Service 244
Taxpayer
Attitudes
Penalty
Study
Settlement
Initiatives
Telephone
Service
Taxpayer
Engagement
Use of IRS
Services
OIC
Program
EITC
Letters
Collection
Agencies
This page intentionally left blank.
