Case Study - Banking

Introducing new types of technology into large organizations is a nice meaty challenge for many reasons. Large organizations are full of smart people, lots of good ideas, and many different realities of how the organization does and should function. This tension is excellent soil for growing great ideas if you know how to tend to the garden.

Situation:
I was the head of Cognitive Experience Design for an AI product company based in NYC. Our company had sold our conversational AI product to the wealth management function of an international bank. The bank bought our product with plans to execute the top tasks of the customer service function in a question/answer format. The highest volume ticket was a simple update request and on paper, this makes perfect sense. A machine can easily read the service database notes and provide as much information as any human service agent. However, none of the leaders could explain why a human agent couldn’t resolve these tickets with ease. The average ticket had a really late close rate in comparison to other ticket types with more complexity. My 4 person team of 2 designers, a linguist, and a solution engineer got to work on the design brief but I pushed for one extra week of discovery research to really understand why these tickets took human agents so long to close. The best deal I could make is that we’d do the research for free if the bank granted my team access to the service floor. In my experience any business looks vastly different between executives and service employees.

Task:
To prepare for the research trip, I came up with a three pronged approach. We did interviews, ethnographic observation, and left the last two days open to move in the direction we needed to based on what we learned on days 1-3. My plan was to have a prototype ready for day 5 to justify the trip but it was a gamble because we had no evidence to support this as a possibility. I also had to think about the confidence and skill level of the members of my team. We were working outside of certainty so the best we could do was practice. We did mock interviews, we brushed up on how to synthesize quickly, and we got crystal clear on who was responsible for what.

Actions:
It took sitting in on 3 interviews to figure out the problem. On the service side of things, we asked questions like “Tell me about your day yesterday” and “Describe the last time you had to escalate a ticket”. On the wealth side, we asked about the current pending number of tickets and hacks to get things done quickly. Within the first 3 meetings I could tell it was a trust issue. The associates and wealth managers had no visibility into the status of their tickets and even worried that a ticket had not been properly submitted. It was a consistent issue across small to large effort tasks and the current system had zero transparency. Our solution was to develop an interface that extracted information quickly, allowed associates and managers to view progress which meant new tickets weren’t opened up for the same task. We also developed a plug in for email and text so that managers could initiate tickets and check their progress from their phone. This population worked outside of the office more than 60% of the time and we learned that they could only access the ticketing system from their laptops in the past.

Results:
By the next quarter, the department service statistics looked much different. The average ticket lifespan decreased by 48%! I’d like to say we figured out how to get the two functions to trust one another, but I do think we helped create a blueprint for cultural change. We invited the service team and the wealth team to participate in the design process and more importantly, the pilot tests. The participation rate was low but the insights were major with participation going all the way up to the global managing director. The two groups found that there were big misunderstandings about what happens during an international wire transfer, for example. This led to meaningful updates to the new hire training experiences, which my company was hired to work on as a bode of trust in our conversational AI product. Wealth managers who didn’t participate probably never would but we considered it a victory to see their time spent on service tickets drop dramatically to be reinvested into more rewarding types of work as reflected in the improved job satisfaction scores and noted in the performance review comments, measured 6 months after our product was launched.