Case Study - Leadership

IPsoft was founded in 1998 and found success in the creation of an automated IT service desk. The Amelia product is considered to be the world’s first chatbot although v1 launched with a command line interface. Amelia uses a combination of algorithms including machine learning, natural language processing (NLP), and most recently generative AI to facilitate a conversational interaction. The product was primarily designed for businesses to build and deploy virtual agents for customer and employee support with the ability to handle conversations across various channels, including phone, chat, and other touchpoints.

I was hired in 2018 as a conversational design lead to work as a practitioner/leader for a team comprised of about 12 designers who focused on user experience, linguistics, and cognitive technology for clients.

Situation:
I joined the design function at a time when the company and the Amelia product was in rapid expansion mode. The head of design had invested two years of hard work to create a baseline of belief in the value of design thinking and also define some big hairy goals for the performance of the product. The company was engineering focused with an emphasis on build fast, learn fast but design could help the teams figure out how to build better things through better insights. My team was spread out geographically with direct reports in NYC, Austin, Chicago, Norway, Germany, Spain, and India. Our clients were also spread out all over the world, over industries, and even the functional purpose our product served. Before I joined, the designers spent the majority of their time on enablement and worked pretty exclusively with engineers. They entered the project just before development and created mockups that resembled process maps, just as the engineers did who used to figure out the dialogue mapping before Design was introduced.

Task:
I was asked to focus on three things: (1) define and operationalize higher performing craft, (2) define and operationalize the process supporting client enablement, (3) evangelize design as a valuable tool for better commercial outcomes to the rest of the company, our clients, and the world.

Actions:
In my first 90 days, I focused on developing a deeper technical understanding of the product including NLP, AI/ML modeling, and large language models in general. I spent time in the classroom with the educators who taught our clients. I sat with R&D to understand the roadmap and the history of past failures, victories. I spent a lot of time being visible, building relationships, and listening to people all over the organization. By the end of the 90 days, I had my own mapping of the capabilities of the product against the skills and capacity of the team. I had formed first draft ideas about where we could hit the gas and where it made more sense to slow down. I focused on standardizing and skill development for the Design team.  

By 6 months, I had established clear expectations for each of the three roles with title changes to match the skillsets: user experience designers, conversational designers, and intent recognition designers. Everyone had design in their title, we had defined team norms, and a universal language with reinforcement of these concepts in how we planned work, solved problems, made decisions, and reviewed our progress as individuals and as a function. I replaced Net Promoter Scores (NPS) as the universal product metric because it’s self-reported nature wasn’t a reliable, repeatable data point. Instead, we started using the duration of the interaction, conversational turns between the machine and the customer, and lastly, the number of errors. We conducted audits of every product instance with our standardized metrics to establish clear expectations for user experience and some semblance of best practices. We authored a guide for Conversational AI to further socialize the perspective of the design team from a non-technical point of view.

The hardest part was yet to come— creating an environment for the psychological safety we needed to work in the ambiguous, uncharted territory that innovation requires. The creation of an innovation mindset and a healthy design culture relied heavily on three things:

• Time - with time came more opportunities to build understanding and conviction around mindset. When the moment presented itself, I could point directly at how this mindset benefitted our team and our projects.

• Role modeling - I saw big leaps in trust after I introduced ‘Creativity Workouts’ and a Slack channel for ‘Kudos’. The workouts boosted creative confidence through art and the Slack channel normalized cheering the risk takers 

• Formal mechanisms - designers made more money when they took risks. All designers presented their “best failure” at their review meeting along with a scorecard. This process was used to justify salary increases and title changes. 

As a leader, growing the craft required some creativity. I didn’t want to put anyone in a position to fail so I invented the opportunities we needed to apply the theoretical learning we did as a group. The biggest success was developing a concierge experience for the company. This was an experience that already existed for clients and prospects visiting an IPsoft office but what about the sales pipeline upstream starting with the company website? We spent 6 months developing a proof of concept for the homepage before we presented the experience to our Chief Commercial Officer. This project was the key change lever I needed to coach the team as individuals and in a group setting. Upon the success of this project, our value increased with the pre-sales engineering team, the R&D engineers developing the core product, and the sales/marketing department.

Results:
Into the second year, I now had the advantage of a confident, engaged team that trusted my leadership and trusted themselves, most of all. Our effectiveness as designers soared, team retention was solid, and commitment to growth came easy. By the 2019 performance cycle, my team had the highest median score of the entire company with perfect retention and 3 promotions that included title changes including my own promotion to Director. It was a proud moment in my career to see people and product succeed at the same rate.