Lessons from Twilio: How To Build Developer Loyalty with Docs and Dev Ed

This post by François Dufour is part of our series on Product-Led Growth Playbooks, an emerging field that pushes the boundaries of conventional sales, marketing, and customer success. In the series, we share insights and advice from leaders who have built successful PLG businesses and offer specific playbooks for founders running high-growth software startups targeting software development teams today.

Andrew Baker reads the docs. As Director of Developer Education at Twilio, it’s Andrew’s job to make sure developers are well-equipped to build whatever they have in mind with Twilio’s APIs. That means keeping an eagle-eye view on the state of the docs as a whole, and a narrow focus on how individual pieces of content are performing. To do just that, Andrew uses equal parts developer empathy, documentation automation, and communication with the developer community he serves. He was kind enough to take a moment to speak with me about how the Developer Education team came together and how they scaled the quality and quantity of Twilio’s docs.

The Types of Developer Documentation Twilio Creates

“Developer documentation doesn’t refer to a single piece of content. Or just to “docs”. It’s more a category of content, geared to help developers at various stages of their journey.”

While an API reference doc might give a developer a general sense of what your API can do, a tutorial shows a developer how to build a particular application step by step. Both types of content are classified as developer documentation but are built to serve different purposes.

When Andrew’s team writes content, they clearly spell out every API library, feature, and dependency they are using so a developer can easily follow along. That principle applies to every type of content the Developer Education team ships:

• API Reference - This content is very detailed, highly specific, and serves to show a developer what they can do with a given API or API feature.
• Quickstarts - This content is meant to get a developer from sign up to Hello World! in the shortest amount of time possible. It assumes that the audience is impatient and simply wants to start coding. Twilio’s SMS Quickstart, for example, is geared to give developers confidence in building with Twilio and demonstrate the value of the platform in just a few minutes.

• Docs - This is the most important piece of content in the Dev Ed team’s eyes. A product’s docs page, like SMS, should be built for developer productivity. It should clearly show what you can build with that particular product, and outline a series of steps to get started using high-quality code and intuitive navigation.

• Tutorials - Feature full sample apps as well as snippets of code to show advanced features. This documentation gives developers a much more contextual, deep experience to show just what they can do with a product.

Dev Ed’s Expanding Scope: from Docs to Training and Demos

“Twilio’s Developer Education team now focuses on everything from API reference docs, to Quickstarts, Tutorials, and interactive training courses like Twilio Quest to give developers the content they need to bring their ideas to life, as well as Demos.”

• Documentation - Equip developers with the narrative framework, code, and lessons they need to be successful with your product.
• Interactive Education  - Twilio Quest isn’t just an online 8-bit educational adventure into Twilio, it’s also used in Superclass, an in-person event.  Using a mix of in-person and online forms of education, Twilio’s Dev Ed team can meet engineers where they are, whether that’s at a conference, meetup, or talk, or simply online. The program lets the Dev Ed team serve engineers regardless of their skill level, stack, or the problem they’re trying to solve.
• Demos  - The five-minute demo is a hallmark of Twilio’s Dev Ed and Developer Evangelism teams. It allows Twilio to demonstrate the power of their product using one of the most powerful storytelling devices for developers — code. By telling the story of a specific project or hack, then showing it in action, Twilio’s Dev Ed team shows domain expertise and developer empathy all at once.

‍

How Twilio Measures Success in Developer Documentation

“Docs are like air. It’s impossible for a developer to be successful without them,” says Baker.

Baker found that Google Analytics metrics of traffic levels, and time on page, while important, didn’t tell the whole story of how developers were experiencing Twilio’s docs. So, he built a feedback system that the Developer Education team uses to improve and iterate on their docs at scale. Here’s how it works:

• The Five Star Widget - Embedded in every Twilio docs page is a widget that gives developers the ability to rate a piece of content from 1 star to 5 stars. After they submit their rating, developers will also be asked for any qualitative feedback they’d like to include.

• The Slack Channel - Once a developer submits a review, their star rating and comments are automatically sent to a Slack channel that the Dev Ed team monitors. From there, the Dev Ed team can follow up with that developer, if they’ve provided contact information, to understand more about their rating, what they were working on, and what Twilio can improve.

• The Follow Up - There’s no substitute for one-to-one interactions with developers. Whether the Dev Ed team is talking to a developer over email, at an event, or on the phone, a key to their success is keeping their finger on the pulse of the developer community to ship technical content before they need it, not after.

“Talking with developers allows the team to identify the questions that will typically become the topic of a blog post or quickstart.”

Navigating Trade-Offs as You Scale Documentation

The more technical documentation you publish, the more you’re equipping developers. But, all that equipping comes at a cost. As your proverbial content footprint grows, you have to build systems to scale and maintain the quality of that content. Here are a few lesson that Andrew Baker learned while scaling content:

• Automate Early - There’s nothing worse than an outdated API reference doc. To keep Twilio API reference documentation up to date across the platform, Andrew’s team uses open source tools, like Open API,  and frameworks that allow them to automate the production of API reference docs. This saves them a tremendous amount of time, eliminates the likelihood of human error, and ensures their docs are always up to date.
• Weigh The Benefits of SEO Against The Maintenance Cost - To attract new users who were more use-case minded, Twilio shipped Quickstarts and content that leveraged similar code snippets. For example, there was a non-trivial amount of shared SMS API code in quickstarts for Appointment Reminders, Delivery Alerts, and Broadcast Alerts. While crafting these Quickstarts in a way that was tailored to their audience was helpful for SEO purposes, it also left the Dev Ed team with more technical real estate to maintain and keep up to date. So, they ended up consolidating a few use cases where the code was similar.

“When you launch SEO-minded technical content, remember to keep in mind your responsibility to maintain it.”

Hiring for Technical Expertise and Developer Empathy

When Andrew hires, he looks for engineering aptitude first followed by writing skills second. That aptitude is actually an indicator of developer empathy. With a deeper understanding of engineering ins and outs, you can see the picture of what a developer needs in higher definition.

“We look for engineering aptitude first followed by writing skills second.”

That new hire might have also worked as an engineer and experienced the siren call of a pager going off in the middle of the night, alerting them to a bug that needs debugging stat. Those types of moments give a Developer Educator a first-hand understanding of how critical code is — it can mean the difference between a product peacefully performing or interrupting an engineer's sleep with an error message.

That sense of ownership and responsibility can show itself within the company as well. So much so that, when the Developer Education team at Twilio saw an opportunity to build a better developer experience, they didn’t wait for their ideal CMS to walk through the door with every feature they needed. They built their own CMS to retain granular control over developers’ experience with Twilio.

The Developer Education team also piloted Twilio’s Command Line Interface after seeing through docs feedback that they welcomed that possibility. A month after prototyping the CLI, it went into R&D, and is now in Twilio’s Developer Toolchain.

The possibility to delight developers is always there if you have the eyes to spot it and the sense of ownership to deliver on it.

‍