With a large global Engineering organization of over 1,000 software developers, Qualtrics' Developer Experience (DevX) team is responsible for delivering solutions and processes to those developers to improve their day-to-day happiness and productivity. DevX also runs programs to raise the bar on engineering quality at Qualtrics.

When generative AI and ChatGPT quickly rose in popularity in 2023, Qualtrics' DevX team immediately saw the potential to use AI within their development teams. The team took a primary initiative to evaluate AI coding assistants in the summer of 2023. Godwin Babu, Sr. Manager and leader of Qualtrics' DevX team, hypothesized that AI usage could significantly improve productivity within their engineering teams.

Qualtrics evaluated several AI solutions in the market before deciding on Cody, and security was a primary priority. Godwin stated, “When choosing a coding assistant, we prioritized DLP [data loss prevention] and assurance of the security of our intellectual property. Cody worked seamlessly with the systems we already had in place.”

Qualtrics' security protocols extend to their code host, which is why they run their own self-hosted GitLab instance. Cody works with all major code hosts, both self-hosted and cloud-based, so it could plug into Qualtrics' existing setup. Per Godwin, “We run our own GitLab instance within our own data centers, and Sourcegraph works seamlessly with it. That made signing up for Cody easy.”

DevX team also wanted a solution supporting the latest LLMs so their teams could always access the fastest and most accurate models. This led Qualtrics back to Cody, which supports several LLMs as options. “Cody's ability to switch backends, from Claude to GPT, is very attractive to us. This is a fast-moving field, and we see updates to LLMs constantly, so having a tool that can react to new LLMs quickly is important to us.”

Lastly, Qualtrics was already a customer of Sourcegraph with extensive usage of Code Search for their engineering teams. According to Gordon Fu, Quality Engineering Manager, Qualtrics' existing trust and confidence in Sourcegraph as a vendor led to Cody quickly bubbling to the top of their AI evaluation.

Qualtrics ultimately decided on Cody as their AI coding solution based on its security, LLM interoperability, and ability to work with their existing code host setup.

Since bringing Cody to the engineering organization, Qualtrics engineers report having to leave their IDE to find information on the web 28% less often when using Cody, and they can understand code 25% faster. The majority of their surveyed engineers also report between 10 and 30 minutes of time savings per day by using Cody, which the DevX team estimates to be roughly 10% of development time.

Looking forward, Godwin also sees places where Cody will drive even more progress across their engineering teams: “One of the places we're thinking of using Cody is connecting it with other initiatives. For example, in DevX, we have one program to improve code coverage: setting goals for all teams to achieve certain coverage with unit tests. We see a direct correlation with Cody here since Cody has proven adept for creating tests and improving code coverage.”

Brendan Doyle, Senior Software Engineer at Qualtrics, already sees value in writing unit tests with Cody. Brendan works on a team building a wrapper library for AWS Lambda and other internal Infrastructure as Code (IaC) services.

“It's not a secret for developers that writing unit tests is a necessary time sink in the day-to-day life of a developer for writing reliable software,” per Brendan. “Generating unit tests, by far, is my number one favorite feature and time saver for Cody. I can spend my brainpower somewhere else instead of spending it figuring out how to write tests.”

Brendan's team also gets value from using Cody to ramp up on big codebases and existing projects, especially for the younger engineers. “One of the most daunting things as a junior engineer is working on a large, existing codebase. There is always a ton of domain knowledge about that code that's restricted to the people who wrote it, no matter how well it's documented. There are always nuances that only the code authors know. But if developers know how to prompt Cody, Cody can find context and explain the code to them.”