Every Team Deserves a Krew: The Platform Behind Docusign's 220+ AI Teammates

Abhishek Chaturvedi opens in a new tab, Senior Principal Engineer, and Nabeel Shahzad, opens in a new tab Director of Engineering, also contributed to this blog.

Every morning, an agent scans Docusign's error logs looking for patterns before our engineers wake up. Another checks every new feature flag rollout for the right environment and test coverage. A third reads every new pull request the moment it opens and flags the risky ones, the kind that touch a public API or 20 files at once, before a human reviewer even looks.

These are all Docusign Krewmates, internal agents that anyone at Docusign can build to take on the repetitive, manual work every engineer knows too well. There are more than 220 of them, built in just the past few months by our engineers, and already returning thousands of hours of engineering time to higher-value work. 

In this post, we’ll look at why we built Docusign Krew, the agent orchestration platform enabling individual Krewmates, how it works, and what it's already doing across the org.

Engineering glue work nobody wants

Engineering teams are often slowed down by tedious and repetitive tasks. Digging through logs to confirm an error is real, checking whether a change breaks a downstream service, following the same review checklist for the hundredth time. Docusign engineers were spending real hours doing this kind of work every week.

As we’ve written about previously, our peer-led AI adoption journey drove the creation of an internal AI coding agent that engineers used for repetitive coding tasks. It's been invaluable, but general purpose: give it a task, it does the task. There was nothing that could watch for a signal, decide something needed attention, and kick off a workflow on its own. That gap is what Krew was built to close.

What we built: no code, no tickets, no waiting

Krew is the platform. A Krewmate is an individual agent built on it. Anyone can create one with zero code or infra setup. Simply provide a system prompt and your own skill files, select from a suite of built-in tools (like GitHub, Jira, Slack, Confluence, Azure DevOps, Grafana, Kubectl, telemetry, browser, memory, or webhooks), and set a trigger, such as a schedule, a native platform event, or a Slack workflow.

The idea started just a few months ago at an internal Docusign Hackathon with a simple question: could we connect an entire workflow, from a signal in our telemetry to an investigation to a fix to a review, instead of automating each step separately.

So we built Krew, a distributed agent platform that orchestrates work across multiple layers:

• Triggers. A scheduled job handles routine work, like a Krewmate that checks telemetry every morning without anyone asking. Everything else runs through an always-on API: a GitHub pull request, a new Jira ticket, or a Slack workflow calling in can all invoke a Krewmate directly.

• Orchestrator. The API gateway validates each incoming request and queues it for processing. The agent orchestrator picks up that work and routes it to the right agent.

• Execution. The agent runtime loads the agent's definition, then hands the task to whichever LLM is configured. The model decides which tools to call: read code, query logs, open a pull request, post to Slack. Tools run in an isolated, sandboxed workspace spun up specifically for that execution.

• Resultspersist. A pull request gets opened, a comment gets posted, a ticket gets updated, a report gets generated — and any state or memory the agent needs for next time gets saved.

This architecture is designed for: 

• Parallelism: Multiple agents run concurrently, managed by CosmosDB distributed state. 

• Reliability: Failed tasks auto-retry, failed agents auto-cleanup and state survives restarts. 

• Auditability: Every run, every tool call, every decision is logged.

None of that required unusual technology. What took real engineering time was making every Krewmate fluent in Docusign: knowing which of our systems to check, which of our own tools to call, which LLM models we can route agents to, and which actions our AI principles allow without a person signing off first. 

Krew services run on the same infrastructure as our AI coding agent, which turned out to matter more than we expected. When a Krewmate needs code written rather than just information gathered, it hands that piece off to the coding agent instead of generating code itself. That keeps Krew from having to run its own, more expensive code-generation loop for something our coding agent already does well.

Krew's architecture layers

What Krewmates actually do

In short, whatever our engineers want them to do. Some examples of real Krewmates built by Docusign engineers:

• Night Watch. The very first Krewmate the team built, and still one of the simplest, scans telemetry for new error patterns and files a task for another agent to pick up rather than fixing anything itself. That handoff shows up again and again across the Krewmates built since: something gets detected – whether it's an exception, a stale PR, an orphaned file, or even a Slack request – and gets turned into a task instead of one agent trying to do everything end to end.

• The feature flag reviewers. Feature flags allow engineering teams to safely roll out new capabilities, but they require rigorous validations and checks before release. The first team to build a Krewmate outside the group that created the platform was our eSign team, automating the manual check every feature flag rollout requires: right environment, metrics and test coverage with no risk to the customer workflows. 

• The on-call handoff. This Krewmate sits at the handoff boundary across global on-call rotations in India, Europe, and the U.S., reviewing the last twelve hours across two GitHub repos. Every merged change gets tagged by risk and environment, every open incident comes with its current hypothesis and mitigation, and anything still worth watching gets called out. It posts straight to the on-call channel, capped at 200 words by design, so the incoming engineer starts the shift already caught up instead of piecing it together from Slack scrollback.

• The PR risk checker. The agent reads every pull request the moment it opens and applies a simple heuristic: touch a public API contract, change more than a handful of files, or leave the description blank, and it gets flagged as high risk with a comment explaining why. Touch a readme and nothing else, and it's left alone. The agent doesn't decide anything, it just makes sure the right pull requests get a closer look before they're merged. 

Other Krewmates handle Jira cleanup, monthly ops reporting, traffic anomaly detection, and weekly incident summaries, different problems, the same underlying pattern: investigate, prepare, hand off to a person.

What Krewmates do

Humans stay in the loop

Docusign's AI principles require a human in the loop, and the Krew platform is built accordingly. A Krewmate can investigate, prepare a report, open a pull request, or flag an issue. But it can't merge a pull request or take an irreversible action without a person deciding to let it. Tools in the Krew toolbox that can take higher-risk actions carry an explicit warning before anyone can use them. 

This is intentional, and it’s the same principle behind every AI tool we've shipped, internally and externally in our AI platform.

Here's a quick video on how to create a new Krewmate:

What we learned building Krew

• Adoption was organic, not forced. Night Watch was the platform team's own first Krewmate, built to prove the concept could work. The first Krewmate anyone else built was our eSign team’s feature flag reviewer, and nobody asked them to build it. That's still the pattern. Teams adopt it when they hit that same kind of tedious, repeatable work, not because of a rollout plan.

• Real workflows reveal better platform requirements. Once engineers applied Krew to their own daily work, the requests followed fast. More tools, multi-repo triggers, richer Slack integration, deeper GitHub access, shared memory, workflow chaining. None of it came from a roadmap, it came from people hitting the platform's actual limits.

• Reuse beats rebuilding. Our telemetry integration got easier once we reused patterns from existing incident-investigation tooling instead of building from scratch. Azure DevOps has gone the other way. There was no existing pattern to borrow from, and it's still the integration we're working hardest on.

• A "successful" run isn't the same as a correct one. Krewmates would report success while quietly leaving duplicate Slack posts, partial tool outputs, or a session that died mid-task. That gap pushed us toward stronger run state, idempotent side effects, and real end-to-end observability, not just a log that says a job finished.

Making Krew work wasn't just about building a generic agent platform. The real differentiator is how deeply Krew integrates into the Docusign ecosystem. Because it embeds natively into our engineering workflows with numerous tool integrations, teams can safely build and share their own Krewmates.

Early impact, what’s next

Krew is only a few months old and the platform is gaining real traction among our engineers. More than 220 unique Krewmates now exist across Docusign engineering, together running more than 28,000 executions, or roughly 2,500 a week. Thousands of hours of engineering time already returned to higher-value work, and counting.

Every execution gets judged, not just logged. An LLM reviews each Krewmate's run, looking at the prompt and the complexity of what it actually did, and uses the most conservative estimate of how much time it saved a skilled engineer. Early signals from our engineers suggest those estimates track with reality: the top Krewmates are saving roughly 1-2 hours of engineering time per execution, time that goes back to harder problems only a software engineer can solve.

Expanding our integrations. We want to keep upgrading our integrations to broaden where Krewmates can be put to work, including with our incident management platforms and with the tools our product and engineering teams operate regularly.

Krewmates can work as a team. Multiple Krewmates can be chained together within a shared memory scope, one agent finishing a task and handing its output to a memory store the next agent picks up from, rather than every Krewmate working in isolation. 

Identity runs through Okta. Every Krewmate is still visible to everyone using the platform today, but access to Krew itself runs through Okta rather than something more ad hoc, laying the groundwork for splitting Krewmates into private and public going forward.

More companies will keep building their own version of this. Ours just happens to already know Docusign, and it's only getting more fluent. Are you building something similar at your own company? I'd love to compare notes, connect with me on LinkedIn opens in a new tab.