Code Insights strategy

Mission

Code Insights helps engineering leaders, managers, and engineers answer questions about how their code is changing or what’s in their code that are difficult or impossible to answer otherwise.

We bring the best practices of big data to challenges of big code, turning a codebase into a queryable database.

We help engineering leaders set and measure quantitative engineering goals, whether they be related to a migration, deprecation, vulnerability, code smell, or high level understanding. We also help engineering leaders understand the shape of their code and how it changes over time. Then we help them tie trends and metrics to owners on their engineering teams.

We help individual engineers measure the day-to-day goals of their teams – how they are progressing on an individual initiative, how quickly other teams are adopting their code – as well as quantify tech debt problems to justify to leaders why certain work is important. We also help them get a historical usage view of functions, changes, or other patterns. Happily, we’ve heard that Code Insights has even helped some engineers get promotions or bonuses, because they use code insights to quantify the impact and spread of their work.

Strategy

To deliver on our mission, we will:

• First, build a robust Code Insights product offering that is configurable yet simple enough to answer many different engineering leaders’ specific questions
• Concurrently, continually track usage and feedback to grow the product in its most valuable directions
• Then, be intentional with how we publicly launch Code Insights to make sure this story is immediately clear
• Next, build towards a Cloud product offering after launching publicly to self-hosted customers
• Eventually, build new major features that help answer the second- and third-stage level of questions about a codebase

Vision

Guiding principles

Code Insights is built first for engineering leaders

The answer to “why are building Code Insights now, ahead of other cool projects Sourcegraph can pursue?” is that Code Insights serves the needs of engineering leaders.

Engineer leaders may not need to use Sourcegraph directly for code search (though their teams do), but they do need to answer questions about their code that Sourcegraph, via Code Insights, is well-positioned to serve.

What this looks like:

• If individual developers want feature A, and engineering leaders want feature B, we’ll likely build feature B first (but that does not mean we won’t later build feature A, especially as the product matures)
• As we collect feedback and case studies towards our GA launch, we are particularly focused on engineering leaders

What this does not look like:

• If engineering leaders want feature C, but feature C would be harmful to the experience of individual developers, we will not build feature C.

Code Insights layers together the Sourcegraph feature ecosystem

Code Insights naturally sits at the intersection of many other key features, and there are validated needs for, for example: customers to have Insights visualizing what they monitor with Code Monitors, Batch Changes to naturally flow into Code Insights for tracking progress and impact, Precise Code Intelligence that powers code insights for any specific symbol across all repositories.

As Sourcegraph becomes an increasingly powerful developer tool, Code Insights will be vital to tracking and understanding how both Sourcegraph and a company’s engineers impact a codebase.

What this looks like:

• Code Insights will be extensible and integrate with other Sourcegraph features

What this does not look like:

• We won’t add integrations just for the sake of integrations, without a validated need and use case

Code Insights is NOT for tracking individual developers

Code Insights is not for trying to track an individual’s output and whether or not someone is a “good” engineer or high performing employee. We believe it is low-signal to track metrics like like number of commits or number of lines of code an individual contributes.

What this looks like:

• We don’t build features that would primarily be used for evaluating individuals’ performance
• On a case by case basis, we might even intentionally build features to block our other features from being used for this purpose, if the need ever arises
• We won’t market or sell code insights under false pretenses that this is a valid use case for this feature

What this does not look like:

• We won’t prevent the positive use cases of individual tracking: if an individual wants to make a code insight demonstrating – for example – the spread of an API they built, they can

Code Insights is an enterprise quality professional tool

For Code Insights to be valuable, it must be precise, exact, and performant. When it comes to codebases, there’s little value in a measurement tool that is 90% correct, or within +/- 5%: our users need exact answers to inform decisions and track goals successfully. Code Insights also has to be performant enough that one can use it to answer questions when the need arises – our users can’t wait weeks for their data, nor can they “ration” use of Code Insights among a Sourcegraph instance.

What this looks like:

• We don’t compromise on precise accuracy; we test thoroughly, often, and at the extremes of possible use
• We often prioritize improvements to speed and database size above more user-facing tweaks

What this does not look like:

• Not every Code Insights feature will complete instantly – our developer audience understands the complexity of the tool and forgives some amount of loading/processing time

Code Insights builds on top of Sourcegraph’s other features

Users do not need to learn a new query language or format to use Code Insights. Nor will Code Insights introduce features that function differently than those same features in other areas of the product, confusing users. Code Insights is built atop Sourcegraph Search and will build with Sourcegraph’s other features rather than implement its own versions of any functionality.

What this looks like:

• If a Code Insights feature shares a name or purpose with another Sourcegraph feature, we’ll adopt the name and constraints of that feature
• If building a feature that might be reused across the Sourcegraph platform, Code Insights consults with other product engineering teams

What this does not look like:

• Code Insights won’t avoid introducing net new features if no other Sourcegraph feature has yet built them

Top customer, support, sales and marketing issues

• The biggest issues customers face, and that we’re addressing, are the yet-to-be-built product gaps. See the docs page for Current Limitations of Code Insights for an up-to-date list.
• Internally, the main task for Code Insights is determining the pricing and packaging

Competitive landscape

While there are many devtool products that use some version of the word “Insights,” and a handful of others that do high level information about code, there are no known direct competitors to Code Insights.

There are no other products that can efficiently and effectively provide the deep historical and realtime analysis that Code Insights can provide, largely because any product that wanted to compete with Sourcegraph Code Insights would first have to catch up to the product moat of Sourcegraph Search (which powers Code Insights), and then also build out a Code Insights-like product.

There may be one-off instances of companies that have glued together in-house solutions to answer some of the same problems, but we have already validated (thus far) every one of those companies would prefer to use Code Insights.

While Code Insights is in Beta, you can find a detailed breakdown of “similar” products in the original Product Document for Code Insights. At some moment pre-GA launch, the product and marketing team will create more standard competitive resources based on the GA product features and merge them into this page.

Strategy for FY22H2:

The next six months are focused on building a product that can quickly prove out its product-market fit via successful customer sales.

Goals

Themes (and What’s next and why)

What we’re not working on & why

In order to focus and have a successful initial launch, the Code Insights team is explicitly not working on any of the following until FY23:

• Code Insights is GA on Cloud: due to the multi-tenant nature of cloud, there’s additional backend work we’ll do in FY23 to make Insights GA on Cloud. Optimizations we plan to make in FY22H2 will help us quickly plan and support Code Insights on Cloud, but we won’t have time for that until FY23H1.
• Custom Insights Data Import: it’s worth exploring what importing data mixed with Code Insights can do, but we first need to stabilize and grow our core product features before adding that complexity. We’ll explore this in FY23H1.
• Custom Extension Insights: while some Sourcegraph extensions do and can contribute Code Insights, we’re not focused on third-party contributed insights because of the limited excitement initial users had for them. We may explore this in FY23H1.
• Integration with Batch changes: some customers want to visualize batch changes progress in Insights. While valuable, Batch changes is also new, so only a subset of customers will get value immediately and we’ve chosen to delay this work and focus on features that benefit all Code Insights users first.
• Integration with Precise Code Intelligence: though making insights for symbols is useful, the feature maturity of cross-repo code intel and the complexity involved is not yet worth the time cost for us to develop first.