Amplitude

Why aren’t we using Looker for this?

Looker is very flexible in that we have the ability to set it up for any purpose we’d like. The downside of this is that to provide a great self-service experience for questions about product usagee, this would take a LOT of work. We get these capabilities out of the box with Amplitude.

What is in Looker vs. Amplitude?

Anything not based directly on analyzing Sourcegraph Cloud events is in Looker. This includes pings from on-prem instances, anything we get from the Cloud Postgres database and any data from third-parties tools (such as Google Analytics and Salesforce).

Any analysis that was conducted in Looker prior to the implementation of Amplitude can still be done in Looker, and existing Looker dashboards and visualizations will still be maintained. Amplitude will help us conduct new and different analysis regarding product analytics.

Overview

Most Sourcegraph Cloud events are being sent to Amplitude. The events not being sent are extremely low traffic events, non-UI events (e.g. backend events for a search that we capture) or ones we have explicitly decided to exclude (such as code insights events because we’re focused on enterprise and Cloud data won’t inform any decisions). The full data map is in Drive.

The data is currently backfilled from 2021-10-01.

It’s sent through a script that runs every hour and pulls from sourcegraph_analytics.amplitude_events_v5, which is loaded by this scheduled query.

Adding event properties

Event properties are attributes of a particular event. These are added to the amplitude_events_v5 table as individual columns from this scheduled query. For example, here’s a snippet that extracts event properties from both the argument field of an event, as well as an event itself.

The steps to adding additional event properties to the query/table:

1. Create a PR to update the amplitude_events_v5 scheduled query with the new event properties.
2. Once another member of the team approves, open a blank BigQuery query editor, run the query, and then save the results to a new test table (e.g. amplitude_test_20210812).
3. Create an issue for Data Engineering to upload a sample of the new test table to a test project in Amplitude.
4. If the data shows up as expected in Amplitude, go ahead and update the amplitude_events_v5 query with your PR.
5. Update the INSERT statement with the new event properties. You can’t backfill event properties to events that already exist in Amplitude; however, you can backfill event properties if you’re backfilling events also.

If you’re adding a new event property, please add it to the data map.

JSON_EXTRACT(argument,
  '$.code_search.results.results_count') AS results_count,
JSON_EXTRACT(argument,
  '$.code_search.results.alert') AS alert,
CASE
  WHEN name = 'ViewRepogroup:python' THEN 'python'
  WHEN name = 'ViewRepogroup:golang' THEN 'golang'
  WHEN name = 'ViewRepogroup:android' THEN 'android'
  WHEN name = 'ViewRepogroup:kubernetes' THEN 'kubernetes'
  WHEN name = 'ViewRepogroup:kubernetes' THEN 'kubernetes'
  WHEN name = 'ViewRepogroup:kubernetes' THEN 'kubernetes'
  WHEN name = 'ViewRepogroup:stanford' THEN 'stanford'
  WHEN name = 'ViewRepogroup:cncf' THEN 'cncf'
ELSE
NULL
END
repogroup_name,

Adding user properties

User properties are the attributes of individual users. The amplitude_user_characteristics table defines all user properties, and this table is joined in with every event that is passed to Amplitude. These user properties include every A/B test from ab_test_users. See more in the A/B testing doc (coming soon).

If an additional user property is added to this table, it will only be applied to events and users going forward. User properties are sent to Amplitude along with events, so new properties will only be added when a user triggers an event that is sent to Amplitude. When adding a user property, please add it to the data map

Adding events to Amplitude

Amplitude is built on top of our existing eventLogger infrastructure, so we consider an event to be anything logged by this. We use object action framework in Proper Case for our naming. For example, in eventLogger this would show up as ‘SearchSubmitted’, and in Amplitude it’s converted to ‘Search Submitted’.

All events from eventLogger are sent to Amplitude except if explicitly added to a denylist in the scheduled query.

How do I find what an event is named in Sourcegraph’s code?

Most times you can search in the ‘Select event’ dropdown and you’ll find the event you’re looking for. If you can’t, or are unsure what the Amplitude event corresponds to, keep reading!

To figure out an event’s name is in code or to confirm whether an existing event in Amplitude is the correct one, follow these steps (or watch this video) :

To find what a button, page view or action is named

1. Open Safari
2. Enable developer mode (preferences -> advanced)
3. Go to sourcegraph.com
4. Show the JavaScript console
5. Enter localStorage.eventLogDebug = "true"
6. Start using Sourcegraph and click/view the event you’re looking for!

Where should I save charts?

There are three levels to where a chart can be saved.

1. To create a private chart, uncheck ‘Make Discoverable’ in the chart’s dropdown (“More”).
2. To create a chart that is discoverable by other teammates but not as easily accessible unless they search (or you send them a link), just create a chart. This is Amplitude’s default.
3. Add charts to Team Spaces if you want them to be easily accessible to teammates.

What are Team Spaces?

Team Spaces are a collection of charts, dashboards and notebooks that are relevant to a specific group (Amplitude’s version of a folder system). By joining one, you’ll be notified of every new analysis that a teammate adds to the space (and they’ll be notified of yours, as well). You can see all of our Team Spaces here.