Data configuration

What metrics does Multitudes provide?

We go into detail on our metrics here: What We Measure and Why

What insights does Multitudes provide and how are these calculated?

In addition to helping you visualize key metrics, we also share insights about those metrics. Insights are made up of the following:

  • Insight text: The message at the top of a chart, describing the metric’s value and trend
  • Insight value: Where the metric is at currently
  • Insight trend: Whether the overall trend over the date range has been up, down, or flat over the selected date range. This is indicated by the arrow. The arrow color changes depending on whether the trend is towards the target or away from it (for example, an upward trend for Merge Frequency will be green because more merges are generally better)
  • Insight status: A combination of the insight value, trend, and the metric’s target. This is represented by thumb icons (see the top right corner of each metric card in the Trend Summary, or throughout the At a glance section):
    • red thumbs down icon

      The insight value is outside of the target and not getting closer to it

    • yellow sideway thumbs icon

      The insight value is outside of the target zone but trending towards it, or within the target zone but close to the threshold

    • green thumbs up icon

      The insight value is well within the target

Any changes to custom targets will immediately affect the insights (text, value, trend, and status) and Actions in the app. Of course, the time range of data will also affect insights and Actions, in this case, it's important to know that the At a glance view and Trend Summary behave differently.

At a glance view

Screenshot


As this view is designed to be a high-level overview, the date range is fixed to the last 6 weeks. The insights and Actions are only affected by changing custom targets.

In other words, changes to selected dates in the Trend Summary section will not affect this view.

How can I set custom targets?

To give an idea of what “good” looks like, our metrics show a default industry benchmark. We also allow teams to set custom targets, because we recognize that teams may have different considerations (e.g., hardware teams may be limited in Lead Time due to physical operations, some teams may want to beat industry standards). 

Targets, whether custom or industry benchmarks, are what drive the insights that Multitudes shows you, whether in app or via our notifications in Slack or email.

There are two ways to customize targets:

  1. Settings > Targets page
    Screenshot

    Navigate to Settings > Targets  or click the “Customize targets” button on the top right of the At a glance section on the My Insights homepage. Customizing a target is as simple as clicking on a target and editing it!

    To be clear, in this table:

    • Rows represent teams (including a row for the organization as a whole at the top)
    • Columns represent the metrics
    • Each cell represents a target


    For a cell with a given team row and metric column, if the text of the cell is colored:

    • Black, this means the current target is unchanged from the industry benchmark (the industry benchmark is shown on the first gray row, for reference)
    • Teal, this means a custom target has been set

    Note that this table currently lets you set targets for 5 of our metrics, Lead Time, Merge Frequency, Change Failure Rate, Out-of-Hours Work, and PR Participation Gap. To set targets on other metrics (or also for these metrics, as a secondary option), see the option below.

  2. From a chart legend

    On a metric-specific pages (as accessed from the left hand menu bar), wherever you see a chart with a green target zone, you can customize the target here by going to the legend and clicking on the small target icon next to the “Target” label. This will show a modal from which you can edit the target. Click “Save Changes” to trigger a chart refresh.

    Note that if you’ve selected multiple teams or have “All” as the team filter at the top right of the page, the target shown in the legend will be the target for the organization as a whole.

    Screenshot with instructions

How do we integrate with Jira Software?

Install our Jira integration to get insights like Types of Work and Feature vs Maintenance Work.

We pull up to 200 Jira projects, ordered by the latest issue updated. We check for new projects and update this list daily.

Note that in order to install this integration, you must have Owner or Manager level permissions.

On the Multitudes app, go to the Integrations page (from the menu, find Account, click Settings, then click the Integrations tab across the top). Find the card that says Jira in the top Integrations section and click ‘Connect’ at top right.

  • After you click 'Connect' on the Jira card on the Integrations page, on the resulting pop-up click ‘Connect Jira projects’

    Screenshot with red arrow and circle showing where to click
  • This opens a new page on Atlassian Marketplace, click ‘Try it free’

    Screenshot with red arrow and circle showing where to click
  • This pops up a modal. In this modal, first, select your site; second, click the button to ‘Start free trial’. Don’t worry this integration is not separately charged beyond your instance of Multitudes!

    Screenshot with red arrow and circle showing where to click
  • This pops up a new modal, click ‘Get started’

    Screenshot with red arrow and circle showing where to click
  • This will close the modals, on the resulting page, a dialog box should pop up on the lower left hand corner indicating that the install is in progress. Once complete, it will show success, from this dialog box, click ‘Get started’

    Screenshot with red arrow and circle showing where to click
  • A new tab will open, on this tab, click ‘Allow access’

    Screenshot with red arrow and circle showing where to click
  • A new tab will open, scroll to the bottom and click ‘Accept’

    Screenshot with red arrow and circle showing where to click
  • This will close the last tab. On the resulting page, click the button ‘Connect with Multitudes’

    Screenshot with red arrow and circle showing where to click
  • Now, you should be brought back to our app, where you’ll see a pop-up asking you to link Jira projects to Multitudes teams. First, make the selections as desired. Second click ‘Finish linking teams’

    Screenshot with red arrow and circle showing where to click
  • Lastly, you’ll see a success message in the pop-up!

    Screenshot

⚠️Please note that once the installation is finished and teams are linked, you will need to wait a few minutes, then refresh the app to see the charts. Until the charts are ready, you will see the message "No data for this time period" on the Value Delivery page:

Screenshot
Screenshot

How do we integrate with Linear?

Install our Linear integration to get insights like Types of Work and Feature vs Maintenance Work.

Note that in order to install this integration, you must have Owner or Manager level permissions.

On the Multitudes app, go to the Integrations page (from the menu, find Account, click Settings, then click the Integrations tab across the top). Find the card that says Linear in the top Integrations section and click ‘Connect’ at top right.

  1. After you click 'Connect' on the Jira card on the Integrations page, on the resulting pop-up click ‘Continue to set up Linear’

    Screenshot
  2. This opens a new page, click 'Authorize Multitudes'

    Screenshot
  3. Now, you should be brought back to our app, where you’ll see a pop-up asking you to link Linear teams to Multitudes teams. First, make the selections as desired. Second click ‘Finish linking teams’

    Screenshot
  4. Lastly, you’ll see a success message in the pop-up!

    Screenshot

How do we integrate with Github Actions?

Set up our GitHub Actions integration to get insights like Deployment Frequency , Lead Time through to deploy, Deploy Time, and Deployment Failure Rate.

NOTE: You can use either our Deployments API or our GitHub Actions integration to provide us with data about deployments. If you already have our Deployments API integration configured, you will need to revoke that action token before connecting with our GitHub Actions integration.

  1. In the Multitudes app, go to Settings > Integrations and click on the Connect button in the Github and Github Actions card

    Screenshot
  2. On the resulting modal, click to configure Github Actions data

    Screenshot
  3. On the next page of the modal, you can choose either the “Environment/Deployments” or “Workflows” option, based on how you use GitHub Actions. See details below the image.

    Screenshot


    Don't know what to pick? Here are some common scenarios:

    • “I have one workflow that deploys to multiple environments

      • This sounds like you are using GitHub Action’s “Environments” feature, and therefore their “Deployments” feature, so you would select "Environment/Deployments"
      • In your case, the best way for us to track a “deployment” is for you to tell us which Environment is the one referring to your production environment in each repository. You'll see this in the next step
      • Note that within one workflow run, you can have more than 1 successful deployment (e.g., you could use the same workflow to deploy to dev, staging, and prod). Therefore it’s important to understand how we calculate deployment metrics when you’ve selected this option.
      • A note on historical data availability: Due to constraints on GitHub’s API, we will not be able to fetch historical deployment data for this option.

        • This is only an issue if you onboarded to Multitudes recently (i.e., if you onboarded to Multitudes a while ago but didn't configure GitHub Actions until today, then today we can still access your historical deployment data).
        • While most charts will have 6 weeks of historical data from before you onboarded onto Multitudes to get you started, charts relating to deployments will only have data from onboarding onwards.
        • Also note that it might look like your Lead Time increases from the date that you onboard, because from that point onwards, lead time will include deploy time based on your deployments data.
        • Here's a concrete example

          • You onboard onto Multitudes (install our GitHub app) on 1 June, from this point on we start accumulating real-time events data
          • We also pull 6 weeks of historical data, back to 16 May, to get you started
          • On 8 June, you configure GitHub Actions
          • We will only show deployment data from June 1 to June 8
    • “I have a specific workflow that deploys only to prod”

      • This sounds like you are not using GitHub Action’s “Environments” feature
      • In your case, the best way for us to track a “deployment” is for you to tell us which Workflow is the one that deploys to your production environment for each repository. You'll see this in the next step
      • A note on historical data availability:  we will have access to historical data (the 6 weeks of data from before you onboarded onto Multitudes), but only the latest attempt in the historical data. This means that for example, if an attempt to deploy to production succeeded at 12pm, but for some reason you re-ran the workflow again successfully at 1pm, 1pm will be what is captured as the Deploy Time for that PR, even though it actually got deployed earlier at 12pm. This is a constraint on the historical data from the GitHub API

    With respect to data availability in either case, once you have integrated GitHub Actions, we will have real-time data, and will take the first successful attempts to deploy to production going forward (since that is when the change is first available to customers).

  4. Once you’ve made a selection, the following page of the modal will prompt you for further configuration. In either case you will:

    • First identify which repository
    • Then within that repository, select either relevant environment or workflow
    • Click the "Save" button
  5. All done!

    Screenshot

How do we integrate with Opsgenie?

Some insights on Multitudes, such as Mean Time to Restore (MTTR) require your organization to install an integration with Opsgenie. After you’ve installed Opsgenie, you’ll be able to see insights for more metrics!

  • In the Multitudes app, go to Settings > Integrations and click on the connect Opsgenie card. This should direct you to your Opsgenie instance (otherwise click here)
  • On Opsgenie navigate to settings > API key management > add a new API key
    Screenshot
  • Once you click add a new API key, you should get a pop-up. Proceed to do 3 things in the pop-up
    Screenshot with instructions
    1. Select both the "Read" and “Configuration access” access scopes
    2. Copy the key - we also recommend you store this API access key in a secure location
    3. Click  "Add API Key" to finalize
  • Back on Multitudes, paste the previously-copied API key into the text field and click save
  • You will then be prompted to link your Opsgenie teams to Multitudes teams, this is so that you can use the team filters. If you do not link any Opsgenie teams all data will be returned unfiltered
  • All done! Multitudes will now start to track your incidents and calculate Mean Time to Restore

Note: Multitudes tracks MTTR by looking at your Opsgenie <code-text>incidents<code-text>. It does not look at Opsgenie <code-text>alerts<code-text>. The Opsgenie <code-text>incidents<code-text> feature is only available to paying Opsgenie customers.

My commits don’t seem to be appearing in Multitudes – what’s going on?

Check that the email address that is in your local git config (<code-text>user.email<code-text> when you do <code-text>git config -l<code-text>) matches the email address(es) that are linked to your GitHub account. You can change this by either changing git config to match an email that’s in GitHub, or by adding your git config email address to your GitHub account under https://github.com/settings/emails. If the emails are different, GitHub won’t know how to match your commits to your GitHub login (although it still links it to the account because of your SSH keys).

What does the “Collaborative PRs”/”All PRs” toggle do?

On the Merge Frequency chart on the Value Delivery page, you’ll see an option to choose between Collaborative PRs orAll PRs in the top-right corner of the chart. This toggle lets you only see PRs that had input from other team members (not just the PR author), like a comment, a review, or a merge. 

  • When Collaborative PRs is selected (default), only PRs that have had a comment, review, or merge from someone other than the PR author are shown. Bot activity is not counted as a collaboration. In other words, “selfie PRs" are excluded.
  • When All PRs is selected, all PRs are shown - even the “selfie PRs" where there was no comment, review, or merge on the PR by anyone other than the PR author.

How does the “exclude weekend hours” toggle work?

On some Trend Summary cards on our My Insights page, and on the Flow of Work > filters bar > “Show more filter” menu, there is an Exclude weekend hours toggle. 

You can turn this toggle on to exclude weekend hours from the calculations like Lead Time. “Weekend hours” are personal to each individual; it includes all hours during their custom non-working days. For example, if someone works part-time Monday to Wednesday, and the toggle is turned on, any hours on Thursday, Friday, Saturday and Sunday would be excluded because this is their weekend. 

You can configure non-working days for each team member in Settings.

How do I change the standard working hours?

Multitudes uses people’s working hours to calculate out-of-hours work. We have a default setting for this (Mon-Fri, 9 am - 5 pm in the timezone of your company’s headquarters), but because people work flexibly, we strongly recommend that you configure this for your individual preferences.To change the working hours for you or a teammate, go to Settings > Team Members, and select the person you want to adjust. Scroll down to the “Working Times” section, make changes as needed to the Timezone, Work days, or Working hours, and then choose “Confirm.”

How do I back-date a team member's working hours?

Some insights on Multitudes, such as out-of-hours work, are based on a team member’s working times. These are individually configurable but we start with the following defaults:

  • Timezone: The organization’s timezone
  • Working days: Monday to Friday
  • Working hours: 8am to 5pm

If you have a team member who works different hours or in a different timezone, you will likely want to update the default settings and back-date the changes so that person's historical data matches their actual working hours. To do that, follow the steps below.

First, go to Settings > Team Members, and select the person you want to adjust. Scroll down to the “Working Times” section and make changes as needed to the Timezone, Work days, or Working hours.

After you make changes to the "Working Times" section, a banner will appear (see screenshot below). Choose “Yes, back-date these changes”.

Screenshot of the working times settings in the Multitudes app, with a banner asking whether to back-date changes.

This will back-date your current working time settings, applying the current settings to the last 12 months of data for this person. This includes recalculating metrics that use these settings, such as Out-of-Hours work.

If you do not want to back-date the changes to Working Times, click “No, update from today” and the app will apply your settings from that point onwards, just like any other setting.

What if we have people who work on multiple teams? Or people who don’t work on any specific teams?

We’ve left the team structure flexible so it can meet your needs: We allow people to be on multiple teams, or on no teams at all.

However, if someone is not on any team, then they won’t appear in our graphs (since our tool is focused on helping teams work better together). As soon as you add them to at least one team, then they will be included in our insights.

Can’t find what you are looking for?

Contact us