Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
49 changes: 0 additions & 49 deletions .github/workflows/codeql.yml

This file was deleted.

Original file line number Diff line number Diff line change
Expand Up @@ -147,6 +147,53 @@ This shows the relative impact of deeper adoption, independent of how many users

For example, if users in the engaged cohorts averaged 20 pull requests/user/month and the passive cohort averaged 10, then the equation is 20 / 10 = 2x multiplier.

### Understanding industry comparisons

The impact dashboard's **Industry comparison** widget compares two measures for your enterprise, or for an organization that isn't owned by an enterprise, with aggregated results from participating peers:

* **Agentic adoption**: The percentage of active users in your enterprise or organization who are agent-first or multi-agent users.
* **Leverage**: The ratio between the pull request merge rate for agentic users and the rate for non-agentic users (per user per month). For example, a leverage score of 4.1x means that agentic users merged pull requests at 4.1 times the rate of non-agentic users during the reporting period.

Each comparison covers the 28-day reporting period shown in the widget. Peer cohorts use the company size and industry selected when you opt in. Company size options are 1–999, 1,000–4,999, and 5,000+ employees, and there are ten industry options. These values are self-reported and aren't verified. You can compare with peers in your selected industry or across all industries.

When enough eligible peers are available, each card shows:

* Where your score ranks within the cohort.
* The difference between your score and the peer median.
* The difference between your score and the top decile.

The top-decile comparison requires a larger peer cohort than the median comparison, so the median can be available when the top decile isn't. Enterprises and organizations with very few active users can view comparisons, but they aren't counted toward any peer cohort.

The chart plots your enterprise's score against the distribution of peer scores and marks the peer median and top decile. Some markers may be unavailable if the cohort does not contain enough eligible peers.

For comparisons within your selected industry, the dashboard expands the peer cohort in this order until enough eligible peers are available:

1. Participating peers with the same company size and industry.
1. Participating peers with the same company size, across all industries.

Comparisons always use peers with the same company size. The widget shows which peer cohort was used, so you can tell when a comparison has been expanded beyond your selected industry.

For {% data variables.product.prodname_ghe_cloud %} with data residency, peer cohorts include only participants hosted in the same data residency region. The dashboard applies the same cohort expansion within that region and does not combine data across regions.

The dashboard does not use a cohort based only on industry. Comparisons across all industries use participating peers, rather than all {% data variables.product.github %} users or enterprises.

Participation is controlled by the **Copilot metrics industry comparison** policy and is disabled by default. Enterprises, and organizations that aren't owned by an enterprise, can participate. Organizations owned by an enterprise participate through their enterprise. Enterprises used only to assign {% data variables.copilot.copilot_business_short %} licenses can't participate, because leverage can't be calculated for them. Enabling the policy both contributes your aggregated metrics to peer cohorts and lets you view comparisons. For details, see [AUTOTITLE](/copilot/how-tos/administer-copilot/view-impact-dashboard#opting-in-to-industry-comparisons).

Rankings appear only after enough enterprises and organizations have opted in to form a peer cohort. If there are not enough eligible peers after the dashboard applies the fallback cohorts, it doesn't show a ranking. A ranking is also unavailable when your own score can't be calculated, such as when there are no active users in the reporting period or no non-agentic users to provide a baseline for leverage.

#### How industry comparisons protect your data

* **Opt-in only.** Nothing is contributed until an enterprise owner, or the owner of an organization that isn't owned by an enterprise, enables the **Copilot metrics industry comparison** policy. Enterprises used only to assign {% data variables.copilot.copilot_business_short %} licenses never contribute.
* **Only two summary scores are contributed.** Each participant contributes its agentic adoption and leverage scores for the 28-day reporting period. No user names, user-level activity, code, repository content, or pull request content is contributed.
* **Results are aggregated.** Your scores are combined with those of other participants to calculate cohort statistics, such as the median and top decile. Other participants can't see your enterprise or organization name, which enterprises and organizations participate, or how many participants are in a cohort.
* **Cohort statistics aren't linked to participants.** When cohort statistics are calculated, participants' scores are combined into results that are stored only by date, company size, and industry. These results contain no enterprise, organization, or user identifiers, so they can't be linked back to a specific participant.
* **Rankings are calculated on {% data variables.product.github %}'s servers.** Your own scores are stored with the rest of your {% data variables.product.prodname_copilot_short %} usage metrics. To show your ranking, {% data variables.product.github %} compares your scores with the cohort statistics for the same reporting period. Other participants' scores are never looked up or compared individually.
* **Small cohorts are suppressed.** A cohort's statistics are shown only when it contains enough participants and users. Enterprises and organizations with very few active users aren't counted toward any cohort, so a small number of developers can't shape the results.
* **Company size and industry stay private.** The company size and industry you select are used only to group participants into cohorts. They aren't shown to other participants or verified by {% data variables.product.github %}.
* **Data residency is respected.** For {% data variables.product.prodname_ghe_cloud %} with data residency, data is never combined across data residency regions.
* **Results stay in the dashboard.** Industry comparisons aren't included in downloads, NDJSON exports, or the {% data variables.product.prodname_copilot_short %} usage metrics API.
* **Aggregated results are short-lived.** Cohort statistics are recalculated daily and deleted after 30 days. If you opt out, the dashboard stops showing comparisons immediately, and your data is excluded from cohorts starting with the next daily update. Each daily calculation uses your current opt-in setting, so if earlier results are recalculated after you opt out, your data isn't added back.

### Estimating potential return on investment

The impact dashboard's **Potential return on investment** section provides a comparison of cost and pull request output between **Phase 0-1 Passive and Code First Users** and **Phase 2-3 Agent First Users**.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,40 @@ For a detailed explanation of what the dashboard shows, including adoption cohor
{% data reusables.copilot.access-copilot-metrics-dashboard %}
1. In the left sidebar, click **{% data variables.product.prodname_copilot_short %} impact**.

## Comparing your impact with peers

The **Industry comparison** widget shows how your agentic adoption and leverage compare with participating peers. Industry comparisons are available for enterprises and for organizations that aren't owned by an enterprise. Organizations owned by an enterprise participate through their enterprise, and enterprises used only to assign {% data variables.copilot.copilot_business_short %} licenses can't participate, because leverage can't be calculated for them.

### Opting in to industry comparisons

Participation is controlled by the **Copilot metrics industry comparison** policy, which is disabled by default. When the policy is enabled, your enterprise or organization contributes aggregated {% data variables.product.prodname_copilot_short %} metrics to peer cohorts and can view comparisons. When it's disabled, you don't contribute data or see comparisons.

Enterprise owners, and owners of organizations that aren't owned by an enterprise, can opt in from the widget or by enabling the policy. The widget and the policy use the same setting.

1. In the **Industry comparison** widget, click **Opt-in for industry comparison** and review the information about how your data is used.
1. Select values from the **Industry type** and **Company size** dropdown menus. Both values are required to create a peer cohort.
1. Click **Opt in**.

### Viewing industry comparisons

1. Use the scope selector to switch between **All industries** and your selected industry.
1. Review the comparison cards for the selected 28-day reporting period:
* **Your agentic adoption** shows the percentage of active users who are agent-first or multi-agent users.
* **Your leverage** compares the pull request merge rate per user per month for agentic users with the rate for non-agentic users.
1. For each card, review your percentile ranking and your difference from the peer median and, if shown, the top decile. The ranking line shows the peer cohort used for the comparison. Use the chart to see your score, the peer median, and the top decile within the peer distribution.

Rankings appear only after enough enterprises and organizations have opted in to form a peer cohort. The median comparison is shown first, and the top-decile comparison is added when the cohort grows larger. If your selected industry doesn't have enough participants yet, the dashboard compares you with peers of the same company size across all industries. If that cohort doesn't have enough participants either, the dashboard doesn't show a ranking. Especially soon after launch, you might see only the median, all industries instead of your selected industry, or no ranking until more peers opt in.

A ranking also isn't shown when your own score can't be calculated for the reporting period. For example, there might be no active users, or all active users might already be agent-first or multi-agent users, leaving no non-agentic users to compare leverage with.

For details about what data is shared and how it's protected, see [AUTOTITLE](/copilot/concepts/copilot-usage-metrics/copilot-metrics#how-industry-comparisons-protect-your-data).

### Changing your peer cohort or opting out

To change your peer cohort, click {% octicon "gear" aria-label="Industry comparison settings" %} in the widget, then choose a different industry or company size. The widget uses your new selection right away, and your data moves to the new peer cohort with the next daily update.

To stop participating, click {% octicon "gear" aria-label="Industry comparison settings" %} in the widget and select **Opt-out** or set the **Copilot metrics industry comparison** policy to **Disabled**. The dashboard stops showing comparisons immediately, and your data is excluded from peer cohorts starting with the next daily update. Previously calculated aggregate results don't identify your enterprise or organization and are deleted after 30 days.

## Estimating potential return on investment

The "Potential return on investment" section provides a directional comparison of cost and pull request output across adoption phases.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ category:
- Track Copilot usage
---

After you’ve viewed the {% data variables.product.prodname_copilot_short %} usage metrics dashboard, you can use this article to interpret each chart and identify opportunities to increase adoption and engagement.
Use this article to interpret usage and adoption metrics across the {% data variables.product.prodname_copilot_short %} usage metrics and impact dashboards, and to identify opportunities to increase adoption and engagement.

## Reviewing overall usage trends

Expand Down Expand Up @@ -105,6 +105,22 @@ The adoption multiplier compares engaged users (Phase 1, 2, or 3) against passiv
> [!NOTE]
> The adoption multiplier compares two different populations of users, not the same users over time. Differences in team composition, project complexity, or seniority between engaged and passive users can influence the comparison as much as {% data variables.product.prodname_copilot_short %} adoption itself.
## Reviewing industry comparisons

The **Industry comparison** widget compares your enterprise's agentic adoption and leverage with aggregated results from participating peers. Review the percentile ranking, peer median, top decile, and peer distribution together. Do not treat any single value as a target.

| Signal | What it suggests | Suggested action |
|:--|:--|:--|
| Agentic adoption is above the peer median | A larger share of your users have adopted agent-first or multi-agent workflows than in the comparison cohort. | Continue the enablement practices that support agentic workflows, and review which teams still have lower adoption. |
| Agentic adoption is below the peer median | Your enterprise has room to expand adoption of agent-first and multi-agent workflows. | Review the adoption cohort distribution, then prioritize onboarding or enablement for teams that remain passive or code-first. |
| Leverage is above the peer median | The ratio of merge rates between agentic and non-agentic users is higher in your enterprise than in the peer cohort. | Review the practices of teams with strong agentic adoption and consider whether those practices can be applied more broadly. |
| Leverage is below the peer median | The ratio of merge rates between agentic and non-agentic users is lower than for the comparison cohort. | Check team composition, project complexity, and how recently agentic workflows were introduced before changing your rollout plan. |
| Your ranking improves across reporting periods | Your enterprise's position has improved relative to the participating peer cohort. | Confirm that the improvement appears alongside sustained adoption and pull request trends before expanding enablement investment. |
| Results are unavailable | There are not enough eligible participating peers of the same company size to provide the comparison, even after the dashboard expands the peer cohort to all industries. Results are also unavailable when your own score can't be calculated, such as when there are no active users or all active users are already agent-first or multi-agent users, leaving no non-agentic users to compare leverage with. | Review your enterprise's or organization's own adoption and leverage trends where available until a peer comparison is available. |
| Your ranking changes but your score doesn't | The participating peer cohort changed, or the dashboard compared you with a different cohort than in the previous period. | Check which peer cohort the ranking line shows before comparing results across periods. |

Industry comparisons show associations, not causation. Differences in team composition, project type, engineering practices, and adoption maturity can affect both measures. The participating peer cohort can also change between reporting periods.

## Reviewing potential return on investment

The **Potential return on investment** section compares {% data variables.product.prodname_copilot_short %} cost, estimated percentage of payroll, and pull requests per developer per month across earlier and agent-first adoption phases. Treat these comparisons as directional associations, not proof that a change in adoption caused a change in output.
Expand Down
3 changes: 3 additions & 0 deletions data/code-languages.yml
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,9 @@ jsonc:
markdown:
name: Markdown
comment: xml
mermaid:
name: Mermaid
comment: percent
perl:
name: Perl
comment: number
Expand Down
10 changes: 10 additions & 0 deletions package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -224,6 +224,7 @@
"mdast-util-to-string": "^4.0.0",
"next": "^16.3.8",
"parse5": "8.0.1",
"pele": "1.3.0",
"quick-lru": "7.0.1",
"react": "^19.2.5",
"react-dom": "^19.2.5",
Expand Down
1 change: 1 addition & 0 deletions src/content-render/stylesheets/index.scss
Original file line number Diff line number Diff line change
Expand Up @@ -4,5 +4,6 @@
@import "article-section-framing.scss";
@import "syntax-highlighting.scss";
@import "alerts.scss";
@import "mermaid.scss";
@import "octicon-table-optimization.scss";
@import "accessibility.scss";
27 changes: 27 additions & 0 deletions src/content-render/stylesheets/mermaid.scss
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
// Maps Pele diagram colors and font to Brand and Primer tokens, so diagrams match the page.
.mermaid-diagram {
margin-bottom: 1rem;
overflow-x: auto;

.pele {
--pele-bg: var(--brand-color-canvas-default);
--pele-fg: var(--brand-color-text-default);
--pele-muted: var(--brand-color-text-muted);
--pele-line: var(--brand-color-text-default);
--pele-surface: var(--brand-color-canvas-subtle);
--pele-surface-alt: var(--bgColor-neutral-muted);
--pele-border: var(--brand-color-border-default);
--pele-accent: var(--fgColor-accent);
--pele-critical: var(--fgColor-danger);
--pele-series-1: var(--data-blue-color-emphasis);
--pele-series-2: var(--data-orange-color-emphasis);
--pele-series-3: var(--data-green-color-emphasis);
--pele-series-4: var(--data-red-color-emphasis);
--pele-series-5: var(--data-teal-color-emphasis);
--pele-series-6: var(--data-yellow-color-emphasis);
--pele-series-7: var(--data-purple-color-emphasis);
--pele-series-8: var(--data-brown-color-emphasis);
// Pele sizes nodes from Helvetica-like metrics on the server. Mona Sans fits inside that size.
--pele-font: var(--brand-body-fontFamily);
}
}
Loading
Loading