Comments Received / Week

Overview

The Comments Received / PR report (also known as "PR Review Depth") measures the average number of comments a developer receives on their pull requests. This metric quantifies code review engagement depth and helps teams understand the thoroughness of their review process.

What Does This Metric Measure?

Definition

Comments Received / PR calculates the average number of review comments per merged pull request that a developer has authored.

Key Characteristics:

• Per merged PR: Only counts pull requests that have been successfully merged

• Review engagement depth: Measures how much discussion/feedback occurs during code review

• Author-focused: Shows feedback received by PR authors, not given by reviewers

• Average metric: Smooths out variation across individual PRs

What It Tells You

This metric indicates:

• ✅ Review thoroughness — How deeply reviewers engage with code changes

• ✅ Code review culture — Level of feedback and discussion on PRs

• ✅ Mentorship intensity — How much guidance developers receive

• ✅ Complexity signals — Higher comments may indicate complex or unfamiliar code

What It Doesn't Tell You

This metric does NOT measure:

• ❌ Quality of comments — Insightful vs. superficial feedback

• ❌ Comment sentiment — Constructive vs. critical tone (see "Review Themes" for this)

• ❌ Silent approvals — Reviews approved without comments

• ❌ Comment length — One-word vs. detailed explanations

• ❌ Comment type — Blocking issues vs. style nits vs. questions

How It's Calculated

Formula

Comments Received / PR = Total PR Comments Received / Total Merged PRs

Components Breakdown

1. Numerator: Total PR Comments Received

• Sums all comments received on merged pull requests

• Includes: Inline review comments, review feedback, general discussion

• Excludes: Comments on unmerged/abandoned PRs, bot comments

2. Denominator: Total Merged PRs

• Counts pull requests with "Merged" status

• Only includes PRs authored by developers in the analysis scope

• Excludes: Draft PRs, closed (non-merged) PRs, abandoned PRs

Example Calculation

Scenario:

• Developer has 25 merged PRs

• Total comments received across all merged PRs: 100

Calculation:

100 comments / 25 PRs = 4.0 comments per PR

Data Filtering

Included:

• All comments on merged PRs

• Comments from all reviewers (teammates, cross-functional partners)

• All comment types (approvals with comments, changes requested, general feedback)

Excluded:

• ❌ Comments on unmerged/abandoned PRs

• ❌ Bot/automated comments

• ❌ Comments on draft PRs (unless merged)

• ❌ Self-comments (developer commenting on their own PR)

Accessing the Report

Navigation Path

Primary Route:

1. Login to Span

2. Under "Insights" in main navigation, click "Productivity"

3. Select "Code Review" tab

4. Find "Comments Received / PR"

Alternative Access:

• Individual Pages: View each developer's received feedback

• Team Pages: See team-level review depth averages

• Metric Explorer: Search for "comments received" or "review depth"

Available Filters

Dimension Filters

Time Filters

• Last 2 weeks

• Last 4 weeks (common default)

• Last 3 months

• Last 6 months

• Last 12 months

• Custom date range

• Fiscal year alignment

Special Filters

• Group By: Break down by people, teams, repositories, or time

• Comparison Period: Compare to previous period or year-over-year

• PR Status: Fixed to "Merged" (cannot be changed)

Interpreting the Data

Benchmark Ranges

Context-Based Interpretation

For Junior Developers

• Expected: 6-12 comments/PR

• Higher values = Good mentorship happening

• Decreasing over time = Developer improving, requiring less guidance

For Senior Developers

• Expected: 2-5 comments/PR

• Lower values = Code quality and clarity improving

• Stable values = Consistent quality and practices

For Complex/High-Risk Code

• Expected: 5-10+ comments/PR

• Higher values = Appropriate scrutiny for risk level

• Review thoroughness matches code criticality

For Simple/Routine Changes

• Expected: 1-3 comments/PR

• Lower values = Efficient process for low-risk changes

• Should be faster cycle times

Pattern Recognition

Positive Signals:

• ✅ High values for junior developers (mentorship)

• ✅ Stable, consistent values over time (mature process)

• ✅ Appropriate variation by code complexity

• ✅ Gradual decrease as developers gain experience

Warning Signals:

• ⚠ Very low values across entire team (< 2) — May indicate insufficient review rigor

• ⚠ Very high values (> 10) consistently — May indicate unclear requirements or communication issues

• ⚠ Sudden spike or drop — Process change or team composition change

• ⚠ High variance within team — Inconsistent review standards

Trend Analysis

Increasing Trend:

• More complex work being tackled

• Onboarding new team members

• Implementing stricter review standards

• Increased scrutiny or quality focus

Decreasing Trend:

• Team shipping more routine work

• Improved code quality requiring less feedback

• Streamlined review process

• Loss of review rigor (if combined with quality issues)

Use Cases

1. Code Review Process Health Assessment

Scenario: Monitor consistency of review practices over time

How to Use:

• Track team average monthly

• Alert on sudden changes (± 30%)

• Compare to PR cycle time — ensure review depth isn't causing delays

• Correlate with defect rates — validate thoroughness improves quality

Example: Team average drops from 5 to 2 comments/PR

• Investigate: Process change? Tool change? Team composition?

• Validate: Are defect rates stable or increasing?

2. Onboarding Effectiveness Tracking

Scenario: Track whether new hires receive adequate feedback

Expected Pattern:

• Weeks 1-4: 8-12 comments/PR (learning phase, heavy guidance)

• Weeks 5-12: 6-9 comments/PR (improving, still learning)

• Weeks 13-24: 4-6 comments/PR (competent, occasional guidance)

• Weeks 25+: 3-5 comments/PR (proficient, peer-level reviews)

Action: If not decreasing, investigate:

• Code quality issues

• Unclear standards

• Need for additional training

3. Team Performance Comparison

Scenario: Compare review depth across different teams

Analysis:

• Team A: 3 comments/PR, low defect rate, fast cycle time

• Team B: 7 comments/PR, low defect rate, slow cycle time

• Team C: 2 comments/PR, high defect rate, fast cycle time

Interpretation:

• Team A: Optimal balance

• Team B: Over-reviewing or complex work

• Team C: Insufficient review rigor

4. Mentoring and Knowledge Sharing

Scenario: Assess quality of mentorship through code review

How to Use:

• Senior engineers should have lower received comments (3-5)

• Junior engineers should have higher received comments (6-10)

• Pair with "Comments Authored / Week" for reviewers

• Validate mentorship is happening

Example: Junior dev has 2 comments/PR avg

• Red Flag: Not receiving adequate guidance

• Action: Assign specific mentors, increase review participation

5. Process Optimization

Scenario: Measure impact of process changes

Example:

• Implement PR size guidelines (< 400 lines)

• Before: 6 comments/PR, 3-day cycle time

• After: 4 comments/PR, 1-day cycle time

• Result: Smaller PRs = faster, more focused reviews

6. Risk Detection

Scenario: Identify areas with inadequate review coverage

Warning Pattern:

• Low review depth (< 2 comments/PR)

• High defect rates or incidents

• Critical/high-risk code areas

Action:

• Mandate additional reviewers

• Implement review checklists

• Require security/performance review

Important Considerations & Limitations

What This Metric Does Well

✅ Quantifies review engagement — Provides objective measure of review depth ✅ Tracks culture changes — Shows evolution of review practices over time ✅ Identifies imbalances — Reveals inconsistent review standards ✅ Benchmarking support — Enables team and organizational comparisons

Critical Limitations

⚠ Does Not Distinguish Comment Quality

• "LGTM" counts same as detailed architectural feedback

• Superficial comments inflate the metric artificially

• Recommendation: Manually review sample comments for quality

⚠ No Context About Comment Sentiment

• Doesn't capture constructive vs. critical tone

• Can't distinguish helpful vs. unhelpful feedback

• Alternative: Use "Review Themes" for sentiment analysis

⚠ Averages Hide Distribution

• Average of 4 could be: all PRs with 3-5 comments, OR mix of 0 and 15

• Outliers affect the average significantly

• Recommendation: Examine P50 (median), P75, P90 percentiles

⚠ Only Merged PRs Counted

• Abandoned PRs with extensive discussion excluded

• Draft PRs not counted (unless merged)

• Impact: May underestimate total review engagement

⚠ Missing Silent Approvals

• Approvals without comments not reflected

• Some reviews are thorough but don't generate comments

• Impact: May undervalue efficient, clear code

⚠ Team Averages Can Mislead

• Team with mixed junior/senior = moderate average

• Hides individual variation

• Recommendation: Always break down by person

⚠ Cannot Distinguish Comment Purpose

• Required feedback vs. optional suggestions

• Blocking issues vs. stylistic preferences

• Limitation: All comments weighted equally

Data Quality Considerations

Requires:

• ✅ VCS integration properly synced (GitHub, GitLab, Azure DevOps)

• ✅ Accurate person/team assignments

• ✅ Active contributor definitions configured

Data Timing:

• Updates daily (typically 24-hour lag)

• Historical data always complete

• Real-time data may have slight delay

Related Metrics for Complete Picture

View these alongside "Comments Received / PR":

Best Practices

For Individual Contributors

As a PR Author:

• Don't fear high comment counts — it's learning opportunity

• Track your trend over time — should decrease as you improve

• Seek clarification on feedback rather than implementing blindly

• Celebrate when comments decrease (sign of growth)

Tips to Reduce Comments:

• Smaller PRs (< 400 lines)

• Clear PR descriptions with context

• Self-review before submitting

• Follow team coding standards

• Include tests upfront

For Reviewers

Provide Effective Comments:

• Be specific and actionable

• Distinguish blocking vs. suggestions

• Explain the "why" behind feedback

• Balance critical feedback with recognition

• Link to documentation or examples

Avoid:

• Excessive stylistic nits

• Scope creep (architectural changes on bug fix PRs)

• Vague feedback ("this is bad")

• Redundant comments

For Team Leads

Monitor for:

• Consistent review standards across team

• Appropriate differentiation by developer level

• Balanced review depth (not too high or low)

• Trend alignment with team goals

Actions:

• Set clear review guidelines

• Provide reviewer training

• Define blocking vs. non-blocking feedback

• Create code style guides to reduce stylistic comments

• Celebrate effective reviews (quality over quantity)

For Engineering Managers

Use for:

• Onboarding progress validation

• Team health assessment

• Process improvement measurement

• Resource allocation (identify bottlenecks)

Avoid:

• Using as sole performance indicator

• Setting arbitrary quotas without context

• Comparing across vastly different code complexity

• Punishing high or low values without investigation

Common Questions

Q: Our Comments Received / PR is 6.5 - is that good?

A: There's no universal "good" value. Consider:

• Team experience level (higher for junior teams is expected)

• Code complexity (infrastructure code may have more comments)

• Team goals (speed vs. quality vs. learning)

• Compare to peer teams and historical baseline

Action: Correlate with quality metrics (defect rate) and velocity (cycle time) to find your optimal balance.

Q: Why did our Comments Received / PR drop by 50%?

A: Investigate these possibilities:

1. Process change (auto-approval for low-risk PRs)

2. Team composition (more experienced developers)

3. Improved code quality (less feedback needed)

4. Smaller PRs (less surface area for comments)

5. New tooling (automated checks reducing manual comments)

6. Different review culture or standards

7. Filter/scope changes in report

Action: Review sample PRs, survey team, check for process changes.

Q: Should I try to increase or decrease this metric?

A: Depends on your goals:

Increase if:

• Defect rates are high (need more thorough reviews)

• Junior team needs more mentorship

• Complex/high-risk code requires more scrutiny

• Knowledge sharing is a priority

Decrease if:

• Reviews causing excessive delays (> 3-day cycle time)

• Comments are mostly superficial/stylistic

• Team is mature and producing quality code

• Automated checks can replace manual feedback

Balance: Find the "sweet spot" for your team's quality, velocity, and culture goals.

Q: How is this different from "Comments Authored / Week"?

A:

• Comments Received / PR (this metric): Author perspective — feedback I get on my code

• Comments Authored / Week: Reviewer perspective — feedback I give to others

Together they show:

• Bidirectional review culture

• Balance of giving vs. receiving feedback

• Individual contribution patterns

Q: This is an average - how do I see variation?

A: Drill deeper:

1. Filter "Group By" → People to see individual developers

2. Examine PR list to see individual PR comment counts

3. Use percentiles (P50, P75, P90) to understand distribution

4. Export data and analyze in spreadsheet for full distribution

Troubleshooting

If Metric Seems Too High (> 10)

Check:

• Are PRs too large? (Review "PR Diff Size")

• Are standards unclear? (Create style guides, use linters)

• Is feedback too nitpicky? (Distinguish blocking vs. suggestions)

• Are requirements unclear? (Improve PR descriptions)

If Metric Seems Too Low (< 2)

Check:

• Are PRs getting reviewed at all? (Check "% PRs Merged Without Review")

• Are reviews thorough? (Manually sample recent PRs)

• Is code very simple? (Check PR types and complexity)

• Are reviewers engaged? (Check "Comments Authored / Week")

If Metric Shows Zero

Possible Causes:

• No merged PRs in the period

• Filter excludes all PRs

• New developer with no completed work

• Data sync issue with VCS integration

Quick Reference Checklist

When analyzing Comments Received / PR:

•  What's the team average? Individual range?

•  Is review depth appropriate for team experience level?

•  How does this correlate with defect rates?

•  Is review depth causing cycle time delays?

•  Are junior developers receiving more feedback than seniors?

•  What's the trend over time (improving or declining)?

•  Do outlier PRs (high comment count) reveal patterns?

•  Are comments constructive and valuable? (sample review)

•  How does this compare to peer teams?

Summary

The Comments Received / PR report provides visibility into code review engagement depth across your team. Use it to ensure appropriate review thoroughness, track mentorship effectiveness, validate onboarding progress, and maintain healthy code quality practices. Remember: this metric measures quantity of feedback, not quality—always combine with qualitative assessment, outcome metrics, and team context for the complete picture.

Healthy Target: 2-5 comments/PR for experienced teams; 6-10 comments/PR for junior developers; adjust for code complexity and risk level.