Skip to content
Skip to assessment
Example review deliverable

Sample AI feature assessment

GTM Healthcare Intelligence

This example shows the format of a focused feature assessment: what failed, why it mattered, the recommended change, and how its correction was checked.

P1

Source claims exceeded the inputs

Wording correctedF01 / evidence →
Evidence

CMS labels could suggest a completed public-data import. The displayed snapshot used synthetic provider records and embedded demographic references.

Recommended
Name those inputs separately from optional public-file ingestion.
P2

A combined question returned individual rows

Totals and display verifiedF02 / evidence →
Evidence

Correct rep figures left the reader to reconstruct the requested total.

Recommended
Return summed bookings and quota, then calculate weighted attainment; keep individual rows when requested.
Verified combined result

$754,000 ÷ $800,000 = 94.25%

Bookings ÷ quota · 94.3% displayed
P2

Equivalent amounts behaved differently

Input checks passedF04 / evidence →
Evidence

“One point five million” required clarification where “$1.5M” succeeded. Both now return Singh’s $1,592,362 revenue for 2025; malformed “one point five point six million” still asks for clarification.

Recommended
Apply the same $1.5 million minimum to complete written and numeric inputs; ask for clarification when the quantity is malformed.

What was verified

The recorded checks confirmed complete combined and individual quota results, matching written and numeric thresholds, and removal of the repeated bookings block.

Full findings and verification evidenceOriginal comparisons, dated follow-ups, acceptance checks, method and sources.

Original review: GTM Healthcare Intelligence — Data provenance and query completeness. The five original requests were checked 2:53 p.m. Eastern · October 25, 2026, at app revision d80dd7d ↗.

The separate October 26 quota follow-up checked three requests at revision a1ce674 ↗. The quota requests did not recheck other findings.

The later October 26 written-amount follow-up, at 3:01 p.m. Eastern, checked written and numeric thresholds plus one malformed-input control. Source disclosure, quarterly ranking and no-show controls retain their original October 25 observations.

The latest October 26 bookings-display follow-up, at 3:51 p.m. Eastern, checked three requests after a separate app release. It verified the repeated block was removed while numerical results and calculation records stayed available.

All sales and practice records used here are synthetic.

Assessment summary

All four selected findings have dated verification: source disclosure and quarterly ranking on October 25, combined quota and written amounts in separate October 26 follow-ups. A later October 26 check confirmed the duplicate bookings block was removed while the combined and individual values stayed intact.

Findings
4
Verified corrections
4
Awaiting recheck
0
October 25 requests
5
October 26 quota requests
3
October 26 written-amount requests
3
October 26 display requests
3

Original review scope

  • Public GTM demo and portfolio data disclosures.
  • Selected source snapshots and recorded before-fix responses.
  • Five targeted requests to the production query endpoints.
  • Expected numbers recalculated directly from fixture rows without the query engine.

Findings and evidence

F01P1The public-data label was broader than the shipped snapshot.Verified · October 25Retain the provenance and period checks
F02P2Combined totals and individual results are complete and shown once.Verified · October 26, 3:51 p.m.Retain the duplicate-output and quota checks
F03P2A valid quarterly ranking was rejected by the scope guard.Verified · October 25Retain the provenance and period checks
F04P2Written and numeric revenue thresholds now match.Verified · October 26Retain written, numeric and malformed-input checks
F01 / P1Data provenance
Verified · October 25

The public-data label was broader than the shipped snapshot.

Original October 25 finding. The source-disclosure correction was not rechecked by the October 26 quota requests.

The earlier project summary described “synthetic and public data” beside a “CMS NPPES · Medicare PUF · Census” badge. The recorded SpectrumIQ input modes were synthetic providers and embedded demographic references.

Review risk. A reader could infer that the displayed provider counts came from a completed CMS import. This is a review risk; no user misunderstanding was measured.

Recorded before

…using synthetic and public data.
CMS NPPES · Medicare PUF · Census

October 25 recheck · verified then

Synthetic demo records.
Embedded references.
Optional public-data ingestion.

Implemented fix

Publish separate disclosures for generated records, bundled references and the optional local-file loader. Avoid using a source name as evidence that ingestion occurred.

Verification and limits

Verified on the public case-study page. All three labels are present and agree with the recorded input modes. Publication correction 43bdf1b; no data or calculation change is claimed.

Acceptance checks
  • Display the three data paths distinctly.
  • Name the default snapshot’s actual input modes.
  • Describe public-file ingestion as optional, not a completed live import.
F02 / P2Query completeness
Verified · October 26, 3:51 p.m.

Combined totals and individual results are complete and shown once.

Original failure recorded October 25. The 1:55 p.m. October 26 quota check verified the numerical fix and recorded a duplicate display block. A separate app release removed that repetition; the 3:51 p.m. live check below verified presentation and preserved values.

The combined-quota question returned only Chen and Kim rows on October 25. The 1:55 p.m. October 26 follow-up verified the combined total and individual quota rows but recorded repeated bookings. After a separate app release, the 3:51 p.m. check confirmed each rep appeared once while the combined and individual values stayed intact.

Review risk. The original response supplied correct individual figures but omitted the requested combined result. Without a total, a reader had to reconstruct weighted attainment from the amounts.

Question checked
What is the combined quota and attainment for Sarah Chen and David Kim in Q1 2025?

Recorded before

Chen: $504,000 bookings / $450,000 quota / 112.0%
Kim: $250,000 bookings / $350,000 quota / 71.4%
No combined result.

October 25 recheck · open then

Fresh production response: the same two rep rows.
No combined quota, bookings or attainment was returned.

October 26 quota recheck · 1:55 p.m.

1:55 p.m. Eastern · October 26, 2026. Serving revision a1ce674 ↗; existing fix 65e1db5 ↗.

Three selected requests checked combined-only, per-rep-only and combined-plus-per-rep answers. The total is $754,000 bookings / $800,000 quota = 94.25% attainment, displayed as 94.3%. Correct individual quota rows were preserved.

Combined only · verified

What is the combined quota and attainment for Sarah Chen and David Kim in Q1 2025?

October 26 observed response
Period: Q1 2025. Reps: Sarah Chen, David Kim.
Total: Bookings (won ACV) $754,000; Full-quarter quota $800,000; Attainment 94.3%.

Per-rep only · verified

Show bookings, quota and attainment by rep for Sarah Chen and David Kim in Q1 2025.

October 26 observed response
Period: Q1 2025. Reps: Sarah Chen, David Kim.
Chen: Bookings (won ACV) $504,000.
Kim: Bookings (won ACV) $250,000.

Period: Q1 2025. Reps: Sarah Chen, David Kim.
Chen: Bookings (won ACV) $504,000; Full-quarter quota $450,000; Attainment 112.0%.
Kim: Bookings (won ACV) $250,000; Full-quarter quota $350,000; Attainment 71.4%.

Combined and per-rep together · verified

Show combined bookings, quota and weighted attainment for Sarah Chen and David Kim in Q1 2025, along with the per-rep results.

October 26 observed response
Period: Q1 2025. Reps: Sarah Chen, David Kim.
Total: Bookings (won ACV) $754,000; Full-quarter quota $800,000; Attainment 94.3%.

Period: Q1 2025. Reps: Sarah Chen, David Kim.
Chen: Bookings (won ACV) $504,000; Full-quarter quota $450,000; Attainment 112.0%.
Kim: Bookings (won ACV) $250,000; Full-quarter quota $350,000; Attainment 71.4%.

Historical display observation from the 1:55 p.m. quota check. The later 3:51 p.m. follow-up below verified its repair.

The per-rep-only response contains a duplicate bookings-only block before the full quota rows. The captured response is retained verbatim; correct quota and attainment rows are present.

45 JavaScript tests passed, including all 63 nonempty rep subsets, aggregate calculations and preservation of individual results.

Download October 26 quota recheck packet ↓

October 26 bookings-display recheck · 3:51 p.m.

October 26, 2026, 3:51 p.m. Eastern. Deployed fix 3bde922 ↗; app PR 28 ↗; presentation tests ↗.

After the separate app release, all three production requests passed. Combined bookings remain $754,000 against $800,000 quota: 94.25% attainment, displayed as 94.3%. Each individual result appears once: Chen $504,000 / $450,000 / 112.0%; Kim $250,000 / $350,000 / 71.4%.

The earlier quota check retains the original repeated bookings response as a dated before record.

Combined total · passed

What is the combined quota and attainment for Sarah Chen and David Kim in Q1 2025?

October 26, 3:51 p.m. recorded response
Period: Q1 2025. Reps: Sarah Chen, David Kim.
Total: Bookings (won ACV) $754,000; Full-quarter quota $800,000; Attainment 94.3%.

Individual results · passed

Show bookings, quota and attainment by rep for Sarah Chen and David Kim in Q1 2025.

October 26, 3:51 p.m. recorded response
Period: Q1 2025. Reps: Sarah Chen, David Kim.
Chen: Bookings (won ACV) $504,000; Full-quarter quota $450,000; Attainment 112.0%.
Kim: Bookings (won ACV) $250,000; Full-quarter quota $350,000; Attainment 71.4%.

Combined and individual results · passed

Show combined bookings, quota and weighted attainment for Sarah Chen and David Kim in Q1 2025, along with the per-rep results.

October 26, 3:51 p.m. recorded response
Period: Q1 2025. Reps: Sarah Chen, David Kim.
Total: Bookings (won ACV) $754,000; Full-quarter quota $800,000; Attainment 94.3%.

Period: Q1 2025. Reps: Sarah Chen, David Kim.
Chen: Bookings (won ACV) $504,000; Full-quarter quota $450,000; Attainment 112.0%.
Kim: Bookings (won ACV) $250,000; Full-quarter quota $350,000; Attainment 71.4%.

The individual-results request produced both performance and quota query records, exercising the overlapping plan. Both underlying results, applied scopes and calculation details remain available; only the redundant display block is suppressed.

50 JavaScript tests passed: five new presentation tests and 45 existing tests, including weighted quota checks for all 63 nonempty rep subsets. Distinct cohorts, periods, groupings, filters, rankings and values retain their separate output.

This follow-up did not recheck source disclosures, quarterly rankings, written amounts, malformed amounts, or no-show controls. Their earlier dated evidence is retained.

Download October 26 bookings-display recheck packet ↓

Implemented fix

The earlier numerical fix returns combined quota and weighted attainment while preserving individual results. The later display fix at 3bde922 suppresses a fully redundant bookings block only when quota results cover the same scope, displayed metrics and values. Distinct results and the underlying calculation records remain available.

Verification and limits

The October 26, 3:51 p.m. production check at revision 3bde922 passed requests for combined totals, individual results, and both together. Combined bookings $754,000 / quota $800,000 still gives 94.25% attainment (94.3% displayed). Chen and Kim each display once with their correct quota rows. The earlier repeated response is preserved in the 1:55 p.m. quota evidence.

Acceptance checks
  • Return one combined result with $800,000 quota and $754,000 bookings.
  • Calculate attainment as 754,000 ÷ 800,000 = 94.25% (94.3% at one decimal).
  • Preserve correct per-rep results and supported cohort restrictions.
F03 / P2Scope validation
Verified · October 25

A valid quarterly ranking was rejected by the scope guard.

Original October 25 finding. The quarterly-ranking repair was not rechecked by the October 26 quota requests.

A recorded production response rejected an annual quarterly ranking. PR #22 changed coverage checks to accept valid period partitions per measure.

Review risk. A supported question could be blocked even when the supplied tool queries covered the requested year. The observed failure was a clarification, not an incorrect number.

Question checked
Who had the most denials in each quarter of 2025?

Recorded before

The shared period must apply to every independent measure.

October 25 recheck · verified then

Q1: Brooks — 81
Q2: Singh — 74
Q3: Brooks — 65
Q4: Yamamoto — 60

Implemented fix

Accept complete quarterly partitions without accepting missing periods, swapped ranges or unsupported cohort links. This repair was implemented before the current assessment.

Verification and limits

One fresh production request returned all four expected winners and counts. Expected values were recalculated directly from monthly fixture rows. This is a targeted recheck, not proof of every possible phrasing or tool plan.

Acceptance checks
  • Return all four requested quarters with their own winners.
  • Match the direct fixture sums.
  • Continue rejecting genuinely incomplete or unsupported scopes.
F04 / P2Input usability
Verified · October 26

Written and numeric revenue thresholds now match.

Original clarification recorded October 25; the separate October 26 written-amount follow-up below verifies the already-deployed fix. The original failed response remains preserved as an October 25 record.

The October 25 written-threshold request stopped for clarification. A separate October 26 recheck returned the same normalized revenue threshold, provider result and displayed answer for written and numeric forms; a malformed quantity still stopped calculation.

Review risk. The original written amount required clarification where the equivalent numeric form succeeded. The recheck tested the complete amount and scale, including an ambiguous quantity that must stop before calculation.

Question checked
Which providers had at least one point five million in revenue in 2025?

Recorded before

The numeric threshold was not preserved. Please clarify the comparison; no calculation was run.

October 25 recheck · open then

Written amount: clarification again.
Numeric control (≥ $1.5M): Singh — $1,592,362.

October 26 written-amount recheck

3:01 p.m. Eastern · October 26, 2026. Serving revision a1ce674 ↗; existing fix 65e1db5 ↗.

Written “one point five million” and numeric “$1.5M” both normalize to revenue ≥ 1,500,000 for January–December 2025, grouped by provider across all providers. Both return only Singh, with $1,592,362 revenue; the displayed answers are identical.

Directly summing the 12 monthly sample-record values per provider, without the query engine, confirms Singh is the only provider at or above $1,500,000. The packet retains all five annual provider totals and the linked fixture source.

Written amount · matched

Which providers had at least one point five million in revenue in 2025?

Source: validated-query. Tool calls: 1.

October 26 recorded response
Period: Jan–Dec 2025.
Revenue:
Singh: $1,592,362
Filter after aggregation: Revenue ≥ $1,500,000.

Numeric amount · matched

Which providers had at least $1.5M in revenue in 2025?

Source: validated-query. Tool calls: 1.

October 26 recorded response
Period: Jan–Dec 2025.
Revenue:
Singh: $1,592,362
Filter after aggregation: Revenue ≥ $1,500,000.

Malformed amount · clarification, no calculation

Which providers had at least one point five point six million in revenue in 2025?

Source: query-clarification. Tool calls: 0.

October 26 recorded response
Please clarify the complete numeric quantity, including its decimal and scale. No calculation was run.

The malformed phrase “one point five point six million” requests clarification with zero tool calls and no calculation. It does not fall back to the leading “one.”

The existing quota and threshold suite passed 7 tests, with 0 failures, using node --test tests/quota-decimal.test.mjs. The earlier quota packet retains its separate 45-test result.

Written and numeric requests were compared for threshold, operator, period, grouping, provider selection, calculated rows and displayed answer. Annual revenue was independently summed from monthly fixture values without the query engine. A third request checked that malformed input caused no calculation.

Source-disclosure, quarterly-ranking and no-show controls were not rechecked by these written-amount requests. The separate quota packet retains its own earlier October 26 scope.

Download October 26 written-amount recheck packet ↓

Implemented fix

The existing written-number parser preserves the complete quantity and scale, and rejects malformed quantities before tool execution. Fix 65e1db5 was already deployed before the follow-up; no app code was changed or deployed for this recheck.

Verification and limits

Verified October 26 at 3:01 p.m. Eastern, serving revision a1ce674. Written and numeric forms both use revenue ≥ 1,500,000 across all 12 months and return only Singh at $1,592,362. Direct monthly sums confirm the expectation. The malformed control requests clarification, with zero tool calls and no calculation.

Acceptance checks
  • Interpret one point five million as 1,500,000, or clearly ask for the supported numeric form.
  • Match the $1.5M control when calculation proceeds.
  • Never silently accept only the leading “one.”

Checks to retain

Retain the numerical, input and display checks after their dated verification.

  1. 01

    Verified · October 26, 3:51 p.m. Eastern · F02

    Retain the duplicate-output and quota checks

    Keep each rep’s bookings, quota and attainment together once, while preserving requested totals and calculation records. Retain checks that preserve distinct cohorts, periods, groupings, filters, rankings and values. The 50-test suite includes five new presentation tests and the existing weighted-quota coverage.

  2. 02

    Verified · October 26, 3:01 p.m. Eastern · F04

    Retain written, numeric and malformed-input checks

    Keep equivalent complete thresholds and provider results for written and numeric amounts. Keep ambiguous quantities from becoming partial numbers or triggering a calculation.

  3. 03

    Verified · original October 25 review · F01 / F03

    Retain the provenance and period checks

    Keep the three data disclosures and quarterly coverage checks. These findings were verified in the original review, not in the October 26 quota follow-up.

Original verification — October 25

The five original requests and their results below are retained verbatim. They describe revision d80dd7d, not the later October 26 quota or written-amount results.

Original requests began 2:53 p.m. Eastern · October 25, 2026. App revision d80dd7d.

Q01

combined quota

Open: no combined result

Expected: $800,000 quota; $754,000 bookings; 94.25% attainment

Observed production response
Period: Q1 2025. Reps: Sarah Chen, David Kim.
Chen: Bookings (won ACV) $504,000; Full-quarter quota $450,000; Attainment 112.0%.
Kim: Bookings (won ACV) $250,000; Full-quarter quota $350,000; Attainment 71.4%.
Q02

quarterly ranking

Verified correction

Expected: Q1 Brooks 81; Q2 Singh 74; Q3 Brooks 65; Q4 Yamamoto 60

Observed production response
Period: Jan–Mar 2025.
Highest Denial counts for Q1: Brooks (81).

Period: Apr–Jun 2025.
Highest Denial counts for Q2: Singh (74).

Period: Jul–Sep 2025.
Highest Denial counts for Q3: Brooks (65).

Period: Oct–Dec 2025.
Highest Denial counts for Q4: Yamamoto (60).
Q03

written decimal

Open usability limitation; safe clarification

Expected: Interpret the complete 1,500,000 threshold, or clearly state the supported numeric form

Observed production response
The numeric threshold was not preserved. Please clarify the comparison; no calculation was run.
Q04

numeric control

Control passed

Expected: Singh $1,592,362 at threshold ≥ $1,500,000

Observed production response
Period: Jan–Dec 2025.
Revenue:
Singh: $1,592,362
Filter after aggregation: Revenue ≥ $1,500,000.
Q05

missing denominator

Safe refusal observed

Expected: Do not calculate no-show percentages without an appointment denominator

Observed production response
That rate cannot be calculated because its required denominator is not supplied.

Method and limits

AI tools supported implementation, testing and analysis. Numerical expectations were recomputed from the synthetic input records; this is not an independent third-party audit.

  • The requests were purpose-selected; they are not a random sample or a model-accuracy benchmark.
  • This review did not test every query, security controls, production scale, or source accuracy.
  • The original before/after strings and five check results are October 25 records at d80dd7d. October 26 checks at a1ce674 covered three quota requests at 1:55 p.m. Eastern and three written-amount requests at 3:01 p.m. Eastern. A later three-request presentation check at 3:51 p.m. Eastern followed the separate app release at 3bde922. Source disclosures, quarterly rankings and no-show controls retain their October 25 evidence; written and malformed amounts were not rechecked by the presentation follow-up.
  • No app code was changed or deployed for the original assessment or its first two follow-ups; those numerical and input fixes were already deployed. The later presentation check followed a separately deployed duplicate-bookings fix. This portfolio update does not deploy app code.
  • No user study, patient-data review, clinical validation or measured commercial outcome is claimed.

Evidence ledger

E1–E9 retain the original record; E10–E11 document the earlier quota follow-up; E12–E13 document the written-amount follow-up and parser source; E14–E17 document the latest display follow-up and separate app release.

  1. E1
    Recorded portfolio wording — before

    Broad public-data summary and source badge in the historical project entry.

  2. E2
    SpectrumIQ input-mode metadata

    Records synthetic providers and embedded_reference demographics.

  3. E3
    Published provenance correction

    Case-study, project-entry and screenshot release. The correction changed publication content, not demo data.

  4. E4
    Period-partition repair — PR #22

    Accepts valid period partitions while preserving coverage and cohort restrictions.

  5. E5
    Synthetic sales records and quota inputs

    Direct inputs for the combined-quota expectation.

  6. E6
    Synthetic practice records

    Direct inputs for quarterly denial winners and annual revenue thresholds.

  7. E7
    Quota calculation and rendering

    Reviewed source for the rep-level quota result.

  8. E8
    Question normalization

    Reviewed source for the written-threshold limitation.

  9. E9
    Verification packet

    Selected recorded-before responses, five fresh requests, expected values and observed results. Synthetic data only.

  10. E10
    Existing combined-quota fix

    Fix 65e1db5 was already deployed before the October 26 recheck; no source change or deployment was performed for the follow-up.

  11. E11
    Quota recheck packet — October 26

    Three complete captured responses, expected weighted calculations, serving revision a1ce674 and the 45-test summary. Other findings were not rechecked.

  12. E12
    Written-amount recheck packet — October 26

    Three complete captured responses at 3:01 p.m. Eastern, normalized thresholds, independent monthly sums and the existing 7-test summary. Original and quota evidence remain separate.

  13. E13
    Written-number parser at existing fix

    Parser source at fix 65e1db5, already deployed before the written-amount recheck. The dated packet also links fixture and targeted-test sources.

  14. E14
    October 26 duplicate-bookings display recheck packet

    Three production requests captured at 3:51 p.m. Eastern after the separate app release; includes raw responses, retained numerical anchors, the earlier duplicate observation and scoped test results.

  15. E15
    Deployed duplicate-bookings display fix

    Pinned revision 3bde922 suppresses only a fully redundant performance display; underlying query and calculation records remain available.

  16. E16
    Duplicate-output presentation tests

    Five new presentation tests in the 50-test JavaScript suite; distinct scopes and values retain their output.

  17. E17
    GTM app PR 28: duplicate bookings display fix

    Separate app release preceding the 3:51 p.m. production check. The assessment update itself does not deploy app code.

End of assessmentKristen Martino / October 25, 2026