Free Network Build Assessment
Score your buildYour HSD tables can pass while members can't get an appointment. Secret shopper testing is how that gets found: directory accuracy, wait times, whether anyone answers. Here's where it is required, what it tests, and how plans can prepare.
Network adequacy compliance has two distinct dimensions that Medicare Advantage plans sometimes treat as the same thing. The first is the paper adequacy of the network: the NPI counts, time-and-distance calculations, and HPMS filings that make up the annual submission process. The second is the functional adequacy of the network, whether members can actually get appointments with in-network providers in a reasonable timeframe, whether the provider directory is accurate when a member calls or goes online, and whether the plan can provide care in a member's preferred language.
Secret shopper testing exists to check the second dimension. CMS's Automated Criteria Check doesn't test whether anyone answers the phone, so a network can pass it and still fail the people trying to use it.
Before planning around "the CMS secret shopper program," be clear about which program you mean. The rules differ by line of business, and only some of them are written down as a formal survey requirement.
Medicaid managed care: Under 42 CFR 438.68(f), an independent entity hired by the state will survey each plan annually, for rating periods beginning on or after July 9, 2028 (a CMS chart lists July 10). See our guide to Medicaid wait-time standards and secret shoppers.
Exchange plans: CMS's April 2024 technical guidance covers appointment wait time secret shopper surveys for qualified health plan issuers in the federally facilitated Exchanges, using a third party hired by the issuer that is not an affiliate, starting with plan year 2025.
Medicare Advantage: We haven't found a published rule that sets up a standing CMS secret shopper survey for MA comparable to those two, so don't rely on anyone's description of its schedule or selection criteria without a source. The rules that do apply are the wait-time standards in 42 CFR 422.112(a)(6)(i) and the directory requirements in 42 CFR 422.111(m). Outside researchers do run these tests. In May 2023, the Senate Finance Committee's majority staff ran a secret shopper study of 12 MA plans in six states: 10 listed mental health providers per plan, 120 calls. Staff got an appointment 22 times out of 120.
The published Medicaid requirement and the Senate study point to the same handful of things a caller can check.
Appointment availability and wait times: Can a new patient get an appointment, and how soon? For MA primary care and behavioral health, 42 CFR 422.112(a)(6)(i) requires urgently needed or emergency services immediately, services that need medical attention but are not urgent within 7 business days, and routine and preventive care within 30 business days. Medicaid's routine standards under 42 CFR 438.68(e) are 10 business days for outpatient mental health and substance use disorder services, 15 for primary care, and 15 for OB/GYN.
Provider directory accuracy: The Medicaid survey under 438.68(f) tests, at minimum, active network status, street address, telephone number, and whether the provider is accepting new enrollees. Under 42 CFR 422.111(m), MA plans must update directory information within 30 days of learning of a change and attest to its accuracy at least once a year.
Language access: 42 CFR 422.111(b)(3)(i) requires MA plans to disclose providers' cultural and linguistic capabilities, including American Sign Language. A directory that lists a language no one at the office speaks is exactly the kind of error a caller can find.
Behavioral health access: This is where public testing has been most pointed. In the Senate study, 39 of the 120 calls were non-working numbers, wrong numbers, or calls never returned. It's a sensible place to start your own testing.
The consequences differ by program. In Medicaid, directory errors found by the survey go to the state within 3 business days and to the plan within 3 more, and the plan corrects them on the 42 CFR 438.10(h)(3) timeline. A plan is deemed compliant with the wait-time standards at 90% or better, and falling short means a state remedy plan under 42 CFR 438.207(f). In the Exchanges, CMS's guidance says an issuer that falls short "would need to add more contracted providers."
In Medicare Advantage, a network access complaint is one of the events that can pull a CMS network review forward, and operating contracts that fail a review may face compliance or enforcement actions. MedPAC's June 2024 report says CMS has never imposed intermediate sanctions or civil money penalties for network adequacy noncompliance, but that isn't a reason to ignore access problems your members can see.
Star Ratings don't publish secret shopper results, but they do include member experience measures from the CAHPS survey, such as Getting Needed Care and Getting Appointments and Care Quickly. Those ask members about the same access that a caller would test from the outside. A plan whose listed providers can't offer appointments is likely to hear about it in those scores as well.
The most effective preparation is not a one-time readiness exercise: it is keeping network information right continuously. A practical program has these parts:
None of these measures guarantee a clean result. But they fix the problems a caller would find, and they give you records to respond with if someone finds one first.
Blueprint automates the network build workflows described in this article: from adequacy modeling to provider outreach tracking. See it with your state and line of business.