Skip to Main Content

Agile Procurement Playbook

A guide on how to apply the principles of agile to procurement.

Appendix F: Procurement options analysis examples

In deciding which procurement strategy would best fit the needs of a product, developing an analysis of alternatives for the different options that are available helps provide data to make decisions.

Example #1: Centers for Medicare & Medicaid Services (CMS) — Medicare Payment Modernization contracting strategy

Background

As modernization efforts transitions from proof of concepts on the Fiscal Intermediary Standard System (FISS) to a production roadmap, serious thought must be put into the contracting strategy to support this effort for the next 5–10 years. Setting up the correct contracting strategy has the largest impact on the probability of success on this effort. The FISS system is the initial target for proof of concepts to be transitioned into production, this effort shouldn't be limited to just that system. This is a unique opportunity to synthesize diverse Medicare payment systems. Including the following:

  • Common Working File (CWF)
  • Multi-Carrier System (MCS)
  • VIPS Medicare Shared System (VMS)
  • Centers for Medicare & Medicaid Services Innovation (CMMI) models

To achieve the goal of a successful payment modernization effort, contracting strategy must enable the following factors:

  • Ability to award new tasks quickly
  • Quality of technical talent available via contract vehicle
  • Contractor digital service experience and approach (this is different than "agile development")
  • Familiarity with CMS systems, specifically complex business logic associated with Medicare FFS payment processing
  • Ability to define development and infrastructure requirements at the blanket purchase agreement (BPA) level so details aren't missed in task orders

There exists a variety of contracting options, each with pros and cons. It's the effort of this document to determine which contracting option best suits needs for this modernization effort. Ultimately, the decision on choosing the right contracting strategy needs to answer the simple question of: Will this vehicle/strategy position the Medicare Payment Modernization effort to have the greatest probability of success?

Pros and cons of different procurement options

Strategic Partners Acquisition Readiness Contract (SPARC)

The Office of Information Technology (OIT) at CMS has an existing indefinite delivery/indefinite quantity (IDIQ) vehicle called SPARC that's the agency's answer to agile, waterfall, and hybrid development methodologies. This vehicle has a $25B ceiling that houses over 140 different vendors. Its goal is to be the one-stop shop for agency development. There are certainly advantages to leveraging work already completed as well as drawbacks to leveraging a generic development contacting vehicle.

Pros Cons
The IDIQ is already existing and vendors have been pre-vetted. The vendor pool isn't specialized to any project.
The vendor pool is diverse based on company size The vendor pool is so large the agile contracting is impossible.
The ceiling on the IDIQ is extremely high — low risk of breaking through it. The probability of protest per contract is substantially higher due to more vendor bids and issues with small business sizes.
Orders under $10M aren't protestable Agile methodologies, user-center design, and DevOps practices aren't baked into the IDIQ and thus need to be included with each proposal. This bloats the proposal with boilerplate language while increasing the surface area of protest.
All contract types permissible giving CMS added flexibility. Most vendors aren't familiar with CMS payment processing and could struggle to meet the quick on boarding and rapidly changing needs of this project.
Helping CMS meet small business goals. Ability to award new tasks quickly.

SPARC with on-ramp

This option has the ability to continue leveraging SPARC while allowing for new vendors to be added to the IDIQ. This list includes all of the Pros and Cons listed previous with SPARC with the following consideration.

Pros Cons
Ability to add new entrants to the cadre of pre-qualified vendors that could compete for agile development work. High-protest probability — would require a competitive process that could be protested. There are two points of protest because we already have a large-vetted pool of contractors plus the source selection decision.
- Additional resources need on Office of Acquisition & Grants Management (OAGM) and program side to manage source selection process.

Governmentwide Acquisition Contract (GWAC) — General Services Administration (GSA) Alliant 2, National Institutes of Health (NIH) CIO-SP3, etc.

Pros Cons
Existing pre-vetted cadre of qualified vendors. Best-in-Class (BIC) designation that'll help CMS meet Office of Management and Budget (OMB) targets.
Orders under $10M not protestable. Agency could receive criticism for not using SPARC that was intended, in part, for agile development; however, climate is pushing agencies toward Category Management and spend under management (SUM) / BIC use.
All contract types permissible. Not all vendors will have subject matter expertise with CMS systems or payment processing.
Alliant 2 explicitly includes agile development services.  

Agency-wide BPA

Pros Cons
Targeted competition possible; however, must provide Request for Quote (RFQ) to any company requesting it. Vendors aren't tailored to the project's specific goals.
Ability to reach new contractors who've entered the market for initial competition. Initial award and any calls are protestable.
BPA holders will be specifically selected based upon demonstrated ability to develop and deploy digital services using agile methodologies, user-centered design, and DevOps. New umbrella contract vehicle is resource intensive versus placing individual orders against existing contract vehicles.
Could receive OMB designation as a SUM vehicle but requires mandatory-use policy and other reporting requirements. If significant overlap to existing SUM/BIC vehicle, modified business case to OMB required.
- If approved by OMB, and BPA put in place, CMS mandatory-use policy and rigorous reporting requirements needed.
- Inability to add new contractors to mix after award of BPA.
- Risk of ending up with a significant number of vendors in pool.
- Agency could receive criticism for not using SPARC which was intended, in part, for agile development.

GSA Schedule task order competitions

The GSA Schedule is available for individual task orders as they are necessary.

Pros Cons
Existing pre-vetted cadre of qualified vendors. Not a SUM or BIC vehicle.
New contractors are constantly being added to the industrial base. Hundreds of vendors; however, Federal Procurement Data System (FPDS) data shows that [x] proposals are received on average.
Targeted competition possible (to reasonably ensure receipt of at least three quotations); must provide RFQ to any company requesting it. Limited to Firm-Fixed-Price (FFP) or Labor-Hour (LH); however, these contract types are consistent with commercial practices for buying agile development.

Project-specific BPA off of GSA Schedule

Pros Cons
The vendors are tailored to the project's needs. Initial award and any calls are protestable.
Contracts can be issued as new problems or requirements are encountered. Multiple program-specific BPAs are resource intensive for the agency to award and manage.
Smaller pool of specialized vendors. If the vendors are smaller companies, there's risk of being unable to meet the needs of the total work necessary for the project.
The pool of vendors all become domain experts as the project matures. If significant overlap to existing SUM/BIC vehicle, modified business case to OMB required.
Agile methodologies, user-center design, and DevOps practices can be baked into the BPA. This avoids proposals with boilerplate language while focusing on the objectives. If approved by OMB, and BPA put in place, CMS mandatory-use policy and rigorous reporting requirements needed.
The base period on awarded contracts is shorter. CMS is put into the position of less risk by requiring results from vendors faster. CMS doesn't have the pressure to award existing vendors on a contract re-compete due to transition time shorten via domain knowledge. This drive competition. -
Avoids single points of failure. Re-competes are actually encouraged. -

Digital services contracting approach

While there are a variety of potential contracting options available, to-date, the most successful approach to complex system development at CMS using an innovative, digital service approach, was done on the Quality Payment Program.

Some high-level stats on the QPP BPA to-date:

  • There have been 16 task orders awarded under Agile Delivery to Execute Legislative Endeavors for Quality Related Initiatives (ADELE)
  • All vendors are small businesses
  • Every ADELE BPA holder has won an award at this point
  • All ADELE task orders have been competitively awarded
  • Zero protests
  • The average procurement action lead time is 67 days
  • The average period of performance for an ADELE task order is 2.2 years
  • Responses were typically limited to 20–25 pages with 2-week turnarounds
  • Low-dollar value allows for Contracting Officer's Representatives (COR) II rather than COR III

Retrospective: SPARC vs. ADELE

As previously listed, the goals for our contracting strategy are:

  • Ability to award new tasks quickly
  • Quality of technical talent available via contract vehicle
  • Contractor digital service experience and approach (this is different than "agile development")
  • Familiarity with CMS systems, specifically complex business logic associated with Medicare FFS payment processing
  • Ability to define development and infrastructure requirements at the BPA level so details are not missed in task orders

CMS has experience over the past year issuing contracts through both SPARC and ADELE and there are some great data points that can be gathered to compare the two vehicles against the goals that we're aiming for.

Ability to award new tasks quickly

Working in an agile fashion means not having all of the business requirements up front. Most of the time this process is impossible and a vast majority of the time the initial business requirement gathering for a complex system is incorrect. As new requirements come to light throughout the modernization process, the ability to issue contracts rapidly is necessary.

Vehicle Ability to award new tasks quickly
SPARC Competitive task orders averaged 139 days (19.9 weeks).
ADELE BPA Competitive task orders averaged 67 days (9.5 weeks).

Quality of technical talent available via contract vehicle

Competition is vital in making sure the government is getting the best possible vendor for the job. More vendors doesn't always mean better competition. Contracts see the best competition when vendors specialize in the work that issued. Overall competition increases as vendors learn through previous proposals. Finding the sweet spot of the vendor pool size that allows for competition and allows vendors to continuously challenge each other to rise the skill level is important.

Vehicle Quality of technical talent available
SPARC The density of high-skilled vendors is variable. Roughly 5% of the total vendors have won a competitive proposal under SPARC.
ADELE BPA 100% of the vendors have won a competitive proposal under ADELE. Each one of these competitive proposals included a design challenge that required working software and designs to be submitted.

Contractor digital service experience and approach (this is different than "agile development")

User-centered design, DevOps, and product management are required to deliver modern digital services. These skillsets along with an agile methodology are necessary in delivering working software in small increments allowing for quickly changing development and design as new requirements are discovered.

Vehicle Contractor digital service experience
SPARC Partners offer agile, waterfall, and hybrid development methodology. Having digital service skillsets weren't prerequisites for joining SPARC.
ADELE BPA 100% of the vendors on ADELE went through market research making sure that modern digital services and agile development practices were core to their business.

Familiarity with CMS systems, specifically complex business logic associated with Medicare FFS payment processing

Having a background with CMS programs allows vendors to hit the ground running when contracts are awarded. The amount of time that's required to either onboard or transition work is diminished when vendors already know systems that they'll be working on. Having these familiarities allows for the period of performance to decrease and thus protecting CMS from getting locked into a vendor that underperforms. In addition to familiarity, having a smaller vendor pool means that contractors and their employees already have accounts, access, and the ability to quickly begin work after contract award.

In some cases it takes over two months for new contractors to get access to CMS systems, this would make any attempt at iterative development dead on arrival.

Vehicle Familiarity with CMS systems
SPARC The average period of performance for a SPARC task order is 3.8 years.
ADELE BPA The average period of performance for an ADELE task order is 2.2 years.

Ability to define development and infrastructure requirements at the BPA level so details are not missed in task orders

Having certain requirements that can be found in all contracts that are intended to go out under the vehicle can benefit from having these requirements baked into the vehicle directly vs. requiring justification on each contract. This protects CMS from potentially missing common contracting needs with each contract issued. This also cuts down on the size of the proposal that needs to be reviewed with each task order.

Vehicle Requirements baked into the vehicle
SPARC None.
ADELE BPA Agile methodologies, digital service requirements.