CivicActions has started working on the General Services Administration Agile Services Blanket Purchase Agreement Request for Quotation and, in the spirit of open knowledge and collaboration, I wanted to share my thoughts to the team (and anyone else) based on the rules of evaluating solicitation responses, and what I know about 18F.
Quoters will be scored against the checklists and questions in the U.S. Digital Services Playbook
My bet is the selection committee is going to score the technical responses as “Exceptional,” “Acceptable,” “Non-Acceptable” against each and every checklist item and question in the U.S. Digital Services Playbook. From that, the Selection Committee can give each Quoter (vendor) an objective score for each play in the Playbook and add up all the points for the technical approach to produce an overall rating and a quantitative score.
The awardees will be those with the highest score (and some consideration of price).
It is procurement law to have an objective process, using objective criteria that is communicated to public. There is surprisingly little detail about the criteria in the proposal except, “the Quoter must demonstrate that they followed the U.S. Digital Services Playbook by providing evidence in the repository.” Hence, my bet they will use the checklist items and questions in the Playbook for criteria.
Quoters demonstrating a user-driven build will score higher than any features or execution
The biggest critique of government IT is that it builds the wrong things that are not useful to real users.
The RFQ describes three pools of services with Pool 2 including all of Pool 1 tasks, and Pool 3 including all of Pool 2 tasks. In other words, user-centric design are the plays that come first and must be done before the others.
To be blunt, the team should demonstrate user focus from day one. That focus should be represented in the Git repo, with every design and development deliverable being clearly associated with an identified user story that has been validated by users. Each deliverable should be presented to users for feedback, and integrated into successive MVP iterations that can be user tested allowing multiple rounds of feedback to be integrated. This would demonstrate a real investment in establishing an agile process that connects with real users.
I think Quoter repos demonstrating the team spent 2 or 3 whole days just focused on user-research at the beginning will score high during evaluation. IMHO, the only technical work at the start should be explorative spikes to get a handle on the API and set up the CI pipeline.
It is classic government IT fail to say, “We don’t have time to spend a lot of time with users, we have to meet this deadline.” Another classic (waterfall) fail is to spend too much time in a three ring circus of stakeholders gathering ad-infinitum requirements on paper that bloat the project. GSA 18F will absolutely be looking for vendors that strike the balance and demonstrate they follow the Agile Manifesto to put people and collaboration over process, tools and contracts while valuing both.
The selection committee will look in the repo to see if and how end-users drove the minimum viable product core feature(s) and whether there was actual testing and iteration.
The Git repo needs to have as much detail as a traditional proposal
The RFQ emphasizes continually how the repo should demonstrate how the organization follows the playbook.
The actual MVP code should be only one part of the repo. Much of the repo should reflect the agile process and assets demonstrating how the MVP grew out of a collaboration and iteration with target users.
Assign a budget and demonstrate how we stuck to it
A quoting team competing to get on the BPA should have a budget for this project and stick to it.
Play #5 is “Structure budgets and contracts to support delivery,” and most Quoters will give only lip service to this play while throwing lots of resources at the deadline (e.g. budget creep).
Quoters providing evidence of assigning a budget and managing the project to constraints will score “Exceptional” for this play.
There’s clearly an implied budget of 10 days in the project. I think a firm can demonstrate within the repo structuring budgets and contracts to support delivery by showing how we selected a team, a leader, stuck with that team, had structured hours when work was done, made choices, etc. Even better if we show timeboxing for different activities and our internal interactions with our own executives to meet the “budget.”
Working software is a must
It doesn’t have to set the world on fire, but Dave Zvenyach has tweeted as saying working software is needed. Agile Methodologies promote the idea of “always being releasable.” You should document when you got the MVP done and how you iterated on it.