Federal IT Contracting 2.0: Solving The Problem With Contractors

The founding principal of federal contracting is competition.  The notion is that through a formal process using both objective and subjective criteria, that the proposal evaluation teams are positioned to pick the best of the competitors.  There is no shortage of detractors of the process.  It’s cumbersome, it’s rigid, it’s bureaucratic, it’s slow, the Federal Acquisition Regulation (FAR) is too big and complex, etc.  Reformers argue that shortcuts of various types need to be taken.  All of that may be true, but I don’t think that those are the things that prevent IT contracts from being much more successful than they currently are.

A typical IT contract awards work to ONE contractor.  I think that this is the basic flaw.  At the time of contract award perhaps the best contractor is the one chosen.  But that contractor doesn’t necessarily STAY the best contractor.  It is no secret that contractors use their “A” business team for writing proposals, and propose their “A” team developers and engineers.  Their actual USE the “A” team on the project is no guarantee except for a few who are officially designated as “key personnel”.  But even that is no guarantee, and there is a process for replacing key personnel.  Even in the rare instance where a contractor delivers the staff exactly as proposed, this ‘best’ team’s performance has a tendency to erode after the ‘honeymoon’ period common to all projects.  This is due to totally normal human behavior.

There is no shortage of material on organizational development, group behavior, management and behavioral sciences, group and individual motivation.  And there is no shortage of methods, consultants, and interventionists to help teams perform to their highest potential. There is not enough of these sources and methods being employed on federal IT contracts.  Of course, for a contractor to employ these tools on a continuing basis, it would add to the costs which would be passed through to the customer via the contractor’s overhead costs.  It’s a thorny problem. 

Here is a solution: All mid to large size IT contracts should award tasks orders of... say... 80% to the ‘best’ contractor and 20% to the ‘next best’ contractor.  Let’s call these the A team and B team contractors.  Simply, the B team is there to keep the A team on their best game.  If the performance of the A team begins to slip, the Program Manager (PM) and the Contracting Officer’s Technical Representative (COTR) and Contracting Officer (KO) have the power and mechanism in place to easily move current or new work tasks from A and award it to B, and vice versa when necessary. Another way would be to have B act as a validation and verification (V&V) team, which allows them visibility into the guts of the project (and ready to take over) This scheme would provide for constant competition among the contractors, and would surely keep their executives and managers engaged in the quality of service on contracts they’ve won – perhaps as much as they worry about the quality of the proposals for the NEXT contract they’ve yet to win.

In the current federal IT contracting environment, competition works well enough…. until there is one winner.  At that point, the game is over.  The Federal IT community needs to figure out how to take advantage of normal human and organizational behavior, along with their survival instincts and revenue goals by incorporating this or other constant competition schemes into their IT contracting playbook. 

EPILOG:  Ironically, amid all of CMS’s failures on the Healtcare.gov system rollout, I think they actually had a similar mechanism in place that saved a chunk of their bacon…or maybe just a few slices. The main contractor’s (A) role was taken over by another contractor (B) who was working on a key part of the system: the ‘data hub’ that routes data between systems and organizations. So contractor B was responsible for getting the system working during the crash effort that was successful at getting – now – millions of folks signed up.  But now CMS has decided to give most of the work to yet another contractor (C).  The next thing to watch on this project is how C behaves, now that they are establishing control and probably recruiting all of the “A” team staff from the original A contractor and B as well.

Federal IT Dashboard: Visibility Into Complex Data or Eye Candy?

The "tree map" below shows the relative size (dollars) of Information Technology (IT) investments in the various Federal agencies. The Defense Department is the largest chunk and the tiny bit in the bottom right corner is the Smithsonian.  This is one of the many displays available to the public on the Federal IT Dashboard.

The idea behind the IT Dashboard is great: Require all major IT investment programs to post and update information about their state and status on a publicly available website.  The information would be available to be scrutinized by interested press, public, auditors, contractors, and those INSIDE the federal government. 

 So, how’s the Federal IT Dashboard working out? 

Two Points.

POINT ONE: the website is a well designed, organized, professional, interactive, and had appealing…. I’d even say ‘gorgeous’ ways of presenting extremely complex, voluminous data.  There are pull down lists for filtering data and for selecting the method of display (tables, graphs, even animated timelines), and there are great visualizations that you can design for yourself that shows the trends in spending, progress and other factors over the last 10+ years.

To understand how (our) ~$76,000,000,000 is being spent on almost 7,400 IT projects, you can and should visit and explore for yourself: https://itdashboard.gov.  You will find information on individual programs and projects as well.  I find that part the most interesting, especially for projects that I was involved in, or read about in the press.  That’s where “POINT TWO” comes in.

POINT TWO: Bureaucracies and the people in them tend NOT to deliver bad news up the chain.  Whether it’s wishful thinking that things will improve, or thinking that nobody is watching, or simply fear of negative consequences is immaterial.  The effect is that rather than having actual insight into a program, those who depend on such data actually have flawed insight.  Well, maybe they have great insight into programs that have no troubles, and little insight into those that NEED intervention from higher up in the chain.  

The data that feeds the IT dashboard is supposed to be reviewed and ‘blessed’ by Chief Information Officers in the reporting agencies, but there also is perhaps a tendency for the CIOs to avoid ‘help’ from up the chain.  If the IT Dashboard had the Healthcare.gov project identified as behind schedule, who cared? If it didn’t identify issues…. What happened?

One way to avoid getting ‘help’ is to modify a program’s basic elements of reporting (cost, schedule, functionality).  All or most reporting systems properly allow changes to these ‘baseline’ factors.  The issue is who keeps up with the rolling and cumulative changes over time?  Too often, a change is the baseline basically becomes a new start for reporting purposes. OR… is so confusing in the end, that it becomes almost impossible to figure out what capability the project will deliver….  at what cost….at what date.

In the project shown below, the baseline has been changed 5 times since 2009 (each Δ is a baseline change).  I found this one by searching for a project that was “ALL GREEN”.  But when I looked into the details and the "Evaluation Explanation", I found that they acknowledge having program management problems and their cost profile is RED (on the green, yellow, red scale). In this case I also notice that an adjustment of the baseline was followed by a DECREASE in score rather than what I would expect to be an INCREASE in score (a baseline lets a project reset it's goals, cost estimates, and schedule).

In another graphic, same program, it appears to me that the part of the project that is active is actually RED, but the future parts of the project that are not yet active are GREEN.  I guess the FIVE GREEN cost/schedule scores overwhelm the ONE RED cost score? Even if the RED one is the same cost as the GREEN ones combined?  Jumpin' Jimminy.

From personal experience I can tell you that there are many dedicated people feeding the many data sources that drive the IT Dashboard.  But as that data gets interpreted or adjusted or scored/colorized as it works it's way up the chain leaves lots of room for creeping, serial … . ummm…errrr…ummm.... overly optimistic reinterpretation of the data by the management chain during the data aggregation phase of reporting.

Fortunately there is a lot of semi-raw data on the website for one to explore.  Unfortunately the eye candy charts and graphs don't always reflect the data and may distract and mislead observers who do not dig deeper into the details. This effort is 5 years old, and I guess is still a crawling or walking toddler.  I hope it matures and begins running soon.

Fixed Price Contracting for a Total System: An Attempt – Part 2 of 2 How The Plan Worked Out.

PART 2

The software development and systems integrator community took interest in the agency’s procurement.  They attended the vendor day briefings, asked questions, reviewed documents in the reading room, requested copies and otherwise showed interest in bidding.  One of the questions most often asked had to do with the engineering change proposal process.  The government made it clear that they did not anticipate using a change process due to the limitation of funds on the program. And made it clear that they understood that it was indeed tradition to refine specifications during development, but would not add funds to the contract or approve changes during the development process. The government made it clear that any changes required would be made AFTER delivery of what was specified, rather than DURING the development of the deliverable.  The government acknowledged that the conventional wisdom and experience was that catching and fixing logic and other errors earlier were cheaper than fixing them AFTER completion.

The bidders bid, the proposal evaluation panel selected a winner and made the announcement.  There were no protests.  After award ceremonies and a little partying on both sides, the government and the contractor held a series of meetings at the highest levels, to discuss the projects goals, philosophy and…. No changes!  No technical spec changes, no documentation requirements changes, no changes to the outlines of the documents, no changes to anything.

The contractor’s team hunkered down in office space in Greenbelt Maryland and got to work.  The government got office space nearby to cut the time required for meetings, status updates and questions or issues that the contractor may have.  The government was under orders to NEVER authorize or even suggest that they WOULD authorize any changes to anything.

After a month or so, the contractor’s project manager began submitting requests for changes.  First it was seemingly small things like changes to the outline of several documents.  Request denied? Then they found several logic errors or inconsistencies in flows between screens and processes. They argued that the cost of fixing the problem now would be orders of magnitude cheaper now, rather than after all was coded.  The government said: ‘implement it with that error, and we’ll deal with it AFTER you delivery everything…with errors.  NO CHANGES! 

The honeymoon was officially over.

The PM made many other attempts to get the government to authorize changes.  The government wouldn’t.  The government had another round of high-level meetings with executives, lawyers, contracting officers from both sides.  The PM was replaced, with apologies from the contractor’s executives.

The contractor acquired and installed hardware and database management software and other tools into the development facility.  That kept the appearance of progress and optimism high.

At some point the contractor started missing key deliverable dates, and the quality of deliverables slipped.  There were more high-level meetings, more apologies, and another project manager.

It became clearer that whatever was happening inside the contractor’s firm was working against the government’s interests, and against the ability and will of the contractor to meet its contractual obligations. Clearly their capture strategy was an underbid and ‘getting well’ on requirement modifications. Memory tells me that they went through three or four project managers before all was said and done.

I don’t remember exactly what event triggered the government’s decision to terminate the contract, but the only remaining decision was to terminate for default or to terminate for the convenience of the government.  The government’s project office kept great records that could have supported a strong legal default process (where the government could recover monies paid). But the decision from the government’s legal shop was that they didn’t have the horsepower to engage in a legal battle with the contractor. So the legal office pushed the program to terminate the contract ‘for convenience’ (where the government would neither recover monies paid nor seek money damages). 

The deal that was eventually negotiated is that the government took ownership of the hardware and some of the commercial software products, and the contractor walked away with whatever revenue they had been paid in the form of progress/deliverables payments.

The hardware was given to another program in the agency and the program began a new attempt at a much more modest effort to automate using internal IT staff.  While the government learned much about the inner workings of its program through the years of analysis and design that it could leverage in it’s ongoing modernization effort.  This effort was a huge failure.

Years later, one of our team met one of the contractor’s deputy project managers.  He told the story of how the legal department had advised the cognizant VP that the contract terms and conditions were too tight and risky to bid on. And advised a no-bid.  The VP decided to ignore that advice.

It is hard to imagine a more diligent government effort at attempting a fixed price IT contract, and a contractors equally diligent effort at attempting to weaken the terms and conditions of the contract to which they were legally bound.  But it is standard fare for contractors to exploit or create weaknesses in contracts.  In this case they didn’t succeed.  Nor did the government get a system. 

For most people, I guess this ONE program, ONE contractor, ONE set of circumstances isn’t proof of anything about the viability of fixed price contracting for IT solutions. But I think that it was a perfect experiment that showed that the contracting community has problems of its own.  These problems and practices do not get enough exposure and discussion.  If they did, the dialog about ‘fixing’ federal IT contracting would be more analytical about the IT contractor side of the federal-private sector equation.  While governments' failures receive much public exposure, the failures of some of the best in the business receive very little exposure (or quickly fade).  

More on comparing government IT failures to private IT failures in a future post.

EPILOG:  The contractor was Martin Marietta Data Systems (MMDS), which no longer exists as far as I can tell. At the time they had a good reputation (so it wasn’t just some little inexperienced company, it was part of a very large corporation). Looking at public records now, it seems that it’s parent company was involved in defending against a hostile takeover during the time of this effort. 

It is not clear to me how independent the MMDS business unit was from the other MM entities, or how, or if they were effected by the takeover activities. But even when the government chooses a reputable firm, nothing guarantees against negative effects of internal corporate issues, incentives, and bad decisions.

Please post comments and ideas on this topic.