I think most in the commercial space industry would agree that COTS is one of the best things the government has done to help promote commercial space in a long time. While I think that overall this program has been run pretty well by government standards, some recent discussions on a few threads at NASASpaceflight.com (starting with this comment by “general” on one thread, and this comment by Gary on another) got me thinking about some issues with the current approach to COTS. My goal in bringing these issues up isn’t to demean the good work that the COTS program has been doing (I know and respect many people involved in COTS both at NASA and at both of the COTS contractors), but more to spur some discussion on things that can be improved.
- Overemphasis on Unmanned Cargo: I’ve previously discussed on this blog what I think is one of the key deficiencies of the current COTS approach–the focus on cargo delivery to ISS while ignoring the crew delivery issue. The problem with a focus on just cargo delivery to ISS is that a cargo delivery capability doesn’t really open up many other markets. Sure, there may be a few flights here and there for DragonLab, but the reality is that without a passenger delivery capability, there just isn’t much need for such capabilities, outside of NASA’s ISS needs. Bigelow, for instance isn’t going to be providing much demand for cargo flights if he can’t get people to his station. The ability to safely fly people to orbit, and to be able to deliver them to/retreive them from space stations is a lot more useful. Not only would NASA be a potential customer, but also Bigelow, and even free-flights. And once you start getting more demand for people flying to space, demand for cargo will increase as well. Basically, by only funding the unmanned part of COTS, NASA is forcing those COTS competitors into markets for which there is little other non-NASA demand.Of course, NASA has lots of reasons for wanting to do things this way. NASA as an institution wants to build and fly its own rockets, and being able to continually point at a US manned spaceflight “gap”, and being able to point at other nation’s anemic manned spaceflight programs as threats, makes it that much easier to continue getting funding for their anachronistic manned spaceflight projects. Think about it. If there were one or two US commercial options for getting people to the space station, do you really think there would be as much urgency for continuing to throw good money after bad on Ares-I? Most congresspeople don’t have much of a vested interest in keeping the Shuttle Workforce humming along sucking up taxpayer dollars. But the idea of there being a “gap” in US manned spaceflight, and US access to the ISS resonates more. Especially when there is no non-NASA alternative. Take that unifying threat away, and all of the sudden convincing Congresspeople in states other than Alabama, Texas, and Florida that it’s a national priority to keep throwing billions of dollars a year keeping people in their district employed, doing something that the market is already providing, is going to be come a lot trickier.
- The “Skin-In-The-Game” Provision: One of the defining features of NASA’s implementation of COTS under Mike Griffin’s tenure was the requirement that COTS companies match NASA funding, to “put some skin in the game”. Jorge Frank (whose opinions I normally agree with) really liked this provision, calling it one of the best things about COTS, and stated that “It limits the field to serious providers that are confident that they have a business case for their spacecraft beyond selling rides to NASA.” The problem is, that speaking from the record, this hasn’t been the case at all. Look at Orbital Sciences. They have pretty much stated that they don’t think that there’s any market for Cygnus other than ISS resupply. SpaceX is trying to do Dragon-Labs, but even then it isn’t a big demand driver. In both cases, they’re not expecting to make money selling their cargo delivery services to anyone else, but are expecting to make most of their money off of the new launch vehicle that will be developed in order to lift the capsule. Orbital wants to go after the Delta II market, and SpaceX wants to edge-in on the EELV market. But let me come back to that point in a second.Basically, when you combine these first two points, you can see all sorts of perverse incentives created. In order to raise a large amount of cash, you need to have a large and solid market to justify it to investors. But as I mentioned before, there’s not a big non-NASA market for cargo deliveries to LEO stations, and even the NASA market isn’t really that big, all things told. And more importantly, at the time the COTS contracts were handed out, it wasn’t obvious if a COTS competitor would actually get any of that follow-on demand, even if they delivered. Without having a realistic non-NASA source of demand for the capsule part of the equation, the skin-in-the-game requirements pretty much killed the case for anyone trying to propose doing a capsule on an existing launch vehicle. Without developing a passenger delivery capability, the only market that could justify the kind of skin NASA wanted in the game was possibly a launch vehicle market. Which is a good part of why all three of the COTS winners (SpaceX, RpK, and then OSC when RpK couldn’t raise money) were basing their actual market case on developing new launch vehicles.So, not only did the skin-in-the-game requirement make it really hard for entities that didn’t have billionaire backing (or large existing lines of business) to compete, but it also drove the technical and execution risk for the program up by biasing selection towards companies that had to develop both a launch vehicle and a prox-ops spacecraft. Low-technical risk approaches that used existing launch vehicles wouldn’t actually develop hardware that would provide enough non-NASA business to justify enough outside investment to meet NASA’s skin-in-the-game requirements. Quite frankly, if COTS fails to deliver, there’s a high probability that it will be due to the fact that both COTS competitors need to develop both a launch vehicle and a capsule.
- Payment for “Soft” Milestones: One of the other distinguishing features of COTS is that it is a firm, fixed-price contract, where payment is only given on achievement of specific milestones. The idea being that in theory this gives the company a lot more flexibility on how to achieve its goals, while the government only has to pay for actual results, not just for effort. Unfortunately, this is also a nice theory that got watered down in practice. If you look at both company’s COTS contracts, you’ll notice that both of them make the vast majority of their money off of meeting “soft” milestones, such as performing design reviews, raising money, etc. By the time you get to most of the hardware milestones, the government has already paid out most of the value of the contract–which greatly reduces the benefit of this approach. In fact, if I read SpaceX’s contract information correctly, they get paid for the first two of three COTS demo flights for just getting the flight off the ground–even if it fails. They don’t have to actually have a succesful COTS mission to collect any but the last payment.One of the key tenets of the original proto-COTS concept Gary Hudson had pitched to O’Keefe’s NASA was that other than an initial pump-priming kickoff payment, all other payments would be for hard technical milestones. That would’ve reduced the government’s risk a lot, since until technical milestones start being achieved, it only has the kickoff capital at risk. Second off, it would emphasize rewarding actual successful development of hardware, not just paying for paper analysis like it has always done in the past.There’s actually a fair deal of danger here for COTS and future COTS follow-ons. The worst thing that could happen would be for OSC and SpaceX to collect most of their money, and then have some high profile failures right at the end. The government would see it as having spent lots of money on small space firms, and then losing their shirt. Something like COTS wouldn’t happen again for a long time. Had they stuck to Gary’s suggestion, technical milestones would’ve been earlier in the program, and therefore, if there was a failure, it would’ve been a lot less costly to the government.
Now, I’m not saying that COTS is doomed to failure or anything like that. I’m hoping and praying that SpaceX and OSC are able to turn this into a success. I am suggesting though that in the future, for COTS-like programs, that it would be a good idea to make sure that “skin-in-the-game” requirements are better matched with realistic expectations on the size of non-NASA markets, that we be more careful not to bias incentives in a way that encourages larger technical risks than ought to be taken, and that rewards actual hardware success instead of paying a lot for paperwork.
[Additional Thoughts: After getting a good night’s sleep, I had a few additional thoughts I wanted to tack on. First, I wanted to point out what I think is one of the best things about the whole COTS approach–the fact that the government is giving the COTS contractors a lot freer hand in how they go about their development projects. When you compare this to how NASA’s running Constellation, you can see how big of a difference this is we’re talking about. Also, by having fixed-price payments based on technical milestones, it removes the need for anywhere near as much direct oversight, both on company accounting (a big headache for cost-plus contracts), and on the technical side. If the company doesn’t take advantage of specialized NASA resources, and ends up botching a technical milestone, they don’t get paid. The incentives all point in a lot closer to the right direction.
One other comment on the skin-in-the-game question, is to remember the X-33 debacle. One of the main reasons why LM was given the award (instead of the DC-X team) was that they were willing to put a lot more skin-in-the-game. The problem is, the willingness to put in money doesn’t necessarily correlate with mission success, competence, or even a desire to see the project succeed! At least one anecdote said that LM put the money in more to prevent the competition from getting something to work than because they really believed on a corporate level that X-33 was going to lead to Venturestar. On the other hand, I am somewhat wary of giving the whole contract with no skin-in-the-game requirements, and no actual requirements to commercialize things. While using the other useful features of COTS (firm milestone based payments, less direct overhead/interference) is better than nothing, a good part of the point of COTS was as a pump-priming exercise. Without incentives clearly placed pushing the COTS winners towards developing these services for commercial applications, a lot of the benefit is wasted.
Lastly, the basic concepts of COTS (fixed-cost milestone-based payments, focusing on areas with a potential for non-NASA customers, etc) could actually be a decent fit for developing other pieces of space infrastructure such as depot, tugs, etc.]