I think most in the commercial space industry would agree that COTS is one of the best things the government has done to help promote commercial space in a long time. While I think that overall this program has been run pretty well by government standards, some recent discussions on a few threads at NASASpaceflight.com (starting with this comment by “general” on one thread, and this comment by Gary on another) got me thinking about some issues with the current approach to COTS. My goal in bringing these issues up isn’t to demean the good work that the COTS program has been doing (I know and respect many people involved in COTS both at NASA and at both of the COTS contractors), but more to spur some discussion on things that can be improved.
- Overemphasis on Unmanned Cargo: I’ve previously discussed on this blog what I think is one of the key deficiencies of the current COTS approach–the focus on cargo delivery to ISS while ignoring the crew delivery issue. The problem with a focus on just cargo delivery to ISS is that a cargo delivery capability doesn’t really open up many other markets. Sure, there may be a few flights here and there for DragonLab, but the reality is that without a passenger delivery capability, there just isn’t much need for such capabilities, outside of NASA’s ISS needs. Bigelow, for instance isn’t going to be providing much demand for cargo flights if he can’t get people to his station. The ability to safely fly people to orbit, and to be able to deliver them to/retreive them from space stations is a lot more useful. Not only would NASA be a potential customer, but also Bigelow, and even free-flights. And once you start getting more demand for people flying to space, demand for cargo will increase as well. Basically, by only funding the unmanned part of COTS, NASA is forcing those COTS competitors into markets for which there is little other non-NASA demand.Of course, NASA has lots of reasons for wanting to do things this way. NASA as an institution wants to build and fly its own rockets, and being able to continually point at a US manned spaceflight “gap”, and being able to point at other nation’s anemic manned spaceflight programs as threats, makes it that much easier to continue getting funding for their anachronistic manned spaceflight projects. Think about it. If there were one or two US commercial options for getting people to the space station, do you really think there would be as much urgency for continuing to throw good money after bad on Ares-I? Most congresspeople don’t have much of a vested interest in keeping the Shuttle Workforce humming along sucking up taxpayer dollars. But the idea of there being a “gap” in US manned spaceflight, and US access to the ISS resonates more. Especially when there is no non-NASA alternative. Take that unifying threat away, and all of the sudden convincing Congresspeople in states other than Alabama, Texas, and Florida that it’s a national priority to keep throwing billions of dollars a year keeping people in their district employed, doing something that the market is already providing, is going to be come a lot trickier.
- The “Skin-In-The-Game” Provision: One of the defining features of NASA’s implementation of COTS under Mike Griffin’s tenure was the requirement that COTS companies match NASA funding, to “put some skin in the game”. Jorge Frank (whose opinions I normally agree with) really liked this provision, calling it one of the best things about COTS, and stated that “It limits the field to serious providers that are confident that they have a business case for their spacecraft beyond selling rides to NASA.” The problem is, that speaking from the record, this hasn’t been the case at all. Look at Orbital Sciences. They have pretty much stated that they don’t think that there’s any market for Cygnus other than ISS resupply. SpaceX is trying to do Dragon-Labs, but even then it isn’t a big demand driver. In both cases, they’re not expecting to make money selling their cargo delivery services to anyone else, but are expecting to make most of their money off of the new launch vehicle that will be developed in order to lift the capsule. Orbital wants to go after the Delta II market, and SpaceX wants to edge-in on the EELV market. But let me come back to that point in a second.Basically, when you combine these first two points, you can see all sorts of perverse incentives created. In order to raise a large amount of cash, you need to have a large and solid market to justify it to investors. But as I mentioned before, there’s not a big non-NASA market for cargo deliveries to LEO stations, and even the NASA market isn’t really that big, all things told. And more importantly, at the time the COTS contracts were handed out, it wasn’t obvious if a COTS competitor would actually get any of that follow-on demand, even if they delivered. Without having a realistic non-NASA source of demand for the capsule part of the equation, the skin-in-the-game requirements pretty much killed the case for anyone trying to propose doing a capsule on an existing launch vehicle. Without developing a passenger delivery capability, the only market that could justify the kind of skin NASA wanted in the game was possibly a launch vehicle market. Which is a good part of why all three of the COTS winners (SpaceX, RpK, and then OSC when RpK couldn’t raise money) were basing their actual market case on developing new launch vehicles.So, not only did the skin-in-the-game requirement make it really hard for entities that didn’t have billionaire backing (or large existing lines of business) to compete, but it also drove the technical and execution risk for the program up by biasing selection towards companies that had to develop both a launch vehicle and a prox-ops spacecraft. Low-technical risk approaches that used existing launch vehicles wouldn’t actually develop hardware that would provide enough non-NASA business to justify enough outside investment to meet NASA’s skin-in-the-game requirements. Quite frankly, if COTS fails to deliver, there’s a high probability that it will be due to the fact that both COTS competitors need to develop both a launch vehicle and a capsule.
- Payment for “Soft” Milestones: One of the other distinguishing features of COTS is that it is a firm, fixed-price contract, where payment is only given on achievement of specific milestones. The idea being that in theory this gives the company a lot more flexibility on how to achieve its goals, while the government only has to pay for actual results, not just for effort. Unfortunately, this is also a nice theory that got watered down in practice. If you look at both company’s COTS contracts, you’ll notice that both of them make the vast majority of their money off of meeting “soft” milestones, such as performing design reviews, raising money, etc. By the time you get to most of the hardware milestones, the government has already paid out most of the value of the contract–which greatly reduces the benefit of this approach.  In fact, if I read SpaceX’s contract information correctly, they get paid for the first two of three COTS demo flights for just getting the flight off the ground–even if it fails. They don’t have to actually have a succesful COTS mission to collect any but the last payment.One of the key tenets of the original proto-COTS concept Gary Hudson had pitched to O’Keefe’s NASA was that other than an initial pump-priming kickoff payment, all other payments would be for hard technical milestones. That would’ve reduced the government’s risk a lot, since until technical milestones start being achieved, it only has the kickoff capital at risk. Second off, it would emphasize rewarding actual successful development of hardware, not just paying for paper analysis like it has always done in the past.There’s actually a fair deal of danger here for COTS and future COTS follow-ons. The worst thing that could happen would be for OSC and SpaceX to collect most of their money, and then have some high profile failures right at the end. The government would see it as having spent lots of money on small space firms, and then losing their shirt. Something like COTS wouldn’t happen again for a long time. Had they stuck to Gary’s suggestion, technical milestones would’ve been earlier in the program, and therefore, if there was a failure, it would’ve been a lot less costly to the government.
Now, I’m not saying that COTS is doomed to failure or anything like that. I’m hoping and praying that SpaceX and OSC are able to turn this into a success. I am suggesting though that in the future, for COTS-like programs, that it would be a good idea to make sure that “skin-in-the-game” requirements are better matched with realistic expectations on the size of non-NASA markets, that we be more careful not to bias incentives in a way that encourages larger technical risks than ought to be taken, and that rewards actual hardware success instead of paying a lot for paperwork.
[Additional Thoughts: After getting a good night’s sleep, I had a few additional thoughts I wanted to tack on. First, I wanted to point out what I think is one of the best things about the whole COTS approach–the fact that the government is giving the COTS contractors a lot freer hand in how they go about their development projects. When you compare this to how NASA’s running Constellation, you can see how big of a difference this is we’re talking about. Also, by having fixed-price payments based on technical milestones, it removes the need for anywhere near as much direct oversight, both on company accounting (a big headache for cost-plus contracts), and on the technical side. If the company doesn’t take advantage of specialized NASA resources, and ends up botching a technical milestone, they don’t get paid. The incentives all point in a lot closer to the right direction.
One other comment on the skin-in-the-game question, is to remember the X-33 debacle. One of the main reasons why LM was given the award (instead of the DC-X team) was that they were willing to put a lot more skin-in-the-game. The problem is, the willingness to put in money doesn’t necessarily correlate with mission success, competence, or even a desire to see the project succeed! At least one anecdote said that LM put the money in more to prevent the competition from getting something to work than because they really believed on a corporate level that X-33 was going to lead to Venturestar. On the other hand, I am somewhat wary of giving the whole contract with no skin-in-the-game requirements, and no actual requirements to commercialize things. While using the other useful features of COTS (firm milestone based payments, less direct overhead/interference) is better than nothing, a good part of the point of COTS was as a pump-priming exercise. Without incentives clearly placed pushing the COTS winners towards developing these services for commercial applications, a lot of the benefit is wasted.
Lastly, the basic concepts of COTS (fixed-cost milestone-based payments, focusing on areas with a potential for non-NASA customers, etc) could actually be a decent fit for developing other pieces of space infrastructure such as depot, tugs, etc.]
Jonathan Goff
Latest posts by Jonathan Goff (see all)
- NASA’s Selection of the Blue Moon Lander for Artemis V - May 25, 2023
- Fill ‘er Up: New AIAA Aerospace America Article on Propellant Depots - September 2, 2022
- Independent Perspectives on Cislunar Depotization - August 26, 2022
In a way, your comments on #2 and #3 seem a little at odds. First you say that the “skin in the game” provision kept out a number of competitors because they couldn’t raise sufficient private capital, but then you also claim that NASA shouldn’t even give up it’s matching funds until the company has achieved nearly all of it’s design, review, development, and manufacturing milestones. How exactly does this latter situation encourage more companies to take on the risk of developing new systems, especially if many of them can’t even raise enough money up front to match NASA’s eventual contribution?
Eric,
Oh, I agree that paying only on technical milestones (after a first kickoff payment to “prime the pump”) does put some sort of skin-in-the-game requirements. The difference is that you’re raising much smaller chunks of money, and instead of raising it to meet some currently unproven market, you’re raising it to make it to the next milestone of a firmly negotiated contract. It’s a lot easier to get a “line of credit” setup to help keep cash reserves while carrying out a government or commercial contract than it is to raise enough equity to do a full development program on speculation.
I wasn’t saying skin-in-the-game was entirely evil, I was just pointing out that it was taken overboard in a dangerous way.
~Jon
Jon,
Your points all make sense to me. What I want to know now is:
1) How do we go about getting these ideas to the right people in government who can make a difference?
2) How do we go about convincing said government officials of the validity of these concepts against the entrenched influence of NASA vested interests?
I think the main point of #2 was that skin-in-the-game requirements need to be well matched to the risk-adjusted net present value of potential revenue sources. If the contract payment for reaching the hard technical milestone is enough to pay for the technical development all by itself then there’s no conflict between #2 and #3.
It’s only when you expect companies to pay for part of the development by raising money based on potential revenue from other markets, and those markets are insufficient, that you crowd new players out of the competition. And that’s true whether the milestones are hard or soft.
Jon,
For a fully funded COTS-D program, what kind of numbers do you think would be needed?
How many teams would you believe would be needed to make it competitive? Personally I would like to see a bigger field of companies and more ideas then just one or two.
Should COTS-D allow someone to just design the crew capsule and buy and off the shelf launcher, like building a capsule then using the Falcon 9?
Vladislaw,
1. Not sure how much would be needed, but somewhere in the $500M-1B range should get you enough to fund at least 2 main efforts, and maybe one or two backup study efforts (in case one of the main efforts hits snags early on). Might be doable with less, but I’m not really sure.
2. I’d like to see two or three primary ones, and maybe some small ~$10-20M secondary ones (especially if the secondary ones have some high-risk/high-payoff approaches they would be maturing as part of that $10-20M). But at least two teams. This is higher risk, and part of the way of mitigating that risk is by having redundancy. And unlike the “cargo to space station” market, the people to orbit market has a lot more potential room for growth. Bigelow for instance would really like to have two options, and quite frankly I think NASA ought to as well. That way, if a) something goes wrong with one supplier’s launch vehicle or capsule, there’s still at least one backup that can be leaned on while the problem is getting sorted out, b) it’s much easier to on-ramp new providers. If you’re already buying services from two providers, it’s much easier to bring on a third if they’re better than it is if you have a sole-provider.
3. Making the rules/incentives such that it doesn’t disincentivize using existing launch vehicles would be a good idea. Just making a crew capsule that can fly on an existing LV is a lot lower risk than building a whole new launch system. But we shouldn’t bias it away from innovative approaches either. If someone wants to propose a small 2-person RLV, they should still have a fair shot at it.
~Jon
Pingback: What’s Working - and Not Working - in NASA’s COTS Program at Parabolic Arc
I think that one reason for the watering down is that NASA staff in are focussed primarily on spending their budget, and hopefully increasing it in the following years.
High-risk and long term COTS contracts represent a lot of personal career risk for such individuals working in NASA. What happens if SpaceX negotiates a true (un-watered down) COTS contract, spends a couple of years developing paper designs and technologies, then changes their mind and pulls out, not pursuing the expensive and difficult developments parts of the contract. Would this not lead to a mad rush where NASA tries to spend this budget somewhere else, quite possibily in a non-COTS program, managed by a competiting bureaucrat in NASA?
Steve,
Having hardware-based milestones doesn’t necessarily imply having no milestones for a long time and then a big rush at the end. Especially if you don’t have an Elon Musk funding you, you have to find a way to spread the milestones out more evenly. I think this can be done, it just requires a more incremental development process than the “we’ll design everything, review everything, then build everything, and then we’ll demonstrate how smart we are” approach that NASA prefers…
~Jon
Pingback: How to Improve COTS « The Four Part Land