Engineering the SpaceX way

I know not everyone is a fan of Elon Musk, especially if their primary source of information is Jalopnik articles on his tweets. But SpaceX is massively transforming the rocket business, and this process is clearly on display. Not all of this applies to someone welding up a race car in their garage, but the first three steps do.

Suggestion: I think this thread would be so much more useful if we discussed the ideas instead of our opinions of the person.

Musk overviewed his five step engineering process, which must be completed in order:

1. Make the requirements less dumb. The requirements are definitely dumb; it does not matter who gave them to you. He notes that it’s particularly dangerous if someone who is smart gives them the requirements, as one may not question the requirements enough. “Everyone’s wrong. No matter who you are, everyone is wrong some of the time.” He further notes that “all designs are wrong, it’s just a matter of how wrong.”
2. Try very hard to delete the part or process. If parts are not being added back into the design at least 10% of the time, not enough parts are being deleted. Musk noted that the bias tends to be very strongly toward “let’s add this part or process step in case we need it.” Additionally, each required part and process must come from a name, not a department, as a department cannot be asked why a requirement exists, but a person can.
3. Simplify and optimize the design. This is step three as the most common error of a smart engineer is to optimize something that should not exist.
4. Accelerate cycle time. Musk states “you’re moving too slowly, go faster! But don’t go faster until you’ve worked on the other three things first.”
5. Automate. An important part of this is to remove in-process testing after the problems have been diagnosed; if a product is reaching the end of a production line with a high acceptance rate, there is no need for in-process testing.

Additionally, Musk restated that he believes everyone should be a chief engineer. Engineers need to understand the system at a high level to understand when they are making a bad optimization.

Disclaimer: I'm not a Musk fan.

He got this right - Everyone’s wrong. No matter who you are, everyone is wrong some of the time

Try very hard to delete the part or process. I disagree - There are reasons for redundancy in aerospace applications.

Accelerate cycle time. I disagree - Haste makes waste.

"Move fast and break things" works for innovating a lot of businesses.  Launching E36 M3 and people into space is not one of them.

In reply to Mr_Asa :

I think that depends on your what qualifications you have for calling something "innovating".

A whole bunch of this sounds an awful lot like the background work that brought about the AK47, which, like, totally works. It also sounds very much like Henry Ford and the Model T, which also totally worked. 

In reply to Mr_Asa :

To counter that - who's launching more E36 M3 into space than the rest of the world put together?

This is not "move fast and break things", it's how to design a better machine. Breaking stuff is what testing is for. 

I'll have a slight counter to his approach.  I wonder how much bureaucracy and process was built into the space programs because they were more publicly funded and mistakes meant programs were shut down. Elon is funding this himself (so to speak) so he doesn't have to appeal to the great unwashed masses for the money. 
 

And as someone who's worked in the software tech industry for 20+ years, applying the same "good enough" process to things like rockets and cars scares me a bit. I've yet to see a software product (from small startups to huge customer base enterprise level) that's not a hot mess on the verge of crashing but has run that way for years. 

-Rob

In reply to Keith Tanner :

Well, the primary reason for that is that NASA has been underfunded for decades.  They no longer have the ability, so it has been outsourced.

Past that, Space X is sending metric tons of crap into space for the same reason any govt contractor is getting away with their ridiculous contracts.  At a certain point, you get momentum and end up as the default.  Then you start getting lawsuits against the govt for favoritism and eventually there is a soft reset.

And it is MF,BT.  Early Space X had multiple very prominent failures.  They seem to have gotten a lot of those out of their system, which is good considering that they are carrying humans and not just human remains or NASA payloads that get dumped into the sea, but if those steps remain in force for the company then it feels like just a matter of time before there is another failure.

As a last note, I know you wanted to separate the man from the company, but it really isn't possible.  "Move fast and break things" is a hallmark of pretty much all of Musk's companies. Tesla is especially bad with it.

The SpaceX failures you're thinking of - other than the first three attempts to get to orbit about 13 years ago - were landing attempts.  I think there was one flight that delivered the secondary cargo to a bad orbit and one satellite destroyed in a ground handling problem. Otherwise they've been highly reliable as an actual launch provider.

The actual launches worked almost every time, and landing the booster is just gravy. Everyone else was throwing them away. Now they've landed dozens of boosters and Boeing can't even get a capsule to reach the space station for any price. 

As for the contracts to launch, you need to take a closer look. They're beating everyone on price by an order of magnitude. It's not due to lawsuits or being entrenched. They're the most reliable and least expensive way to get stuff to orbit because they were the first to optimize for cost instead of performance and the first to actually attempt rapid reuse. 

And Boeing, not a government entity (spare me... ) is still trying to get their rocket into space while Space X has resupplied the ISS several times.  When NASA was properly funded, they got us on the moon in 8 years. 8. That's a government entity.

Teh E36 M3 said:

When NASA was properly funded, they got us on the moon in 8 years. 8. That's a government entity.

I'm not sure if you think this is a good thing or a bad thing

Mr_Asa said:

And it is MF,BT.  Early Space X had multiple very prominent failures. 

Uh... Have you ever looked at anyone else's spaceflight development program?  Spectacular failures at first is par for the course, that's why "rocket science" is one of the stereotypical hard things.

And no, it's not "Move fast, break things".  That phrase comes from Zuckerberg at Facebook and specifically referred to testing new stuff on the production site, rather than doing in-house testing before dumping it on unsuspecting users.  SpaceX does not do that, at least not outside the industry standard of offering substantial discounts for payloads on relatively untested rockets (historically there have been a lot of people willing to risk that because launch is so expensive).  This was only possible at Facebook because the users are not the customers.

Musk and SpaceX have transformed the launch industry.  The cost to get something to LEO on a standard disposable rocket was around $20K per pound for decades (doing it on the Shuttle cost about 3x that).  SpaceX charges about a tenth of that.

Mr_Asa said:

Teh E36 M3 said:

When NASA was properly funded, they got us on the moon in 8 years. 8. That's a government entity.

I'm not sure if you think this is a good thing or a bad thing

The difference between NASA in the 60s and NASA today isn't just funding, it's much more about culture and the degree to which they were left alone to get the job done back then.  NASA today is a Congressional jobs program more than a space program.

It's also open for debate about how useful the 1960s moon landings actually were, but that gets into politics.

codrus (Forum Supporter) said:

Mr_Asa said:

Teh E36 M3 said:

When NASA was properly funded, they got us on the moon in 8 years. 8. That's a government entity.

I'm not sure if you think this is a good thing or a bad thing

The difference between NASA in the 60s and NASA today isn't just funding, it's much more about culture and the degree to which they were left alone to get the job done back then.  NASA today is a Congressional jobs program more than a space program.

It's also open for debate about how useful the 1960s moon landings actually were, but that gets into politics.

But it's also the exact same- Apollo was a series of subcontractors building all of the parts and subcomponents.  So that part is exactly the same as it was.  The amazing part of Apollo was how they all managed to work together so well and get to the moon in 8 years.  Took some serious cubic dollars to do it.

Keith Tanner said:

I know not everyone is a fan of Elon Musk, especially if their primary source of information is Jalopnik articles on his tweets. But SpaceX is massively transforming the rocket business, and this process is clearly on display. Not all of this applies to someone welding up a race car in their garage, but the first three steps do.

Suggestion: I think this thread would be so much more useful if we discussed the ideas instead of our opinions of the person.

Musk overviewed his five step engineering process, which must be completed in order:

1. Make the requirements less dumb. The requirements are definitely dumb; it does not matter who gave them to you. He notes that it’s particularly dangerous if someone who is smart gives them the requirements, as one may not question the requirements enough. “Everyone’s wrong. No matter who you are, everyone is wrong some of the time.” He further notes that “all designs are wrong, it’s just a matter of how wrong.”
2. Try very hard to delete the part or process. If parts are not being added back into the design at least 10% of the time, not enough parts are being deleted. Musk noted that the bias tends to be very strongly toward “let’s add this part or process step in case we need it.” Additionally, each required part and process must come from a name, not a department, as a department cannot be asked why a requirement exists, but a person can.
3. Simplify and optimize the design. This is step three as the most common error of a smart engineer is to optimize something that should not exist.
4. Accelerate cycle time. Musk states “you’re moving too slowly, go faster! But don’t go faster until you’ve worked on the other three things first.”
5. Automate. An important part of this is to remove in-process testing after the problems have been diagnosed; if a product is reaching the end of a production line with a high acceptance rate, there is no need for in-process testing.

Additionally, Musk restated that he believes everyone should be a chief engineer. Engineers need to understand the system at a high level to understand when they are making a bad optimization.

 

To me, a better way to look at the entire process is to understand why things are the way they are, including questioning if that "part" is needed.  And by "part" I mean requirement, process, parts, etc.  

Requirements are not dumb, they are more misunderstood.  Assuming they are dumb dismisses the experience that put the requirement that went into place originally.  1000% in questioning the requirements to make sure they apply and/or are useful- buy they are not dumb.  

And the idea of accelerating the cycle time is laudable, assuming that you are not missing steps.  Many times I see this attempted where re-design happens prior to any testing- so you have no actual input to what needs changed. Fixing problems that you "skip" takes longer and more money.

Based on training I had 25 years ago- 2 and 3 are the same thing.  And oddly enough, 5 is almost a subset of 2 and 3.    When designing for assembly, one finds that automation eliminates the more variability in hand making stuff.

But, IMHO, the REAL key to all of the above is to just follow the process, questioning it where you can.  Too many times, I see steps skipped for *reasons* and more often comes back to bite someone than not.  Question requirements, question design, question process, etc- but don't skip steps just for the sake of skipping them.  If you need to speed up, spend more money allocating the resources to not skip steps.

The Boring Company used to have a good writeup on its website about how they approached the innovation problem.  They had a goal of (i think) making tunnel construction 10x as fast as it was before and turned it into ~10 discrete (high level) problems to solve.  Its pretty interesting.

I'm fully on board with #1 and #2.  I work in aerospace and there are lots of requirements that are misunderstood, or legacy performance requirements that exist because the company has slowly developed them over the years.  As Alfa said, these need to be questioned.

On a related note, there is also an interesting article somewhere on how Ford innovated with the Fiesta 1.0t by approaching like 10 discrete friction/efficiency issues independently and each shooting for a very small improvement.  When all was said and done they assembled the processes/technologies/designs and ended up with that hyper-efficient motor.

I work in government, there are people poking at the correct view of it.  If we get it right 999 times out of 1000 you will only hear about the one time. Each of the 1000 times, we spend an inordinate amount of time documenting the smallest thing to be accountable to the public (which is most of the governmental waste). That ONE time things go wrong? Results in OIG audits, hearings, etc that waste MORE time and taxpayer money as well as congressional hit jobs that might slash the budget while we get MORE requirements documenting small steps to be held MORE accountable (while we had already looked at that one time and pinpointed where it had gone wrong pretty much instantly and already applied a fix) 

The snowball rolling down that hill hadnt grown to nearly the size it is today back in the 60's and there is a ton more "anything government is inherently bad" mentality from the public today. 

In reply to Apexcarver :

Does it seem safe to think that Boeing is designing to not fail whereas SpaceX is designing to succeed?  

People brush off any failures by SpaceX, even when the impact is probably significant, but if Boeing fails- it's a big deal for NASA to deal with.

I am torn on Musk, but overall I think he is very smart, and i see no problem with this list.

I wish it was more adopted, even in things that arent engineering. Ive seen in different industries, the lack of questioning a process or requirement and how it stifles innovation and efficiency. People dont understand why a requirement or process is actually needed, so they skip it to their detriment or since no one has fleshed a requirement for them, they arent in a place to innovate and say this is a more efficient way to meet that requirement.

alfadriver said:

In reply to Apexcarver :

Does it seem safe to think that Boeing is designing to not fail whereas SpaceX is designing to succeed?  

People brush off any failures by SpaceX, even when the impact is probably significant, but if Boeing fails- it's a big deal for NASA to deal with.

there is a lot of interesting back and forth to it. It depends on which of the spheres that Boeing is working in. If there are human lives in the loop, its one thing. If it is an unmanned system you can afford to put more risk on the table for the reward of accelerated development. If we set out to never fail, we miss out on the learning opportunity that failure brings.   There was a recent failure for Boeing with the FAA and type modification approvals, that was definitely an area that a process improvement for implementation was drastically needed in a mass-life critical application for example. 

I think his best point is point #6 - That everyone needs to understand the big picture. So much uselessness happens because people get to bickering (even with themselves) over junk that just doesn't matter. Good example: Me going to bed last night thinking about how to find the optimum angle for my diffuser with respect to horizontal. What I should have been thinking is what is the best way to strongly and simply attach a diffuser to the car, and then let the angle work itself out as long as it is in the realm of normal. A solid, light, and strong diffuser at the slightly wrong angle  > a floppy diffuser that is heavy and at the 'perfect' angle. This also plays directly into the first point, don't make requirements dumb. 

#3 is genius. The smarter you are, the worse you get about obsessing over the useless. 

Finally, one more good example of #1. My buddy called me last night. He let his two camper batteries drain all the way flat and he wanted to know how to test them to see if they were ruined or not. I launched into looking up voltage tables and amp-hrs and getting a known load like a fan to run for a specific time and check voltage before and after blah blah blah. Then I realized he should simply 'camp' for the night in his driveway and see if the batteries had enough reserve left to make him comfortable. Yes, you can test like crazy but at the end of the day you're designing for a real life purpose. Don't forget that.  

alfadriver said:

In reply to Apexcarver :

Does it seem safe to think that Boeing is designing to not fail whereas SpaceX is designing to succeed?  

People brush off any failures by SpaceX, even when the impact is probably significant, but if Boeing fails- it's a big deal for NASA to deal with.

I think that's a good summation. SpaceX is running tests where it's OK to fail - I'm specifically talking about the Starship development we are seeing right now, but also the early landing attempts. They're doing things that just haven't been done, like flying a design of engine that nobody has ever made work outside the lab, or testing out a different way of handling re-entry and of course landing. With Starship, they're working incrementally, starting with the minimum viable test item and working from there. They've lost some hardware along the way which may be viewed as significant by bystanders, but it's not critical hardware - it's test hardware.

Boeing - here we're talking about the Starliner capsule - is not doing live tests. They're relying heavily on simulation and various integration tests. This means that every flight has to work perfectly, there's no room for any sort of failure. At least one of their major problems in their first flight came from a poor test design. They split the integration test into two halves, and the failure happened at the handoff between those two flight modes. That's a really important lesson - your testing is only as good as your test design.

It's worth noting that the Crew Dragon (the SpaceX crew capsule) and Starliner were both part of the NASA Commercial Crew program. Boeing got a lot more money for their version because...well, because they're better at writing proposals that cost a lot of money, I think...and both were delayed for various reasons. But when it came to the first actual flight test, SpaceX had a bunch of practical incremental tests behind them and Boeing did not. Crew Dragon performed perfectly and is now providing taxi service to the ISS while Boeing had a number of critical failures and almost lost the capsule. You can't blame NASA funding or publicly traded companies for this difference.

My point in the original post was what we can take from this as racers. I did not explicitly  state this which I should have because my purpose was not necessarily to discuss rockets. My takeaways:

- make sure you've properly defined the problem you're trying to solve. For example, are you trying to make your car lighter or are you trying to lower your lap times?
- simplify your design. "The best part is no part". It's lighter, cheaper, easier to maintain and has fewer failure modes.
- do real testing. Get your car out there, turn laps. You may not set the lap record but if you're testing properly you'll learn something. You may find failure modes you never expected.
- iterate. 
- anyone working on the design should have a view of the whole machine and understand how any changes might affect other systems. This means both negative and positive effects.

L5wolvesf said:

Disclaimer: I'm not a Musk fan.

He got this right - Everyone’s wrong. No matter who you are, everyone is wrong some of the time

Try very hard to delete the part or process. I disagree - There are reasons for redundancy in aerospace applications.

Accelerate cycle time. I disagree - Haste makes waste.

About deleting parts - it's about removing things that unnecessary, not removing required redundancy. For example, it's looking like the new SpaceX booster is going to use the ullage gas release as a reaction control system on descent. This gas has to be vented anyway, so why not take advantage of that to remove a complex system that needs a very large amount of support mechanisms? The ullage venting will be more reliable than the thrusters in all likelihood. The reliability of the Falcon 9 is showing that this philosophy is working.

Haste may make waste, but failing to iterate means you're stagnant. It's possible to walk the line in the middle. The context of this quote is the very, very slow-paced launch industry where it can take six months to orchestrate a test firing of some proven engines bolted to a fuel tank. But if we look at it as racers - F1 iterates on designs from race to race. If the teams only update their car every few months, they visibly fall behind the pack. SpaceX is mostly in a race against themselves, but it's the same concept.

Some of these are valuable. Some are not. Some are harmful. Elon is not an engineer, and I see nothing in his history to prove that he is particularly intelligent. He made some good decisions and some huge risks which paid off. The rest are a series of variables which cannot be distilled down into some magic formula. The business section of the bookstore always has the current hot ticket, and it always has, and it always changes, because very little which drove success for that person is applicable in any other field in any other industry at any other time. He's just a man, same as so many others.

Which are harmful? Which are valuable? Get past the source, discuss the content. That's a lot more useful than offering opinions on someone's personality or intelligence level.