The death penalty is not expensive because of how we kill people (thus putting the lie to Jarhyn's ignorant "gram of heroin" suggestion). The death penalty is expensive because we take an insane amount of precautions to be sure we're not doing something very, very bad. Even given those measures, we still occasionally do something very, very bad.
How would you suggest we make it the cheaper option?
Assume you don't know ahead of time that the person is 100% guilty and utterly incapable of rehabilitation. What's the bargain method for making that judgement?
Instead of using it as punishment, use it as toxic waste disposal. Generally that means someone chooses it themselves, through consent or action which is clearly communicated as being consent at the far end of an escalation of force. Choose how you want to be treated: rehabilitation through education? Exile to a separate community where you will have a roof, and food and are expected to do what work you can find there? Asylum in a foreign country? Rehabilitation through chemical or surgical means? Or death? If we can't educate you, choose again! If you are tired of exile, retry education! If it still doesn't work, see if anyone else wants you, or see if we can go under the hood to fix the problem. And if you try to circimvent the walls, or the system, then you choose death. No long waits, no appeals. Just disposal.