No material object can stand up to 11 km/sec (Theia's minimum possible speed) impact velocity. "Bounce" isn't a meaningful concept at such energy levels.
Furthermore, there's a problem with orbital mechanics. While you can conceive of an object that just barely crept in over the fence and lost enough energy in a glancing collision to attain capture you're left with the problem that barring gravitational effects your orbit always contains the last place an impulse was applied. Do an aerocapture maneuver and your orbit dips into the atmosphere--you have to burn to actually attain orbit. In a hypothetical lithocapture you have the same problem--your orbit intersects the ground. The only way you could pull this off is if you had great mountains on both sides, ground levels did not intersect.
However, even if you managed to pull that off you're facing another problem: You're far inside the Roche limit of the body you're hitting. Something the size of Theia is only meaningfully bound by gravity--and thus it comes apart as it enters the Roche limit. Even if you somehow managed a lithocapture it ends up scattered across the skies, not as an intact body. (Spacecraft can do such captures because they are held together chemically rather than gravitationally.)
In practice all captures are due to gravity. There are two possible scenarios:
1) Energy is transferred to some other body. The other body (in this case, a moon) rises, the object being captured loses enough energy to go into orbit. (When NASA plays cosmic billiards we don't care about the effect on the object we are using for a turn because it's so great compared to our spacecraft. When it's a moon vs moon the change can be considerable.)
2) A passing body (typically, another planet) adds/subtracts just enough energy to get the newcomer loosely bound. Prograde objects will generally remain loosely bound (and someday depart), retrograde objects will spiral in (Deimos, Phobos.) Prograde objects can spiral in if they manage to have an orbital period of less than one day--which can only happen with a body rotating so slowly that it has no orbits of more than one day. (ex: Venus. You'll be past the Hill Sphere before you have a 1 day orbit.)
Passing planets perturb orbits, this is typically random but if the planets involved are in a resonance it can cause substantial shifts.