I think it's a good idea - ultimately if it were widely accepted, I bet that there would be a shit-ton fewer accidental deaths. Even if there are a few glitches/hacking/whatever, is that really any different from some drunk idiot ramming into you head on at full speed? I think a lot of people have an illusion of total control when driving which can be very misleading. Excellent defensive drivers are killed on the road every day, through no fault of their own.
I'm not in favour of eliminating (optional) manual control though, at least not in the foreseeable future.