Good idea until the “hacker” in said scenario is using your phone. The authentication code will be delivered to the device (s)he is using.
Added email authentication sent to the address you used when signing up would solve the issue or having new cards added or old cards removed.
Granted it’s not as secure as requiring email authentication on every withdrawal, but in my scenario, the worst that can happen is the hacker withdraws money back into your own bank account. Perhaps at a loss, yes, but my personal risk tolerance wouldn’t mind that. I’d feel it a bit of a nuisance having to check my email every time I withdraw.