I think so - I'm doing the left/right cables at the moment and without removing the console, access to the pin and clip that secure the centre cable to the lever would be very tight, to the point of being almost impossible. I'd be very worried about dropping something and just wouldn't risk it.
The console is a few torx screws and some pulling/swearing, took about 30 minutes to remove it in my case. Make sure the cable adjuster nut is eased off all the way back without coming off, get the handbrake lever as upright as you can once all the screws are out and the console comes away fairly easily.