Like David says, there's little to compare in monetary terms...
For me too, the A2 feels special. I look at Prius's and think ha, I'm as economical as you, and I'm not having to drive a frickin' Prius! While the suspension on the A2 could have been better, (demonstrated loud and clear when I use the wife's Honda Jazz) I do feel like I'm driving in something very special indeed.
I've got leather/alcantara, I've got Dension iPhone integration (thanks timmus-Tom), I've got climate, I've got parking sensors even (thanks eBay). I haven't got £20k tho.
As much as I admire the A1s I have seen, I don't know if it would have the same appeal. A different, posh, swish appeal no doubt... But a few years down the line I imagine that would have changed.
It's your call, your money. Why not, if you want to?!