For those doing this on a 1.4 petrol you only need to screw the top bolt back in and tap it a few times to move the top bush out. The bushes are made of steel so a bit of WD40 did the trick.
You can then use a piece of wood to hammer outwards on the alternator housing and you'll see it pivoting forward on the betom bush. Once the top mounting is free you can wiggle the bottom free. The hammering did break a fin or two so be careful if you want to return your old alternator for a discount.
2004 Audi A2 1.4 TDI 75 SE - Mods - Remapped, Red sports leather, RNSE, Votex bodykit, 18" TT v6 alloys, H&R 25mm springs, Custom grille, Alcantara Doorpockets with Bentley grilles, Cruise, ATE powerdiscs and ceramic pads...more to come