It seems that the problem is potentiometer in the mirror assembly. Potentiometer is variable-resistor kind of device that enables mirror to actually "remember" the exact position -- just like the potentiometer for the sun roof. Potentiometer contacts may be dirty/worn/broken.
Unfortunantely, Audi don't supply mirror adjusting unit separately for cars with mirror memory function. The cost of whole unit for my car (4D2 858 531 BD 3FZ) is 315 UK pounds. I understand this doens't win the award for the most ridiculously expensive audi part, but still I'm not prepared to shell out this much.
I found a writeup about cost effective cleaning of potentiometers for Jeep vehicle, but couldn't find any writeup/info for Audis, though I've seen many posts about mirror return problem after the tilt function.
http://www.wjjeeps.com/mirrors2.htm#REPAIR
If anyone have experience of deep diving into the mirror insides please post here. Thanks.