This was with the Citrix ICA client (XenApp). I resolved this by temporarily renaming my Mozilla registry keys during the install and then renaming them back. It seems to be a conflict with Firefox.
I closed Firefox and renamed
(notice the "xMozilla") and it hid the keys from the ICA client install and worked fine.