E-type perovskite magnet HoMnO3 is considered to be a prototype Mn-Mn exchange-striction driven multiferroic. We have grown orthorhombic HoMnO3 single crystals and studied the directional dependence of their electric polarization P. The direction of P is not along the crystallographic a axis, as predicted, but along the c axis. The magnetic state is incommensurate. The direction P along the c axis is primarily induced by Ho-Mn exchange striction, and Mn-Mn striction in the incommensurate spin state leads to an antiferroelectric order along the a axis. We predict the rotation of P with temperature in commensurate E-type RMnO3 (R denotes magnetic rare earth).