In this research, a small humanoid robot and a bicycle of comparable size are constructed. Like the human rider, the robot is designed to pedal, balance and steer the bicycle. We particularly focus on the design of control system for the robot to balance and steer the bicycle using the handlebar. The control system is novel that it is capable of estimating the uncertain center of gravity of the robot-bicycle system and then incorporating such an estimation to enhance control performance. The control system design is based on a general control framework which can establish asymptotic stability under unknown measurement biases. The stability of the control system is theoretically proved and a systematic procedure to compute the control parameters is given. Both simulations and experiments verify that the proposed controller can automatically counteract the mass imbalance in the system and allow the robot to perform straight-line steering.