Did you setup the associated GPIO port and enabled the clocks for GPIOB and GPIOH? Never tried CAN but the GPIO port default register configuration might not be the one the CAN configuration requires. Default conf :
-analog mode
-push pull
-no pull up pull down
-...