Is your load DC? Do you need the relay to open while the load is drawing full power? If the answer is "no" to either of the above, the relay is probably going to be OK, otherwise it's insufficient and will risk welding closed due to an inability to break the load arc (the contacts don't move far enough apart).
Your problem is supply voltage not load current, so putting them in parallel will not help. You need to put them in series; that will help a lot because you now have twice the contact spacing available. It is only a delaying tactic though because any slight timing error between the relays (e.g. one has a tiny bit stronger spring) means that one relay will take most of the arc energy. It may not be as prone to welding closed, but the contacts will erode quickly (which increases contact resistance...) and if one relay does weld closed, you probably won't notice because the other one will continue working for a little while.
If your load is inductive (e.g. a motor), the relay is in NO WAY sufficient. With inductive loads, you need to design a snubber circuit to go across the relay, compute the voltage peak that the snubber produces with the load in question, and then choose your relay ratings from there. Otherwise it will weld closed very quickly.