(deleted)
It's fine.
You will find that super glue goes a frosty and will not stick to the coating on the back of the PCB. Also it is not flat because of all the through hole components so you are better off using stand off pillars. Super glue those instead.
I've gone to using low temp hot glue for gluing things together. It is strong yet generally removable, and sets in a minute or so. For small spots, apply using some on a tooth pick. Doesn't dry out in a tube or go bad either.
Have you tried that Gorilla glue? I find it the best at the moment.
In the right places, tie it down with thin zip ties.
I have always been a fan of epoxies.
+1 for epoxy.
I was thinking of trying epoxy but wasn't sure how well it would work. I've had epoxy crack when I've used it in other circumstances. If you pull hard can you pop off something that's been attached to a PCB with epoxy?
I used neutral cure silicone adhesive for attaching some small components recently. I first used the adhesive to attach a small piece of thin (1mm) silicone sheet to the PCB, in order to insulate the component from exposed circuitry. Then I used the same adhesive to glue the component to the sheet. These weren't under any mechanical stress however.
I also used this same approach to attach a 3.5mm jack to a board. The silicone retains flexibility when it is cured and the slight give when I push/pull on the jack could be a good thing. Or maybe not. I'm not sure that it forms an especially strong bond with the PCB.
I have made six projects that have a small PCB epoxied into a battery cage. The PCB has a 12mm pushbutton on it. The gadgets are used at least once a day. One of them is used three times a day. The gadgets have been in use for several years. The epoxy has never failed.
A few months ago the push button stopped working on one of them; debris got into it. I tried separating the PCB from the battery cage. The only thing that did not break was the epoxy. The PCB broke. The button broke. The battery cage broke.
Epoxy typically does not "stick" to other plastics #. That problem can be overcome by "roughing up" the plastic. I typically use course sandpaper.
# Plastic, like HDPE, makes a great surface when working with epoxy. Just let the epoxy cure then peel it off.
I'll have to epoxy next time... although there are instances where removability is a desirable trait.
Try mixing fibers, even chopped twine into epoxy, use it like putty. Fibers add tensile strength. 
I think that the thing to do is make mounts to attach the board to by less permanent means.
Maybe attach a clip or clips, something with V's to catch edges and hold? Or a turn-cam pinch action?
Try mixing fibers, even chopped twine into epoxy, use it like putty. Fibers add tensile strength.
Maybe have some fiberglass cloth in the tool box for those rainy days.
Cut as needed.
.
Sugru is a putty like substance that sets into hard rummer. It is also good for this sort of thing and is removable.
Silicone bonds excellently to PCB material. Needless to say, insulation is excellent and it absorbs vibration to some extent.
In fact (apart from taking 12 hours or so to cure definitively, especially when sandwiched between two large surfaces), the only problem with it, is removing it. It will come loose with persistence - and careful use of a knife - and with less damage than epoxy.
I have used it to mount a display monitor (directly) to the glass front door of my professional rooms. And I have had to replace the monitor once so far.
An adhesive product I'm fond of and applicable in many instances is 3M's double sided mounting tape. Super sticky, instant, has a little flex, easy to separate, and stackable. - Scotty
Hi,
At times I use four buttons of neutral cure silicon and hot glue to stabilize the PCB while the silicon sets.
This one, doesn't set rock hard but seals and holds, can be cut with knife when set.
Tom...... 
The application should determine the adhesive. All the above are valid suggestions, but for example, using silcone based adhesive to glue down a dc power jack would not be a good idea if that jack is to be used often AND soldered to a PCB. It would be better to use a hard epoxy, that will flex much less, and spread the stress of the jack over a larger footprint than just solder pads. Yet, soldering the same jack to a set of wires to go to the board, may be best glued to the enclosure with silicone! So, it depends...
You really have to use some common sense when selecting an adhesive. I am not a big hot glue fan, but even it has uses, particularly when temporarily attaching something, potting something, or quick sealing. Silicone is GREAT for sealing, not bad for adhering with flexibility, but may cause some slight corrosion while curing as it out gasses acetic acid, (that stops when fully cured though). I just wouldn't put a mess of it into a sealed enclosure with electronics...let it cure before sealing. Epoxy comes in many flavors, with many fillers and temp ratings. The type has to match what you want to do. Double sided tapes will often work for years, again, depending on application.
HDPE is notoriously hard to adhere to long term. Other plastics can be tricky too, so you have to know your materials and your adhesives to make a good descision.
Anywhere sunlight or UV is present requires special care in selecting, as does high heat, very low temps, and solvent or fume concentrations. (water IS the universal solvent!)
Considerations also include shear strength (super glue BAD), tensile strength (super glue EXECELLENT!), flexibility, and abrasion resistance.
Finally, no matter what you are trying to 'glue', clean surfaces are a MUST, and your cleaner cannot leave a residue on the surfaces.
You just gotta think it through!
-fab
P.S. I too am a BIG fan of Gorilla Glue, but haven't found much use for it in electronics...yet!
fabelizer:
...using silcone based adhesive to glue down a dc power jack would not be a good idea if that jack is to be used often AND soldered to a PCB. It would be better to use a hard epoxy, that will flex much less, and spread the stress of the jack over a larger footprint than just solder pads. Yet, soldering the same jack to a set of wires to go to the board, may be best glued to the enclosure with silicone! So, it depends...
I don't understand your example. Why not use epoxy for the wired jack as well?
fabelizer:
Silicone is GREAT for sealing, not bad for adhering with flexibility, but may cause some slight corrosion while curing as it out gasses acetic acid, (that stops when fully cured though).
There are neutral cure silicones.
What about E6000? Does it adhere well to a PCB?