Building suspension for omni-directional wheels?

So I have a robot with 4 wheels, one on each side of it, the wheels allow sideways motion: Omnidirectional bot - YouTube

I'm trying to improve grip and ensure all wheels always touch the ground, and avoid it getting stuck and spinning wheels in the air by adding suspension.
I got the smoothest and best independent spring motion by simply doing this:

However, my wheels are proving a challenge.

722×962 76.6 KB

I cannot use a setup that allows rotation! Rotation allows the metal between the rollers on the wheels to touch the ground, getting it stuck rather than allowing sideways movement. The wheels need to stay completely vertical.

Currently I use this setup, 2 vertical shock absorbers with internal springs, threaded into a shaft. It seems to be on the right track, but the problem is making the shaft exact enough to allow smooth up and down movement without restricting it, but also not allowing wiggleroom. Currently any sideways force on the wheels will push the dampeners against the sides of the shafts which seizes them.

I recently tried prototyping a 3 point system:

1283×962 109 KB

It works great in of itself, as it allows for some rotation due to ball joint sockets, forcing the bottom to flatten out, even if the surface is uneven and angled. It would be fantastic for standard wheels. However, this is counter to my issue, as I want only the roller of the wheel to touch the ground, if the whole bottom of the wheel reaches the ground by slightly angling it forwards or backwards, that means the metal sections will always hit the ground.

I suspect there are all sorts of mechanical parts and fittings that would solve what I am trying to do, but I don't know where to look or what terms to search for that I haven't already tried.
I have some ideas, but they would require a fish-eye rod mount or shock absorber that has bearing or similar that allows rotation in only one axis, everything I can find has ball sockets, which allows for way too much motion for me. I also struggle finding ways to mount rods with small enough parts. Pillow block mounted bearings are gigantic for some of the finer mechanisms i would need to build.

What's your suggestions on how to solve this?

If you want all wheels to touch the ground, the simple method is to have three wheels. If you only have limited access to machine tools, and if you want a diy solution, then it is easier to get reliable up and down movement by means of parallel pivoted rods, instead of sliding rod in tube. Think leaf spring. If the robot was bigger/heavier, then more options are available.

I assume you did not design the wheels you're using? Why is the material between rollers EVER able to touch the ground? I can't picture a scenario where this would occur.