Yeh the problem with that scenario is that Arduino is not necessarily a Machine Learning powerhouse. Now that EdgeImpulse.com is fully supporting the Portenta, they might be able to help with the problem. The Forum is fairly active. https://forum.edgeimpulse.com/
I have occasionally had the red flash of death and it has taken a few times to reboot the Portenta. Hopefully you did the System update to protect against a boot issue. Arduino is fairly good about the board, they may replace it if you enquire.