E.g. FORTH interpreter/compiler is available also for Arduinos. When a terminal is required for user interaction or updates, nowadays a PC will be used for that, so that also the IDE can be used to update a program.
In a RISC architecture both stack and register emulation is very time consuming. Reserving special registers for the interpreter doesn't look simple to me, requires at least compiler switches for register reservation.
Also take into account how interrupts can be handled, so that serial communication or other (I2C, SPI...) functionality and driver boards (motor...) can be used with the interpreter. Otherwise use an ordinary microprocessor, with address and data bus and external RAM and ROM, instead of a microcontroller.