Cool embedded PHP engine in C

Hello.

There is a cool PHP engine called PH7 which compiles under Windows or Unix environments.

Project seemed to be frozen but codebase is ok.

I ported it to ESP32, it works.

Changes made can be found by looking for "ESP32" in the source code, there are few of them, thanks to well-written code.

ESP32 port is here

<?php
 echo 'Welcome guest'.PHP_EOL;
 echo 'Current system time is: '.date('Y-m-d H:i:s').PHP_EOL;
 echo 'and you are running '.php_uname();
?>

thanks for sharing

Memory requirements are relatively low. The code itself relies on its "memory management backend", which uses malloc() under the hood, to allocate big chunks (for slicing and managing). This malloc() call can be replaced with heap_caps_malloc() call, with a parameter saying to perform allocations in PSRAM. Currently small allocations are made in DRAM.

Guys made PH7 long time ago for embedded routers: the router had to have a web-interface, not just static pages.

Programs can be executed from files or from memory. Compiled VM code can be executed directly or stored as a file to execute it later.

Using malloc() as a memory backend (~2KiB of DRAM is used by the command line):

Full advantage of PH7 engine could be achieved if your filesystem layer supports mmap(): no RAM is used, scripts can be of any size.

Unfortunately, VFS layer of ESP32 does not support full POSIX set of operations. PH7 can execute scripts without loading them into ram, if mmap() is supported by the platform. On platforms without mmap(), PH7 uses malloc()/open()/read()/close()/free() approach.

I am developing (it is finished, in testing now) an immutable (readonly) fast FS for embedded applications. With better POSIX support and mmap(). This will allow for huuuuge webservers (~30Mb on ESP32S3).

24 byte per inode, O(log N) inode lookup. O(1) links resolution. Lockless. Support hardlinks, symlinks, UTF8 long names. Supports open("dirname/"), dirfd(), pread(), mmap() etc all the stuff usually required when porting something from big computers :). Hopefully I can release it next month.

Then I plab to add this FS support to PH7 as a VFS backend.

:)

Hello,

There are some updates on the project: right now I slowed down with Arduino/ESP-IDF/FreeRTOS bindings and trying
to make the language more PHP7/8-like.

Compiler changes:

  1. functions now can have a return type (e.g. :int or :mixed), like in modern php:
function Hello(int $arg) :int {

  return $arg + 1;
}

Type hint for arguments (int) is an extension to the language, which performs automatic cast, so code inside function will be guaranteed to get an int and not what was passed

function Hello(int $arg) :int {

  var_dump($arg);                      // <--- will display "int(666)"
  return $arg + 1;
}

Hello('666'); 

Return type casting is not done yet, but i see how it could be done. Right now the code below compiles and works by ignoring 'int' type on 'return'; This is a bad behaviour and it will be fixed

Function return value is autocasted to its function type. Code below will return int(666), not string('666')

function func() :int {
  return '666';
}
  1. mixed type is recognized and compiled correctly. mixed type is a valid type for enum language construct unlike classic PHP

  2. enum language construct has been added:
    Standard PHP supports two types of enumerations: pure enums and backed enums.
    A pure enum contains only named cases, for example

enum Status {
  case Draft;
  case Published;
  case Archived;
}

//A backed enum additionally declares its type (e.g.`int` or `string`) and a //value for each case: 


enum Status: string {
  case Draft = 'draft';
  case Published = 'published';
  case Archived = 'archived';
}
```

PH7 implements an extended enum model. Enum cases may have values of different types, for example

enum Example {
  case Number = 123;
  case Text = 'hello';
  case Flag = true;
  case F = 66.6;
}

Cases without an explicitly assigned value support automatic numbering:

enum Example {
  case A;
  case B;
  case C = 10;
  case D;
}

where A, B, and D receive the values 0, 1, and 2, respectevely.
Typed enums are also supported:

enum Example: int {
  case A;
  case B;
  case C = '123';
}

function test(Example $arg) {
  var_dump($arg);
}

test(Example::C);  // Displays: "int(123)"

Also comma operator is supported for enums with explicit values:

enum Name {  // No type - ":mixed" is assuned, no autocasting for enum values
  case A = 0, B= 1, C = 10;
  case STR = 'String';
}

Enum values (and constants as well) can have complex initializer values:

case A = 'Hello' . ' ' . rand_str(3);

PS:

digitalWrite($gpio, 1);
digitalWrite($gpio, 0);

are showing ~300KHz in a tight loop on ESP32-S3. Didn't measure it on P4, but I think it should be 2x or more.

Added nullable types (e.g. ?string or ?int)

Спойлер
<?php

function test1(mixed $arg): string {
  return $arg === null ? null : $arg;
}

function test2(mixed $arg): ?string {   // nullable return value
  return $arg === null ? null : $arg;
}

function test3(?string $x, int $y){ // nullable argument
  var_dump($x);
  var_dump($y);
}


var_dump(test1('hello'));   // 'hello'
var_dump(test1(11));        // '11'  number converted to a string
var_dump(test1(null));      // ''    null is converted to an empty string (string)

var_dump(test2('hello'));   // 'hello'
var_dump(test2(11));        // '11'
var_dump(test2(null));      // null    null is NOT converted and left intact (?string) 

test3('hello', null); // 'hello', 0  --> null is converted to int(0)
test3(null, '666'); // null, 666  --> '666' is converted to int(666)

Added callable type. Just a syntactic sugar: callables are either strings or arrays under the hood.

Спойлер
<?php


function test(callable $callback = function() { echo "Hello World!\n";})
{
   /* Check if we are dealing with a callback */
  if( !is_callable($callback) ){
     die("Expecting a callback");
  }
  /* Invoke the callback */
  $callback();
}

/* Call without arguments */
test(); /* You should see: Hello World */

/* Call with a single argument */
test(function(){ echo "Welcome guest\n";}); /* You should see: Welcome guest */

test(10); /* You should see: Expecting a callback */



?>

Added support for :never function attribute:

A return from a function which is marked as 'never returning' (i.e. :never return type) now has two level of checks: Any return statement is forbidden at compile time. Compiler emits a HALT opcode at the end of every such function. When runtime hits such HALT instruction the VM is stopped, with diagnostics saying Warning: Return from a :never returning function is detected

Спойлер
<?php
function dummy(): never {
}

dummy();

echo "This will not be printed\n";
?>

I am sorry for turning this post into personal project progress :slight_smile:

Syntax update: Support for ?-> (nullsafe) operator:

$a = null;
$a?->method(); // this is safe
$a->method(); // throws an exception

Null coalesce operator ?? :

$arr = array('First','Second','Third');
$a = $arr[999] ?? 'not set';

Added support for the "arrow function" syntax (lambda function):

$a = fn(int $arg):int => $arg * $arg; // creates a 'callable'

Exceptions are thrown by VM in runtime now, e.g. division by zero, or null dereference and so on can be caught and handled. Original engine was using "report error and continue as if nothing happened" algorithm, which was replaced with proper exception mechanism.

<?php

try {

  $a = null;
  $a(1,2,3);


} catch (Error $a) {
    echo 'Caught: '.($a?->getMessage()).PHP_EOL;
}

echo 'Still alive!'.PHP_EOL;
?>

Function overloading is now fixed and works as intended: there is a ranking system to find an overloaded function that matches most (including mixed typed/untyped arguments).

Assortment of memory leaks and memory access bugs was found and (hopefully) fixed: it is super hard to fix something in 2MiB source code without breaking some hidden crazy invariants.

There are still problems with calling object destructors when script ends: they are not called :). Working on it.

Type-handling philosophy is different from what Zend PHP does.

Hopefully I can release 0.1.0 of the engine in January. Right now it runs on ESP32S3, ESP32CAM, ESP32P4 (all equipped with 4..16MiB of PSRAM), Linux (untested but should) and WIndows (compiles in Cygwin environment).

Three random scripts generated by ChatGPT, modern PHP8 syntax; Executed in PH8 engine with no errors :slight_smile:

Matrix multiplication
<?php

function matmul(array $a, array $b): array
{
    $c = [
        [0, 0, 0],
        [0, 0, 0],
        [0, 0, 0]
    ];

    for ($i = 0; $i < 3; $i++) {
        for ($j = 0; $j < 3; $j++) {
            for ($k = 0; $k < 3; $k++) {
                $c[$i][$j] += $a[$i][$k] * $b[$k][$j];
            }
        }
    }

    return $c;
}

function matprint(array $m): void
{
    foreach ($m as $row) {
        printf(
            "| %8.2f %8.2f %8.2f |\n",
            $row[0], $row[1], $row[2]
        );
    }
}

$a = [
    [1, 2, 3],
    [4, 5, 6],
    [7, 8, 9]
];

$b = [
    [9, 8, 7],
    [6, 5, 4],
    [3, 2, 1]
];

echo "Matrix A:\n";
matprint($a);

echo "\nMatrix B:\n";
matprint($b);

$c = matmul($a, $b);

echo "\nA x B =\n";
matprint($c);

echo "\nCheck C[0][0]: ";
echo "1*9 + 2*6 + 3*3 = ";
echo $c[0][0], "\n";

echo "\nCheck C[2][2]: ";
echo "7*7 + 8*4 + 9*1 = ";
echo $c[2][2], "\n";

echo "\nDone.\n";
?>

Kalmann Filter
<?php

class KalmanFilter 
{
    private float $r; // Process noise covariance
    private float $q; // Measurement noise covariance
    private float $a; // State transition matrix
    private float $b; // Control matrix
    private float $c; // Measurement matrix
    
    private float $Cov;
    private ?float $x = null; // Estimated signal value

    /**
     * KalmanFilter Constructor
     * 
     * @param float $r Process noise (how much we expect the system to wiggle on its own)
     * @param float $q Measurement noise (how much jitter/noise your sensor has)
     * @param float $a State vector
     * @param float $b Control vector
     * @param float $c Measurement vector
     */
    public function __construct(
        float $r = 1, 
        float $q = 1, 
        float $a = 1, 
        float $b = 0, 
        float $c = 1
    ) {
        $this->r = $r;
        $this->q = $q;
        $this->a = $a;
        $this->b = $b;
        $this->c = $c;


        $this->Cov = 0;
        
    }

    /**
     * Filters a new noisy measurement to return a clean prediction.
     * 
     * @param float $z The raw measurement value
     * @param float $u Optional control vector input
     * @return float The estimated/filtered value
     */
    public function filter(float $z, float $u = 0): float 
    {
        // 1. Initialize state if this is the first execution

        if (is_null($this->x)) {

            $this->x = (1 / $this->c) * $z;
            $this->Cov = (1 / $this->c) * $this->q * (1 / $this->c);

            return $this->x;
        }

        // 2. Prediction step
        $predX = ($this->a * $this->x) + ($this->b * $u);
        $predCov = (($this->a * $this->Cov) * $this->a) + $this->r;

        // 3. Update / Correction step
        $kalmanGain = $predCov * $this->c * (1 / (($this->c * $predCov * $this->c) + $this->q));
        
        $this->x = $predX + ($kalmanGain * ($z - ($this->c * $predX)));

        $this->Cov = $predCov - ($kalmanGain * $this->c * $predCov);

        return $this->x;
    }

    /**
     * Get the current state estimate
     */
    public function getState(): ?float 
    {
        return $this->x;
    }
}


// Tuning parameters:
// R (Process Noise) = 0.01: We assume the true value is mostly stable.
// Q (Measurement Noise) = 1.00: We trust our raw data less because it's noisy.
$filter = new KalmanFilter(0.01, 1.00/*, 1.0, 0.0, 1.0*/);

// Simulated noisy incoming data stream
$noisyData = [10.5, 9.2, 11.1, 8.9, 10.8, 9.5, 10.1, 11.5, 9.0, 10.2];

echo "Raw Value -> Filtered Value\n";
echo "---------------------------\n";

foreach ($noisyData as $reading) {
    $corrected = $filter->filter($reading);
    printf("%6.2f    -> %6.4f\n", $reading, $corrected);
}
?>

Linear equations, Gauss method
<?php

function solve(array $a, array $b): array
{
    $n = count($b);

    // Forward elimination
    for ($i = 0; $i < $n; $i++) {
        // Find pivot
        $max = $i;

        for ($k = $i + 1; $k < $n; $k++) {
            if (abs($a[$k][$i]) > abs($a[$max][$i])) {
                $max = $k;
            }
        }

        if (abs($a[$max][$i]) < 1e-12) {
            throw new Exception("Singular matrix");
        }

        // Swap rows
        if ($max != $i) {
            $tmp = $a[$i];
            $a[$i] = $a[$max];
            $a[$max] = $tmp;

            $tmp = $b[$i];
            $b[$i] = $b[$max];
            $b[$max] = $tmp;
        }

        // Eliminate column
        for ($k = $i + 1; $k < $n; $k++) {
            $factor = $a[$k][$i] / $a[$i][$i];

            for ($j = $i; $j < $n; $j++) {
                $a[$k][$j] -= $factor * $a[$i][$j];
            }

            $b[$k] -= $factor * $b[$i];
        }
    }

    // Back substitution
    $x = array_fill(0, $n, 0.0);

    for ($i = $n - 1; $i >= 0; $i--) {
        $sum = $b[$i];

        for ($j = $i + 1; $j < $n; $j++) {
            $sum -= $a[$i][$j] * $x[$j];
        }

        $x[$i] = $sum / $a[$i][$i];
    }

    return $x;
}

// System:
//  2x +  y - z =  8
// -3x -  y + 2z = -11
// -2x +  y + 2z = -3

$a = [
    [ 2,  1, -1],
    [-3, -1,  2],
    [-2,  1,  2]
];

$b = [8, -11, -3];

echo "Gaussian elimination\n\n";

$x = solve($a, $b);

for ($i = 0; $i < count($x); $i++) {
    printf("x[%d] = %.6f\n", $i, $x[$i]);
}

echo "\nExpected: 2, 3, -1\n";
echo "Done.\n";
?>

Example "15" game (204 lines of code) in PHP. Runs on my test setup on ESP32S3 and ESP32P4:

15 Game source code
<?php

$solved = '123456789ABCDEF_';
$allowed = '123456789ABCDEF_';

/* Generate a solveable position by making moves */
function new_game()
{
    $fields = '123456789ABCDEF_';
    $blank = 15;
    $previousBlank = -1;

    for ($step = 0; $step < 200; ++$step) {
        $moves = array();

        for ($i = 0; $i < 16; ++$i) {
            if ($i == $blank || $i == $previousBlank) {
                continue;
            }

            $horizontal =
                ($i - ($i % 4)) == ($blank - ($blank % 4))
                && abs($i - $blank) == 1;

            $vertical = abs($i - $blank) == 4;

            if ($horizontal || $vertical) {
                $moves[] = $i;
            }
        }

        $next = $moves[mt_rand(0, count($moves) - 1)];

        $fields[$blank] = $fields[$next];
        $fields[$next] = '_';

        $previousBlank = $blank;
        $blank = $next;
    }

    return $fields;
}

/* Validate the complete board before using it. */
$valid = false;

if (isset($_GET['fields']) && is_string($_GET['fields'])) {
    $candidate = $_GET['fields'];

    if (strlen($candidate) == 16) {
        $seen = array();
        $valid = true;

        for ($i = 0; $i < 16; ++$i) {
            $tile = $candidate[$i];

            if (strpos($allowed, $tile) === false
                || isset($seen[$tile])) {
                $valid = false;
                break;
            }

            $seen[$tile] = true;
        }

        if ($valid) {
            $fields = $candidate;
        }
    }
}

if (!$valid) {
    $fields = new_game();
}

/* A tile is identified by its label, not its position. */
if (isset($_GET['pressed']) && is_string($_GET['pressed'])
    && strlen($_GET['pressed']) == 1) {

    $tile = $_GET['pressed'];

    if ($tile != '_' && strpos($allowed, $tile) !== false) {
        $tilePos = strpos($fields, $tile);
        $blankPos = strpos($fields, '_');

        $horizontal =
            ($tilePos - ($tilePos % 4))
                == ($blankPos - ($blankPos % 4))
            && abs($tilePos - $blankPos) == 1;

        $vertical = abs($tilePos - $blankPos) == 4;

        if ($horizontal || $vertical) {
            $fields[$blankPos] = $tile;
            $fields[$tilePos] = '_';
        }
    }
}

$won = ($fields === $solved);

?>
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<title>Fifteen</title>
<style>
    body {
        font-family: sans-serif;
        text-align: center;
        margin: 30px auto;
        max-width: 440px;
        padding: 0 12px;
        background: #f4f5f7;
    }

    h1 { margin-bottom: 8px; }

    .board {
        display: grid;
        grid-template-columns: repeat(4, 1fr);
        gap: 6px;
        margin: 24px auto;
    }

    .tile {
        display: flex;
        align-items: center;
        justify-content: center;
        aspect-ratio: 1;
        border-radius: 9px;
        background: #356b9a;
        color: white;
        font-size: clamp(22px, 7vw, 36px);
        font-weight: bold;
        text-decoration: none;
    }

    a.tile:hover { background: #24537b; }

    .empty {
        background: #dfe3e8;
    }

    .message {
        color: #26743b;
        font-size: 20px;
        font-weight: bold;
    }

    .restart {
        display: inline-block;
        margin: 8px;
        padding: 10px 18px;
        background: #333;
        color: white;
        text-decoration: none;
        border-radius: 6px;
    }
</style>
</head>
<body>

<h1>Fifteen</h1>

<?php if ($won) { ?>
    <div class="message">Congratulations!</div>
<?php } ?>

<div class="board">
<?php
for ($i = 0; $i < 16; ++$i) {
    $tile = $fields[$i];

    if ($tile == '_') {
        echo '<div class="tile empty"></div>';
        continue;
    }

    $blankPos = strpos($fields, '_');

    $horizontal =
        ($i - ($i % 4)) == ($blankPos - ($blankPos % 4))
        && abs($i - $blankPos) == 1;

    $vertical = abs($i - $blankPos) == 4;

    if (!$won && ($horizontal || $vertical)) {
        $href = '15.php?fields=' . rawurlencode($fields)
              . '&amp;pressed=' . rawurlencode($tile);

        echo '<a class="tile" href="' . $href . '">'
           . $tile . '</a>';
    } else {
        echo '<div class="tile">' . $tile . '</div>';
    }
}
?>
</div>
<a class="restart" href="15.php">New game</a>
</body>
</html>

Regular expressions were added. The regexp engine is based on SLRE. I fixed it (there were bugs) and added a layer of standart PHP preg_ functions:

preg_match(...);
preg_match_all(...);
preg_replace(...);
preg_split(...);
preg_grep(...);
preg_quote(...);
preg_last_error();
preg_last_error_msg()

An extension to the PHP language was added: "a compile time include":

Classic PHP has two language constructs: "include" and "require" which are similar to #include in C/C++ but executed at runtime. I.e. PHP compiles included file after execution is started.

Sometimes it is required to include file BEFORE compiling takes place . So I added two language constructs: compile "/path/to/file.php" and compile_once "/path/to/file.php" which work as C/C++ #include construct

Added a HTTP server, written in plain C, uses pico http parser. Able to serve static files (via mmap() where possible) and execute .php scripts.

It is just proof-of-concept, no mbedSSL yet. The 15 game from the post above running in browser (ESP32S3)

SQLite3 engine is imported to the codebase: sqlite3 API is available in two flavors: native sqlite3 functions and SQLite3 class.