Go as a dynamic plugin language for Go

Go as a dynamic plugin language for Go

why would you want to do this?

We're going to be using yaegi, an embedded Go interpreter for this experiment.

All the code is here

layout

Our trivial demo program will call scripts that change one int, and the change will be reflected in the runner's state. There are many possibilities but I'm keeping it simple here.

Since yaegi scripts can't change the runner's state directly, the scripts will change the integer by calling wrapper functions. Also, to affect the runner's state, the function will have to run inside the runner. Therefore, the script must return a func() (or func() error if you want to handle with errors) that we can execute.

scripts

Our script will look like this:

package main

import (
    "fmt"
    "lib"
    "math/rand"
)

func Func() func() {
    return func() {
        fmt.Println(lib.GetX()) // Print the integer
        lib.SetX(rand.Int()) // Set it to something random
    }
}

Notice the lib import here. This is a package we add to the interpreters set of symbols. It will contain wrapper functions around our integer.

The returned func() runs in the runner.

We'll store this script in scripts/one.go

runner

Our runner will look like this:

// imports omitted for brevity

func main() {
    x := 0 // the all-important integer

    // all Go files under scripts
    files, err := filepath.Glob("scripts/*.go")
    if err != nil {
        log.Fatal(err)
    }

    for _, filename := range files {
        // Clean interepreter for each script
        i := interp.New(interp.Options{})
        // standard library
        i.Use(stdlib.Symbols)
        // wrapper functions for the integer
        // You could put these somewhere more convenient but they're short enough to fit here
        i.Use(interp.Exports(map[string]map[string]reflect.Value{
            "lib": map[string]reflect.Value{
                "IncX": reflect.ValueOf(func() { x++ }),
                "DecX": reflect.ValueOf(func() { x-- }),
                "SetX": reflect.ValueOf(func(n int) { x = n }),
                "GetX": reflect.ValueOf(func() int { return x }),
            },
        }))
        // run the script
        _, err := i.EvalPath(filename)
        if err != nil {
            log.Fatalf("error running %s: %s", filename, err)
        }
        // Extract main.Func(), check its type
        symbols := i.Symbols("")
        main, ok := symbols["main"]
        if !ok {
            panic("main not found")
        }
        runFuncValue, ok := main["Func"]
        if !ok {
            log.Fatalf("%s: Func not found", filename)
        }
        runFunc, ok := runFuncValue.Interface().(func() func())
        if !ok {
            log.Fatalf("%s: Run is not of type func() func()", filename)
        }

        // run the function it returns
        runFunc()()
        log.Printf("x is now %d", x)
    }
}
2021/01/04 18:20:26 running scripts/one.go
0
2021/01/04 18:20:26 x is now 5577006791947779410

We can see one.go printing 0 (the zero value for our integer) and after setting it to a random number, the change is reflected.

This has some potential - you could build an HTTP server with script-configurable routes, a text editor automated in Go, or even a window manager with layouts and a bar configured in Go without re-compiling.

should you do this?

Probably not.

yaegi has quite a few bugs, and hasn't reached a stable version yet. Go modules are unsupported.

Packages must be wrapped to be used - I haven't investigated this yet. To use non-standard-library packages, they must be in the GOPATH/GOROOT (go get needs to be run) and symbols need to be extracted (every external package used by scripts would have to be configured in the runner)

The library isn't particularly friendly. Extracting symbols doesn't error when the package isn't found.

Completions won't be available for the lib package, and there will be a big error as you write your scripts because lib isn't a real package. Scripts will have to be 1 file (perhaps the interpreter could be improved to accomodate this)

Therefore, you should probably stick with compiling modules into the binary (like Caddy), Lua or JavaScript. Or shell integration. Or filesystems. Or HTTP.

Thanks for reading


.
├── my
│   ├── matrix
│   ├── email
│   ├── sourcehut
│   ├── lobsters
│   └── github
├── code
│   ├── i
│   ├── feef
│   ├── greedy
│   ├── typeup
│   ├── metro
│   └── point
├── blog
│   ├── Shell sins
│   ├── DWM tips
│   ├── How this site was made (old site)
│   └── Go as a dynamic plugin language for Go
├── wiki
├── notes
│   └── Tree-style blogs
├── pictures
├── webring
│   ├── bakpakin.com
│   ├── macwright.com
│   ├── arp242.net
│   ├── icyphox.sh
│   ├── zdsmith.com
│   ├── christine.website
│   ├── wiki.alopex.li
│   ├── erkin.party
│   ├── dave.cheney.net
│   ├── technomancy.us
│   ├── rxi.github.io
│   ├── hisham.hm
│   ├── junglecoder.com
│   ├── kivikakk.ee
│   ├── bernsteinbear.com
│   ├── 100r.co
│   ├── acha.ninja
│   ├── hillelwayne.com
│   ├── jvns.ca
│   ├── research.swtch.com
│   ├── eigenstate.org
│   └── drewdevault.com