Introduction

The GoTEE framework implements concurrent instantiation of
TamaGo based unikernels in
privileged and unprivileged modes, interacting with each other through monitor
mode and custom system calls.

With these capabilities GoTEE implements a TamaGo
based Trusted Execution Environments (TEE), bringing Go memory safety,
convenience and capabilities to bare metal execution within TrustZone Secure
World or equivalent isolation technology.

GoTEE can supervise pure Go, Rust or C based freestanding Trusted Applets,
implementing the GoTEE API, as well as any operating system capable of running
in TrustZone Normal World such as Linux.

A compatibility layer for
libutee
is planned, allowing execution of/as OP-TEE
compatible applets.

Documentation

The main documentation, which includes a tutorial, can be found on the
project wiki.

The package API documentation can be found on
pkg.go.dev.

Supported hardware

The following table summarizes currently supported SoCs and boards.

SoC Board SoC package Board package
NXP i.MX6ULZ USB armory Mk II imx6 usbarmory/mark-two
NXP i.MX6ULL MCIMX6ULL-EVK imx6 mx6ullevk

Example application

In TEE nomenclature, the privileged unikernel is commonly referred to as
Trusted OS, while the unprivileged one represents a Trusted Applet.

The GoTEE example
demonstrate concurrent operation of Go unikernels acting as Trusted OS,
Trusted Applet and Main OS.

:warning: the Main OS can be any "rich" OS (e.g. Linux), TamaGo is simply
used for a self-contained example. The same applies to the Trusted Applet
which can be any bare metal application capable of running in user mode and
implementing GoTEE API, such as freestanding C or Rust programs.

A Rust example
can be used replacing trusted_applet_go with trusted_applet_rust when building.

The example trusted OS/applet combination performs basic testing of concurrent
execution of three TamaGo
unikernels at different privilege levels:

  • Trusted OS, running in Secure World at privileged level (PL1, system mode)
  • Trusted Applet, running in Secure World at unprivileged level (PL0, user mode)
  • Main OS, running in Normal World at privileged level (PL1, system mode)

The Main OS yields back with a monitor call.

The Trusted Applet sleeps for 5 seconds before attempting to read privileged OS
memory, which triggers an exception handled by the supervisor which terminates
the Trusted Applet.

The GoTEE syscall
interface is implemented for communication between the Trusted OS and Trusted
Applet.

When launched on the USB armory Mk II,
the example application is reachable via SSH through
Ethernet over USB
(ECM protocol, supported on Linux and macOS hosts):

$ ssh [email protected]
PL1 tamago/arm (go1.16.5) • TEE system/monitor (Secure World)

  help                                   # this help
  reboot                                 # reset the SoC/board
  stack                                  # stack trace of current goroutine
  stackall                               # stack trace of all goroutines
  md  <hex offset> <size>                # memory display (use with caution)
  mw  <hex offset> <hex value>           # memory write   (use with caution)

  gotee                                  # TrustZone test w/ TamaGo unikernels
  dbg                                    # show ARM debug permissions
  csl                                    # show config security levels (CSL)
  csl <periph> <slave> <hex csl>         #  set config security level  (CSL)
  sa                                     # show security access (SA)
  sa  <id> <secure|nonsecure>            #  set security access (SA)

>

The example can be launched with the gotee command, it will spawn the Main OS
twice, to demonstrate behaviour before and after TrustZone restrictions are in
effect in using real hardware peripherals.

Compiling

Build the TamaGo compiler
(or use the latest binary release):

wget https://github.com/f-secure-foundry/tamago-go/archive/refs/tags/latest.zip
unzip latest.zip
cd tamago-go-latest/src && ./all.bash
cd ../bin && export TAMAGO=`pwd`/go

Build the example trusted applet and kernel executables:

git clone https://github.com/f-secure-foundry/GoTEE-example
cd GoTEE-example && make nonsecure_os_go && make trusted_applet_go && make trusted_os

:warning: replace trusted_applet_go with trusted_applet_rust for a Rust
TA example, this requires Rust nightly and the armv7a-none-eabi toolchain.

All compilation outputs are written under a bin subdirectory.

Executing and debugging

Native hardware

The PoC can be executed on the USB armory Mk II
by loading the compilation output bin/trusted_os.imx in
SDP mode.

gotee

Emulated hardware

An emulated run of the previously compiled executables under QEMU can be
executed as follows:

make qemu
...
00:00:00 PL1 tamago/arm (go1.17.1) • TEE system/monitor (Secure World)
00:00:00 PL1 loaded applet addr:0x94000000 size:4093296 entry:0x9406e3a4
00:00:00 PL1 loaded kernel addr:0x80000000 size:3715053 entry:0x8006cd34
00:00:00 PL1 waiting for applet and kernel
00:00:00 PL1 starting mode:USR ns:false sp:0x96000000 pc:0x9406e3a4
00:00:00 PL1 starting mode:SYS ns:true sp:0x00000000 pc:0x8006cd34
00:00:00 PL1 tamago/arm (go1.17.1) • system/supervisor (Normal World)
00:00:00 PL1 in Normal World is about to yield back
00:00:00    r0:00000000  r1:814223f0  r2:00000001  r3:00000000
00:00:00    r4:00000000  r5:00000000  r6:00000000  r7:00000000
00:00:00    r8:00000007  r9:00000034 r10:814000f0 r11:802885a9 cpsr:600001d6 (MON)
00:00:00   r12:00000000  sp:81441f88  lr:80146ac8  pc:801444c0 spsr:600000df (SYS)
00:00:00 PL1 stopped mode:SYS ns:true sp:0x81441f88 lr:0x80146ac8 pc:0x801444c0 err:exit
00:00:00 PL0 tamago/arm (go1.17.1) • TEE user applet (Secure World)
00:00:00 PL0 obtained 16 random bytes from PL1: 431ad651c0c8fb66f929df143eed6411
00:00:00 PL0 requests echo via RPC: hello
00:00:00 PL0 received echo via RPC: hello
00:00:00 PL0 will sleep for 5 seconds
00:00:01 PL1 says 1 missisipi
00:00:01 PL0 says 1 missisipi
...
00:00:05 PL1 says 5 missisipi
00:00:05 PL0 says 5 missisipi
00:00:05 PL0 is about to read PL1 Secure World memory at 0x90010000
00:00:05    r0:90010000  r1:948220c0  r2:90010000  r3:00000000
00:00:05    r4:00000000  r5:00000000  r6:00000000  r7:00000000
00:00:05    r8:00000007  r9:00000044 r10:948000f0 r11:942cba81 cpsr:600001d7 (ABT)
00:00:05   r12:00000000  sp:9484ff04  lr:94168868  pc:9401132c spsr:600000d0 (USR)
00:00:05 PL1 stopped mode:USR ns:false sp:0x9484ff04 lr:0x94168868 pc:0x9401132c err:ABT
00:00:05 PL1 says goodbye

:warning: the emulated run performs partial tests due to lack of full
TrustZone support by QEMU.

Debugging

make qemu-gdb
arm-none-eabi-gdb -ex "target remote 127.0.0.1:1234" bin/trusted_os.elf
>>> add-symbol-file bin/trusted_applet.elf
>>> b main.main

Authors

Andrea Barisani
[email protected] | [email protected]

Andrej Rosano
[email protected] | [email protected]

License

Copyright (c) F-Secure Corporation

This program is free software: you can redistribute it and/or modify it under
the terms of the GNU General Public License as published by the Free Software
Foundation under version 3 of the License.

This program is distributed in the hope that it will be useful, but WITHOUT ANY
WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A
PARTICULAR PURPOSE. See the GNU General Public License for more details.

See accompanying LICENSE file for full details.

GitHub

https://github.com/f-secure-foundry/GoTEE