terraform-provider-goplugin

CI Go Report Card Apache 2 licensed GitHub release (latest SemVer) Terraform regsitry

A Terraform provider to create terraform providers 🤯, but easier and faster!

Terraform go plugin provider is a Terraform provider that will let you execute Go plugins (using yaegi) in terraform by implementing a very simple and small Go API.

Why

Sometimes I want to manage resources in Terraform that don’t have a provider, however, creating a Terraform provider takes time and a lot of effort, including understanding low level concepts. So this poor resource will not end in Terraform.

Unless… in the cases where we don’t need to manage tons of resources or its a simple API, A small go plugin would be enough to manage them in Terraform.

Use cases

When to use it

  • A terraform provider doesn’t have support of the resource you need (e.g Github provider doesn’t have gist support).
  • You want to manage private/internal APIs with Terraform.
  • You don’t want/need to understand low level Terraform concepts.
  • A simple plugin that communicates with an API and marshal/unmarshal JSON is enough for you.
  • Prototyping, MVPs and exploring ideas around terraform provider development.
  • Implement private terraform providers for your company/organization

When NOT to use it

  • You need performance, interpreted code will be less efficient and slower.
  • Your provider is complex and with tons of resources.
  • You need Go third party libraries (this smells like a complex use case).
  • You need to provide official Terraform support for a product.

Examples

Check examples

Plugins v1

Resource

Example of a NOOP plugin:

package terraform

import (
 "context"

 apiv1 "github.com/slok/terraform-provider-goplugin/pkg/api/v1"
)

func NewResourcePlugin(opts string) (apiv1.ResourcePlugin, error) {
 return plugin{}, nil
}

type plugin struct{}

func (p plugin) CreateResource(ctx context.Context, r apiv1.CreateResourceRequest) (*apiv1.CreateResourceResponse, error) {
 return &apiv1.CreateResourceResponse{}, nil
}

func (p plugin) ReadResource(ctx context.Context, r apiv1.ReadResourceRequest) (*apiv1.ReadResourceResponse, error) {
 return &apiv1.ReadResourceResponse{}, nil
}

func (p plugin) DeleteResource(ctx context.Context, r apiv1.DeleteResourceRequest) (*apiv1.DeleteResourceResponse, error) {
 return &apiv1.DeleteResourceResponse{}, nil
}

func (p plugin) UpdateResource(ctx context.Context, r apiv1.UpdateResourceRequest) (*apiv1.UpdateResourceResponse, error) {
 return &apiv1.UpdateResourceResponse{}, nil
}

Important concepts

IDs

Resources will have 2 ids:

The common Terraform ID that it’s used internally by terraform to identify the terraform resource, to refer to that tf resource in the HCL code and to import the resource into terraform.

And the resource ID itself, the one that identifies teh resource Id outside terraform (e.g a User ID in a rest API). Normally this ID is the one you want to use to get information of the resource by using it in a datasource.

Warning plugin_id it’s part of the Terraform identifier, this attribute should not change, if it changes, resource will be recreated.

Plugin design and limitations

Plugin have some limitations, some imposed by the engine itself, Yaegi, and other ones imposed by this provider design in favor of simplicity and portability:

  • No third party packages (external libraries) supported.
  • Flat source code (no nested packages) and in a single package.
  • Allow splitting code in multiple files.
  • Small and simple API: Less features, more reliable and easy to maintain.
  • Automatically ignore plugin tests (package _test) on plugin load.

JSON input/output

Instead of using interface{}/any for the data that is being passed and returned in the plugins, we decided to treat the plugins as another remote API, and use a common way that its an standard on communication, JSON.

This although less performant and a bit more verbose, benefits the plugin reliability and portability making them less brittle to changes and unknown side effects of magical auto encode/decode. Apart from this:

  • Go standard library has native support and is well tested.
  • Terraform has native support and by using jsonencode/jsondecode to use it in HCL code and see changes on plans.

No computed data from plugins

Computed attributes are static attributes that are generated at the creation or the import phase of a resource, this data once generated can’t change.

Giving the ability the user to return this data from the plugins, could make the plugins return different data on each run, making Terraform break.

So, to ease the user plugin development and usage, we decided to avoid computed data on plugins, and instead add support for plugin data sources in case users need to get extra data from a resource.

This is less performant, as a data source will fetch data every time, but its more reliable and less brittle, avoiding shoot ourselves in the foot.

Requirements

  • Terraform >=1.x.

Terraform cloud

This provider supports terraform cloud.

Development

To install your plugin locally you can do make install, it will build and install in your ${HOME}/.terraform/plugins/...

Note: The installation is ready for OS_ARCH=linux_amd64, so you make need to change the Makefile if using other OS.

Example:

cd ./examples/local
rm -rf ./.terraform ./.terraform.lock.hcl
cd -
make install
cd -
terraform plan

GitHub

View Github