Transcript

transcript is a CLI tool for snapshot testing other CLI tools.

TODO: Video here.

Usage

Automatically record a shell session or type-out a transcript file by hand,
then use the check command!

cat > demo.cmdt <<EOF
$ echo stdout
1 stdout

$ echo stderr 1>&2
2 stderr

# Non-zero exit codes.
$ false
? 1
EOF

transcript check ./demo.cmdt

Install

go get -u github.com/deref/transcript

NOTE: Transcript is not Go-specific. It is simply written in Go and Go provides
a convenient distribution mechanism. If there is expressed interest, it may be
re-packaged for various additional distribution channels.

Record

Initial authoring of tests is performed in an interactive shell.

To record an interactive session to a file, run:

transcript shell -o example.cmdt

The interactive shell supports standard readline behaviors and can be exited
with ^d or exit like most other shells.

Check

To interpret a transcript file and validate that the results (stdio output and
exit codes) have not changed, run the following:

transcript check example.cmdt

Check returns a non-zero exit code if any check failures or other errors occur.

Update

When the CLI tools under test are modified, the quickest way to update test
files is to use the automated update process:

transcript update example.cmdt

This will interpret a command transcript file, but does not check any output or
exit status expectations. Instead, the given file will be rewritten with the
newly observed results.

Edit

While transcript files can be edited by hand, more advanced edits can be made
using an interactive update session. The experience should be familiar to users
of git rebase --interactive.

NOTE: Not yet implemented.

“Command Transcript” File Format

Transcript files represent recorded shell sessions.

.cmdt is the recommended file extension.

This format is intended to be human-editable, but sacrifices some ease of
hand-authoring in exchange for added functionality. Users are expected to
primarily use the transcript tool to create and update transcripts.

Structure

Cmdt files are line-oriented. Each line represents an instruction to the
Transcript interpreter. Each instruction begins with an opcode, followed by a
space. The remainder of an instruction line forms arguments to the operation
specified by the opcode.

Operations

Operations with the following opcodes are supported:

# — comment

Comments may appear anywhere in a .cmdt file and are ignored
by the interpreter.

A space is not required after the # opcode.

Blank lines are also treated as comments.

$ — command

Run a shell command.

Supports the subset of Bash syntax provided by
mvdan/sh.

1, 2 — output

Match a line of output from a particular stdio stream of the previously
run command.

The opcodes are named after the file descriptors of stdout
(1) and stderr (2) respectively.

Output lines are matched exactly. More flexible matching may be
configured by % directives in a future version of
Transcript.

Transcript checking assumes that the interleaving of stdout and stderr
lines is significant and that output lines are written atomically.
The ordering of concurrent writes to both streams is undefined, which
will lead to flakey tests. Incrementally written lines will be buffered,
which may mask text interleaving issues that would affect users. Both of
these shortcomings may be mitigated in the future.

? — exit-code

Exit code of the previously run command.

If omitted, the exit code defaults to 0.

% — directive

Reserved for future use by Transcript.

Go API

In addition to the transcript CLI, there is a Go API for users who wish to
embed cmdt scripts in to their existing Go test suites.

import (
  _ "embed"

  "github.com/deref/transcript/cmdtest"
)

//go:embed test.cmdt
var fs embed.FS

func TestCLI(t *testing.T) {
  f, _ := fs.Open("test.cmdt")
  defer f.Close()
  cmdtest.Check(f)
}

NOTE: Assuming that ./test.cmdt uses the CLI tool you are developing, you
must first build your tool and ensure it is on PATH.

There is also a CheckString function for small, inline tests. However, prefer
to use .cmdt files so that the transcript tool can assist with updates and
edits.

GitHub

View Github