Staffjoy

Build Status Godoc Reference

The main purpose of Staffjoy is to make creating employee work schedules easy and quick. This project is very inspiring and I learned a lot from the implementation and engineering and monorepo structure.

This is a monorepo, so all of the code for all of the services are in this repo. The core technologies are the Bazel build system, Kubernetes (including its DNS for internal service discovery), Redux, Go, Protocol Buffers, gRPC, and Yarn. In staging and production, we used Google Container Engine and their managed databases.

Services

Staffjoy V2 Architecture

  • faraday – Proxies all traffic from external services to internal ones. It also tells backend services whether a user is logged in. It’s the only service that has a public IP address!
  • www – [www.staffjoy.com] is the main marketing website. It also handles login and logout.
  • myaccount – [myaccount.staffjoy.com] is a single-page javascript app that lets users modify their accounts
  • account-gateway [account.staffjoy.com] is the externally-available REST api for modifying accounts. It converts REST to gRPC for the accounts-datastore
  • accounts-server is the internal system that processes gRPC calls and stores/retrieves information using the account database.
  • company-gateway [company.staffjoy.com] is the externally-available REST api for modifying companies. It converts REST to gRPC for the companys-datastore
  • company-server is the internal system that processes gRPC calls and stores/retrieves information using the company database.
  • whoami [whoami.staffjoy.com] is a website that sends information about the current web session for easy access in the front-end.
  • superpowers [superpowers.staffjoy-v2.local] is a development-only website that lets you gain super user powers across Staffjoy (denoted as “support” flag on user accounts)
  • ical [ical.staffjoy-v2.local] is service serving up a worker’s shift list through ical

External API Standards

  • Services should be RESTful JSON over HTTPS
  • Serve the spec at /swagger.json
  • Use the apidocs package to serve a swagger UI at /ui/

Dev

Getting started

Welcome to Staffjoy!

We use a monorepo that stores all of our code in this single repo. We use Vagrant to run a Minikube(Kubernetes) cluster locally on your laptop. This makes it easy to run all of Staffjoy’s services.

Setting up your Gopath

If you are running Go code, you should set up your $GOPATH, then clone this repository into the v2.staffjoy.com package:

mkdir -p $GOPATH/src/
git clone [email protected]:staff-joy/staffjoy.git $GOPATH/src/v2.staffjoy.com/

Development station setup – One-time dependencies on host machine (laptop)

  • Virtualbox
  • Vagrant, then run vagrant up to boot the dev server.
  • Vagrant Plugins:
    • Vagrant host manager: vagrant plugin install vagrant-hostmanager
    • Vagrant resize disk: vagrant plugin install vagrant-disksize
  • modd

Running the Environment

Run make dev. Code will boot and run at staffjoy-v2.local. Note that the first time you do this could take up to ~30 minutes in order to provision the VM!

Changes will trigger an automatic rebuild and redeployment. (Check deployment progress at kubernetes-dashboard url). End the dev server with control + c (and it will automatically shut off the VM).

If you run into issues with stuck deployments in development – then run make dev-k8s-fix then re-run make dev.

Accessing the environment

Access the VM by running vagrant ssh. Code is located in /home/vagrant/golang/src/v2.staffjoy.com/ (aliased to $STAFFJOY, i.e. cd $STAFFJOY).

To build code and run it locally, in vagrant go to the code directory $STAFFJOY in vagrant, then run make dev-build for a one-time build.

If things are really goofing, run vagrant destroy -f then rebuild.

PHPMyAdmin

To view the DB locally you can pop a docker image of PHPMyAdmin:

Login Info:

$ docker run --name myadmin -d -e PMA_HOST=10.0.0.100 -p 8080:80 phpmyadmin/phpmyadmin

Development resources

Go

Assets in Go

If you are loading assets like templates, CSS, JS, etc – you need to package the the data into the binary. Otherwise, the app will ship and it won’t be able to find the assets!

To do this, use the go.rice project. If you modify any of the asset files, you will need to rebuild them then commit the resulting rice-box.go file and commit it. You have been warned!

Most services provide a build.sh file that compiles all the data that needs to be committed.

Development tools

The tool GoConvey is great for seeing tests.

Environment variables for configuration

  • ENV: Set to development,staging, or production. Null defaults to development
  • SENTRY_DSN: Set to the Sentry api key in every Go service for proper error tracking and reporting

WSL (Optional)

  • Create new file /etc/wsl.conf, with the following content:

[automount]
enabled = true
options = "metadata,umask=022,fmask=11"
  • Install latest Vagrant inside the WSL and the Host
  • Install latest Virtualbox
  • Run dev env

Enable Create symbolic links:

- Open Local Security Policy (gpedit.msc)
- Navigate to: Computer Configuration\Windows Settings\Security Settings\Local Policies\User Rights Assignment
- Open 'Create Symbolic Links' (SeCreateSymbolicLinkPrivilege)
- Add your username (or a group you are assigned to)
- Restart PC

> bash
$ vagrant up
$ vagrant ssh
$ cd $STAFFJOY
$ make dev-build

Protocol Buffers

If you modify the files in protobuf/, run make protobuf to recompile all of the generated files.

⚠️ Please make sure that the version of protobuf matches the runtime version (see this issue):

go install github.com/golang/protobuf/...
cd $GOPATH/src/github.com/golang/protobuf/
# Switch to version that is packaged in app
# v1.4.2 May 14, 2020 (LATEST OFFICIAL RELEASE)
git checkout d04d7b157bb510b1e0c10132224b616ac0e26b17
# Re-install
go install github.com/golang/protobuf/proto
cd $GOPATH/src/v2.staffjoy.com/
make protobuf

@todo consider updating, many changes

If you’re getting started with protocol buffers, here are some resources:

Working Offline

  • Email will break, but you can look at the system logs for the email service to see what would have been sent. (Useful for grabbing account activation links!)
  • See all Go documentation installed on the host machine with godoc -http=":8080". You’ll be able to see all docs at localhost:8080

Fork Todos

  • Convert internal go build.sh to makefile.
  • Deprecate Glide for god modules.
  • Added request tracing for Faraday, current middleware implemntation (faraday/trace_mw.go) was depricated in https://github.com/googleapis/google-cloud-go (v0.46.0). Using latest, broke since upgrade.
  • Migrate Go dependencies to be go.mod based via Bazel. Due to this dependencies currently maintained twice in go.mod and WORKSPACE (bazel).
  • Verify GCP deployment to stage and production works with Google Container Engine the same it works in local Kubernetes 1.18+.
  • Sentry after upgrading to Go 1.12 threw exceptions – which were fixed but not tested – check still works correctly.

Credit

The authors of the original code were @philipithomas, @samdturner, @andhess, and some contractors. @kootommy designed the application and most of the marketing pages, and worked closely with engineering on implementation. This is a fork of the internal repository. For security purposes, the Git history has been squashed.

GitHub

View Github