FluxPipe is a very experimental stand-alone Flux API for ClickHouse and other embedded datasources

Flux is a lightweight scripting language for querying databases and working with data. 1

Instructions

Download a binary release or build from source

📦 Download Binary

curl -fsSL github.com/lmangani/fluxpipe/releases/latest/download/fluxpipe-server -O && chmod +x fluxpipe-server

🔌 Start Server w/ Options

./fluxpipe-server -port 8086

Run with -h for a full list of parameters

🐛 Usage Examples

HTTP API

Fluxpipe serves a simple REST API loosely compatible with existing flux integrations and clients

Grafana Flux 1

Usage with native Grafana InfluxDB/Flux datasource (url + organization fields are required!)

⭐ ClickHouse SQL

import "sql" 

sql.from(
  driverName: "clickhouse",
  dataSourceName: "clickhouse://default:@clickhouse-host:9000/system",
  query: "SELECT database, total_rows FROM tables WHERE total_rows > 0"
) 
|> rename(columns: {database: "_value", total_rows: "_data"})
|> keep(columns: ["_value","_data"])

image

image

⭐ CURL POST

Usage with curl

curl -XPOST localhost:8086/api/v2/query -sS \
  -H 'Accept:application/csv' \
  -H 'Content-type:application/vnd.flux' \
  -d 'import g "generate" g.from(start: 2022-04-01T00:00:00Z, stop: 2022-04-01T00:03:00Z, count: 3, fn: (n) => n)'

#datatype,string,long,dateTime:RFC3339,long
#group,false,false,false,false
#default,_result,,,
,result,table,_time,_value
,,0,2022-04-01T00:00:00Z,1
,,0,2022-04-01T00:00:36Z,2
,,0,2022-04-01T00:01:12Z,3

STDIN CMD

Fluxpipe can be used as a command-line tool and stdin pipeline processor

Generate CSV

echo 'import g "generate" g.from(start: 2022-04-01T00:00:00Z, stop: 2022-04-01T00:03:00Z, count: 5, fn: (n) => 1)' \
| ./fluxpipe-server -stdin

#datatype,string,long,dateTime:RFC3339,long
#group,false,false,false,false
#default,_result,,,
,result,table,_time,_value
,,0,2022-04-01T00:00:00Z,1
,,0,2022-04-01T00:00:36Z,1
,,0,2022-04-01T00:01:12Z,1
,,0,2022-04-01T00:01:48Z,1
,,0,2022-04-01T00:02:24Z,1
Parse CSV
cat scripts/csv.flux | ./fluxpipe-server -stdin
Query SQL
cat scripts/sql.flux | ./fluxpipe-server -stdin

Status

  • Fluxlib
    • parser
    • executor
  • ClickHouse driver by @adubovikov
    • driver tests
  • STDIN pipeline
  • HTTP api
    • plaintext
    • json support
    • api doc

Footnotes

    ↩2

GitHub

View Github