SciCoFK: A Decentralized Scientific Computing Framework Embraced with Consensus and Incentive Mechanism

This repo is the primary code of SciCoFK. In this research work, we propose a novel decentralized computing framework SciCoFK to address collaborative scientific computing tasks, especially for large-scale matrix multiplication, by adopting consensus, incentive and verification mechanisms.

Requirements

Platform

  • Linux server
  • Intel CPUs
  • Nvidia GPUs

Required Packages

  • Latest Go programming Language
  • Latest Intel OneAPI HPC Toolkit
  • Latest Nvidia CUDA Toolkit

Function Demonstration of Each Directory

SciCoFK
├── Proof_of_Real_Contribution
│   
├── matrix_multiplication_OpenMP
│
├── matrix_multiplication_cuda
│
├── matrix_multiplication_multi_GPUs
│
└── matrix_multiplication_verification
  1. The codes in Proof_of_Real_Contribution implement the consensus mechanism — Proof-of-Real-Contribution.
  2. The codes in matrix_multiplication_OpenMP implement the OpenMP-optimized matrix multiplication.
  3. The codes in matrix_multiplication_cuda implement the CUDA-optimized matrix multiplication running on one GPU.
  4. The codes in matrix_multiplication_multi_GPUs implement the CUDA-optimized matrix multiplication running on multiple GPU.
  5. The codes in matrix_multiplication_verification implement the verification mechanism — GVFA — for validating matrix multiplication.

Running of Each Function

  1. The way of running PoRC consensus mechanism:

Proof_of_Real_Contribution
├── WinnerInEachRound.txt
├── count.sh
├── go.mod
├── main.go
├── principal
│   └── principal.go
├── system
│   └── system.go
├── task
│   └── task.go
└── woker
    ├── multiplier.go
    └── worker.go
  1. Modify the values of variable numberOfComputingWorkers and numberOfTasks in file main.go to specify the number of workers and tasks for each consensus round.
  2. Modify the values of variable num in file principal/principal.go to specify the dimension of square matrices.
  3. Run the program: 
    $ ./count.sh numberofConsensusRounds

    The outputs show the number of bookking rights that each worker obtains.

  4. The winner ID of each consensus round is recorded in file WinnerInEachRound.txt.
  1. The way of running OpenMP-optimized matrix multiplication program:

matrix_multiplication_OpenMP
├── bin
│   └── run
├── clean.sh
├── data
├── include
│   └── matrix.h
├── make
├── run.sh
└── src
    ├── change_C_output.c
    ├── main.c
    └── matrix.c


$ ./make
$ ./run.sh firstMatrixDimension secondMatrixDimension

The generated matrices is stored in data directory. The output shows the running time.

  1. The way of running CUDA-optimized matrix multiplication program running on one GPU:

matrix_multiplication_cuda
├── bin
│   └── run
├── data
├── include
│   └── matrix.h
├── makefile
├── run.sh
└── src
    ├── main.cu
    └── matrix.cu


$ make
$ ./run.sh numberOfRowsofFirstMatrix numberOfColumnsofFirstMatrix numberOfColumnsofSecondMatrix

The generated matrices is stored in data directory. The output shows the running time.

  1. The way of running CUDA-optimized matrix multiplication program running on multiple GPU:

matrix_multiplication_multi_GPUs
├── bin
│   └── run
├── data
├── include
│   └── matrix.h
├── makefile
├── run.sh
└── src
    └── main.cu


$ make
$ ./run.sh numberOfRowsofFirstMatrix numberOfColumnsofFirstMatrix numberOfColumnsofSecondMatrix

The generated matrices is stored in data directory. The output shows the running time.

  1. The way of running the program of GVFA verification mechanism:

matrix_multiplication_verification
├── bin
│   ├── change
│   └── run
├── clean.sh
├── data
├── include
│   └── matrix.h
├── make
├── run.sh
├── src
│   ├── change_C_output.c
│   ├── freivald.c
│   ├── isProduct.c
│   └── main.c
└── transfer_data.sh
  1. Copying matrices data from above three matrix multiplication programs to data directory by running script transfer_data.sh
$ ./transfer_data.sh 1/2/3
  1. Executing verification for matrix multiplication

$ ./make
$ ./run.sh numberOfRowsofFirstMatrix numberOfColumnsofFirstMatrix numberOfColumnsofSecondMatrix

Note: the value of three parameters in above command must be equal to the dimension of matrices in data directory. The outputs show if the result of matrix multiplication is correct and running time.

  1. The funciton of src/change_C_output.c is to produce wrong result matrices.

GitHub

View Github