Code challenge

The aim of this project is to provide an url hash retriever worker pool for getting hash digest for a collection of urls. At the moment, it makes a Get HTTP call for each url, then retrieves MD5 digest of the body of the response.

  • The project is developed in a way that it is flexible for changing the hashing algorithm. The only thing to do is to change the default hashing algorithm using an option.
  • It is flexible to change how we should calculate the digest regarding an url. The only thing to do is to implement httphash.HashRetriever
  • It is possible to specify the number of workers that work in parallel to accomplish the task.

Getting started

In order to use the worker pool follow the below instructions.

1- Install Go programming language (installation manual)

2- Change the working directory to the project directory

3- use following command to build an executable for using the tool

go build -o build/tool cmd/url_hash_retriever/main.go

Then you can use the tool by following command

build/tool 5  http://google.com http://time.ir

The first argument is the number of workers that should accomplish the task, and then you can pass as many urls as you want by other argument.

if you do not provide number of workers its default number is 10.

result

incorrecturl: Get "incorrecturl": unsupported protocol scheme ""
http://google.com: 4232d22639faa16ec9218307919e371b
http://time.ir: 2542908a294bf906f28cc7ee01c24c9e

in the case of occurring an error, instead of hash digest, the error description will be printed after the url.

Structure

The following directory tree is the representation of the structure in which project files and directories are located.

project
│   README.md       project description and guide
│   .gitignore      for specifying files that git must ignore   
│   go.mod          for dependency management
│   go.sum          for dependency management
│      
└─── cmd            Entry point for using the worker pool
│   
└─── tests          unit tests 
│   
└─── tools          packages for having the url hash retriever worker pool

Tests

To run the tests in your development environment that has Go installed, run following command:

go test ./tests/...

GitHub

View Github