Varnam is an Indian language transliteration library. GoVarnam is a Go port of libvarnam with some core architectural changes. Not every part of libvarnam is ported.
It is stable to use daily as an input method. See it in action here: https://varnam.subinsb.com/
You will need to install GoVarnam library in your system for any app to use Varnam.
- Download a recent GoVarnam version.
- Extract the zip file
- Open a terminal and go to the extracted folder by using this command :
- Now run this command to install GoVarnam :
sudo ./install.sh install
It will ask for your password, enter it.
- Installation is finished
To check if installation is successful, try this command :
varnamcli -s ml enthaanu
It should give malayalam output if installation is successful.
- To make Varnam give better suggestions, you will need to import some words. Download a
.vlf(Varnam Learnings File) file from here [TODO LINK].
- Import it:
varnamcli -s ml -import file.vlf
Now, you may install the IBus engine to use Varnam system wide: https://github.com/varnamproject/govarnam-ibus
Test it out:
varnamcli -s ml namaskaaram
Learn a word:
varnamcli -s ml -learn കുന്നംകുളം
Train a word with a particular pattern:
varnamcli -s ml -train college കോളേജ്
Learning Words From A File
You can import all language words from any text file. Varnam will separate english words and non-english words and learn accordingly.
varnamcli -s ml -learn-from-file file.html
You can download news articles or Wikipedia pages in HTML format to learn words from them.
This repository have 3 things :
- GoVarnam library
- GoVarnam Command Line Utility (CLI)
- Go bindings for GoVarnam
GoVarnam is written in Go, but to be a standard library that can be used with any other programming languages, we compile it to a C library. This is done by :
go build -buildmode "c-shared" -o libgovarnam.so
(Shortcut to doing above is
libgovarnam.so is a shared library that can be dynamically linked in any other programming languages. Some examples :
- Go bindings for GoVarnam: See govarnamgo folder in this repo
- Java bindings for GoVarnam: IN PROGRESS
Wait, it means we need to write another Go file to interface with GoVarnam library ! This is because we’re interfacing with a shared library and not the Go library.
Files & Folders
govarnam– The library files
main.go, c-shared*– Files that help in making the govarnam a C shared library
govarnamgo– Go bindings for the library. For use with other Go projects
cli– A CLI tool for varnam. Uses
govarnamgoto interface with the library.
symbol-frequency-calculator– For populating the
weightcolumn in VST files
CLI (Command Line Utility)
The command line utility (CLI) is written in Go, uses govarnamgo to interface with the library.
You need to separately build the CLI:
cd cli # Show the path to libgovarnam.so export LD_LIBRARY_PATH=$(realpath ../):$LD_LIBRARY_PATH go build -o varnamcli .
This section is straight on getting your hands in. Explanation of how GoVarnam works is at the bottom.
- Clone of course
- You will need a
.vstfile. Get it from
schemesfolder in a release. Paste it in
make libraryto compile
When you make changes to govarnam source code, you will need to do
make library for the changes to build on and then test with CLI.
You can run tests (to make sure nothing broke) with :
Changes from libvarnam
ml.vsthas been changed to add a new
symbolstable. Get the new
ml.vsthere. The symbol with the least weight has more significance. This is calculated according to popularity from corpus. You can populate a
ml.vstwith weight values by a Python script. See that in the subfolder. The previous ruby script is used for making the VST. That is the same.
ml.vstfrom libvarnam is incompatible with govarnam.
patterns_contentis renamed to
patternstable in learnings DB won’t store malayalam patterns. Instead, for each input, all possible malayalam words are calculated (from
symbolsVARNAM_MATCH_ALL) and searched in
words. These are returned as suggestions. Previously,
patternwould store every pattern to a word. english => malayalam.
patternsin govarnam is used solely for English words.
Computer => കമ്പ്യൂട്ടർ. These English words won’t work out with our VST tokenizer cause the words are not really transliterable in our language. It would be
kambyoottar => Computer
To build without SQLite :
go build -tags libsqlite3 -buildmode=c-shared -o libgovarnam.so
- git tag
- make build release
Pack ibus engine:
- make build-ubuntu18 release