This is a server that allows translations from text to sign language glosses. The server loads a pre-trained translation model compatible to the sequence-to-sequence Huggingface interface and exposes a translate function as an XMLRPC server.
A machine with a GPU and 32 GB of RAM.
- Clone this code on your computer
git clone https://github.com/DFKI-SignLanguage/text-to-gloss-machine-translation.git .
- Go to the sub-directory named "models" and download the translation model
cd models
wget https://cloud-affective.dfki.de/s/4XAH22BD6ttjTD9/download/result_fold_3_best_model.pt
cd ..
- Create a virtual machine and install the requirements
python -m venv venv
source venv/bin/activate
pip install -r requirements.txt
- Start the translation server
venv/bin/python translator.py
The XML-RPC server can be accessed by any compatible client. An example on how to use from a commandline can be found here:
curl -s \
-H "Content-Type: text/xml" \
-d '<?xml version="1.0"?>
<methodCall>
<methodName>translate</methodName>
<params>
<param>
<value><string>der zug ICE 234 aus mainz fährt um 4 : 40 Uhr .</string></value>
</param>
</params>
</methodCall>' \
http://localhost:8000/RPC2
Please not that the text needs to be lowercased.
Alternatively, you can try directly from within Python code with the provided example:
python translator_client.py
The translator script takes the parameter --config and allows parameters to be specified in a YAML configuration file. Two sample configuration files are provided. Example:
venv/bin/python translator.py --config config/translator.yaml
The following parameters can be configured:
- pretrained model: the name of the huggingface pre-trained model that needs to be loaded. By default it is
facebook/nllb-200-distilled-600M. It doesn't need to be downloaded. - finetuned_model: the fine-tuned model that needs to be downloaded as instructed above. By default it resides in the subdirectory
model - port: the port where the server shall run
- rpc_path: the rpc path that the server will respond to
- exposed_function: the name of the function that will be exposed to XML_RPC
There is also the commandline option --listen-all-interfaces. If true, theserver will listen on all network interfaces, otherwise only on localhost.
Based on the models by Maithri Rao, expanded by Eleftherios Avramidis, with the help of Cristina Espana Bonet and Fabrizio Nunnari. German Research Center for Artificial Intelligence (DFKI GbmbH).
Initial work funded by the research project BIGECO (BMBF, 2023-2026). Adaptation to fit the extended reality animation engine done by SignReality (EU 101070631 - UTTER open call: Development and application of deep models for eXtended Reality).