HTRbyMatching

Description

A pytorch implementation of the paper A Few-shot Learning Approach for Historical Ciphered Manuscript Recognition and its extension Few Shots Are All You Need: A Progressive Few Shot Learning Approach for Low Resource Handwriting Recognition. The proposed model can be used for low resource handwriting recognition in a few-shot learning scenario.

Download Code

clone the repository:

git clone https://github.com/dali92002/HTRbyMatching
cd HTRbyMatching

Requirements

Create your evironment, with the file htrmatching.yml, then activate it.

conda env create -f htrmatching.yml
conda activate fewdet

Models

Download the desired weights. These are following the training that was done in A Few-shot Learning Approach for Historical Ciphered Manuscript Recognition.

Training Dataset	Fine Tuning Dataset	URL
Omniglot	--	model
Omniglot	Borg	model
Omniglot	Copiale	model
Omniglot	Vatican	model

Training

Prepare Your alphabet and Data:

Check this instructions file to prepare your data for training:

https://docs.google.com/document/d/1c9waNt2vNq_WJiRLBU81hqCcTEaPjQ0-I8xybyyJ8Uc/edit?usp=sharing

Trainig / Fine Tuning:

Then, you canbtrain the model from scratch or fine tune it by taking the omniglot model weights from the table above. Here, I am training for the cipher named runic my training data path is ./few5/ my validation data path is ./data_validation , I am training with a batch_size of 5 and a threshold of 0.4 (it is recommended to keep it 0.4) and here I am doing a fine tuning over the model trained on omniglot so I specify train_type as "fine_tune". If you want to train from scratch specify the train_type as "scratch".

python train.py --cipher runic --data_path ./few5/ --val_data_path data_validation --batch_size 5 --shots 5 --alphabet alphabet --thresh 0.4 --train_type fine_tune

Testing

Download the desired pretrained weigts from the section Models. Then run the following command. Here we are choosing to recognize the lines of the cipher "borg", in a 1 shot scenario, with the model finetuned on the borg (as will be stated in the testing model path). We specify the input data path: lines and alphabet. As well ass the desired output path, here we want in in a floder named output in this same directory and the threshold 0.4.

python test.py --cipher borg --testing_model /MODEL_WEIGHTS_PATH/  --lines /LINES_PATH/ --alphabet ./ALPHABET_PATH/  --output ./OUTPUT_PATH/ --shots 1 --thresh 0.4

Please, check the folders named lines and alphabet to realize how you should provide your input data. After running you will receive the results in a 3 subfolders of your output folder.

Training in a progressive way (Coming soon)

This part is related to the paper Few Shots Are All You Need: A Progressive Few Shot Learning Approach for Low Resource Handwriting Recognition. ...

Citation

If you find this useful for your research, please cite it as follows:

@inproceedings{souibgui2021few,
  title={A few-shot learning approach for historical ciphered manuscript recognition},
  author={Souibgui, Mohamed Ali and Forn{\'e}s, Alicia and Kessentini, Yousri and Tudor, Crina},
  booktitle={2020 25th International Conference on Pattern Recognition (ICPR)},
  pages={5413--5420},
  year={2021},
  organization={IEEE}
}

@article{souibgui2022fewShots,
  title={Few Shots Are All You Need: A Progressive Few Shot Learning Approach for Low Resource Handwritten Text Recognition},
  author={Souibgui, Mohamed Ali and Forn{\'e}s, Alicia and Kessentini, Yousri and Megyesi, Be{\'a}ta},
  journal = {Pattern Recognition Letters},
  volume = {160},
  pages = {43-49},
  year = {2022},
  issn = {0167-8655},
  doi = {https://doi.org/10.1016/j.patrec.2022.06.003}
  }