Refactor tokenizer bootstrap (!240) · Merge requests · GitLab.org / ModelOps / AI Assisted (formerly Applied ML) / Code Suggestions / AI Gateway · GitLab

Tan Le requested to merge refactor-tokenizer-bootstrap into main Jul 21, 2023

Some small refactor following up !238 (merged)

Inline init_tokenizer method call
Make bootstrap.py a stand-alone script

Edited Jul 21, 2023 by Tan Le