Update required tokenizers version in setup.py to 0.6.0 as suggested in

huggingface#2831
lkemkes · Mar 17, 2020 · 75d7f3d · 75d7f3d
1 parent 68ef0a1
commit 75d7f3d
Showing 1 changed file with 1 addition and 1 deletion.
diff --git a/setup.py b/setup.py
@@ -92,7 +92,7 @@
     packages=find_packages("src"),
     install_requires=[
         "numpy",
-        "tokenizers == 0.5.2",
+        "tokenizers == 0.6.0",
         # accessing files from S3 directly
         "boto3",
         # filesystem locks e.g. to prevent parallel downloads