CodeGen is a family of open-source model for program synthesis. Trained on TPU-v4. Competitive with OpenAI Codex.

This is the official code for the paper CodeRL: Mastering Code Generation through Pretrained Models and Deep Reinforcement Learning (NeurIPS22).

[NeurIPS'22 Spotlight] A Contrastive Framework for Neural Text Generation

Calculate perplexity on a text with pre-trained language models. Support MLM (eg. DeBERTa), recurrent LM (eg. GPT3), and encoder-decoder LM (eg. Flan-T5).

Code for "Critique Fine-Tuning: Learning to Critique is More Effective than Learning to Imitate"

高性能小模型测评 Shared Tasks in NLPCC 2020. Task 1 - Light Pre-Training Chinese Language Model for NLP Task

Embeddings: State-of-the-art Text Representations for Natural Language Processing tasks, an initial version of library focus on the Polish Language

TurkishBERTweet: Fast and Reliable Large Language Model for Social Media Analysis

Code for "Semi-supervised Formality Style Transfer using Language Model Discriminator and Mutual Information Maximization"

A 78.5% word sense disambiguator based on Transformers and RoBERTa (PyTorch)

Informal to formal dataset mask MLM

translatorlab: a machine translation tool that uses artificial intelligence models to provide accurate and fast translations between different languages

Language Model with RNN in tensorflow

Simple next word prediction model from scratch, implemented using only numpy.