Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dora_datacollector_updated #2197

Merged
merged 39 commits into from
Nov 4, 2024
Merged

Dora_datacollector_updated #2197

Changes from all commits
Commits
Show all changes
39 commits
Select commit Hold shift + click to select a range
3fddcd2
final script added
Jun 20, 2024
da14858
dora img added
Jun 20, 2024
eda9cf5
ephoc increased
Jun 20, 2024
091f467
PEFT CONFIQ updated
Jun 20, 2024
7a87bbc
model hub id removed
Jun 20, 2024
4fedd83
tokenizer and model comments added
Jun 20, 2024
cb1b7cb
README updated on HUB integration
Jun 20, 2024
e73e3b6
README updated on device map
Jun 20, 2024
de00eda
epochs updated
Jun 20, 2024
6ba1dfe
output dir changed
Jun 20, 2024
e577aa1
readme updated
Jun 20, 2024
a4dcd8d
readme updated
Jun 20, 2024
762e6af
torch.compile added
Jun 24, 2024
1374011
torch.compile added to RM
Jun 24, 2024
ddfcd4c
args updated to user
Jun 24, 2024
71aa469
Merge branch 'huggingface:main' into main
shirinyamani Jun 24, 2024
e4cfd34
Update examples/dora_finetuning/README.md
shirinyamani Jun 25, 2024
ea121c8
Update examples/dora_finetuning/README.md
shirinyamani Jun 25, 2024
058a617
Update examples/dora_finetuning/README.md
shirinyamani Jun 25, 2024
d137e67
Update examples/dora_finetuning/QDoRA_finetuning.ipynb
shirinyamani Jun 25, 2024
5d3e12a
Update examples/dora_finetuning/QDoRA_finetuning.ipynb
shirinyamani Jun 25, 2024
d07814a
use_compile removed from .py + readme
Jun 25, 2024
8a3af1c
Qdora output dir updated
Jun 25, 2024
fb5f317
use_dora added
Jun 26, 2024
2d63c3a
use_peft removed from readme and use_dora added
Jun 26, 2024
7371a85
readme updated + epoch num increased
Jun 26, 2024
1cc5afd
prepare_model_for_kbit_training added + gradient cheack set to true
Jun 26, 2024
c3b0110
readme updated python ---> bash
Jun 26, 2024
8d7d802
notebook successfully re-ran
Jun 26, 2024
47eefc1
notebook successfully re-ran
Jun 26, 2024
4041d2c
notebook successfully re-ran
Jun 26, 2024
1a7dadd
quantize config updated
Jun 26, 2024
97397c4
reqs added
Jun 27, 2024
8424e8d
makefile added
Jun 27, 2024
3864ad7
make style added
Jun 27, 2024
280e760
make style ran successfully
Jun 27, 2024
1e2ceb7
ruff check && fromat examples tried
Jun 27, 2024
7e394b3
Merge branch 'huggingface:main' into main
shirinyamani Nov 3, 2024
63295b3
datacollector updated, mlm=false
shirinyamani Nov 3, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions examples/dora_finetuning/dora_finetuning.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
AutoModelForCausalLM,
AutoTokenizer,
BitsAndBytesConfig,
DataCollatorWithPadding,
DataCollatorForLanguageModeling,
Trainer,
TrainingArguments,
)
Expand Down Expand Up @@ -95,7 +95,7 @@ def tokenize_function(examples):
tokenized_datasets = dataset.map(tokenize_function, batched=True, remove_columns=dataset["train"].column_names)

# Data collator to dynamically pad the batched examples
data_collator = DataCollatorWithPadding(tokenizer)
data_collator = DataCollatorForLanguageModeling(tokenizer, mlm=False)

# Define training arguments
training_args = TrainingArguments(
Expand Down
Loading