Fix VNNGP with batches #2375

LuhuanWu · 2023-07-09T07:11:04Z

Fix VNNGP in batch settings VNNGP with Batches #2300
In addition, set default jitter value to 1e-3 and variational stddev init value to 1e-2.
Change initialization strategy
Special-case behaviour when training_batch_size = M (ensure we only have one minibatch and that we compute the full KL)
Remove the option to set k == M

gpytorch/variational/nearest_neighbor_variational_strategy.py

test/variational/test_nearest_neighbor_variational_strategy.py

Create only a single taining minibatch with all the data, and use the appropriate kl_helper

LuhuanWu mentioned this pull request Jul 9, 2023

VNNGP with Batches #2344

Open

gpleiss requested changes Jul 17, 2023

View reviewed changes

LuhuanWu and others added 9 commits August 15, 2024 10:02

fix vnngp batch mode

87c98a3

fix format issue

1e0c505

fix nearest_neighbor_variational_strategy batch compatibility

0dd0908

fix nearest_neighbor_variational_strategy type error

dac2099

Improve type hints

6e304a1

Allow x to be broadcasted to inducing points batch shape

f58f56e

Fix behaviour for training_batch_size == M

45ed860

Create only a single taining minibatch with all the data, and use the appropriate kl_helper

Remove unnecessary settings

d28f599

Remove M=k case

dd65761

gpleiss force-pushed the master branch from dc4657b to dd65761 Compare August 15, 2024 18:17

gpleiss added 2 commits August 15, 2024 13:47

Handle mixed model/inducing point batch sizes with KL helper

9602969

Test cases catch batch VNNGP errors

4d4740b

gpleiss approved these changes Aug 16, 2024

View reviewed changes

gpleiss enabled auto-merge (squash) August 16, 2024 23:42

gpleiss merged commit c8dde64 into cornellius-gp:main Aug 16, 2024
7 checks passed

Provide feedback