keras_nlp.models.GemmaPreprocessor No Longer Exists

[rlcauvin]

Describe the bug
The keras_nlp.models.GemmaPreprocessor class described here no longer exists.

To Reproduce

!pip install -U keras-nlp
!pip install -U tensorflow

import keras_nlp
import tensorflow as tf

gemma_backbone = keras_nlp.models.GemmaBackbone.from_preset("gemma_2b_en", trainable = True)
gemma_preprocessor = keras_nlp.models.GemmaPreprocessor(gemma_backbone.vocabulary)

inputs = tf.keras.layers.Input(shape=(None,), dtype=tf.string)
x = gemma_preprocessor(inputs)
x = gemma_backbone(x)
sentence_embeddings = x['sequence_output'][:, 0, :]
x = tf.keras.layers.Dense(64, activation='relu')(sentence_embeddings)
x = tf.keras.layers.Dropout(0.2)(x)
outputs = tf.keras.layers.Dense(1, activation='sigmoid')(x)
model = tensorflow.keras.models.Model(inputs=inputs, outputs=outputs)

Output of the line that attempts to create the preprocessor:

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
Cell In[75], line 1
----> 1 gemma_preprocessor = keras_nlp.models.GemmaPreprocessor(gemma_backbone.vocabulary)

AttributeError: module 'keras_nlp.api.models' has no attribute 'GemmaPreprocessor'

I get the same results if I replace keras_nlp with keras_hub.

Expected behavior

As described here, this class used to exist.

Additional context

I am trying to use Gemma as a sentence encoding layer in a binary classifier. How should I do so if not with keras_nlp.models.GemmaPreprocessor and the code above?

[jeffcarp]

That was removed in dd0651e. We'll update the docs. The new one is here:
https://keras.io/keras_hub/api/models/gemma/gemma_causal_lm_preprocessor/#gemmacausallmpreprocessor-class

[Gopi-Uppari]

Hi @rlcauvin,

I have reproduced the issue, please refer to this gist file. To avoid it could you please use keras_hub.models.GemmaCausalLMPreprocessor class as mentioned by @jeffcarp and also please refer this keras documentation.

Thank you.

[rlcauvin]

Thank you. For reference, here is what @jeffcarp suggested for accessing token embeddings directly in KerasHub models.

[jeffcarp]

The original link points to the keras_nlp docs. In the updated keras_hub docs the GemmaPreprocessor class (correctly) no longer exists:
https://keras.io/keras_hub/api/models/gemma/

Closing this as the keras_nlp docs should remain (unlinked in the navigation) for legacy reference.