Implement remaining tasks in InferenceClient

[Wauplin]

InferenceClient is a new user-friendly client to deal with inference on HF products. It can seamlessly interact with both the Inference API, the free service to quickly discover and evaluate hosted models, and Inference Endpoints, the paid product for production-ready cloud-based inference.

The InferenceClient (see #1510) offers a simple API with 1 method per task. It handles input/output serialization out of the box to make it easy to use. Similarly, AsyncInferenceClient provides the same interface but for async calls (see #1524). In the initial implementation, only a subset of tasks have been implemented. It is now time to implement the rest!

This issue is meant to centralize the development of the other tasks. The idea is to create 1 PR for each new task, depending on the demand on each task. Here is a good example from @dulayjm who implemented the zero_shot_image_classification task.

Requirements for each task:

• Implement method in _client.py
  • inputs and output must be type annotated to help documentation
  • usually comes down to a self.post call with correct parameter + deserialization
• Add docstring including sections description, args, returns, raises and example.
• Add a test in ./tests/test_inference_client.py
• Generate test record with pytest tests/test_inference_client.py -k <test_name> --vcr-record=all => generates a YAML file to mock HTTP call to avoid using the InferenceAPI prod for CI tests (see comment)
• Update table in inference.md
• Run make style => will format code correctly AND generate the asyncio version of the code out of the box
• Run make quality => make sure the code is correctly formatted/typed => don't hesitate to ask for help if needed

Remaining tasks:

• object-detection Add object detection to inference client #1548 (thanks @dulayjm)
• document-question-answering Add document question answering to InferenceClient #1620 @martinbrose
• visual-question-answering Add visual question answering to inference client #1621 @martinbrose
• fill-mask Add fill mask to inference client #1613 @martinbrose
• question-answering Add question answering to inference client #1609 @martinbrose
• table-question-answering Add table question answering to inference client #1612 @martinbrose
• text-classification Add text classification to inference client #1606 @martinbrose
• token-classification Add token classification to inference client #1607 @martinbrose
• translation Add translation to inference client #1608 @martinbrose
• tabular-classification Add tabular classification to inference client #1614 @martinbrose
• tabular-regression Add tabular regression to inference client #1615 @martinbrose
• zero-shot-classification Add support for zero shot classification task #1644 @Wauplin

[dulayjm]

Well-written issue! I'll take a look and see if I can implement a few others potentially.

[martinbrose]

Hi @dulayjm / @Wauplin,

I ended up going through the list and doing all of the remaining tasks as little challenges.

Thanks @dulayjm for providing a good template with your object detection PR. I hope you weren't working on any of the remaining ones. I'm sorry if you did...

[dulayjm]

Thank you for writing them!!

[martinbrose]

@Wauplin, many thanks for all the PR reviews and the review points provided.
It's been a great learning experience.

You've been on 🔥 after your holiday!

[Wauplin]

Marathon is now over!🏃😴

Thanks @martinbrose @dulayjm for the massive community work on this issue 🤗 I never thought it would be closed that quickly. We now support all main tasks available on the Hub which is a very nice improvement 🔥 🚀

[martinbrose]

@Wauplin, there is one more thing I would like to discuss:

When looking at the tasks that don't have inference models readily available and a model needs to start up, like tabular classification, I ran into some problems in my Jupyter Notebook playground.

I believe the implementation is sending API requests in quick succession while the model is starting up, and the user gets rate-limited super quickly.

Should that be mentioned in the doc string, and advice added on how to avoid this from happening?

[Wauplin]

Hmm, that's a good question. Have you been rate-limited yourself? In general rate limits happen quickly if the user is not logged in (e.g. not registered with a token). What we could do is to catch HTTP 429 errors here and here and if:

1. a HTTP 429 happen
2. and no token in the header
3. and uses InferenceAPI (e.g. url starts with "https://api-inference.huggingface.com")

=> Then we raise append a message to the raised error saying "hey, you should use a token".

This way it would be seen much more easily by user (than completing the docstring). WDYT?

Also maybe we should add a warning on tabular-classification/tabular-regression that they are not working on InferenceAPI at the moment (which is not so problematic given how little usage we have for tabular data on the Hub).

[martinbrose]

Yeah, I think some sort of error-catching and warning might be needed. I'll have a look at the above links tonight.

I was logged in with a token and even signed up for Pro for a few days. In all cases, I got rate-limited myself very quick.
I haven't had the time to understand exactly why it's happening. I did add the additional parameter "wait_for_model": True", but that didn't seem to help much either.

[Wauplin]

Hmm interesting feedback, I didn't know about that. Let's open a new issue then to address it specifically. Would you mind doing it and report your use case + error?

For what I understand, in the worst case scenario InferenceClient should make 1 call per second which shouldn't lead rate-limits too quickly. I saw the "wait_for_model": True but to be honest this is a bit outdated now that we've set a retry mechanism in InferenceClient directly (or at least I thought so). I'll investigate further when I have time.

[martinbrose]

Sure, no problem.
Will attend to this over the weekend.