How to get pixel perfect location of text?

[thebest132]

So bounding boxes can be off a lot of the time, almost always a few pixels off. I want a perfect bounding box fit around the text, or at least snugly on the highest pixel of the text and the leftmost pixel, so that I can get an accurate coordinate for exact text reproduction. How do I do this? It should also work for angled text, perspective warped text, stroked text, etc.

[GreatV]

Getting pixel-perfect bounding boxes around text, especially for angled, warped, or stroked text, is challenging. Based on the discussion referenced (Getting character level bounding boxes), the main issue is that PaddleOCR provides line-level bounding boxes rather than character-level ones.

Possible Approaches:

1. Tesseract with Character-Level Bounding Boxes

   • Tesseract OCR can be configured to provide character-level bounding boxes using:

     tesseract image.png output --psm 6 --oem 1 -c tessedit_char_whitelist="ABCD..."

   • Use hocr or TSV output format to extract character positions.

     tesseract image.png output -c tessedit_create_tsv=1

   • The downside is that Tesseract struggles with accuracy on small distortions and spacing variations.

2. Enhancing PaddleOCR for Character-Level Bounding Boxes
   Since PaddleOCR currently provides line-level bounding boxes:

   • You can segment the line-based output further by applying connected component analysis (CCA) via OpenCV.
   • Convert the line bounding boxes into individual character bounding boxes by clustering detected pixels based on color variations and intensity.

3. Deep Learning-Based Text Segmentation

   • Use models like CRAFT (Character Region Awareness for Text Detection), which provides character-level bounding boxes.
   • This method can handle rotated, warped, and stylized text better than traditional OCR methods.

4. Post-Processing with Morphological Operations

   • If you're working with printed text, you can use morphological transformations in OpenCV, like dilation and erosion, to refine bounding boxes.
   • Use edge-detection techniques (like Canny edge detection) to align bounding boxes snugly on character edges.

5. Perspective Correction for Warped Text

   • If the text is perspective-warped, use homography estimation or document skew correction before OCR processing.
   • OpenCV provides functions such as cv2.findHomography and cv2.warpPerspective for this purpose.

Best Approach:

If you need the highest accuracy for bounding box refinement:

• Use CRAFT for initial character detection.
• Apply CCA and morphological post-processing to further enhance character boundaries.
• If needed, fine-tune bounding boxes using deep-learning-based pixel segmentation models.

Would you like step-by-step guidance on implementing any of these methods? 🚀

---

Response generated by 🤖 feifei-bot | chatgpt-4o-latest

⚠️ This is an AI-generated response. Please verify any critical information.

[thebest132]

@GreatV Hi human GreatV, this is a tough problem. Any ideas? I want to use PaddleOCR because it is by far the best OCR with the best language support. I want to ask you because you have a deep understanding of these topics. I don't know if there's a best CV method that will always find the top left coordinate no matter the style of the text (not top left of first character, top left of line).

[GreatV]

You may need to use the det_db_unclip_ratio parameter to adjust bounding box tightness. For more details, visit https://paddlepaddle.github.io/PaddleOCR/latest/en/ppocr/blog/inference_args.html.

[thebest132]

@GreatV

This is the result of setting det_db_unclip_ratio to 2. As you can see, for some text there is a lot of padding, and for other text there is not enough. You can also observe that the bottom large subtitle line has text coming outside of the bounding box.

I don't think it's possible to get a pixel perfect coordinate using just this parameter. The top subtitle line is closest to perfect but is still possibly 1-2 pixels off. I need 100% perfect left-most and top-most coordinate for reproducing the text myself.

It will also become even tougher for text with drop shadow, or glow, or perspective warp.

[GreatV]

you may need fune-tune your model, please refer to https://paddlepaddle.github.io/PaddleOCR/latest/en/ppocr/model_train/finetune.html#2-text-detection-model-fine-tuning