如何使用Mobile Vision API获取图像中文本的位置？

Question

如何使用Mobile Vision API获取图像中文本在屏幕上的位置，并绘制矩形框将其框起来？

示例：

- VINNUSAURUS

---

在返回的“文本”中，您可以提取包括“边界框”在内的组件。Google有一个CodeLab与此完全相同的示例，只需要将其翻译为C＃。https://codelabs.developers.google.com/codelabs/mobile-vision-ocr/#6 - SushiHangover

好的，我现在会检查。 - VINNUSAURUS

GraphicOverlay是示例中的一个类，它只是通过GraphicOverlay.Graphic抽象类扩展了View。我相信已经有一段时间没有引用它来在Xamarin中使用Mobile Vision了（顺便说一句：如果您正在编写新代码，您可能想看看“ML Kit”，因为它已经取代了Google的Mobile Vision API）。 - SushiHangover

ML.Kit的设备处理与M.Vision一样是“免费”的，但更高精度的云服务（如OCR/文本识别器）则需要通过Firebase付费使用，因为它们不仅限于移动优化的ML模型，而且可以与Xamarin的Android/iOS API很好地配合使用。 - SushiHangover

ML.Kit：本地=免费，云= $ - SushiHangover

Answer 1

如何实现

在布局中添加一个 ImageView 元素。

<LinearLayout xmlns:android="http://schemas.android.com/apk/res/android"
xmlns:app="http://schemas.android.com/apk/res-auto"
xmlns:tools="http://schemas.android.com/tools"
android:layout_width="match_parent"
android:layout_height="match_parent"
android:orientation="vertical">
<ImageView
    android:layout_width="match_parent"
    android:layout_height="250.0dp"
    android:minWidth="25px"
    android:minHeight="25px"
    android:id="@+id/imageView1" />
</LinearLayout>

在onCreate方法中实例化ImageView

ImageView imgView;
    protected override void OnCreate(Bundle savedInstanceState)
    {
        base.OnCreate(savedInstanceState);
        SetContentView(Resource.Layout.activity_main);

        imgView = FindViewById<ImageView>(Resource.Id.imageView1);
        OCR();
    }

这是获取文本并在其上绘制矩形的重要代码

请阅读代码顶部的注释

public void OCR()
    {
        //Convert The Image To Bitmap
        Bitmap bitmap = BitmapFactory.DecodeResource(ApplicationContext.Resources, Resource.Mipmap.lineindent);

        TextRecognizer textRecognizer = new TextRecognizer.Builder(ApplicationContext).Build();

        if (!textRecognizer.IsOperational)
        {

            Log.Error("Main Activity", "Dependencies not available");

            // Check android for low storage so dependencies can be loaded, DEPRICATED CHANGE LATER
            IntentFilter intentLowStorage = new IntentFilter(Intent.ActionDeviceStorageLow);

            bool hasLowStorage = RegisterReceiver(null, intentLowStorage) != null;

            if (hasLowStorage)
            {

                Toast.MakeText(this, "Low Memory On Disk", ToastLength.Long);
                Log.Error("Main Activity", "Low Memory On Disk");
            }

        }
        else
        {
            Frame frame = new Frame.Builder().SetBitmap(bitmap).Build();


            SparseArray items = textRecognizer.Detect(frame);
            List<TextBlock> blocks = new List<TextBlock>();

            TextBlock myItem = null;
            for (int i = 0; i < items.Size(); ++i)
            {
                myItem = (TextBlock)items.ValueAt(i);

                //Add All TextBlocks to the `blocks` List
                blocks.Add(myItem);

            }
            //END OF DETECTING TEXT

            //The Color of the Rectangle to Draw on top of Text
            Paint rectPaint = new Paint();
            rectPaint.Color = Color.White;
            rectPaint.SetStyle(Paint.Style.Stroke);
            rectPaint.StrokeWidth = (4.0f);

            //Create the Canvas object,
            //Which ever way you do image that is ScreenShot for example, you 
            //need the views Height and Width to draw recatngles 
            //because the API detects the position of Text on the View
            //So Dimesnions are important for Draw method to draw at that Text 
            //Location
            Bitmap tempBitmap = Bitmap.CreateBitmap(bitmap.Width, bitmap.Height, Bitmap.Config.Rgb565);
            Canvas canvas = new Canvas(tempBitmap);
            canvas.DrawBitmap(bitmap, 0, 0, null);

            //Loop through each `Block`
            foreach (TextBlock textBlock in blocks)
            {
                IList<IText> textLines = textBlock.Components; 

                //loop Through each `Line`
                foreach (IText currentLine in textLines)
                {
                    IList<IText>  words = currentLine.Components;

                    //Loop through each `Word`
                    foreach (IText currentword in words)
                    {
                        //Get the Rectangle/boundingBox of the word
                        RectF rect = new RectF(currentword.BoundingBox);
                        rectPaint.Color = Color.Black;

                        //Finally Draw Rectangle/boundingBox around word
                        canvas.DrawRect(rect, rectPaint);

                        //Set image to the `View`
                        imgView.SetImageDrawable(new BitmapDrawable(Resources, tempBitmap));


                    }

                }
            }

        }
    }

结果

如果您想在Lines上显示矩形，则需要从words循环中删除代码并将其放入Lines循环中，对于块也是如此。

- VINNUSAURUS