CameraX 分析器不在每一帧上运行。

Question

我尝试使用Android开发者文档，通过CameraX实现MLKit文本分析器。我期望分析器可以在每个帧上运行，但实际上只有当Preview被初始化时它才会运行一次。虽然MLKit对文本进行了分析，但只有当我旋转屏幕并重新启动Fragment时才会执行分析。为什么会这样呢？

在XML中，我使用了<androidx.camera.view.PreviewView .../>。

在Fragment中：

private lateinit var cameraProviderFuture: ListenableFuture<ProcessCameraProvider>

private fun startCamera() {
    context?.let { ctx ->
        cameraProviderFuture = ProcessCameraProvider.getInstance(ctx)
        cameraProviderFuture.addListener(Runnable {
            val cameraProvider = cameraProviderFuture.get()
            bindPreview(cameraProvider)
        }, ContextCompat.getMainExecutor(ctx))
    }
}

private fun bindPreview(cameraProvider: ProcessCameraProvider) {
    val preview: Preview = Preview.Builder()
        .build()

    val cameraSelector: CameraSelector = CameraSelector.Builder()
        .requireLensFacing(CameraSelector.LENS_FACING_BACK)
        .build()

    val analyzer = ImageAnalysis.Builder()
        .setTargetResolution(Size(1280, 720))
        .setBackpressureStrategy(ImageAnalysis.STRATEGY_KEEP_ONLY_LATEST)
        .build().apply {
            setAnalyzer(Executors.newSingleThreadExecutor(), TextAnalyzer())
        }

    val camera = cameraProvider.bindToLifecycle(viewLifecycleOwner, cameraSelector, analyzer, preview)

    preview.setSurfaceProvider(preview_view.createSurfaceProvider(camera.cameraInfo))
}

分析器实现：

class TextAnalyzer : ImageAnalysis.Analyzer {

val detector = FirebaseVision.getInstance().onDeviceTextRecognizer

@SuppressLint("UnsafeExperimentalUsageError")
override fun analyze(imageProxy: ImageProxy) {
    val mediaImage = imageProxy.image
    val rotation = degreesToFirebaseRotation(imageProxy.imageInfo.rotationDegrees)
    mediaImage?.let { image ->
        val firebaseImage = FirebaseVisionImage.fromMediaImage(image, rotation)
        detector.processImage(firebaseImage)
            .addOnSuccessListener { firebaseVisionText ->
                val text = firebaseVisionText.text
                val textBlock = firebaseVisionText.textBlocks
                Log.i("TEXT_ANALYZER", "success text $text")
                textBlock.forEach {
                    Log.i("TEXT_ANALYZER", "success text BLOCK $it")
                }
            }
            .addOnFailureListener { e ->
                Log.i("TEXT_ANALYZER", "failed ex $e")
            }
    }
}

private fun degreesToFirebaseRotation(degrees: Int): Int = when (degrees) {
    0 -> ROTATION_0
    90 -> ROTATION_90
    180 -> ROTATION_180
    270 -> ROTATION_270
    else -> throw Exception("Rotation must be 0, 90, 180, or 270.")
}
}

- vrgrg

Answer 1

我已经找到了答案。我只需要更仔细地阅读文档，文档中说明必须关闭图像代理以避免阻塞线程和预览 :-)

  override fun analyze(imageProxy: ImageProxy) {
        val mediaImage = imageProxy.image
        val rotation = degreesToFirebaseRotation(imageProxy.imageInfo.rotationDegrees)
        mediaImage?.let { image ->
            val firebaseImage = FirebaseVisionImage.fromMediaImage(image, rotation)
            detector.processImage(firebaseImage)
                .addOnSuccessListener { firebaseVisionText ->
                    val text = firebaseVisionText.text
                    val textBlock = firebaseVisionText.textBlocks
                    // IMPORTANT
                    imageProxy.close()
                }
                .addOnFailureListener { e ->
                    // IMPORTANT
                    imageProxy.close()
                }
        }
    }

- vrgrg

---

没有阅读手册：一旦通过调用ImageProxy.close()关闭正在分析的图像。 - Peter Chaula