In a past game, before the VisionPortal, I helped some students use JNI to call OpenCV code written in c++. At that time we had to import OpenCV as a module into our FTC Android Studio project. But now with the VisionPortal I don’t see an OpenCV module, although OpenCV is available in Java, so I don’t know how to reference it from CMakeLists.txt. Can you help?
Hi there - author of Vision Portal and EasyOpenCV here.
You’re not going to be able to directly build OpenCV C++ code against against the modern SDK without some additional hoops. At minimum, you need the header files to build against, and these are not packaged with the SDK. Linking against the shared library may also prove difficult because it will be located in some obscure gradle dependency cache directory instead of in the build tree.
I actually have a quickstart repository for using OpenCV from C++ with EOCV located here: GitHub - OpenFTC/EasyOpenCV-Cpp-Quickstart: A quickstart FTC Robot Controller project for running C++ OpenCV code in conjunction with EasyOpenCV
I see now however that I haven’t updated it since SDK 8.0 and EOCV v1.5.2. Some things have changed in recent years that means just bumping version numbers isn’t going to work. I might try to get it updated in the next couple weeks if I have time.
We are still interested in this topic. The header files were not a problem and we built an OpMode that reads an image file on the RobotController and accesses our c++ code via JNI. But when we run even an unrelated VisionPortal OpMode such as ConceptAprilTag we get a very obscure link error at runtime. We can supply more details if you need them.
In case it might help I am including the matchlog from the unsuccessful run. In this latest attempt to isolate the problem we didn’t include any Java/JNI classes; all we did was build the project, which invoked the c++ CMakeLists.txt, and then attempted to run ConceptAprilTag.
I’m sure you’re busy right now but we appeciate any help you can give.
Phil Young
Volunteer mentor, FTC Team 4348
09-16 07:16:47.144 3117 3231 V Robocol : received command: CMD_SET_MATCH_NUMBER(10207) 0
09-16 07:16:47.146 3117 3231 V Robocol : received command: CMD_INIT_OP_MODE(10208) Concept: AprilTag
09-16 07:16:47.171 3117 3274 V RobotCore: thread: …terminating ‘OpModeThread’
09-16 07:16:47.176 3117 3255 I RobotCore: Attempting to switch to OpMode Concept: AprilTag
09-16 07:16:47.183 3117 3255 I RobotCore: ******************** START - OPMODE Concept: AprilTag ********************
09-16 07:16:47.318 3117 3255 D HardwareMap: Clearing which device instances have been retrieved
09-16 07:16:47.321 3117 3277 V RobotCore: thread: ‘OpModeThread’ starting…
09-16 07:16:47.328 3117 3277 I System.out: AprilTag plugin v2.1.0
09-16 07:16:47.331 3117 3277 D AprilTagDetectorJNI: Creating 36h11 tag family
09-16 07:16:47.332 3117 3277 D AprilTagDetectorJNI: Initializing april tag detector
09-16 07:16:47.359 3117 3235 V Robocol : sending CMD_NOTIFY_INIT_OP_MODE(322), attempt: 0
09-16 07:16:47.454 3117 3277 V RobotCore: thread: …terminating ‘OpModeThread’
09-16 07:16:47.454 3117 3277 E ThreadPool: exception thrown in thread pool; ignored
09-16 07:16:47.463 3117 3277 E ThreadPool: java.lang.UnsatisfiedLinkError: dlopen failed: cannot locate symbol “_ZN2cv16cvtColorTwoPlaneERKNS_11_InputArrayES2_RKNS_12_OutputArrayEi” referenced by “/data/app/com.qualcomm.ftcrobotcontroller-2/lib/arm64/libEasyOpenCV.so”…
09-16 07:16:47.465 3117 3277 E ThreadPool: at java.lang.Runtime.loadLibrary0(Runtime.java:989)
09-16 07:16:47.466 3117 3277 E ThreadPool: at java.lang.System.loadLibrary(System.java:1562)
09-16 07:16:47.468 3117 3277 E ThreadPool: at org.openftc.easyopencv.OpenCvCameraBase.(OpenCvCameraBase.java:1071)
09-16 07:16:47.471 3117 3277 E ThreadPool: at org.openftc.easyopencv.OpenCvCameraFactoryImpl.createWebcam(OpenCvCameraFactoryImpl.java:104)
09-16 07:16:47.473 3117 3277 E ThreadPool: at org.firstinspires.ftc.vision.VisionPortalImpl.createCamera(VisionPortalImpl.java:254)
09-16 07:16:47.475 3117 3277 E ThreadPool: at org.firstinspires.ftc.vision.VisionPortalImpl.(VisionPortalImpl.java:157)
09-16 07:16:47.477 3117 3277 E ThreadPool: at org.firstinspires.ftc.vision.VisionPortal$Builder.build(VisionPortal.java:340)
09-16 07:16:47.479 3117 3277 E ThreadPool: at org.firstinspires.ftc.teamcode.samples.ConceptAprilTag.initAprilTag(ConceptAprilTag.java:181)
09-16 07:16:47.481 3117 3277 E ThreadPool: at org.firstinspires.ftc.teamcode.samples.ConceptAprilTag.runOpMode(ConceptAprilTag.java:87)
09-16 07:16:47.481 3117 3277 E ThreadPool: at com.qualcomm.robotcore.eventloop.opmode.LinearOpMode.internalRunOpMode(LinearOpMode.java:199)
09-16 07:16:47.483 3117 3277 E ThreadPool: at com.qualcomm.robotcore.eventloop.opmode.OpModeInternal.lambda$internalInit$1$com-qualcomm-robotcore-eventloop-opmode-OpModeInternal(OpModeInternal.java:184)
09-16 07:16:47.483 3117 3277 E ThreadPool: at com.qualcomm.robotcore.eventloop.opmode.OpModeInternal$$ExternalSyntheticLambda0.run(D8$$SyntheticClass:0)
09-16 07:16:47.484 3117 3277 E ThreadPool: at com.qualcomm.robotcore.util.ThreadPool.logThreadLifeCycle(ThreadPool.java:737)
09-16 07:16:47.485 3117 3277 E ThreadPool: at com.qualcomm.robotcore.eventloop.opmode.OpModeInternal.lambda$internalInit$2$com-qualcomm-robotcore-eventloop-opmode-OpModeInternal(OpModeInternal.java:182)
09-16 07:16:47.485 3117 3277 E ThreadPool: at com.qualcomm.robotcore.eventloop.opmode.OpModeInternal$$ExternalSyntheticLambda1.run(D8$$SyntheticClass:0)
09-16 07:16:47.486 3117 3277 E ThreadPool: at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1133)
09-16 07:16:47.486 3117 3277 E ThreadPool: at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:607)
09-16 07:16:47.487 3117 3277 E ThreadPool: at com.qualcomm.robotcore.util.ThreadPool$ThreadFactoryImpl$1.run(ThreadPool.java:793)
09-16 07:16:47.487 3117 3277 E ThreadPool: at java.lang.Thread.run(Thread.java:761)
--------- beginning of crash
09-16 07:16:47.489 3117 3277 E AndroidRuntime: FATAL EXCEPTION: OpModeThread
09-16 07:16:47.489 3117 3277 E AndroidRuntime: Process: com.qualcomm.ftcrobotcontroller, PID: 3117
09-16 07:16:47.489 3117 3277 E AndroidRuntime: java.lang.UnsatisfiedLinkError: dlopen failed: cannot locate symbol “_ZN2cv16cvtColorTwoPlaneERKNS_11_InputArrayES2_RKNS_12_OutputArrayEi” referenced by “/data/app/com.qualcomm.ftcrobotcontroller-2/lib/arm64/libEasyOpenCV.so”…
09-16 07:16:47.489 3117 3277 E AndroidRuntime: at java.lang.Runtime.loadLibrary0(Runtime.java:989)
09-16 07:16:47.489 3117 3277 E AndroidRuntime: at java.lang.System.loadLibrary(System.java:1562)
09-16 07:16:47.489 3117 3277 E AndroidRuntime: at org.openftc.easyopencv.OpenCvCameraBase.(OpenCvCameraBase.java:1071)
09-16 07:16:47.489 3117 3277 E AndroidRuntime: at org.openftc.easyopencv.OpenCvCameraFactoryImpl.createWebcam(OpenCvCameraFactoryImpl.java:104)
09-16 07:16:47.489 3117 3277 E AndroidRuntime: at org.firstinspires.ftc.vision.VisionPortalImpl.createCamera(VisionPortalImpl.java:254)
09-16 07:16:47.489 3117 3277 E AndroidRuntime: at org.firstinspires.ftc.vision.VisionPortalImpl.(VisionPortalImpl.java:157)
09-16 07:16:47.489 3117 3277 E AndroidRuntime: at org.firstinspires.ftc.vision.VisionPortal$Builder.build(VisionPortal.java:340)
09-16 07:16:47.489 3117 3277 E AndroidRuntime: at org.firstinspires.ftc.teamcode.samples.ConceptAprilTag.initAprilTag(ConceptAprilTag.java:181)
09-16 07:16:47.489 3117 3277 E AndroidRuntime: at org.firstinspires.ftc.teamcode.samples.ConceptAprilTag.runOpMode(ConceptAprilTag.java:87)
09-16 07:16:47.489 3117 3277 E AndroidRuntime: at com.qualcomm.robotcore.eventloop.opmode.LinearOpMode.internalRunOpMode(LinearOpMode.java:199)
09-16 07:16:47.489 3117 3277 E AndroidRuntime: at com.qualcomm.robotcore.eventloop.opmode.OpModeInternal.lambda$internalInit$1$com-qualcomm-robotcore-eventloop-opmode-OpModeInternal(OpModeInternal.java:184)
09-16 07:16:47.489 3117 3277 E AndroidRuntime: at com.qualcomm.robotcore.eventloop.opmode.OpModeInternal$$ExternalSyntheticLambda0.run(D8$$SyntheticClass:0)
09-16 07:16:47.489 3117 3277 E AndroidRuntime: at com.qualcomm.robotcore.util.ThreadPool.logThreadLifeCycle(ThreadPool.java:737)
09-16 07:16:47.489 3117 3277 E AndroidRuntime: at com.qualcomm.robotcore.eventloop.opmode.OpModeInternal.lambda$internalInit$2$com-qualcomm-robotcore-eventloop-opmode-OpModeInternal(OpModeInternal.java:182)
09-16 07:16:47.489 3117 3277 E AndroidRuntime: at com.qualcomm.robotcore.eventloop.opmode.OpModeInternal$$ExternalSyntheticLambda1.run(D8$$SyntheticClass:0)
09-16 07:16:47.489 3117 3277 E AndroidRuntime: at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1133)
09-16 07:16:47.489 3117 3277 E AndroidRuntime: at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:607)
09-16 07:16:47.489 3117 3277 E AndroidRuntime: at com.qualcomm.robotcore.util.ThreadPool$ThreadFactoryImpl$1.run(ThreadPool.java:793)
09-16 07:16:47.489 3117 3277 E AndroidRuntime: at java.lang.Thread.run(Thread.java:761)
09-16 07:16:47.499 3117 3117 V RCActivity: onPause()
09-16 07:16:48.088 3117 3117 V OnBotJavaService: onTrimMemory()
09-16 07:16:48.088 3117 3117 V MTPMonitorService: onTrimMemory()
09-16 07:16:48.089 3117 3117 V RCActivity: onStop()
09-16 07:16:48.090 3117 3117 V AppUtil : rootActivity=FtcRobotControllerActivity destroyed
09-16 07:16:48.091 3117 3117 V AppUtil : rootActivity=FtcRobotControllerActivity
09-16 07:16:48.091 3117 3117 V RCActivity: onDestroy()
09-16 07:16:48.091 3117 3117 V Robocol : EventLoopManager.shutdown()
09-16 07:16:48.104 3117 3117 I RobotCore: ******************** STOP - OPMODE /storage/emulated/0/FIRST/matchlogs/Match-0-Concept:_AprilTag.txt ********************
09-16 07:16:48.114 3117 3255 V EventLoopManager: EventLoopRunnable interrupted
09-16 07:16:48.115 3117 3255 E EventLoopManager: java.lang.InterruptedException
09-16 07:16:48.116 3117 3255 E EventLoopManager: at java.lang.Thread.sleep(Native Method)
09-16 07:16:48.117 3117 3255 E EventLoopManager: at java.lang.Thread.sleep(Thread.java:371)
09-16 07:16:48.118 3117 3255 E EventLoopManager: at java.lang.Thread.sleep(Thread.java:313)
09-16 07:16:48.121 3117 3255 E EventLoopManager: at com.qualcomm.robotcore.eventloop.EventLoopManager$EventLoopRunnable$1.run(EventLoopManager.java:278)
09-16 07:16:48.122 3117 3255 E EventLoopManager: at com.qualcomm.robotcore.util.ThreadPool.logThreadLifeCycle(ThreadPool.java:737)
09-16 07:16:48.123 3117 3255 E EventLoopManager: at com.qualcomm.robotcore.eventloop.EventLoopManager$EventLoopRunnable.run(EventLoopManager.java:267)
09-16 07:16:48.124 3117 3255 E EventLoopManager: at com.qualcomm.robotcore.eventloop.EventLoopManager$1.run(EventLoopManager.java:620)
09-16 07:16:48.125 3117 3255 E EventLoopManager: at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1133)
09-16 07:16:48.126 3117 3284 I RobotCore: saving match logcat to /storage/emulated/0/FIRST/matchlogs/Match-0-Concept:_AprilTag.txt
09-16 07:16:48.126 3117 3284 I RobotCore: logging command line: exec logcat -d -T ‘9-16 7:16:47.000’ -f ‘/storage/emulated/0/FIRST/matchlogs/Match-0-Concept:_AprilTag.txt’ -n4 -v threadtime UsbRequestJNI:S UsbRequest:S art:W ThreadPool:W System:W ExtendedExtractor:W OMXClient:W MediaPlayer:W dalvikvm:W *:V
09-16 07:16:48.127 3117 3255 E EventLoopManager: at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:607)
09-16 07:16:48.128 3117 3255 E EventLoopManager: at com.qualcomm.robotcore.util.ThreadPool$ThreadFactoryImpl$1.run(ThreadPool.java:793)
09-16 07:16:48.129 3117 3255 E EventLoopManager: at java.lang.Thread.run(Thread.java:761)
09-16 07:16:48.129 3117 3255 V RobotCore: EventLoopManager state is STOPPED
09-16 07:16:48.130 3117 3255 V RobotCore: Robot Status: stopped
09-16 07:16:48.130 3117 3255 I FtcEventLoop: ======= TEARDOWN =======
09-16 07:16:48.156 3285 3285 I sh : type=1400 audit(0.0:349): avc: denied { read } for name=“/” dev=“rootfs” ino=1 scontext=u:r:untrusted_app:s0:c512,c768 tcontext=u:object_r:rootfs:s0 tclass=dir permissive=1
09-16 07:16:48.159 3285 3285 I sh : type=1400 audit(0.0:350): avc: denied { open } for path=“/” dev=“rootfs” ino=1 scontext=u:r:untrusted_app:s0:c512,c768 tcontext=u:object_r:rootfs:s0 tclass=dir permissive=1
09-16 07:16:48.196 3117 3235 V Robocol : sending CMD_NOTIFY_ROBOT_STATE(332), attempt: 0
09-16 07:16:48.214 3117 3255 V LynxUsb : 0x0ad104ee on 0x01b0f8d2: releasing delegate to [(embedded)]
09-16 07:16:48.223 3117 3255 V LynxModule: sending LynxFtdiResetControlCommand(false) wasInterrupted=false
09-16 07:16:48.226 3117 3255 V LynxModule: close(#173)
I have updated the EOCV C++ Quickstart Repo to be current for SDK v11.0 and confirmed that the C++ pipeline example contained therein runs correctly.
If you prefer not to clone the quickstart, you can see the diff of the changes you need to make here.
Thank you very much for taking the time to do this. We’ll get to it right after our Meet 0, which is in less than 2 weeks.
I downloaded your quickstart and attempted to build it but got an error on the first try: ninja: error: ‘…/…/…/…/…/…/OpenCV-Repackaged/OpenCV-Android-SDK/src/main/jniLibs/arm64-v8a/libopencv_java4.so’, needed by ‘…/…/…/…/build/intermediates/cxx/Debug/2n3y1f4q/obj/arm64-v8a/libTeamCode.so’, missing and no known rule to make it
Did you also clone OpenCV-Repackaged according to the directions in the quickstart readme?
My mistake for not reading far enough in the readme. After I followed the instructions I was able to get the sample to work as expected. Thank you for taking the time to look into this. I hope that any changes you made will work their way into the next release of the FTC SDK.
Phil Young
Software Mentor, FTC Team 4348