Using EasyOpenCV and AprilTags in autonomous frequently (25% of the time) causes the WebCam to disappear until a physical power cycle of the robot

Software
1

Hello there, we are team 23260 (it’s our first year!) and we are using both EasyOpenCV (to detect our team prop position) and AprilTags (to align with the right place in the backdrop for delivery) in our autonomous.
Unfortunately every 3 of 4 times we run the autonomous code, the AprilTag camera opening fails and puts the system in a state that every new attempt to run any code (autonomous or teleop) will fail to find the webcam.
Even if we go to Edit Configuration and press “Scan”, the webcam dissapears.
The only way to recover from this state is a full power cycle of the control hub. Even the reboot that gets trigger automatically fails to get out of that bad state.

We are able to get a match log for the autonomous when the issue does NOT reproduce (here’s an example: Autonomous-works well.txt - Google Drive). Unfortunately, when the issue happens the ControlHub seems to hang and the match log fails to be written.

Any suggestions? Maybe this is happening because we are mixing both EasyOpenCV and AprilTags.
Does anyone know how we can use just the VisionPortal for both the EasyOpenCV pipeline and the AprilTags?

Today we first use EasyOpenCV to find the prop position, then close that, and create an instance of the VisionPortal for the AprilTag part.

Here is the EasyOpenCV part:

int cameraMonitorViewId = hardwareMap.appContext.getResources().getIdentifier(“cameraMonitorViewId”, “id”, hardwareMap.appContext.getPackageName());
webcam = OpenCvCameraFactory.getInstance().createWebcam(hardwareMap.get(WebcamName.class, “Webcam 1”), cameraMonitorViewId);
circleDetection = new CircleDetection(isRed);
webcam.setPipeline(circleDetection);
webcam.setMillisecondsPermissionTimeout(5000); // Timeout for obtaining permission is configurable. Set before opening.
webcam.openCameraDeviceAsync(new OpenCvCamera.AsyncCameraOpenListener() {
u/Override
public void onOpened() {
webcam.startStreaming(1280, 720, OpenCvCameraRotation.UPRIGHT);
}
u/Override
public void onError(int errorCode) {
}
});

The circleDetection is our OpenCV pipeline:

public class CircleDetection extends OpenCvPipeline {

…

public Mat processFrame(Mat input) {

…

}

}

Then, as soon as we find the team prop, we no longer need the OpenCV part, so we close it like this:

webcam.stopStreaming();
webcam.closeCameraDevice();

And then we create the AprilTagDetection class like this:

aprilTag = new AprilTagProcessor.Builder().build();

// Adjust Image Decimation to trade-off detection-range for detection-rate.
// eg: Some typical detection data using a Logitech C920 WebCam
// Decimation = 1 … Detect 2" Tag from 10 feet away at 10 Frames per second
// Decimation = 2 … Detect 2" Tag from 6 feet away at 22 Frames per second
// Decimation = 3 … Detect 2" Tag from 4 feet away at 30 Frames Per Second
// Decimation = 3 … Detect 5" Tag from 10 feet away at 30 Frames Per Second
// Note: Decimation can be changed on-the-fly to adapt during a match.
aprilTag.setDecimation(2);

// Create the vision portal by using a builder.
visionPortal = new VisionPortal.Builder()
.setCamera(lom.hardwareMap.get(WebcamName.class, “Webcam 1”))
.addProcessor(aprilTag)
.build();

setManualExposure(6, 250); // Use low exposure time to reduce motion blur

Any ideas what might be happening?

Thanks a lot!!

2

Hi - author of EOCV and VisionPortal here.

Note that VisionPortal is simply a wrapper around EOCV. You can either convert your code to a VisionProcessor and use it with VisionPortal (the LearnJava4FTC book has a section on it), or you can call the AprilTag VisionProcessor from an EOCV pipeline (see my post here).

In any case, what you are doing should in theory work. I may try to reproduce the issue as I have time. But, you definitely want to move everything to one API to avoid closing and re-opening the camera if possible.

1 Like
3

That makes a lot of sense, thanks so much.
We are going to first try converting the code to a Vision Processor.
We’ll update this with the results.

1 Like
4

Hey there, thanks so much, we adapted our OpenCV pipeline to work with the VisionPortal and now everything works great. We stressed tested the autonomous and teleop transitions and the old issue didn’t repro a single time.
Thanks!

5

My attention calls to the “setManualExposure(6,250)” line at the end of the code snippet in this post. We’re having trouble with that very line seeming to cause the same symptoms described in this post.

Tonight while working on our team’s robot we found that setManualExposure(…) often but not always causes our camera (C920) to stop functioning, requiring power cycles, reboots, or unplugging/replugging the camera to get it functioning again.

After removing the setManualExposure() call, we didn’t experience any further camera failures whatsoever. April tag recognition performance is degraded since we’re now using the longer default exposure time, but the camera no longer produces control hub resets, reboots, freezes, etc.

I’m wondering if this is related somehow.

FWIW, we’re also using a custom vision processor, but it’s already written using VisionProcessor as recommended by @Windwoes .

Any thoughts from the group as to whether setManualExposure() might be triggering resets/crashes somehow?

Pm

  • Coach, FTC #7172 “Technical Difficulties”
6

Without logs, impossible to say. That being said, assuming you’re re-using that method from the ConceptAprilTagOptimizeExposure.java OpMode, the implementation there is a little weird.

7

When the team meets again today we’ll put setExposure() back in and see if we can grab a log.

Yes, we’re using the method from ConceptAprilTagOptimizeExposure . It seems a little weird to me too, but we don’t know enough about the internals yet to make it “less weird”. Any suggestions for doing that?

8

Interesting, after we made the change to use VisionPortal only (for both OpenCV pipeline and AprilTags) everything worked great for us, without any camera issues at all (we did some stress testing, trying both autonomous and teleop and didn’t repro the issue at all).