After uploading several videos we drew a bounding box around our shipping element in the first frame of each, labeled the box with “ShipElem”, and then used the “Tracking with OpenCV” feature (with the CSRT algorithm) to box and label the remaining frames. This worked well with the exception of these two relatively small issues:
First: We drew the bounding box from upper left to lower right. When we released the mouse on the lower right, the resulting box drawn on the screen jumped down about 50 pixels. We then just repositioned it around the shipping element and all was well. This happened every time we drew a bounding box.
Second: After labeling 36 videos of our shipping element with only the label “ShipElem”, we generated a dataset that resulted in four labels which were:
ShipE
ShipEl
ShipEle
ShipElem
We then went back to the videos and went through each frame one at a time to find and fix the instances of the first three incorrect labels. After that only the “ShipElem” label was in the dataset. All of the bad labels were in the first frame of a video.
We did this twice to be sure we were not typing in the bad labels ourselves accidentally. After being sure that we only typed in “ShipElem”, the resulting dataset again had labels in which the end of the “ShipElem” string was truncated in several spots.
In a nutshell, I think the labels are sometimes being truncated by the software.
We are using Chrome version 95.0.4638.69 (Official Build) (x86_64) on a MacBook Pro running MacOS 10.15.7 (Catalina).
Thanks for making this tool available!
Confidential Robotics - Team 13243, Eagan, Minnesota
Follow up: We just created our first model and noticed one issue that may be a result of the first bug reported in this thread (the bounding box jumping issue). When viewing the “Images” tab in the model section and viewing the TensorFlow-labeled image next to the user-labeled image, we noticed that the user-labeled image shows the bounding boxes too high (and hence the model was trained to see that image as our shipping element).
I’ll attach a picture below. Notice that the bounding box was drawn too high. However, when we drew the box in the labeling section of the UI, it appeared to exactly surround the shipping element after we adjusted it upward due to the initial downward shift reported in this bug.
We’ll be creating a new set of videos and trying this all again after trying out our new model with the flawed bounding boxes. By the way, the training timing guidance was very good in the manual. We requested 1000 steps (360 total images) and it took 21 minutes and 34 seconds to finish (the manual estimated 60 minutes for 3000 steps which is pretty much what we experienced).
What is the width/height of your video that has the bounding box issue?
Also, in the labeling page, you can click the - + buttons above the right side of the image to shrink/zoom the image on screen. Can you tell me if the bounding box jump happens at all the different shrink/zoom settings?
The video is from a Logitech C310 camera which emits a 1280 x 720 image.
Your suggestion led me to try a few other things and I think I know what the general issue is. When we did labeling initially, we used the MacBook Pro’s laptop screen with no external monitor. With no zoom in or zoom out, the issue occurs. When we zoom out (the - button) the issue does not occur at all three zoom out levels. When we zoom in (the + button) the issue occurs at all zoom in levels.
Thinking this is likely a CSS/layout issue, I attached a high resolution external monitor to the MacBook Pro and tried again. The issue did not occur at the “no zoom” level nor did it occur at any of the zoom out levels (the - button). The issue also did not occur at the first several zoom in levels (the + button). But eventually it did occur at high enough zoom.
The bounding box moves down incorrectly after releasing the mouse likely because the table containing the X1, Y1, X2, Y2, and Label fields gets populated thus pushing the screen wider. The CSS controlling the bounding box is likely interacting with the other controls in an unexpected way.
Most likely this is only occurring for people using small laptop screens (ours is 13" - 2560 x 1600) or low res monitors. We now know we can get around it by using our external monitor or zooming out when using the laptop screen.
I attached one screen capture showing the issue and another showing it working. Hope this helps!
I had a similar problem with the labels truncating. To fix it, I deleted the video, uploaded it again, labeled the first bounding box, hit enter and clicked out of the text box (not sure if I needed to do both of those), refreshed the page, verified the correct label, then ran tacking. No more truncated labels.
