Detect semi-transparent rectangular overlays on images











up vote
0
down vote

favorite












I have images that contain transparent rectangular overlay similar to the following images: Image1 Image2. My goal is to detect if these rectangular boxes exist (location doesn't matter). These boxes will always have edges parallel to the sides of the image.



Assumptions:




  • The transfer function of how the transparent rectangles are drawn is now known

  • The sides of the rectangles will also be parallel to the image


Attempted Solution 1: Color Detection



So far, I've tried color detection via cv.threshold as well as using band-pass filters with the cv2.inRange() on multiple color spaces (HSV, LUV, XYZ etc). The issue with color detection is that I am also capturing too much noise to effectively just tune for the pixels for the transparent area. I tried laying the masks using cv2.bitwiseAnd but still can't tune the noise down to a negligible state. I tried only isolating for large groups of pixels using morphological transformations but this still fails.



Attempted Solution 2: Edge Detection + Edge Validation



My second try at detecting the box involved applying cv2.bilateralFilter and then generating hough lines via cv2.Canny,cv2.HoughLinesP. Although I am detecting a significant number of edges related to the transparent box, I also get many miscellaneous edges.



To filter out false edges, I take each line segment and check a few sample pixels to the left and right sides. By applying something something similar to what I believe the transfer function is (cv2.addWeighted) I checked to see if if I can reproduce the similar values. Unfortunately, this also doesn't work well enough to tell the difference between edges from the transparent box vs "real edges." Result From Edge Detection



Any thoughts on how I might detect these boxes is highly appreciated!










share|improve this question






















  • Are these images with alpha channels?
    – Dave W. Smith
    Nov 9 at 0:09










  • @Dave W. Smith No, they should be just RGB images
    – imptr
    Nov 9 at 1:31










  • If I were trying to tackle this, I'd be tempted to start by converting to HSV, isolating the V channel, and looking for first for potential corners, observing that there's a very high likelyhood of 'lighter' pixels with the corner, and darker ones on the outside. Then I'd try matching up the corners and sanity checking along the edges. It wouldn't be perfect. A piece of paper properly aligned would give a false positive.
    – Dave W. Smith
    Nov 9 at 3:42















up vote
0
down vote

favorite












I have images that contain transparent rectangular overlay similar to the following images: Image1 Image2. My goal is to detect if these rectangular boxes exist (location doesn't matter). These boxes will always have edges parallel to the sides of the image.



Assumptions:




  • The transfer function of how the transparent rectangles are drawn is now known

  • The sides of the rectangles will also be parallel to the image


Attempted Solution 1: Color Detection



So far, I've tried color detection via cv.threshold as well as using band-pass filters with the cv2.inRange() on multiple color spaces (HSV, LUV, XYZ etc). The issue with color detection is that I am also capturing too much noise to effectively just tune for the pixels for the transparent area. I tried laying the masks using cv2.bitwiseAnd but still can't tune the noise down to a negligible state. I tried only isolating for large groups of pixels using morphological transformations but this still fails.



Attempted Solution 2: Edge Detection + Edge Validation



My second try at detecting the box involved applying cv2.bilateralFilter and then generating hough lines via cv2.Canny,cv2.HoughLinesP. Although I am detecting a significant number of edges related to the transparent box, I also get many miscellaneous edges.



To filter out false edges, I take each line segment and check a few sample pixels to the left and right sides. By applying something something similar to what I believe the transfer function is (cv2.addWeighted) I checked to see if if I can reproduce the similar values. Unfortunately, this also doesn't work well enough to tell the difference between edges from the transparent box vs "real edges." Result From Edge Detection



Any thoughts on how I might detect these boxes is highly appreciated!










share|improve this question






















  • Are these images with alpha channels?
    – Dave W. Smith
    Nov 9 at 0:09










  • @Dave W. Smith No, they should be just RGB images
    – imptr
    Nov 9 at 1:31










  • If I were trying to tackle this, I'd be tempted to start by converting to HSV, isolating the V channel, and looking for first for potential corners, observing that there's a very high likelyhood of 'lighter' pixels with the corner, and darker ones on the outside. Then I'd try matching up the corners and sanity checking along the edges. It wouldn't be perfect. A piece of paper properly aligned would give a false positive.
    – Dave W. Smith
    Nov 9 at 3:42













up vote
0
down vote

favorite









up vote
0
down vote

favorite











I have images that contain transparent rectangular overlay similar to the following images: Image1 Image2. My goal is to detect if these rectangular boxes exist (location doesn't matter). These boxes will always have edges parallel to the sides of the image.



Assumptions:




  • The transfer function of how the transparent rectangles are drawn is now known

  • The sides of the rectangles will also be parallel to the image


Attempted Solution 1: Color Detection



So far, I've tried color detection via cv.threshold as well as using band-pass filters with the cv2.inRange() on multiple color spaces (HSV, LUV, XYZ etc). The issue with color detection is that I am also capturing too much noise to effectively just tune for the pixels for the transparent area. I tried laying the masks using cv2.bitwiseAnd but still can't tune the noise down to a negligible state. I tried only isolating for large groups of pixels using morphological transformations but this still fails.



Attempted Solution 2: Edge Detection + Edge Validation



My second try at detecting the box involved applying cv2.bilateralFilter and then generating hough lines via cv2.Canny,cv2.HoughLinesP. Although I am detecting a significant number of edges related to the transparent box, I also get many miscellaneous edges.



To filter out false edges, I take each line segment and check a few sample pixels to the left and right sides. By applying something something similar to what I believe the transfer function is (cv2.addWeighted) I checked to see if if I can reproduce the similar values. Unfortunately, this also doesn't work well enough to tell the difference between edges from the transparent box vs "real edges." Result From Edge Detection



Any thoughts on how I might detect these boxes is highly appreciated!










share|improve this question













I have images that contain transparent rectangular overlay similar to the following images: Image1 Image2. My goal is to detect if these rectangular boxes exist (location doesn't matter). These boxes will always have edges parallel to the sides of the image.



Assumptions:




  • The transfer function of how the transparent rectangles are drawn is now known

  • The sides of the rectangles will also be parallel to the image


Attempted Solution 1: Color Detection



So far, I've tried color detection via cv.threshold as well as using band-pass filters with the cv2.inRange() on multiple color spaces (HSV, LUV, XYZ etc). The issue with color detection is that I am also capturing too much noise to effectively just tune for the pixels for the transparent area. I tried laying the masks using cv2.bitwiseAnd but still can't tune the noise down to a negligible state. I tried only isolating for large groups of pixels using morphological transformations but this still fails.



Attempted Solution 2: Edge Detection + Edge Validation



My second try at detecting the box involved applying cv2.bilateralFilter and then generating hough lines via cv2.Canny,cv2.HoughLinesP. Although I am detecting a significant number of edges related to the transparent box, I also get many miscellaneous edges.



To filter out false edges, I take each line segment and check a few sample pixels to the left and right sides. By applying something something similar to what I believe the transfer function is (cv2.addWeighted) I checked to see if if I can reproduce the similar values. Unfortunately, this also doesn't work well enough to tell the difference between edges from the transparent box vs "real edges." Result From Edge Detection



Any thoughts on how I might detect these boxes is highly appreciated!







image-processing computer-vision object-detection cv2






share|improve this question













share|improve this question











share|improve this question




share|improve this question










asked Nov 8 at 23:24









imptr

61




61












  • Are these images with alpha channels?
    – Dave W. Smith
    Nov 9 at 0:09










  • @Dave W. Smith No, they should be just RGB images
    – imptr
    Nov 9 at 1:31










  • If I were trying to tackle this, I'd be tempted to start by converting to HSV, isolating the V channel, and looking for first for potential corners, observing that there's a very high likelyhood of 'lighter' pixels with the corner, and darker ones on the outside. Then I'd try matching up the corners and sanity checking along the edges. It wouldn't be perfect. A piece of paper properly aligned would give a false positive.
    – Dave W. Smith
    Nov 9 at 3:42


















  • Are these images with alpha channels?
    – Dave W. Smith
    Nov 9 at 0:09










  • @Dave W. Smith No, they should be just RGB images
    – imptr
    Nov 9 at 1:31










  • If I were trying to tackle this, I'd be tempted to start by converting to HSV, isolating the V channel, and looking for first for potential corners, observing that there's a very high likelyhood of 'lighter' pixels with the corner, and darker ones on the outside. Then I'd try matching up the corners and sanity checking along the edges. It wouldn't be perfect. A piece of paper properly aligned would give a false positive.
    – Dave W. Smith
    Nov 9 at 3:42
















Are these images with alpha channels?
– Dave W. Smith
Nov 9 at 0:09




Are these images with alpha channels?
– Dave W. Smith
Nov 9 at 0:09












@Dave W. Smith No, they should be just RGB images
– imptr
Nov 9 at 1:31




@Dave W. Smith No, they should be just RGB images
– imptr
Nov 9 at 1:31












If I were trying to tackle this, I'd be tempted to start by converting to HSV, isolating the V channel, and looking for first for potential corners, observing that there's a very high likelyhood of 'lighter' pixels with the corner, and darker ones on the outside. Then I'd try matching up the corners and sanity checking along the edges. It wouldn't be perfect. A piece of paper properly aligned would give a false positive.
– Dave W. Smith
Nov 9 at 3:42




If I were trying to tackle this, I'd be tempted to start by converting to HSV, isolating the V channel, and looking for first for potential corners, observing that there's a very high likelyhood of 'lighter' pixels with the corner, and darker ones on the outside. Then I'd try matching up the corners and sanity checking along the edges. It wouldn't be perfect. A piece of paper properly aligned would give a false positive.
– Dave W. Smith
Nov 9 at 3:42

















active

oldest

votes











Your Answer






StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");

StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);

StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});

function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});


}
});














 

draft saved


draft discarded


















StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53217667%2fdetect-semi-transparent-rectangular-overlays-on-images%23new-answer', 'question_page');
}
);

Post as a guest















Required, but never shown






























active

oldest

votes













active

oldest

votes









active

oldest

votes






active

oldest

votes
















 

draft saved


draft discarded



















































 


draft saved


draft discarded














StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53217667%2fdetect-semi-transparent-rectangular-overlays-on-images%23new-answer', 'question_page');
}
);

Post as a guest















Required, but never shown





















































Required, but never shown














Required, but never shown












Required, but never shown







Required, but never shown

































Required, but never shown














Required, but never shown












Required, but never shown







Required, but never shown







Popular posts from this blog

Schultheiß

Verwaltungsgliederung Dänemarks

Liste der Kulturdenkmale in Wilsdruff