The problem working with color is there is no such thing as red or blue or green. Is red a color with rgb == 255, 0, 0? For most images there won't be any pixels with that rgb value. How low can the red portion be before you start calling this black. Is 100, 0, 0 "red enough". How much additional brightness can there be and still call a color red? Is 255, 100, 100 too pink?
When looking for particular colors does it make more sense to use HSV color encoding? In the HSV color model red has a Hue of 0, green 120 and blue 240 with 360 being red again. Saturation is how saturated the color appears, 1 (or 100%) being all color and 0 being no color. Value is how bright or dark the color appears with 1 (or 100%) being very bright and 0 being black. The HSV equivalent of 255, 0, 0 is 0, 1, 1 (Hue = 0 degrees, Saturation = 1 or 100%, Value = 1 or 100%).
But even though HSV makes it easier to specify redness using hue (0 +/-10 degrees say), you still need to make rather arbitrary selections for saturation and value.