It sounds like you're looking for a way to quantify the "distance" between colors - in mathematics they will call it a metric. For example, many people are intuitively quite comfortable with the Euclidean metric - it's just the distance between two points measured by a ruler. In the case of flowers, things are more complicated due to the subjective perception of different colors.
There's a pretty mother Wikipedia article on color difference , which includes links to various implementations.
The difference or distance between two colors is a metric of interest to the science of colors. This allows people to quantify a concept that would otherwise be described with adjectives, to the detriment of someone whose work is critical in color. General definitions use Euclidean distance in a device-independent color space.
In particular, there is Python Colormath , an implementation in python that converts various color encodings, and also appears to have a function to calculate the distance between two colors. If you happen to code in python, this sounds useful, although unfortunately I have no personal experience with this tool. Similar resources are also available there for MATLAB and Excel, provided by the authors of CIE2000 , the leading color formula.