How to sort colors in two dimensions?

I am currently working on a hobby to automatically solve the puzzle from the popular mobile game I Love Hue. The game is available here .

In principle, the whole point of the game is that you are given a bunch of colored rectangular blocks organized in a grid. You can swap most blocks, with the exception of a few fixed blocks marked with black dots. The goal of the game is to swap blocks to get a two-dimensional color spectrum. Colors are sorted so that the color of each block roughly corresponds to the average value of the colors around it. (Sorry, I don’t know any color theory, but there is probably a word for what I'm looking for.) Here is a typical puzzle:

img

I already managed to take screenshots via adb, extract the RGB matrix from the blocks and note which blocks are “fixed”. I am having problems with the actual algorithmic part of this problem.

Here is what I have done so far:

  • Convert RGB to HSV and sort colors by hue in a one-dimensional list. This gives me a spectrum, but I don’t know how to convert this result into two dimensions.
  • We leave the colors in RGB and try to work with a special color. There probably is a multivariate calculus that I could do here, but the difficulty is that some colors share one or more RGB values. It would be necessary to consider all three colors.
  • Using Euclidean distance to find the distance between each pair of colors. I understand that the ultimate goal is to make this distance the smallest among adjacent colors, but a two-dimensional grid makes this difficult.
  • Using the Euclidean distance, I developed a metric on how perfect a particular grid is if you look at the Euclidean distance of the colors of neighboring blocks. However, I cannot find an effective algorithm that can determine the swaps necessary to achieve the ideal state.
+5
source share
2 answers

If you have more images solved , you can create a graph of RGB plots

plot a 3D graph, where x,y is the pixel position, and z is the color channel (R, G, or B). From it you can define some properties of gradients. If the graph is a plane than all you need is just normal (taken from 3 known cells). If this is a curved surface, depending on how many infections it received, you can determine how a large polynomial was used for it. From all this, you can start to solve it.

I would start with something simple (assuming not too large spaces or fancy polynomials):

Process each color channel separately. I would use only static tiles and interpolate the grid colors only from them. Something like:

Without seeing the graphs of R, G, B , I can not evaluate what kind of interpolation you need. If the graphs are linear, use bilinear or linear interpolation. If you do not use higher polynomials.

So, fill in all the grid cells that you can (have neighbors with a known color). After that, find the closest moving tile to the calculated color (if the cell has all 3 interpolated channels) and place them (and set them as static).

Now just repeat the process until all cells are calculated.

[Edit1 Dec 14 2017] some additional notes and more

It was curious and some time passed, so I gave him a chance. First I create a game in C ++ / VCL that takes your image as input (cropped and resized). Then I sorted the fragments manually and plotted the graphs:

RGB graphics

White dots mean that the tile is installed correctly (corresponds to the interpolated color). The colored circles around the dots are interpolated colors (for visual comparison you need to enlarge to see them).

As you can see, the graphs of R, G, B 3D look linear, so (bi) linear interpolation should be sufficient.

If I tried only linear interpolation for strings, then the solver immediately solves the puzzle. However, when I encoded the same thing for columns (more unknown cells between the known ones), the solver began to make several incorrect placements (all material invalid, therefore, incorrect white dots).

column solver

I also tried HSL , but after a while I throw it away due to a wall hit, because the Hue can cross degrees 0 and 360 at any point that is no different from cases that did not overlap. To do this, he will need some heuristics or cross-correlation from neighboring allowed areas, and this will be too much coding for my taste. Without it, results are worse than when using RGB .

So, now I’m thinking about using bilinear interpolation or first allowing interpolation to short distances, and then solving the rest ...

[Edit2 Dec 14 2017] bilinear interpolation

It seems that bilinear RGB interpolation solves all the problems. Therefore, if your board is closed with fixed cells, it should work. If not, you need to solve this problem iteratively, and then use the newly allowed cells as a new binding to unresolved areas. I also realized that I have RGB , and I also repaired it :).

Here is the C ++ / VCL source for the game (it is not optimized at all):

 //$$---- Form CPP ---- //--------------------------------------------------------------------------- #include <vcl.h> #include <math.h> #pragma hdrstop #include "Unit1.h" //--------------------------------------------------------------------------- #pragma package(smart_init) #pragma resource "*.dfm" //--------------------------------------------------------------------------- TForm1 *Form1; bool _update=false; //--------------------------------------------------------------------------- const _ILoveHue_state_fixed =255<<24; const _ILoveHue_state_unsolved= 0<<24; const _ILoveHue_state_solved = 1<<24; const _ILoveHue_render_board=0; const _ILoveHue_render_graph=1; //--------------------------------------------------------------------------- int rgbdist(DWORD c0,DWORD c1) // AABBGGRR { int r0,g0,b0,r1,g1,b1; r0=( c0 &255); r1=( c1 &255); g0=((c0>> 8)&255); g1=((c1>> 8)&255); b0=((c0>>16)&255); b1=((c1>>16)&255); r0-=r1; g0-=g1; b0-=b1; return (r0*r0)+(g0*g0)+(b0*b0); } //--------------------------------------------------------------------------- class ILoveHue { public: // variables bool _redraw; // redraw needed? Graphics::TBitmap *bmp; // screen buffer int sxs,sys,mxs,mys,gxs,gys;// screen,map,grid cell resolution DWORD **map,**imap; // map[y][x] actual and interpolated int mx,my,mx0,my0; // mouse position state actual and last TShiftState sh,sh0; // mouse buttons and spec keys state actual and last int render_mode; // class constructors and destructors ILoveHue() { bmp=new Graphics::TBitmap; bmp_resize(1,1); map=NULL; imap=NULL; mxs=0; mys=0; mx=-1; my=-1; mx0=-1; my0=-1; gxs=1; gys=1; render_mode=_ILoveHue_render_board; } ~ILoveHue() { map_free(); if (bmp) delete bmp; } ILoveHue(ILoveHue& a) { *this=a; } ILoveHue* operator = (const ILoveHue *a) { *this=*a; return this; } //ILoveHue* operator = (const ILoveHue &a) { ...copy... return this; } // game/Window API and stuff void map_free() // relese map { if ( map) { if ( map[0]) delete[] map[0]; delete[] map; } map=NULL; mxs=0; mys=0; if (imap) { if (imap[0]) delete[] imap[0]; delete[] imap; } imap=NULL; } void map_resize(int x,int y) // resize/allocate map { _redraw=true; if ((x==mxs)&&(y==mys)) return; map_free(); map=new DWORD*[y]; if ( map==NULL) return; map[0]=new DWORD[x*y]; if ( map[0]==NULL) return; imap=new DWORD*[y]; if (imap==NULL) return; imap[0]=new DWORD[x*y]; if (imap[0]==NULL) return; mxs=x; mys=y; for (x=mxs,y=1;y<mys;y++,x+=mxs) { map[y]=map[0]+x; imap[y]=imap[0]+x; } if (mxs) gxs=sxs/mxs; else gxs=1; if (mys) gys=sys/mys; else gys=1; } void bmp_resize(int x=-1,int y=-1) // resize bmp { _redraw=true; if ((x>=0)&&(y>=0)) bmp->SetSize(x,y); bmp->HandleType=bmDIB; bmp->PixelFormat=pf32bit; sxs=bmp->Width; sys=bmp->Height; if (mxs) gxs=sxs/mxs; else gxs=1; if (mys) gys=sys/mys; else gys=1; } void bmp_load(AnsiString file) // init game from image (map must be resized already) { _redraw=true; // load file bmp->LoadFromFile(file); bmp_resize(); // convert to map int x,y; DWORD *p,c; for (y=0;y<mys;y++) for (p=(DWORD*)bmp->ScanLine[(y*gys)+(gys>>1)],x=0;x<mxs;x++) { c=p[(x*gxs)+(gxs>>1)+4]&0x00FFFFFF; // near mid point (0<<24 is unsolved state) c=((c>>16)&0x000000FF) // RGB -> BGR (file has reverse RGB order than bmp) |((c<<16)&0x00FF0000) |( c &0x0000FF00); map[y][x]=c; c=p[(x*gxs)+(gxs>>1)]&0x00FFFFFF; // mid point if ((((c)|(c>>8)|(c>>16))&255)<64) // ~max(R,G,B)<32 map[y][x]|=_ILoveHue_state_fixed; } } void mouse(int x,int y,TShiftState s) // handle mouse { _redraw=true; mx=x/gxs; my=y/gys; sh0=sh; sh=s; bool q0=sh0.Contains(ssLeft); bool q1=sh .Contains(ssLeft); if ((!q0)&&( q1)){ mx0=mx; my0=my; } // mouse left button down if (( q0)&&(!q1)) // mouse left button up (swap) { // swap if valid coordinates if ((mx0>=0)&&(mx0<mxs)&&(my0>=0)&&(my0<mys)) if (DWORD(map[my0][mx0]&0xFF000000)!=_ILoveHue_state_fixed) if ((mx >=0)&&(mx <mxs)&&(my >=0)&&(my <mys)) if (DWORD(map[my ][mx ]&0xFF000000)!=_ILoveHue_state_fixed) { DWORD c=map[my0][mx0]; map[my0][mx0]=map[my][mx]; map[my][mx]=c; // swap cells map[my0][mx0]&=0x00FFFFFF; map[my0][mx0]|=_ILoveHue_state_unsolved; // set them as unsolved map[my ][mx ]&=0x00FFFFFF; map[my ][mx ]|=_ILoveHue_state_unsolved; map_solve(false); // check for solved state } // clear selection mx0=-1; my0=-1; } } void draw() // render game { _redraw=false; int x,y,z,x0,x1,x2,y0,y1,y2,r; DWORD c; if (render_mode==_ILoveHue_render_board) { for (y0=0,y1=gys,y2=gys>>1,y=0;y<mys;y++,y0+=gys,y1+=gys,y2+=gys) for (x0=0,x1=gxs,x2=gxs>>1,x=0;x<mxs;x++,x0+=gxs,x1+=gxs,x2+=gxs) { c=map[y][x]; bmp->Canvas->Pen->Color=TColor(c&0x00FFFFFF); if ((x==mx )&&(y==my )) bmp->Canvas->Pen->Color=clYellow; if ((x==mx0)&&(y==my0)) bmp->Canvas->Pen->Color=clGreen; bmp->Canvas->Brush->Color=TColor(c&0x00FFFFFF); bmp->Canvas->Rectangle(x0,y0,x1,y1); if (DWORD(c&0xFF000000)!=_ILoveHue_state_fixed) { r=10; bmp->Canvas->Pen->Color=imap[y][x]&0x00FFFFFF; bmp->Canvas->Brush->Style=bsClear; bmp->Canvas->Ellipse(x2-r,y2-r,x2+r,y2+r); bmp->Canvas->Brush->Style=bsSolid; } if (DWORD(c&0xFF000000)!=_ILoveHue_state_unsolved) { if (DWORD(c&0xFF000000)==_ILoveHue_state_fixed ) c=clBlack; if (DWORD(c&0xFF000000)==_ILoveHue_state_solved) c=clWhite; r=4; bmp->Canvas->Pen->Color=c; bmp->Canvas->Brush->Color=c; bmp->Canvas->Ellipse(x2-r,y2-r,x2+r,y2+r); } } } if (render_mode==_ILoveHue_render_graph) { bmp->Canvas->Pen->Color=clBlack; bmp->Canvas->Brush->Color=clBlack; bmp->Canvas->Rectangle(0,0,sxs,sys); r=13; x0=15; y0=sys-15; int c=r*double(256.0*cos(55.0*M_PI/180.0)); int s=r*double(256.0*sin(55.0*M_PI/180.0)); bmp->Canvas->Pen->Color=clRed; for (y=0;y<mys;y++) for (x=0;x<mxs;x++) { z=(map[y][x])&255; x1=x0+(x*r)+((y*c)>>8); y1=y0 -((y*s)>>8); bmp->Canvas->MoveTo(x1,y1); bmp->Canvas->LineTo(x1,y1-z); } x0=x1+5; bmp->Canvas->Pen->Color=clGreen; for (y=0;y<mys;y++) for (x=0;x<mxs;x++) { z=(map[y][x]>>8)&255; x1=x0+(x*r)+((y*c)>>8); y1=y0 -((y*s)>>8); bmp->Canvas->MoveTo(x1,y1); bmp->Canvas->LineTo(x1,y1-z); } x0=x1+5; bmp->Canvas->Pen->Color=clBlue; for (y=0;y<mys;y++) for (x=0;x<mxs;x++) { z=(map[y][x]>>16)&255; x1=x0+(x*r)+((y*c)>>8); y1=y0 -((y*s)>>8); bmp->Canvas->MoveTo(x1,y1); bmp->Canvas->LineTo(x1,y1-z); } } } // Solver void map_solve(bool _solve) // check for solved state and try to solve if _solve is true { _redraw=true; const int _thr=10; // color comparison threshold int x,y,x0,x1,y0,y1,xx,yy; int r0,g0,b0,r,g,b; int r1,g1,b1; int r2,g2,b2; int r3,g3,b3; DWORD c; // compute interpolated colors to imap (wanted solution) for (x=0;x<mxs;x++) for (y=0;y<mys;y++) if (DWORD(map[y][x]&0xFF000000)!=_ILoveHue_state_fixed) { for (x0=-1,xx=x;xx>= 0;xx--) if (DWORD(map[y][xx]&0xFF000000)==_ILoveHue_state_fixed){ x0=xx; break; } for (x1=-1,xx=x;xx<mxs;xx++) if (DWORD(map[y][xx]&0xFF000000)==_ILoveHue_state_fixed){ x1=xx; break; } for (y0=-1,yy=y;yy>= 0;yy--) if (DWORD(map[yy][x]&0xFF000000)==_ILoveHue_state_fixed){ y0=yy; break; } for (y1=-1,yy=y;yy<mys;yy++) if (DWORD(map[yy][x]&0xFF000000)==_ILoveHue_state_fixed){ y1=yy; break; } c=0; if (int(x0|x1|y0|y1)>=0) { // bilinear interpolation c=map[y0][x0]; r0=c&255; g0=(c>>8)&255; b0=(c>>16)&255; c=map[y0][x1]; r1=c&255; g1=(c>>8)&255; b1=(c>>16)&255; c=map[y1][x0]; r2=c&255; g2=(c>>8)&255; b2=(c>>16)&255; c=map[y1][x1]; r3=c&255; g3=(c>>8)&255; b3=(c>>16)&255; r0=r0+(r1-r0)*(x-x0)/(x1-x0); g0=g0+(g1-g0)*(x-x0)/(x1-x0); b0=b0+(b1-b0)*(x-x0)/(x1-x0); r1=r2+(r3-r2)*(x-x0)/(x1-x0); g1=g2+(g3-g2)*(x-x0)/(x1-x0); b1=b2+(b3-b2)*(x-x0)/(x1-x0); r =r0+(r1-r0)*(y-y0)/(y1-y0); g =g0+(g1-g0)*(y-y0)/(y1-y0); b =b0+(b1-b0)*(y-y0)/(y1-y0); c=(r)+(g<<8)+(b<<16); } imap[y][x]=c; } // compute solved state for (x=0;x<mxs;x++) for (y=0;y<mys;y++) if (DWORD(map[y][x]&0xFF000000)!=_ILoveHue_state_fixed) { map[y][x]&=0x00FFFFFF; if (rgbdist(map[y][x],imap[y][x])<_thr) map[y][x]|=_ILoveHue_state_solved; else map[y][x]|=_ILoveHue_state_unsolved; } // solver/checker if (_solve) { // process all unsolved cells for (x=0;x<mxs;x++) for (y=0;y<mys;y++) if (DWORD(map[y][x]&0xFF000000)==_ILoveHue_state_unsolved) // find match in unsolved cells for (xx=0;xx<mxs;xx++) for (yy=0;yy<mys;yy++) if (DWORD(map[yy][xx]&0xFF000000)==_ILoveHue_state_unsolved) if (rgbdist(map[yy][xx],imap[y][x])<_thr) { // swap if found c=map[yy][xx]; map[yy][xx]=map[y][x]; map[y][x]=(c&0x00FFFFFF)|_ILoveHue_state_solved; } } } } gam; //--------------------------------------------------------------------------- __fastcall TForm1::TForm1(TComponent* Owner):TForm(Owner) { gam.map_resize(7,9); gam.bmp_load("map.bmp"); gam.map_solve(false); _update=true; ClientWidth=gam.sxs; ClientHeight=gam.sys; _update=false; } //--------------------------------------------------------------------------- void __fastcall TForm1::FormDestroy(TObject *Sender) { gam.render_mode=_ILoveHue_render_board; gam.draw(); gam.bmp->SaveToFile("map.bmp"); } //--------------------------------------------------------------------------- void __fastcall TForm1::FormPaint(TObject *Sender){ gam.draw(); Canvas->Draw(0,0,gam.bmp); } void __fastcall TForm1::FormResize(TObject *Sender){ if (_update) return; gam.bmp_resize(ClientWidth,ClientHeight); } void __fastcall TForm1::Timer1Timer(TObject *Sender){ if (gam._redraw) FormPaint(Sender); } void __fastcall TForm1::FormMouseMove(TObject *Sender, TShiftState Shift, int X, int Y){ gam.mouse(X,Y,Shift); } void __fastcall TForm1::FormMouseUp(TObject *Sender, TMouseButton Button, TShiftState Shift, int X, int Y){ gam.mouse(X,Y,Shift); } void __fastcall TForm1::FormMouseDown(TObject *Sender, TMouseButton Button, TShiftState Shift, int X, int Y){ gam.mouse(X,Y,Shift); } //--------------------------------------------------------------------------- void __fastcall TForm1::FormKeyDown(TObject *Sender, WORD &Key, TShiftState Shift) { if (Key=='S') gam.map_solve(true); // try to solve if (Key=='M') { gam.render_mode^=1; gam._redraw=true; } // swap render modes if (Key==115) gam.bmp->SaveToFile("screenshot.bmp"); // [F4] screenshot } //--------------------------------------------------------------------------- 

This is one form application in BDS2006 with one 40 ms timer on it. So just add events ... You can ignore VCL visualization and window files. The important thing is that it has a class and a solve() function. It is used both for the correct verification of the placement and for the solution (depending on the _solve bool). This is the input image map.bmp

map.bmp

I did not encode the corresponding save / load functions, instead, I decided to use a bitmap directly (a waste of space, but hardly enhances the code).

The card itself is a 2D 32bit DWORD array with the form SSBBGGRR hex , where SS is the flag of the cell (fixed / resolved / unresolved).

Here's a compiled demo with source code

Read readme.txt for more information. Here is the result after solving (pressing [S]):

resolved state

As you can see (no), the circles vanish because the bilinearly interpolated color matches your input more closely.

The program expects a grid size of 7x9, image resolution does not matter. Color is selected from the midpoint of the cell (black dot) and slightly to the right (tile color)

To do this efficiently, you can do 2 things:

  • add / use list containing undisclosed cells

    instead of itearting over the entire iteration map only through a list of unresolved cells.

  • convert T(N^2) searches for T((N^2)/2) in a triangular search

    It is still O(N^2) , but the constant time is less.

  • use the 3D RGB LUT table

    for large grids, you can create 32K 3D LUT entries to find the matching cell found in O(1) . Just convert RGB to 15-bit color and use

     DWORD LUT[32][32][32]; 

    where LUT[r][g][b]=row+(column<<16); This will let you know where each color is located. All unused colors are set to 0xFFFFFFFF . Here is an example of using this technique for a similar purpose:

    Look for recolor[32][32][32] in the code ... A coarse 15-bit color may not be enough for this purpose, so it may take more bits, for example, 18 bits, which will result in 256 KB writing, which is still manageable.

    Creating this LUT will take O(N) , but use and support is just O(1) time.

+4
source

I have no idea if this will work or not. I just wrote this for fun and could not apply the real test on it. It would be nice if you kindly ask him and comment on this.

  struct pixel { public int R; public int G; public int B; public bool Fixed; public pixel(int r, int g, int b, bool _fixed) { this.R = r; this.G = g; this.B = b; this.Fixed = _fixed; } public int DistanceSQ(pixel px) { int r = this.R - px.R; int g = this.G - px.G; int b = this.B - px.B; return r * r + g * g + b * b; } public override string ToString() { return string.Format("{0} {1} {2} {3}", this.R, this.G, this.B, this.Fixed); } public override int GetHashCode() { return this.R.GetHashCode() ^ this.G.GetHashCode() ^ this.B.GetHashCode(); } public override bool Equals(object obj) { pixel px = (pixel)obj; return this.R == px.R && this.G == px.G && this.B == px.B; } } static void sort(pixel[,] img) { List<pixel> lst = new List<pixel>(); foreach (pixel px in img) if (!px.Fixed) lst.Add(px); int rows = img.GetLength(0); int cols = img.GetLength(1); while (lst.Count > 0) for (int row = 0; row < rows; row++) for (int col = 0; col < cols; col++) if (!img[row, col].Fixed) { pixel[] neighbors = getFixedNeighbors(img, row, col, rows, cols).ToArray(); int min = int.MaxValue; pixel nearest = new pixel(); foreach (pixel n in lst) { int dist = neighbors.Select((a) => a.DistanceSQ(n)).Sum(); if (dist < min) { min = dist; nearest = n; } } nearest.Fixed = true; img[row, col] = nearest; lst.Remove(nearest); if (lst.Count == 0) return; } } private static IEnumerable<pixel> getFixedNeighbors(pixel[,] img, int row, int col, int rows, int cols) { for (int r = Math.Max(0, row - 1); r < Math.Min(row + 2, rows); r++) for (int c = Math.Max(0, col - 1); c < Math.Min(col + 2, cols); c++) if (img[r, c].Fixed) yield return img[r, c]; } //test { bool b0 = false; bool b1 = true;//for easy editing { pixel[,] img = new pixel[3, 4]; img[0, 0] = new pixel(0, 0, 0, b1); img[1, 0] = new pixel(0, 1, 0, b0); img[2, 0] = new pixel(0, 2, 0, b1); img[0, 1] = new pixel(1, 0, 0, b0); img[1, 1] = new pixel(1, 1, 0, b0); img[2, 1] = new pixel(1, 2, 0, b0); img[0, 2] = new pixel(2, 0, 0, b0); img[1, 2] = new pixel(2, 1, 0, b0); img[2, 2] = new pixel(2, 2, 0, b0); img[0, 3] = new pixel(3, 0, 0, b1); img[1, 3] = new pixel(3, 1, 0, b0); img[2, 3] = new pixel(3, 2, 0, b1); sort(img); } { pixel[,] img = new pixel[3, 4]; img[0, 0] = new pixel(0, 0, 0, b1); img[1, 0] = new pixel(0, 1, 0, b0); img[2, 0] = new pixel(0, 2, 0, b1); img[0, 1] = new pixel(2, 0, 0, b0); img[1, 2] = new pixel(2, 1, 0, b0); img[2, 2] = new pixel(2, 2, 0, b0); img[0, 2] = new pixel(1, 0, 0, b0); img[1, 1] = new pixel(1, 1, 0, b0); img[2, 1] = new pixel(1, 2, 0, b0); img[0, 3] = new pixel(3, 0, 0, b1); img[1, 3] = new pixel(3, 1, 0, b0); img[2, 3] = new pixel(3, 2, 0, b1); sort(img); } } 

The code is simple. It saves non-price items in the list and deletes them each when a location is found for it. To determine which color should be selected for the location, a color that has a minimum sum of square distances is selected. Sqrt is not required, since we only need this for comparison.

"sorting" is the main function that changes the location of unfixed pixels. The input for this function is an array of column columns. The sort function makes changes to this array.

0
source

Source: https://habr.com/ru/post/1273983/


All Articles