Android/iPhone Get pixel level data

I'm interested in getting the rgb values of a given pixel of a photo when it's tapped by the user. Is this even possible?

6 Answers

I have seen many people asking for this: How to get RGB values from a photo.
I just developed a method that seems to just do that.
Unfortunately it does not work well consistently; I do not know why.
By posting the method I hope that someone else who is more advanced with Titanium than I am can get rid of the errors that are apparently still in it.
If you are looking for a clear and working piece of code that answers the RGB-question, then please stop reading because I cannot offer it.

In words, my method is as follows.
When the user had made the photo and touches it at pixel position x,y then I first make a cropped image of the photo-image, and show that in an imageview. The cropped image is only 1x1 pixel to make subsequent processing fast.
Another view shows the cropped image, but stretched to normal pixel-size, so the user sees a patch with uniform color, being equal to the color of the pixel that he touched in the original photo.

When the user agrees that the right color was chosen, he clicks a button. A button-eventlistener then writes a (png) file with the information of the 1x1 cropped image. Next, an App.fireEvent is sent from the Titanium code to a webview. The webview is not visible to the user.

The webview is based on an html file, that loads an image from a specified file location to a canvas - of course, the location refers to the file representing the 1x1 cropped image. With the image on the canvas, the rgb values of the image are easily determined in the html page, and saved as a string. Using another App.fireEvent this string is sent out.

Back in the Titanium code, the string is received and the rgb parameters are read from it.
That's the method, basically. I really hope that one of you smart people on this forum can make it flawless!

If you want a

— answered January 25th 2013 by Eric Kirchner
permalink

0 Comments

0 Votes

Let me start with showing the html code, that does the actual work of determining the rgb values:

<html>
<head>
  <meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
  <title>Pick RGB From Image</title>
</head>
<body>
<table border="0" width="100%"><tr><td align="left" valign="top" >
<h1 style="margin:0;padding:0;font-size:22px">Pick RGB From Image</h1>
</td><td align="right" valign="top" width="150px">
</td></tr></table>

<table width="100%" border="0"><tr>
<td valign="top" width="200px">
<canvas id="myCanvas" width="277" height="272" style="border:1px solid #c3c3c3;cursor: crosshair;">
Your browser does not support the HTML5 Canvas element.
Please update your browser.
</canvas>
</td>
<td valign="top">
<br>
</td>
</td></tr>
</table>

<div id="Mo">Pick.</div>
<div id="dm">yes</div>

<script type="text/javascript">
var c=document.getElementById("myCanvas");
var cxt=c.getContext("2d");
var canvasCopy = document.createElement("canvas");
var copyContext = canvasCopy.getContext("2d");
var str5;
var i = 0;

var img = new Image();
img.onload = function() {
    var maxWidth = 550;
    var maxHeight = 380;
    var ratio = 1;
    if(img.width > maxWidth) ratio = maxWidth / img.width;
    if(img.height > maxHeight) ratio = maxHeight / img.height;
    canvasCopy.width = img.width;
    canvasCopy.height = img.height;
    copyContext.drawImage(img, 0, 0);
    c.width = img.width * ratio;
    c.height = img.height * ratio;
    //cxt.drawImage(canvasCopy, 0, 0, canvasCopy.width, canvasCopy.height, 0, 0, c.width, c.height);
    cxt.drawImage(canvasCopy, 0, 0, canvasCopy.width, canvasCopy.height, 0, 0, c.width, c.height);

    var cData = cxt.getImageData(0, 0, 1, 1); //this takes a 1x1 rectangle of pixels starting at 0,0
    var pixel = cData.data;
    var mr = pixel[0]; //these are the r g and b values we are looking for
    var mg = pixel[1];
    var mb = pixel[2];
    str5 = 'rg b; ' + mr + ';' + mg + ';' + mb;
};

img.src = 'file:///data/data/com.appname//app_appdata/smallpic.png';
//One issue remaining unsolved is that of course it would be better to pass the file location, as 
//determined in the Titanium code, to the webview and use that here. But I could not get it working.
//The location hardcoded here was determined by using an alert in the Ti code showing the filenm.nativePath

Ti.App.addEventListener("app:fromTitanium", function(e) {
          Ti.App.fireEvent('app:fromWebView', { text: str5});
});

</script>
</body>
</html>

The html code shown above is stored in a file web_page.html.

Now I turn to the Titanium code.
I am sorry it is so lengthy, I don't know how to make it shorter while still working.

var win = Ti.UI.currentWindow;
win.backgroundColor = 'gray';

var width3; //these simply define dimensions on the mobile device. L=landscape, P=portrait
var width3L;
var width3P;
var hor2;
var hor2P;
var hor2L;
var vert8;
var vert8P;
var vert8L;
var height1;
var height1P;
var height1L;
var stringImage = [];
screenwidth = Titanium.Platform.displayCaps.platformWidth;
screenheight = Titanium.Platform.displayCaps.platformHeight;

width3P = screenwidth-50;
hor2P = 5;
vert8P = 70 + 23 + Math.round(2.1*(screenheight/9));
height1P = Math.round(0.4*screenheight) -40;
width3 = width3P;
hor2 = hor2P;
vert8 = vert8P;
height1 = height1P;

function toHex(n) {  //Proudly copied from somewhere on the internet - sorry I don't remember from where
    n = Math.max(0,Math.min(n,255));
     return "0123456789ABCDEF".charAt((n-n%16)/16)
      + "0123456789ABCDEF".charAt(n%16);
} 

var fontscalesize = 16;
if (screenwidth > 450) fontscalesize = 24;    

var win1 = Ti.UI.createWindow({
    left: 20, top: 30,
    width: screenwidth-40, height: screenheight-120,
    color: 'gray',
});

var photo = null;

var buttonUseCamera = Ti.UI.createButton({ //with this button, the user enters the camera API
    title: 'Take picture',
    top: 5,
    width: '100%',
    height: 64,
    center: 0,
    font: {fontSize: 13},
})

if (screenwidth > 450) buttonUseCamera.height = 96;    

var photoPicture = Ti.UI.createImageView({ //imageView that will show the image taken by the user
        top: 80,
        width: width3,
        height: screenheight - 400,
        left: hor2,
        backgroundColor: 'black',
        borderColor: 'gray',
        borderWidth: 1,
        borderRadius: 10
});

var crosshairImage = Ti.UI.createImageView({
    image: '../images/crosshair_30.png', //just use any 30x30 image of a crosshair
    width: 'auto',
    height: 'auto',
    zIndex: 10,
    left: 100,
    top: 100,
});

//cropping technique copied from http://developer.appcelerator.com/question/72431/crop-imageview 

var baseImage = Ti.UI.createImageView({ //baseImage contains the full photo
    image: photoPicture.image,
    width: width3,
    height: screenheight - 400,
});

var cropSmallView = Ti.UI.createView({
    width: 1,
    height: 1, //this defines the 1x1 image that we will transfer to the webview
    backgroundColor: '#00fd00',
    backgroundColor: 'gray',
});

baseImage.left = -100; //some default values that will be changed when the user touches the screen
baseImage.top = -100;

var croppedImage = cropSmallView.toImage().media; //I found that the .media is necessary for android

var smallView = Ti.UI.createImageView({  
//this will be the stretched 200x90 view of the cropped 1x1 image of the complete photo!
        image: croppedImage,
        top: 80 + screenheight - 400 + 25,
        left: 10,
        width: 200,
        height: 90,
        backgroundColor: 'black',
        borderColor: 'gray',
        borderWidth: 1,
        borderRadius: 10,
});

And the same Titanium file continues with:

var buttonUseColor = Ti.UI.createButton({ //this button is clicked when the user thinks he touched the right pixel
    title: 'Use this color',
    top: 80 + screenheight - 400 + 25,
    width: 200,
    height: 90,
    right: 10,
    font: {fontSize: 16},
})

var view1 = Ti.UI.createView({    
    width: width3,
    left: hor2,
    top: 80 + screenheight - 400 + 15,
    height: 120,
    backgroundColor: '#0ff',
    borderRadius: 10,
    zIndex: 1
});

photoPicture.addEventListener('click', function(e) {
//Actually, I use exactly the same EventListener for the touchstart, touchmove and touchend events
    hor = e.x;
    vert = e.y;
    baseImage.left = -1*hor; //Thus the photo is cropped from e.x, e.y position onwards
    baseImage.top = -1*vert;
    croppedImage = cropSmallView.toImage().media;
    smallView.image = croppedImage;
    if ((vert > 15) && (vert < 415)) {     
//vert>15 because it must be larger than win1.top-0.5*(crosshairImage.height)
//vert<415 because it must be smaller than win1.top – 0.5*(crosshairImage.height) + photoPicture.height
        crosshairImage.left = hor - 15; 
//this is a shift over 0.5*(crosshairImage.height)
        crosshairImage.top = vert + 50;
    };
    buttonUseCamera.title = 'Vert=' + vert;
});

win1.add(photoPicture);
win1.add(crosshairImage);
win1.add(smallView);
win1.add(buttonUseColor);

buttonUseCamera.addEventListener('click', function() {
    Titanium.Media.showCamera({

        success:function(event)    {
            photo = event.media;
            photoPicture.image = event.media; //use the photo and put it in the window
            baseImage.image = event.media;
            cropSmallView.remove(baseImage);
            cropSmallView.add(baseImage);
            croppedImage = cropSmallView.toImage().media;
            smallView.image = croppedImage;
            win1.remove(smallView); 
                           //smallView is a stretched view of the 1x1 cropped image.
            win1.add(smallView);  //make sure the smallView is refreshed here
        },

        cancel:function() {
        },

        error:function(error) {
// make a warning window
            var a = Titanium.UI.createAlertDialog({ title: 'Camera' });
// set message for dialog
            if (error.code == Ti.Media.NO_CAMERA)    {
                a.setMessage('Camera not available');
            } else {
                a.setMessage('Error : ' + error.code);
            }
// show alert
            a.show();
        },
//        overlay: crosshair,
        allowEditing: true,
        mediaTypes: Ti.Media.MEDIA_TYPE_PHOTO,
        showControls: true
    });

});



var webview = Ti.UI.createWebView({
    url: 'web_page.html',
    visible: false, //the webview will not be visible. For bug fixing I sometimes make it visible
    height: 302,  
    top: 0,       });

var j = 0;
win1.add(webview);

var fromWebViewHandler = function(e) {
    var str5 = e.text;  //e.text contains the string that the webview produced, with rgb's.
    var rgb = e.text.split(';'); //the string has the rgb values separated by ;
    alert(' R.=' + rgb[1] + ' G=' + rgb[2] + " B=" + rgb[3]);
    Ti.App.removeEventListener('app:fromWebView', fromWebViewHandler);
    var filenm = Ti.Filesystem.getFile(Ti.Filesystem.applicationDataDirectory, 'smallpic.png');
    if (filenm.exists()) { filenm.deleteFile(); } //delete waste!
};

buttonUseColor.addEventListener('click', function() {
    the buttonUse Color is clicked by the user when he wants the touched pixel to be RGB-ed.    
    var filenm = Ti.Filesystem.getFile(Ti.Filesystem.applicationDataDirectory, 'smallpic.png');
    if (filenm.exists()) {
        filenm.deleteFile();
    };
    var img2 = smallView.image;
    filenm.write(img2); //if you put an alert here showing filenm.nativePath, you know the
        filelocation that you need to specify in the html code //
    webview.reload(); //make sure the latest version of the image file is loaded in the webview
    var str5 = ' 3'; //just a fake input
    j = j + 1;
    buttonUseCamera.title = ' j =' + j;
    Ti.App.addEventListener('app:fromWebView', fromWebViewHandler);
    Ti.App.fireEvent('app:fromTitanium', { text: str5});
});

win.add(buttonUseCamera);

win1.open();

As mentioned above, this method often works: after clicking on the "Use this color" button, you get an alert with the RGB-values of the pixel you touched on the photo.

What still troubles me (greatly), is that if you found RGB values in this way, and then proceed on the photo to touch another pixel and again click the "Use this color" button, then you do not get an alert with RGB values. cropped That surprises me, because the updated cropped image is correctly uploaded to the webpage. I think (but I am not sure) the trouble is that apparently the FireEvents working only once on my Android 2.2.1.

Another issue is that sometimes the RGB values are not presented even not when the "use this color" button is clicked for the first time. Can that be due to a file being still present on the application folder? I don't understand it.
I hope someone can find the error(s), because it would really have great added vaklue if we can flawlessly read RGB values inside Titanium.

image
— answered January 25th 2013 by Eric Kirchner
permalink

1 Comment
- Hey eric, I'm trying to figure out how to do this same thing; but I can't seem to get yours to work at all. Can you possibly put this in a github account and then I can pull from it? Or can you email it to me perhaps? I'd like to a) figure out why your fireevent is only working once, and b) figure out a modification of this for another app not related to eyedropper stuff.
  
  -Ryley (ryleyherrington at gmail)
  
  — commented April 25th 2013 by Ryley Herrington
0 Votes
Hello I'm interested to have the color of the pixel where we click on the screen.. Have you find a solution ?? Thank you

— answered June 18th 2013 by François Coppey
permalink

0 Comments

0 Votes

Titanium Community Questions & Answer Archive

We felt that 6+ years of knowledge should not die so this is the Titanium Community Questions & Answer Archive

Android/iPhone Get pixel level data

6 Answers