An image histogram in 30 lines of code

The source image ("Lena").


Its pixel-distribution histogram.

According to Wikipedia, "An image histogram is a type of histogram which acts as a graphical representation of the tonal distribution in a digital image. It plots the number of pixels for each tonal value. By looking at the histogram for a specific image a viewer will be able to judge the entire tonal distribution at a glance."

It occurred to me that it shouldn't be that hard to get Google Charts to produce an image histogram, with just a few lines of code. And that turns out to be true. Around 30 lines of server-side JavaScript will do the trick.

If you have JDK 6, run the command "jrunscript" in the console (or find jrunscript.exe in your JDK's /bin folder and run it). Then you can cut and paste the following lines into the console and execute them in real time. (Alternatively, download js.jar from the Mozilla Rhino project, and run "java -cp js.jar org.mozilla.javascript.tools.shell.Main" in the console.)

The first order of business is to open and display an image in a JFrame. The following 9 lines of JavaScript will accomplish this:

imageURL = "http://wcours.gel.ulaval.ca/2009/a/GIF4101/default/8fichiers/lena.png";
IO = Packages.javax.imageio.ImageIO;
image = IO.read( new java.net.URL(imageURL) );
frame = new Packages.javax.swing.JFrame();
frame.setBounds(50,80,image.getWidth( )+10,
    image.getHeight( )+10);
frame.setVisible(true);
pane = frame.getContentPane();
graphics = pane.getGraphics();
graphics.drawImage( image,0,0,null );

The next order of business is to set up a histogram table, loop over all pixel values in the image, tally the pixel counts, and form the data into a URL that Google Charts can use:

function getMaxValue( array ) {

for (var i = 0,max = 0; i < array.length; i++ )
    max = array[ i ] > max ? array[ i ] : max;

return max;
}

// get pixels
width = image.getWidth();
height = image.getHeight();
pixels = image.getRGB( 0,0, width, height, null, 0, width );

// initialize the histogram table
table = (new Array(257)).join('0').split('');

// populate the table
for ( var i = 0; i < pixels.length; i++ )
    table[ ( pixels[ i ] >> 8 ) & 255 ]++;

maxValue = getMaxValue( table );

data = new Array();

for ( var i = 0; i < table.length; i++ )
    data.push( Math.floor( 100 * table[ i ] / maxValue ) );

data = data.join(",");

url = "http://chart.apis.google.com/chart?chxt=y&chbh=a,0,0&chs=512x490&cht=bvg&chco=029040&chtt=histogram&chd=t:"

// call Google Charts
image = IO.read( new java.net.URL( url + data ) );

// draw the resulting image
graphics.drawImage( image,0,0,null );

Note that we actually tally only the green pixel values. (But these are the most representative of tonal values in an RGB image, generally.) Table values are normalized against maxValue, then multiplied by 100 to result in a number in the range 0..100. Google obligingly plots the data exactly as shown in the above graphic.

And that's about all there is to say, except: Why can't all graphics operations be this easy? :)

reade more... Résuméabuiyad

Compiled languages are too complex

In a talk Thursday at the O'Reilly Open Source Conference, Google distinguished engineer Rob Pike blasted C++ and Java for being overly verbose and too complex.

"I think these languages are too hard to use, too subtle, too intricate," Pike averred. "They're far too verbose and their subtlety, intricacy and verbosity seem to be increasing over time. They're oversold, and used far too broadly."

I tend to agree. Where else but in a language like C would you ever come up with something like:

(*((*(srcPixMap))->pmTable))->ctSeed =
(*((*((*aGDevice)->gdPMap))->pmTable))->ctSeed;

This monstrous line of code is one I used very often in my days of graphics programming on the Mac (circa 1996). On the Mac, the all-important CopyBits() routine always examines the ctSeed field of the source and destination color tables to see if they differ. If the two seed values are not the same, QuickDraw will waste time translating color table info, which you don't want (if you're interested in performance). Hence, you use this line of code to coerce the ctSeed field of the source and destination color tables to the same value. I wrote about this and other tricks for speeding up graphics on the Mac in a 1999 MacTech article.

Of course, the answer to Pike's Complaint is to use dynamic languages like JavaScript or Ruby instead of C++ or Java. But that's not always possible (as when trying to do high-performance graphics programming).

Still, it's surprising how much you can do in JavaScript these days. At the USENIX annual conference last month, Google engineer Adam de Boor raised an eyebrow or two in the audience when he pointed out that Google's Gmail service (443,000 lines of code) is written entirely in JavaScript.

Pike and others at Google are promoting the Go language as a solution to the compiled-language complexity problem.

Go figure.

reade more... Résuméabuiyad

Democrats have many good ideas. Republicans have no ideas whatsoever.

This is a partisan post. Which is not to say I'm writing this because I like Democrats better in general. I just like Democrats better right now, because they have ideas to fight our country's problems. Republicans do not.

Note that this was not always the case. In the Reagan era, Republicans had a lot of ideas, and Democrats largely promised more of the same. Voters seemed to realize this when they made their choice. but Republicans dropped the mantle of the "party of ideas" about a decade ago, and show no signs of picking it back up.

What are the main problems facing our country today? Well, there's the ongoing depression. Republicans' best idea has been to cut unemployment benefits, which reduces structural unemployment slightly but raises cyclical unemployment a lot. We are not in a recession because people have suddenly decided that they don't want to work; people are looking for jobs, and simply not finding ones that match their skills.

Then there's that long-term deficit. A simple breakdown shows that most of that deficit comes as a result of the Bush tax cuts; reversing these cuts would help stabilize our public finances (though in the long run, big cuts in Medicare are needed as well). But Republicans absolutely refuse to rescind those unsustainable tax cuts. Instead, if they are in power, they will probably just threaten to shut down the government unless Obama comes up with ideas for big spending cuts; then, if Obama capitulates (and he is kind of a capitulating sort of dude), the GOP will slam him for the very same spending cuts they forced him to make.

And then there's health care. Republicans have essentially no ideas for how to cut healthcare costs, unless they decided to support Medicare cuts, which they are afraid to do. When pressed, Republicans suggest tort reform and "allowing people to buy health insurance across state lines"; the former is a good idea but a drop in the bucket, and the latter is basically a meaningless red herring.

Financial reform? Republicans recognize the moral hazard problem - the idea that implicit government promises of bailouts encourage banks to take crazy risks - but they have no solution to this other than to have the government say, very sternly, "We won't bail you out the next time!" Which is a laugh, because yes you will, and you know it.

And finally, there's energy. Republicans have nothing on this. "Drill here drill now"? Not a solution, since that oil can and will be sold anywhere. What else? Deafening silence.

Compare this to the Democrats.

On the depression: Democrats are split on additional stimulus spending, but they definitely favor rebuilding our infrastructure, which is important in the short-term (because it adds to demand) and the long-term (because it improves our competitiveness). They favor having the Fed do more quantitative easing (printing money and buying stuff), although legislators cannot tell the Fed what to do. These ideas are good, although the other thing I think we need to do - pressure China to immediately revalue its currency - is unlikely to happen.

On the deficit: Democrats favor rescinding the Bush tax cuts. This is the right thing to do, since those tax cuts were unsustainable. In the short and medium terms that will make a huge difference, but in the long term we'll need to contain health care costs and cut health care spending a lot. This, sadly, is something Democrats are not yet talking about much.

On health care, Democrats finally switched us to a universal coverage nation. That was an important first step - it removed the division of the country into health-care "haves" and "have-nots," which should make future cost-cutting measures more politically possible. It implemented various small cost-control measures, any of which could be ramped up hugely in a few years if it is found to work. This is almost certainly the best health care fix that we could expect in the short term; it remains to be seen if the Dems will come through with serious cost-control ideas in the future.

On financial reform, the Democrats have had a bunch of good ideas, many of which - a resolution authority to reduce the moral hazard problem, a consumer protection agency to reduce lending scams, rules on derivative trading to cut unnecessary complexity from the system - are in the process of being put into law, thanks to Dems and no thanks to the GOP. And on energy, Democrats are correctly investing in alternative energy technology.

Now, I am not saying that I agree with all of the Democrats' ideas. A cap-and-trade system, for example, seems pretty pointless to me (especially since U.S. emissions are falling anyway). And on some issues (China trade, health cost control) they need to go much farther. Nor is having a lot of ideas automatically good; Chairman Mao had a whole little red book full of bad ideas.

But the point is that the Democrats' ideas are mostly good ones. They are a clear improvement over the status quo - ballooning deficits, a collapsing health care system, a bloated unproductive financial sector, vulnerability to peak oil. The Republicans offer no such improvement. They have no ideas whatsoever.

For this reason, stumping for the Democrats in the fall elections is not partisan, and it is not ideological. It is simply patriotic. A vote for a Republican is, at this juncture, a vote for sclerosis and decline.

reade more... Résuméabuiyad

Learning about ESP pages in Sling

Lately I've been doing a fair amount of server-side scripting using ESP (ECMAScript Pages) in Sling. At first blush, such pages tend to look a lot like Java Server Pages, since they usually contain a lot of scriptlet markup, like:

<%
// script code here
%>

and

<%=
// stuff to be evaluated here
%>

So it's tempting to think ESP pages are simply some different flavor of JSP. But they're not. From what I can tell, ESP pages are just server pages that get handed to an EspReader before being served out. The EspReader, in turn, handles the interpretation of scriptlet tags and expression tags (but doesn't compile anything into a servlet). Bottom line, ESP is not JSP, and despite the availability of scriptlets tags, things work quite a bit differently in each case.

Suppose you want to detect, from an ESP page or a JSP page, what kind of browser a given page request came from. In a Sling JSP page you could do:

<%@taglib prefix="sling" uri="http://sling.apache.org/taglibs/sling/1.0" %><%

%><sling:defineObjects/>

<html><body>

<%

java.util.Enumeration c = request.getHeaders("User-Agent");

String s = "";

while ( c.hasMoreElements() )

s += c.nextElement();

%>


<%= s %>

</body></html>

But what do you do in ESP? Remember, <sling:defineObjects/> is not available in ESP.

It turns out that Sling automatically (without the need for any directives) exposes certain globals to the JavaScript Context at runtime, and one of them is a request object. Thus, in ESP you'd simply do:

<%

c=request.getHeaders("User-Agent");

s = "";

while ( c.hasMoreElements() )

s += c.nextElement();

%>

<%= s %>

Very similar to the JSP version.

So the next question I had was, what are the other globals that are exported into the JavaScript runtime scope by Sling? From what I can determine, the Sling globals available in ESP are:

currentNode
currentSession
log
out
reader
request
resource
response
sling


currentNode is the JCR node underlying the current resource; currentSession is what it sounds like, a reference to the current Session object; log refers to the org.slf4j.Logger; reader, I'm not sure about (is it a reference to the EspReader?); request is a reference to the SlingHttpServletRequest; resource is the current Resource; response is, of course, a reference to the SlingHttpServletResponse; and sling is a SlingScriptHelper. All of these are available all the time, throughout the life of any ESP script in Sling.

For more information, try the Sling Javadocs here or Day's page of resources here (note, in particular, the list of References on the right).

reade more... Résuméabuiyad

Crashing the Crash Report Tool

Lately I've been crashing the Mobile Device Crash Report Tool a fair amount. But so, too, have a lot of other people, judging from the buzz on the Googlesphere. Evidently iTunes is fussy (on Vista, especially) when it comes to indiscriminate plugging/unplugging of iPods from USB ports while a sync is in progress (yeah, I know).

For crying out loud can we not have a meaningful error message, though? Repeat after me: Whisky, Tango, Foxtrot.

reade more... Résuméabuiyad

How to hack an OpenOffice spreadsheet with JavaScript

One of the most difficult aspects of learning to write OpenOffice macros (in JavaScript) is that there is so little example code available. OpenOffice comes with dozens of useful code examples written in OpenOffice BASIC, but for Beanshell, Python, and JavaScript (the other supported macro languages) there's hardly anything. Beyond that, there's only the inscrutable 1200-page Developer's Guide (no longer available in PDF), which is inscrutable not so much because it is not well written (it is, in fact, quite well written) but because the Byzantine OpenOffice C++ and Java APIs are themselves so impenetrable. If you've ever tried to write an OO.o macro "in anger," you know what I'm talking about. The whole experience is exhausting.

So without further whining, I thought I'd present the most quintessential of OO.o code examples, showing how to parse an OO.o spreadsheet with JavaScript. I don't claim that the code is particularly stellar; only that it works. (That's the main thing, right?)

The following code will parse through a spreadsheet and display data from the desired numbers of rows and columns in a Swing JEditorPane.

// When a spreadsheet is open in OO.o, this macro will
// loop over a given number of rows and columns and
// summarize those cells in a JEditorPane
//
// Copyleft 2010 by Kas Thomas
// http://asserttrue.blogspot.com/

importClass(Packages.com.sun.star.uno.UnoRuntime);
importClass(Packages.com.sun.star.sheet.XSpreadsheetDocument);

// go thru the sheet one row at a time
// and collect cell data into an array of
// records, where each record is an array
// of cell data for a given row
function harvestCells( sheet, rows, columns ) {
   var masterArray = [];
   for (var i = 0; i < rows; i++) {
           var ar = [];
           for (var k = 0; k < columns; k++) {
                   var cell = sheet.getObject().getCellByPosition( k,i );
                   var content = cell.getFormula();
                   if ( content.indexOf(",") != -1)
                   ar.push( "\"" + content + "\"" );
                   else
                   ar.push( content );
           }
           masterArray.push( ar );
   }

   return masterArray;
} // harvestCells()

// a Swing UI for displaying the data
function EditorPane( ) {

   Swing = Packages.javax.swing;
   this.pane = new Swing.JEditorPane("text/html","" );
   this.jframe = new Swing.JFrame( );
   this.jframe.setBounds( 100,100,500,400 );
   var editorScrollPane = new Swing.JScrollPane(this.pane);
   editorScrollPane.setVerticalScrollBarPolicy(
   Swing.JScrollPane.VERTICAL_SCROLLBAR_ALWAYS);
   editorScrollPane.setPreferredSize(new java.awt.Dimension(150, 150));
   editorScrollPane.setMinimumSize(new java.awt.Dimension(10, 10));
   this.jframe.setVisible( true );
   this.jframe.getContentPane().add( editorScrollPane );

   // public methods
   this.getPane = function( ) { return this.pane; }
   this.getJFrame = function( ) { return this.jframe; }
}

( function main( ) {

   //get the document object from the scripting context
   oDoc = XSCRIPTCONTEXT.getDocument();

   //get the XSpreadsheetDocument interface from the document
   xSDoc = UnoRuntime.queryInterface(XSpreadsheetDocument, oDoc);

   // get a reference to the sheets for this doc
   var sheets = xSDoc.getSheets();

   // get Sheet1
   var sheet1 = sheets.getByName("Sheet1");

   // construct a new EditorPane
   var editor = new EditorPane( );
   var pane = editor.getPane( );

   // harvest cell data (from sheet, rows, cols)
   var masterArray =  harvestCells( sheet1, 100,8 );

   // display the data
   var text =  masterArray.join("\n") ;
   pane.setText( text  );
})();

The main() method comes last, so it may help to read the code from the bottom up. First, we get a reference to the current document with

oDoc = XSCRIPTCONTEXT.getDocument();

(a standard OO.o idiom), then we use that to get a reference to the XSpreadsheetDocument. (I should caution that there is no error-checking code here. It's assumed that a spreadsheet document is already open and frontmost in OpenOffice.) Once you've got the XSpreadsheetDocument, you can peel through its Sheets. With a Sheet, you can do:

var cell = sheet.getObject().getCellByPosition( k,i );

to get cell data for the cell at column k, row i. If the spreadsheet is a converted .csv doc, the cells will all have text data, and (here's the weird part) to get the data you have to do:

var content = cell.getFormula();

since getValue() will only work if the cell contains a double-precision floating point number.

To test this macro, I looked online for a sizable .csv file that I could play with. I found a free/open movie database at George Tech's website. It's a comma-delimited file with data for 1742 movies ranging from 1925 to the early 1990s. It's by no means a definitive movie listing (apparently it represents one of the school's collections) but it suits my purposes just fine.

When I run the macro, it takes less than a second to display all of the spreadsheet's cell data (for all 1742 x 8 cells) in a JEditorPane window:



Ideas for further exploration:

• Style the data in some interesting way
• Save the data as HTML
• Push the data into a Sling repository (blog coming)

reade more... Résuméabuiyad

Please, no more Whisky Tango Foxtrot error messages

I've been seeing this monolog box once in a while in Acrobat these days (not only Acrobat 10 Professional, which is still in development, but Acrobat 9 Pro as well). I can't really call it a dialog box, since it is, in fact, more of a monolog. This is what I sometimes call a Whisky Tango Foxtrot message.

No doubt the underlying error condition happened due to something I did in a script. That's fine, I accept responsibility for the crash. What I don't accept responsibility for is the lameness of the error message. It's a completely content-free error message. It tells me nothing about what went wrong. It's a billboard that says, in effect, "Acrobat sometimes barfs and doesn't know why."

The point of error messages should be to tell you what went wrong, so that you can fix it or work around it, or at least know that it involves something more specific than a nameless internal error. I should at least be given a code of some kind, although even that can be awfully lame, as iTunes users know all too well. Have you ever gotten the infamous iTunes -9812 error? (It's another example of a Whisky Tango Fox monolog box.) Plenty of people see this error, and they turn to Google in desperation to find out what it means, since Apple's web site is of scant help. My September 26, 2009 blog on this subject ("A Fix for the Dreaded iTunes -9812 Error") continues to bring upwards of 2000 hits a month. Why? Because Apple displays no meaningful troubleshooting information whatsoever in conjunction with the -9812 error message. It's just a billboard for iTunes' ability to barf. People have to come to my blog to find out what it means. How lame is that?

Please, UI designers and programmers, provide meaningful error messages, or at least point me to a log file somewhere, where I can find out what went wrong. If the program ran out of memory, followed a null pointer, had a stack-heap collision, entered an endless loop, or choked or bad data (or whatever), just tell me straight out. I can live with the news, I promise.

reade more... Résuméabuiyad