An image histogram in 30 lines of code

The source image ("Lena").


Its pixel-distribution histogram.

According to Wikipedia, "An image histogram is a type of histogram which acts as a graphical representation of the tonal distribution in a digital image. It plots the number of pixels for each tonal value. By looking at the histogram for a specific image a viewer will be able to judge the entire tonal distribution at a glance."

It occurred to me that it shouldn't be that hard to get Google Charts to produce an image histogram, with just a few lines of code. And that turns out to be true. Around 30 lines of server-side JavaScript will do the trick.

If you have JDK 6, run the command "jrunscript" in the console (or find jrunscript.exe in your JDK's /bin folder and run it). Then you can cut and paste the following lines into the console and execute them in real time. (Alternatively, download js.jar from the Mozilla Rhino project, and run "java -cp js.jar org.mozilla.javascript.tools.shell.Main" in the console.)

The first order of business is to open and display an image in a JFrame. The following 9 lines of JavaScript will accomplish this:

imageURL = "http://wcours.gel.ulaval.ca/2009/a/GIF4101/default/8fichiers/lena.png";
IO = Packages.javax.imageio.ImageIO;
image = IO.read( new java.net.URL(imageURL) );
frame = new Packages.javax.swing.JFrame();
frame.setBounds(50,80,image.getWidth( )+10,
    image.getHeight( )+10);
frame.setVisible(true);
pane = frame.getContentPane();
graphics = pane.getGraphics();
graphics.drawImage( image,0,0,null );

The next order of business is to set up a histogram table, loop over all pixel values in the image, tally the pixel counts, and form the data into a URL that Google Charts can use:

function getMaxValue( array ) {

for (var i = 0,max = 0; i < array.length; i++ )
    max = array[ i ] > max ? array[ i ] : max;

return max;
}

// get pixels
width = image.getWidth();
height = image.getHeight();
pixels = image.getRGB( 0,0, width, height, null, 0, width );

// initialize the histogram table
table = (new Array(257)).join('0').split('');

// populate the table
for ( var i = 0; i < pixels.length; i++ )
    table[ ( pixels[ i ] >> 8 ) & 255 ]++;

maxValue = getMaxValue( table );

data = new Array();

for ( var i = 0; i < table.length; i++ )
    data.push( Math.floor( 100 * table[ i ] / maxValue ) );

data = data.join(",");

url = "http://chart.apis.google.com/chart?chxt=y&chbh=a,0,0&chs=512x490&cht=bvg&chco=029040&chtt=histogram&chd=t:"

// call Google Charts
image = IO.read( new java.net.URL( url + data ) );

// draw the resulting image
graphics.drawImage( image,0,0,null );

Note that we actually tally only the green pixel values. (But these are the most representative of tonal values in an RGB image, generally.) Table values are normalized against maxValue, then multiplied by 100 to result in a number in the range 0..100. Google obligingly plots the data exactly as shown in the above graphic.

And that's about all there is to say, except: Why can't all graphics operations be this easy? :)

reade more... Résuméabuiyad

Compiled languages are too complex

In a talk Thursday at the O'Reilly Open Source Conference, Google distinguished engineer Rob Pike blasted C++ and Java for being overly verbose and too complex.

"I think these languages are too hard to use, too subtle, too intricate," Pike averred. "They're far too verbose and their subtlety, intricacy and verbosity seem to be increasing over time. They're oversold, and used far too broadly."

I tend to agree. Where else but in a language like C would you ever come up with something like:

(*((*(srcPixMap))->pmTable))->ctSeed =
(*((*((*aGDevice)->gdPMap))->pmTable))->ctSeed;

This monstrous line of code is one I used very often in my days of graphics programming on the Mac (circa 1996). On the Mac, the all-important CopyBits() routine always examines the ctSeed field of the source and destination color tables to see if they differ. If the two seed values are not the same, QuickDraw will waste time translating color table info, which you don't want (if you're interested in performance). Hence, you use this line of code to coerce the ctSeed field of the source and destination color tables to the same value. I wrote about this and other tricks for speeding up graphics on the Mac in a 1999 MacTech article.

Of course, the answer to Pike's Complaint is to use dynamic languages like JavaScript or Ruby instead of C++ or Java. But that's not always possible (as when trying to do high-performance graphics programming).

Still, it's surprising how much you can do in JavaScript these days. At the USENIX annual conference last month, Google engineer Adam de Boor raised an eyebrow or two in the audience when he pointed out that Google's Gmail service (443,000 lines of code) is written entirely in JavaScript.

Pike and others at Google are promoting the Go language as a solution to the compiled-language complexity problem.

Go figure.

reade more... Résuméabuiyad

Democrats have many good ideas. Republicans have no ideas whatsoever.

This is a partisan post. Which is not to say I'm writing this because I like Democrats better in general. I just like Democrats better right now, because they have ideas to fight our country's problems. Republicans do not.

Note that this was not always the case. In the Reagan era, Republicans had a lot of ideas, and Democrats largely promised more of the same. Voters seemed to realize this when they made their choice. but Republicans dropped the mantle of the "party of ideas" about a decade ago, and show no signs of picking it back up.

What are the main problems facing our country today? Well, there's the ongoing depression. Republicans' best idea has been to cut unemployment benefits, which reduces structural unemployment slightly but raises cyclical unemployment a lot. We are not in a recession because people have suddenly decided that they don't want to work; people are looking for jobs, and simply not finding ones that match their skills.

Then there's that long-term deficit. A simple breakdown shows that most of that deficit comes as a result of the Bush tax cuts; reversing these cuts would help stabilize our public finances (though in the long run, big cuts in Medicare are needed as well). But Republicans absolutely refuse to rescind those unsustainable tax cuts. Instead, if they are in power, they will probably just threaten to shut down the government unless Obama comes up with ideas for big spending cuts; then, if Obama capitulates (and he is kind of a capitulating sort of dude), the GOP will slam him for the very same spending cuts they forced him to make.

And then there's health care. Republicans have essentially no ideas for how to cut healthcare costs, unless they decided to support Medicare cuts, which they are afraid to do. When pressed, Republicans suggest tort reform and "allowing people to buy health insurance across state lines"; the former is a good idea but a drop in the bucket, and the latter is basically a meaningless red herring.

Financial reform? Republicans recognize the moral hazard problem - the idea that implicit government promises of bailouts encourage banks to take crazy risks - but they have no solution to this other than to have the government say, very sternly, "We won't bail you out the next time!" Which is a laugh, because yes you will, and you know it.

And finally, there's energy. Republicans have nothing on this. "Drill here drill now"? Not a solution, since that oil can and will be sold anywhere. What else? Deafening silence.

Compare this to the Democrats.

On the depression: Democrats are split on additional stimulus spending, but they definitely favor rebuilding our infrastructure, which is important in the short-term (because it adds to demand) and the long-term (because it improves our competitiveness). They favor having the Fed do more quantitative easing (printing money and buying stuff), although legislators cannot tell the Fed what to do. These ideas are good, although the other thing I think we need to do - pressure China to immediately revalue its currency - is unlikely to happen.

On the deficit: Democrats favor rescinding the Bush tax cuts. This is the right thing to do, since those tax cuts were unsustainable. In the short and medium terms that will make a huge difference, but in the long term we'll need to contain health care costs and cut health care spending a lot. This, sadly, is something Democrats are not yet talking about much.

On health care, Democrats finally switched us to a universal coverage nation. That was an important first step - it removed the division of the country into health-care "haves" and "have-nots," which should make future cost-cutting measures more politically possible. It implemented various small cost-control measures, any of which could be ramped up hugely in a few years if it is found to work. This is almost certainly the best health care fix that we could expect in the short term; it remains to be seen if the Dems will come through with serious cost-control ideas in the future.

On financial reform, the Democrats have had a bunch of good ideas, many of which - a resolution authority to reduce the moral hazard problem, a consumer protection agency to reduce lending scams, rules on derivative trading to cut unnecessary complexity from the system - are in the process of being put into law, thanks to Dems and no thanks to the GOP. And on energy, Democrats are correctly investing in alternative energy technology.

Now, I am not saying that I agree with all of the Democrats' ideas. A cap-and-trade system, for example, seems pretty pointless to me (especially since U.S. emissions are falling anyway). And on some issues (China trade, health cost control) they need to go much farther. Nor is having a lot of ideas automatically good; Chairman Mao had a whole little red book full of bad ideas.

But the point is that the Democrats' ideas are mostly good ones. They are a clear improvement over the status quo - ballooning deficits, a collapsing health care system, a bloated unproductive financial sector, vulnerability to peak oil. The Republicans offer no such improvement. They have no ideas whatsoever.

For this reason, stumping for the Democrats in the fall elections is not partisan, and it is not ideological. It is simply patriotic. A vote for a Republican is, at this juncture, a vote for sclerosis and decline.

reade more... Résuméabuiyad

Learning about ESP pages in Sling

Lately I've been doing a fair amount of server-side scripting using ESP (ECMAScript Pages) in Sling. At first blush, such pages tend to look a lot like Java Server Pages, since they usually contain a lot of scriptlet markup, like:

<%
// script code here
%>

and

<%=
// stuff to be evaluated here
%>

So it's tempting to think ESP pages are simply some different flavor of JSP. But they're not. From what I can tell, ESP pages are just server pages that get handed to an EspReader before being served out. The EspReader, in turn, handles the interpretation of scriptlet tags and expression tags (but doesn't compile anything into a servlet). Bottom line, ESP is not JSP, and despite the availability of scriptlets tags, things work quite a bit differently in each case.

Suppose you want to detect, from an ESP page or a JSP page, what kind of browser a given page request came from. In a Sling JSP page you could do:

<%@taglib prefix="sling" uri="http://sling.apache.org/taglibs/sling/1.0" %><%

%><sling:defineObjects/>

<html><body>

<%

java.util.Enumeration c = request.getHeaders("User-Agent");

String s = "";

while ( c.hasMoreElements() )

s += c.nextElement();

%>


<%= s %>

</body></html>

But what do you do in ESP? Remember, <sling:defineObjects/> is not available in ESP.

It turns out that Sling automatically (without the need for any directives) exposes certain globals to the JavaScript Context at runtime, and one of them is a request object. Thus, in ESP you'd simply do:

<%

c=request.getHeaders("User-Agent");

s = "";

while ( c.hasMoreElements() )

s += c.nextElement();

%>

<%= s %>

Very similar to the JSP version.

So the next question I had was, what are the other globals that are exported into the JavaScript runtime scope by Sling? From what I can determine, the Sling globals available in ESP are:

currentNode
currentSession
log
out
reader
request
resource
response
sling


currentNode is the JCR node underlying the current resource; currentSession is what it sounds like, a reference to the current Session object; log refers to the org.slf4j.Logger; reader, I'm not sure about (is it a reference to the EspReader?); request is a reference to the SlingHttpServletRequest; resource is the current Resource; response is, of course, a reference to the SlingHttpServletResponse; and sling is a SlingScriptHelper. All of these are available all the time, throughout the life of any ESP script in Sling.

For more information, try the Sling Javadocs here or Day's page of resources here (note, in particular, the list of References on the right).

reade more... Résuméabuiyad

Crashing the Crash Report Tool

Lately I've been crashing the Mobile Device Crash Report Tool a fair amount. But so, too, have a lot of other people, judging from the buzz on the Googlesphere. Evidently iTunes is fussy (on Vista, especially) when it comes to indiscriminate plugging/unplugging of iPods from USB ports while a sync is in progress (yeah, I know).

For crying out loud can we not have a meaningful error message, though? Repeat after me: Whisky, Tango, Foxtrot.

reade more... Résuméabuiyad

How to hack an OpenOffice spreadsheet with JavaScript

One of the most difficult aspects of learning to write OpenOffice macros (in JavaScript) is that there is so little example code available. OpenOffice comes with dozens of useful code examples written in OpenOffice BASIC, but for Beanshell, Python, and JavaScript (the other supported macro languages) there's hardly anything. Beyond that, there's only the inscrutable 1200-page Developer's Guide (no longer available in PDF), which is inscrutable not so much because it is not well written (it is, in fact, quite well written) but because the Byzantine OpenOffice C++ and Java APIs are themselves so impenetrable. If you've ever tried to write an OO.o macro "in anger," you know what I'm talking about. The whole experience is exhausting.

So without further whining, I thought I'd present the most quintessential of OO.o code examples, showing how to parse an OO.o spreadsheet with JavaScript. I don't claim that the code is particularly stellar; only that it works. (That's the main thing, right?)

The following code will parse through a spreadsheet and display data from the desired numbers of rows and columns in a Swing JEditorPane.

// When a spreadsheet is open in OO.o, this macro will
// loop over a given number of rows and columns and
// summarize those cells in a JEditorPane
//
// Copyleft 2010 by Kas Thomas
// http://asserttrue.blogspot.com/

importClass(Packages.com.sun.star.uno.UnoRuntime);
importClass(Packages.com.sun.star.sheet.XSpreadsheetDocument);

// go thru the sheet one row at a time
// and collect cell data into an array of
// records, where each record is an array
// of cell data for a given row
function harvestCells( sheet, rows, columns ) {
   var masterArray = [];
   for (var i = 0; i < rows; i++) {
           var ar = [];
           for (var k = 0; k < columns; k++) {
                   var cell = sheet.getObject().getCellByPosition( k,i );
                   var content = cell.getFormula();
                   if ( content.indexOf(",") != -1)
                   ar.push( "\"" + content + "\"" );
                   else
                   ar.push( content );
           }
           masterArray.push( ar );
   }

   return masterArray;
} // harvestCells()

// a Swing UI for displaying the data
function EditorPane( ) {

   Swing = Packages.javax.swing;
   this.pane = new Swing.JEditorPane("text/html","" );
   this.jframe = new Swing.JFrame( );
   this.jframe.setBounds( 100,100,500,400 );
   var editorScrollPane = new Swing.JScrollPane(this.pane);
   editorScrollPane.setVerticalScrollBarPolicy(
   Swing.JScrollPane.VERTICAL_SCROLLBAR_ALWAYS);
   editorScrollPane.setPreferredSize(new java.awt.Dimension(150, 150));
   editorScrollPane.setMinimumSize(new java.awt.Dimension(10, 10));
   this.jframe.setVisible( true );
   this.jframe.getContentPane().add( editorScrollPane );

   // public methods
   this.getPane = function( ) { return this.pane; }
   this.getJFrame = function( ) { return this.jframe; }
}

( function main( ) {

   //get the document object from the scripting context
   oDoc = XSCRIPTCONTEXT.getDocument();

   //get the XSpreadsheetDocument interface from the document
   xSDoc = UnoRuntime.queryInterface(XSpreadsheetDocument, oDoc);

   // get a reference to the sheets for this doc
   var sheets = xSDoc.getSheets();

   // get Sheet1
   var sheet1 = sheets.getByName("Sheet1");

   // construct a new EditorPane
   var editor = new EditorPane( );
   var pane = editor.getPane( );

   // harvest cell data (from sheet, rows, cols)
   var masterArray =  harvestCells( sheet1, 100,8 );

   // display the data
   var text =  masterArray.join("\n") ;
   pane.setText( text  );
})();

The main() method comes last, so it may help to read the code from the bottom up. First, we get a reference to the current document with

oDoc = XSCRIPTCONTEXT.getDocument();

(a standard OO.o idiom), then we use that to get a reference to the XSpreadsheetDocument. (I should caution that there is no error-checking code here. It's assumed that a spreadsheet document is already open and frontmost in OpenOffice.) Once you've got the XSpreadsheetDocument, you can peel through its Sheets. With a Sheet, you can do:

var cell = sheet.getObject().getCellByPosition( k,i );

to get cell data for the cell at column k, row i. If the spreadsheet is a converted .csv doc, the cells will all have text data, and (here's the weird part) to get the data you have to do:

var content = cell.getFormula();

since getValue() will only work if the cell contains a double-precision floating point number.

To test this macro, I looked online for a sizable .csv file that I could play with. I found a free/open movie database at George Tech's website. It's a comma-delimited file with data for 1742 movies ranging from 1925 to the early 1990s. It's by no means a definitive movie listing (apparently it represents one of the school's collections) but it suits my purposes just fine.

When I run the macro, it takes less than a second to display all of the spreadsheet's cell data (for all 1742 x 8 cells) in a JEditorPane window:



Ideas for further exploration:

• Style the data in some interesting way
• Save the data as HTML
• Push the data into a Sling repository (blog coming)

reade more... Résuméabuiyad

Please, no more Whisky Tango Foxtrot error messages

I've been seeing this monolog box once in a while in Acrobat these days (not only Acrobat 10 Professional, which is still in development, but Acrobat 9 Pro as well). I can't really call it a dialog box, since it is, in fact, more of a monolog. This is what I sometimes call a Whisky Tango Foxtrot message.

No doubt the underlying error condition happened due to something I did in a script. That's fine, I accept responsibility for the crash. What I don't accept responsibility for is the lameness of the error message. It's a completely content-free error message. It tells me nothing about what went wrong. It's a billboard that says, in effect, "Acrobat sometimes barfs and doesn't know why."

The point of error messages should be to tell you what went wrong, so that you can fix it or work around it, or at least know that it involves something more specific than a nameless internal error. I should at least be given a code of some kind, although even that can be awfully lame, as iTunes users know all too well. Have you ever gotten the infamous iTunes -9812 error? (It's another example of a Whisky Tango Fox monolog box.) Plenty of people see this error, and they turn to Google in desperation to find out what it means, since Apple's web site is of scant help. My September 26, 2009 blog on this subject ("A Fix for the Dreaded iTunes -9812 Error") continues to bring upwards of 2000 hits a month. Why? Because Apple displays no meaningful troubleshooting information whatsoever in conjunction with the -9812 error message. It's just a billboard for iTunes' ability to barf. People have to come to my blog to find out what it means. How lame is that?

Please, UI designers and programmers, provide meaningful error messages, or at least point me to a log file somewhere, where I can find out what went wrong. If the program ran out of memory, followed a null pointer, had a stack-heap collision, entered an endless loop, or choked or bad data (or whatever), just tell me straight out. I can live with the news, I promise.

reade more... Résuméabuiyad

Server-side E4X in Apache Sling

Over the past couple of days, I've been blogging a fair amount about Mozilla Rhino. One of the surprising (to me) things about Rhino is how much faster it is on its own (i.e., when you include js.jar in your classpath) than when you use the Rhino-based scripting engine that comes embedded in the JRE. (See previous blog.)

In the past few days I've also been spending a lot of time with Apache Sling. Imagine my relief to find that Apache Sling uses Rhino proper (js.jar version 1.6R6) rather than relying on the JRE's onboard scripting engine. This means server-side EcmaScript runs much faster in Sling than it otherwise would. But Sling's use of Rhino 1.6R6 is a big win in another way as well. It turns out 1.6R6 is the first Rhino build to feature onboard support for E4X (the EcmaScript extensions for XML, otherwise known as ECMA-357).

Rhino has had E4X support for some time, but until recently it's been a patched-on kind of support relying on the external xbean.jar (which was originally created by BEA, in pre-Oracle days). Prior to Rhino 1.6R6, you had to have xbean.jar in your classpath in order to have E4X support. Now it's built-in. No more need for xbean.jar.

So it turns out you can use E4X grammar in your server-side scripts for Sling, which, I gotta say, is a huge turn-on (if you're as big a geek as I am).

I'll be blogging more about E4X in Sling in coming days at dev.day.com. In the meantime, I thought I'd leave you with a quick example of what you can do with E4X on the server side.

Recently, I ran into a situation where I had the following bit of markup in an .esp (server-side EcmaScript) file:

<fields>

    <field name="Director">
        <value><%= currentNode["Director"] %></value>
    </field>
    <field name="Genre">
        <value><%= currentNode.Genre %></value>
    </field>
    <field name="Language">
        <value><%= currentNode.Language %></value>  
    </field>
    <field name="Movie">
        <value><%= currentNode.Movie %></value>
    </field>
    <field name="Released">
        <value><%= currentNode.Released %></value>
    </field>
    <field name="Runtime">
        <value><%= currentNode.Runtime %></value>
    </field>
    <field name="Starring">
        <value><%= currentNode.Starring %></value>
    </field>
    <field name="Writers">
        <value><%= currentNode.Writers %></value>
    </field>

</fields>

Now mind you, there's absolutely nothing wrong with having markup that looks like this in an .esp file; it's fine as-is. But if you're an XML scripting geek, you see a situation like this and you inevitably start looking at ways to "roll up" all this verbosity into 2 or 3 lines of E4X. And sure enough, this is what I came up with:

<%
fields = <fields/>;

names = ["Movie","Director","Genre",
    "Language","Released","Runtime",
    "Starring","Writers"];

for (var i = 0; i < names.length;i++) {
   field = <field>
         <value>{currentNode[names[i]]}</value>
    </field>;
   field.@name = names[i];
   fields.* += field;
}
%>

 <%=  fields.toXMLString() %>

Concise to a fault. Arguably, it's not as readable as the fully unrolled markup (particularly if you're not a scriptomaniac), but if you're well-versed in E4X, it's perfectly clear what's going on, and it shortens the .esp file to where all code fits on one screen without scrolling. (Always a good thing, in my book.)

As I say, I'll be writing more about this sort of thing on dev.day.com soon. Watch that space.

In the meantime, if you're new to E4X, I recommend taking a look at this article on IBM's Developerworks site. It'll get you up-to-speed quickly.

reade more... Résuméabuiyad

More JavaScript performance data

After yesterday's exercise comparing the execution speed of Rhino-standalone versus Rhino-embedded-in-JRE, I decided to carry out a couple more performance investigations.

I was curious to find out, first of all, how much slower Rhino (a pure-Java implementation of JavaScript) would be than, say, Spidermonkey (the C++ implementation in Firefox). I repeated the "numerical efficiency" test from yesterday (Euler's expansion series for pi) and got the following results:


Execution time (not speed, but time; higher values are worse) is plotted vertically. The surprising result? Firefox's C++ interpreter is not all that much faster than standalone Rhino. The Y-axis here goes from zero to 1200 milliseconds. Trials, on the X-axis, run from zero to values of 200000, 400000, 600000, 800000, and 1000000 for the terms argument to yesterday's pi() function. So in other words, it took Rhino 1212 milliseconds to calculate pi from a million terms, whereas it took Firefox/Spidermonkey 984 ms to do the same thing, making Firefox's interpreter about 20% faster. Rhino comes out looking quite good, considering it is, after all, "only Java."

A more shocking comparison emerged when I tested Rhino against Acrobat. As you may know, Adobe Acrobat comes with a JavaScript engine, presumably written in C++. I expected it to blow the doors off Rhino. Not so! Here are the results:

Astonishingly, Acrobat's interpreter was a full order of magnitude slower than Rhino, requiring 11765 milliseconds to calculate pi from a million terms (versus 1212 ms for Rhino). Why so slow? I have no idea. Perhaps someone from Adobe can comment.

Bottom line: Don't assume that because something is written in Java, it's (automatically) slow. It depends on the runtime environment and what it is, exactly, you're trying to do.

After this, my respect for Rhino only rises. (And it was high to begin with.)

reade more... Résuméabuiyad

How to double the speed of server-side JavaScript

If you're like me, you were probably thrilled to see JavaScript support come bundled into JDK/JRE 6. But if you're using that implementation straight out of the box, you can do far better for execution speed. The trick? Get the Rhino JAR (js.jar) directly from Mozilla and include it in your classpath. Don't use the built-in Rhino support in JDK/JRE 6. It's too slow.

I first noticed the speed difference between regular Rhino and JDK Rhino back in February when I was testing a script that manipulates images. (See my blog post, "Voronoi tessellation in linear time.") In one situation, I found that running my script using the JRE's interpreter took two full minutes, whereas the same script finished in six seconds when I used js.jar.

I decided to follow up on this by running a couple of (highly unscientific) tests. In one case, I came up with a script designed to test the interpreter's speed at handling numeric operations. In another case, I wrote a script designed to test string handling and memory management. The results were revealing.

To test numerical efficiency, I implemented one of Euler's remarkable expansion series for calculating pi. Euler famously found that if you summed the squares of the reciprocals of all positive integers, the result asymptotically approaches pi-squared-over-6:



The JavaScript implementation looks like this:

function pi(terms) {

 sum=0;

 n=1;

  while(terms--)

   sum+=1/(n*n++);

  return Math.sqrt(6*sum);

}

I ran a number of trials with each interpreter, using values for terms of 200000, 400000, 600000, 800000, and 1000000. The graph below shows the results (with execution time plotted vertically; the scale ranges from zero to 1800 milliseconds).



Numerical efficiency test. Vertical axis represents execution time.

Notice that the JRE starts out slower and execution time rises faster than with Rhino (that is, the red line has a steeper slope).

Incidentally, with a million terms, the function produces a value of 3.141592605841622 for pi. (Not bad -- off by only a few parts per hundred million -- but the series doesn't converge quickly enough to be of practical use on modern computers.)

The second test is designed to test string handling and memory management in a kludgy combined fashion. (Folks, this isn't meant to be highly scientific. It's all SWAG.) The code looks like this:

function strings( iterations ) {

   str="abc";

   a=0;

   while(a++<iterations)

     str += str;

   while(a--)

     str.split("").join("");

}

This bit of silliness grows a string by concatenating it with itself over and over again, then repeatedly splits the string into an array of characters and rejoins the array to form a string again. If you pass in an iterations value of 20, you'll make the string double in size 20 times, crashing any known version of Rhino with an OOME. Thus, I tested with values of 12 to 17. Execution time topped out at 5047 millisec for the JRE and 2634 ms for Rhino.


Again notice that the red curve starts out higher and rises at a steeper slope.

(BTW, if you want to see the raw data for these graphs, just inspect the URLs for the images; these are Google Chart dynamic images and the raw numbers are in the URLs.)

I'll say it once more: This is not intended to be a highly scientific set of tests. I believe it to be representative of reality, though, and the reality is, bare-naked Rhino is significantly faster than embedded-in-the-JRE Rhino. You can do whatever you want; I've seen enough data and I've made my decision. Bare-naked is the way to go.

reade more... Résuméabuiyad

Using E4X in Acrobat JavaScript

With all the talk of AJAX in yesterday's blog, I somehow managed to never once talk about XML. (How ironic is that?) But it turns out that one of the pleasures of doing JavaScript programming in Acrobat is the ease with which you can manipulate XML, thanks to the Acrobat interpreter's built-in support for E4X.

If you're not familiar with E4X -- the EcmaScript extensions for XML (otherwise known as ECMA-357) -- you really should take time to look into it. It's a very handy grammar for working with XML. You'll find that E4X is implemented in SpiderMonkey (Gecko's JavaScript engine) since version 1.6.0 and in Rhino (Mozilla's other JavaScript engine) since version 1.6R1. It's also supported in ActionScript, hence is available in Flash CS3, Adobe AIR and Adobe Flex. Add to that list Adobe Acrobat.

How or why would you ever want to use E4X in Adobe Acrobat? Let me give a brief example. The other day, I wrote a short script that harvests all of the annotations from a PDF document. I wanted to put the data in XML format. The script that formats the data as XML looks like this:

// Pass this function an array of Annotations
function getAnnotationsAsXML( annots ) {
    
    var xmlOutput = <annots></annots>;
    for ( var i = 0; i < annots.length; i++ )
    {
       // get the properties for each annotation
        var props = annots[i].getProps();
      
        // add an <annot> element
        xmlOutput.* += <annot/>;

       // start adding child elements
        var parent = xmlOutput.annot[i];
        parent.* = <author>{props.author}</author>;
        parent.* += <contents>{props.contents}</contents>;
        parent.* += <page>{props.page}</page>;
        parent.* += <creationDate>{props.creationDate}</creationDate>;
        parent.* += <type>{props.type}</type>;
    }
    
    return xmlOutput.toXMLString()
}

To test the above code, first open a PDF (using Acrobat Pro) that already contains some annotations. Next, open the JavaScript console in Acrobat (Control-J), copy and paste the code into the console, then add a line:

// Use AcroJS API call 'getAnnots()'
// to harvest all annotations
getAnnotationsAsXML( this.getAnnots( ) );

Highlight (select) all of the code and execute it in the console by hitting Control-Enter. Assuming the document you've got open contains annotations, you should see some XML appear in the console. In my case, I got:

<annots>
 <annot>
   <author>Admin</author>
   <contents>We need to strike this.</contents>
   <page>729</page>
   <creationDate>Tue Jul 06 2010 14:43:57 GMT-0400 (Eastern Daylight Time)</creationDate>
   <type>Highlight</type>
 </annot>
 <annot>
   <author>Admin</author>
   <contents>I am underlining this.</contents>
   <page>729</page>
   <creationDate>Tue Jul 06 2010 14:44:12 GMT-0400 (Eastern Daylight Time)</creationDate>
   <type>Underline</type>
 </annot>
 <annot>
   <author>Admin</author>
   <contents>I liked this.</contents>
   <page>57</page>
   <creationDate>Tue Jul 06 2010 15:04:21 GMT-0400 (Eastern Daylight Time)</creationDate>
   <type>Highlight</type>
 </annot>
 <annot>
   <author>Admin</author>
   <contents>This does not seem right.</contents>
   <page>57</page>
   <creationDate>Tue Jul 06 2010 15:04:32 GMT-0400 (Eastern Daylight Time)</creationDate>
   <type>Text</type>
 </annot>
 <annot>
   <author>Admin</author>
   <contents>Is this the correct copyright date?</contents>
   <page>1</page>
   <creationDate>Tue Jul 06 2010 18:22:39 GMT-0400 (Eastern Daylight Time)</creationDate>
   <type>Highlight</type>
 </annot>
</annots>

Of course, there are many more properties on Annotations than I've captured here. But you get the idea.

Not bad for a dozen lines of code. Sure beats messing around with DOM methods, DOM serialization, etc.

reade more... Résuméabuiyad

Doing AJAX from Acrobat: Tips, Tricks, and Travails

Praise ye mighty gods on Mt. Olympus: It is, in fact, possible to do AJAX from Acrobat. That's the good news. The rest of the news is (how shall we say?) not entirely salutary, and certainly not well documented. But it's pretty interesting nonetheless.

While it's certainly good news that you can do AJAX from Acrobat, Adobe (for whatever reason) has chosen not to follow the well-accepted idiom (in the Web world) of allowing AJAX code to run in the context of a web document. In other words, you can't just put your AJAX code in a PDF (as a field script in a form, say), then serve the PDF and expect to phone home to the server while the user is interacting with the PDF document. Instead, Adobe requires that you put your AJAX calls in a folder-level script, which is to say a static file that lives on your hard drive in a special subpath under your /Acrobat install path. This is roughly the equivalent of Firefox requiring that all AJAX be done in the context of a Greasemonkey script, say, or in the context of Jetpack. Hardly convenient.

The magic comes in a method called Net.HTTP.request(), which is part of the Acrobat JavaScript API. (You'll find it documented on page 548 of the JavaScript for Acrobat API Reference, April 2007 edition.) Due to security restrictions (supposedly), this method cannot be used in PDF forms, nor in a "document context," nor even in the JS console. It must specifically be used in a folder script.

If you look in your local Acrobat install hierarchy, you'll find a folder under /Acrobat called /Javascripts. What you need to do is create an ordinary text file, put your code inside it, and save that file (with a .js extension) in your /Javascripts folder. Acrobat will then load that file (and execute its contents) at program-launch time.

If you're paying attention, you'll notice right away that this means developing AJAX scripts for Acrobat is potentially rather tedious in that you have to restart Acrobat every time you want to test a change in a script.

Something else you're going to notice when you actually get around to testing scripts is that Acrobat pukes (gives a security error) if you don't explicitly tell Acrobat to trust the particular document that's open while you're running the script. This makes relatively little sense to me; after all, if it's a folder script (running outside the document context), why do I have to have a document open at all, and why do I now have to designate that doc as trusted? As we say in aviation, Whiskey Tango Foxtrot.

Whatever. Jumping through the hoops is easy enough to do in practice: To specify the doc as trusted, go to the Edit menu and choose Preferences (or just hit Control-K). In the dialog that appears, choose Security (Enhanced) from the list on the left, then click the Add File button and navigate to the document in question. Once you do this, you can run the AJAX code in your folder-level script.

But wait. How do you run the script? What's the user gesture for triggering a folder script? The answer is, you need to include code in the script that puts a new (custom) menu command on the File menu. The user can select that command to run the script.

Without further head-scratching, let me just show you some code that works:

ajax = function(cURL) {
    var params =
    {
            cVerb: "GET",
            cURL: cURL,
            oHandler:
            {
                    response: function(msg, uri, e,h){
                            var stream = msg;
                            var string = "";
                            string = SOAP.stringFromStream( stream );
                            app.alert( string );
                    }
            }
    };

    Net.HTTP.request(params);
}

app.addMenuItem({ cName: "AJAX", cParent: "File",
    cExec: 'ajax( "http://localhost/mypage");',
    cEnable: "event.rc = (event.target != null);",
    nPos: 0
});

Read the code from the bottom up. The app.addMenuItem() call at the bottom adds a new menu command, "AJAX", to Acrobat's File menu. When the command fires, it executes the code in cExec. For now, you can ignore the code in cEnable, which simply tests if a document is open. (The AJAX menu command will dim if there's no open PDF doc.)

Before going further, let's take note of the fact that the magical Net.HTTP.request() method needs one parameter: a parameter-block object. The parameter block, in turn, needs to have, at a bare minimum, a cURL property (containing a URL string pointing to the server resource you're trying to hit) and a cVerb property (containing one of 'GET', 'POST', 'PUT', 'DELETE', 'OPTIONS', or 'HEAD', or one of the allowed WebDAV verbs, or 'MKCALENDAR'). Optionally, the request block can also have a property called oHandler that will have its response() method called -- asynchronously, of course -- when the server is ready to respond.

So the basic notion is: Craft a param block, hand it to the Net.HTTP.request() method, and let params.oHandler.response() get a callback.

So far, so good. But what should you do inside response()? Well, when response() is called, it's called with four arguments. The first is the response body as a stream object (more about which in a minute). The second is the request URL you used to get the response. The third is an exception object. The fourth is an array of response headers. This is all (sparsely) documented in Adobe's JavaScript for Acrobat API Reference.

What's not so well documented by Adobe is what the heck you need to do in order to read a stream object. I'll spare you the suspense: It turns out the stream object is a string containing hex-encoded response data. The easiest way to decode it is to call SOAP.stringFromStream() on the stream, as illustrated above.

There's more -- lots more -- to doing AJAX from Acrobat (I haven't yet touched on authentication, for example, or WebDAV, or even how to do POST instead of GET), but these are the basics. If you end up doing something interesting with AcroJAX, be sure to add a comment below. And if you want to know how to do Acrobat AJAX against an Apache Sling repository, watch my blog space at dev.day.com. I'll be writing about that soon.

reade more... Résuméabuiyad

In defense of PDF

Can you identify the living fossil?


There seems to be a growing perception, not unlike the controversies surrounding Flash, that Adobe's Portable Document Format is (in the modern world of the Web) a legacy format, something of a living fossil, a technological Coelacanth that refuses to become extinct. Some would go so far as to say PDF doesn't belong on the Web. Some would be wrong, however.

My current employer, Day Software, has (how shall I say this in politically correct form?) a strong prejudice in favor of HTML as the one true and proper Web format for documents. This is reflected in the fact that all of Day's product documentation (like just about everything else Day produces, information-wise) is available online as HTML. There are those at Day, I'm sure, who would like to see PDF disappear from Web sites, if not from planet earth. HTML is a bit of a religion around Day.

And yet, when I joined Day as an employee (two weeks ago), not one document in the half-inch-thick stack of new-employee paperwork that I was asked to fill out was based on an HTML form. Every single document was based on a PDF.

So what is this walking fossil called PDF and why is it still so pervasive?

In the beginning, there was Postscript. Far and away the most successful Adobe technology ever created, Postscript was the first commercially successful page-description language based on vector graphics. It's a Turing-complete language with subroutines, looping, branching, and all the rest. Amazingly, it continues to inhabit printers worldwide. (You probably rely on a Postscript driver to get your printer to work.) It's a brilliant bit of technology, describing, as it does, fonts and shapes and whole pages in resolution-independent terms, in a plain-text (non-binary) interpreted language. Write once, rasterize anywhere.

Since fonts themselves can be described in Postscript, and since Postscript is just text, you'd think PS would be the ideal self-contained document format. Alas, it is not. It's far too verbose, and laborious to render onscreen. (This is what killed Display Postscript.) Still, its inherent portability made Postscript a compelling basis for a document format. So Adobe went on a quest to make Postscript smaller and more amenable to quick screen rendering. They reduced the number of operators (and made their names smaller), and cut out subroutines, and eliminated loops, and did a bunch of other things designed to make Postscript small and screen-friendly, yet without sacrificing portability. The result was PDF.

The first generation of PDF was ASCII-based, and if you looked inside it you basically saw thinly disguised Postscript commands, with loops unrolled and major page elements described as objects. References to objects were maintained in offset tables. Fonts could be embedded, or not. It was still a somewhat verbose format, but at least it could be interpreted and rendered quickly, onscreen or to a printer. (The translation from PDF to Postscript is extremely straightforward.) Adobe came up with a free Reader program and put PDF files out there for anyone who wanted to give them a try. Lo and behold, the format took off.

Why? Why does the world need something like PDF? The (sad, to some) answer is that there is still a need in the world for electronic documents that mimic paper documents. There are industries (such as insurance) in which the physical size and placement of certain pieces of text is regulated by law, and many forms have to fit on a certain size piece of paper when printed out -- there's zero tolerance for text autowrap variations. Form 1040 from IRS has to look like Form 1040, every time. (Imagine the pandemonium at IRS if every tax form that arrived in the mail looked different because it was printed out on a different printer at a different resolution, with text wrapping every which way, all manner of font substitutions, etc.) Like it or not, certain documents have to look a certain way every time, without fail. This is where PDF shines.

Is PDF right for every occasion? No. It's not. No more than HTML is.

Is PDF going to become obsolete? Not any time soon. Not any more than Postscript.

Can/should PDF coexist with markup languages in the world of the Web? I think the answer is yes. Adobe has done a good job of making online PDF forms (for example) REST-friendly and user-friendly. Having to load Reader (or a Reader plug-in for your browser) is a bit of a hassle, but you do get a lot of bang for the buck. The advantages of PDF tend to balance out the disadvantages -- for certain users, in certain situations.

And that's the key. It's not about religion -- it's not about whose document format is inherently better or worse. It's about diversity and choice: choosing the right tool for the job and letting the user (or customer) choose what's right for her. This is the part that the HTML zealots don't get. Some customers want PDF. Some users demand to have documents in a format that looks nice onscreen and prints out nicely (and predictably) on a printer. By not providing those users with that choice, we're (in essence) forcing a technology decision on people. We're forcing our religion on non-converts. And historically, that's always been a dangerous thing to do.

reade more... Résuméabuiyad

A workaround for Acrobat JavaScript's lack of a Selection API

Acrobat has a mind-destroyingly rich JavaScript API (with hundreds upon hundreds of methods and properties, on dozens of object types), but one thing it sorely lacks is a Selection object. Bluntly put, you can't write a line of code that fetches user-selected text on a page. Which sucks massively, because in any modern browser I can do the equivalent of

document.getSelection( )

to get the text of the user's selection (if any) on the current page. In Acrobat, alas, there's no such thing (using JavaScript, at least). If you want to write a plug-in to do the requisite magic using Acrobat's famously labyrinthine C++ API, be my guest (I'll see you at Christmas). But it seems like overkill (doesn't it?) to have to write a C++ plug-in to do the work of one line of JavaScript.

Fortunately, there's a workaround. I nearly fell off my barstool when I happened onto it.

It turns out (follow me now) you can get the exact location on a page of an annotation (such as a Highlight annotation) using the Acrobat JavaScript API; and you can programmagically get the exact location on a PDF page of any arbitrary word on that page. I thought about those two facts for awhile, and then a light bulb (~40 watt Energy Saver) went off in my head: If you're willing to use the Highlight annotation tool to make selections, and if you are willing to endure the indignity of iterating over every word on a page to compare each word's location to the known location of the Highlight annotation, you can discover exactly which words a user has highlighted on a page. It's a bit of a roundabout approach (and wins no awards for elegance), but it works.

The first thing you have to know is that Acrobat lets you do

getAnnots( page )

to obtain a list of Annotation objects (if any exist) on the specified page. (Just specify the zero-based page number: 0 for page one of the document, etc.)

The second thing you have to know is that every Annotation has a gazillion properties, one of which is the list of quads for the annotation. A quad simply contains the annotation's location in rotated page space. The key intuition here is that every annot has a bounding box (think of it as a rectangle) with an upper left corner, an upper right corner, and so on (duh). Each corner has x,y coordinates in page space (double duh). Since there are four corners and two coords for each, we're talking about eight floating-point numbers per quadrilateral (i.e., per quad).

Okay. Cache that thought.

Now consider that in the land of PDF, every word on a page also has a bounding box. And Adobe kindly provides us with a JS method for obtaining the bounding-box coordinates of the Nth word on a page:

getPageNthWordQuads( page, N )

Therefore it's possible to build a function that accepts, as arguments, an Annotation and a page number, and returns an array of all words on that page that intersect the quad space of the annot in question. Such a function looks like this:

function getHighlightedWords( annot, pagenumber ) {
       var annotQuads = annot.quads[0];
       var highlightedWords = new Array;
       // test every word on the page
       for (var i = 0; i < getPageNumWords(pagenumber); i++) {
               var q = getPageNthWordQuads( pagenumber ,i )[0];
               if ( q[1] == annotQuads[1])
                  if ( q[0] >= annotQuads[0] &&
                       q[6] <= annotQuads[6] )
                           highlightedWords.push(getPageNthWord( pagenumber ,i ));
       }
       return highlightedWords;
}


// Test the function:
// Note that this test assumes there is at least one
// annotation on the current page:
page = this.pageNum; // current page
firstAnnot = getAnnots( page )[0];
words = getHighlightedWords( firstAnnot, page );

We can safely compare quad coords for exact equality thanks to the fact that when Acrobat puts a Highlight annot on a page, it sets the annot's quad location to (exactly) the word's location. There's no "off by .0000001" type of situation to worry about.

Something to be aware of is that functions that return quad lists actually return an array of quads, not a single quad; you're usually interested in item zero of the array. (And recall that a quad is, itself, an array -- of eight numbers.)

I tested the above function in Acrobat 9 using Highlight annotations and (voila!) it seems to work.

Now if Adobe will get busy and add a proper Selection object to its JavaScript API, I can turn off the 40-watt bulbs and hop back on my barstool, and get some real work done.

reade more... Résuméabuiyad

Designers, please don't make this UI mistake

While we're on the subject of UI faux pas, let me mention another type of blunder I'm running into more and more these days. See if you can figure out from the following screen shot what I'm talking about. (Click the graphic to see a larger version of it.)



Is it just me, or does anyone else see the illogical nature of having a menu command that leads to a submenu command of exactly the same name? Save As > Save As makes no sense to me. Does it make sense to you?

In the above case, the perpetrator is Adobe. I entered bug 2642169 in the appropriate bug-tracking system; hopefully it will get fixed. There are other problems with the above menus. Some of the commands need to have an ellipsis ("...") after them because they lead to dialogs rather than having an immediate effect. But that's minor (IMHO) compared to Save As > Save As.

Here is yet another instance of the Same > Same problem:



The perpetrator in this case is Microsoft. Somehow (and I don't remember what I did, exactly), I got to this weird menu situation in Vista's Explorer while doing a desktop operation. Obviously, New > New makes no sense.

I could cite a third example, but I think you get the point.

Don't have menu commands that go to submenu commands of the same exact name. It's illogical and it looks funny. It's not what nature intended.

reade more... Résuméabuiyad