Article Link About Batch Importing with GRAILS

Ted Naleid has a nice write-up on his optimizations of a database batch loading process using GRAILS. It’s a few months old but if I can help spread the information, so much the better. I do tend to agree with the comment that GRAILS might not be the correct solution for bulk loading of records into a database. Still, I see Ted’s solution as quick, easy, and works just fine for the size of the import he has. So why not?

I certainly would not have the same GRAILS application instance that is doing the bulk upload also taking normal traffic. That seem like a recipe for problems. Otherwise it is a good read with a decent solution. Enough so I thought to write a short post here. 😉

Were I to go beyond Ted’s volume, I’d look at a dedicated GRAILS app using the same domain objects or perhaps using GORM outside of GRAILS (if that is still possible). In some ways I see this as a good justification (or rationalization) for keeping Hibernate mappings separate, versioned, and in (gasp) XML. You can then use them in GRAILS or in any other code that can link to Hibernate.  Most of my domains predate GRAILS and GORM so mapping them in GORM is not straight forward. That costs me some of the GORM goodness to begin with. Of course I have the XML mappings already so the choice is easy to make. That’s the cool thing about GRAILS, you can use your legacy domain objects and/or mappings. For validations there is always Hibernate Validator.

Reference: http://naleid.com/blog/2009/10/01/batch-import-performance-with-grails-and-mysql/

Advertisement

Does Groovy Know You Are Seeing Clojure?

With all due credit to Venkat Subramaniam for the paraphrased title of this posting.

I’m not sure exactly when I decided to learn Clojure. I know I’d heard about it as one of four JVM languages with a lot of buzz in 2009 (yes I know there are other JVM languages). At the time of  ‘discovery’, I was in New Orleans at last October’s (2009) SpringOne/2GX swooning with others over our love of Groovy and all things Groovy. The other three I am thinking of are a Ruby clone JRuby, a weird language called Scala, and Clojure. Yes I am aware calling Scala weird is a bit of the pot and tea kettle syndrome but I find Scala to be the C or APL syntax of the JVM languages. All things considered, in my not so humble opinion, there is no other language I would rather code with than Groovy. It’s just the most natural and powerful progression from Java. Maybe that is controversial, but since this is my blog, it’s my opinion.

Now I am a bit of a language junkie and always have been since I was a tyke in 1979 coding on a TRS-80 Model 1 in Basic and Z-80 assembler. I’ve toyed around with, or coded for a good living in, Z-80, 6502, 68000, x86 assembler, Modula 2, Pascal, Forth, C, C++, Java, Ruby ,Groovy, and now Scheme (a tad) and Clojure. Pretty decent list if I do say so myself. Oh and sitting on my bookshelf is a book on Erlang. However I can say with some certainty: In the intervening 30 years since starting out on a TRS-80 Model 1 nothing has been more difficult in terms of solving even the most trivial of problems than learning Clojure.

Of course there is a good chance this has nothing to really do with Clojure at all. It is really Functional Programming (FP) that is giving me fits. After all, if you take away FP, learning Clojure is just learning Lisp (it’s Clojure’s immutable paradigms that differ from Lisp – as far as I can tell). I have been doing pretty well with the “Little Schemer” for learning a dialect of Lisp. But alas, I don’t find that translates into Clojure. Not sure why. For two months, I’ve been reading blogs, reading MEAPs (Clojure in Action and The Joy of Clojure), and reading source.

At first reading Clojure was the problem (and still is to some extent – reading is not writing) as I had no Lisp or Scheme exposure. The only thing I knew about Lisp was the oft repeated refrain that truly great programmers know Lisp is the secret weapon. Like Paul Graham’s Beating the Averages post. Did I believe it? Do I believe it? I suppose I did believe it and I did not care. There was much more money in Java and C++ than Lisp, so why worry about that I am using so-called second rate languages. (A strawman there but you know what I mean.) I am in this profession to make a living, not impress people with the tools I use.

Do I still believe it? Umm… not right this minute. I am having such a heck of a time coding anything beyond Hello World in Clojure I have begun to wonder if only certain level guru’s can ever ‘get it’. Notice I am not making a distinction between coding in Clojure/Lisp and Functional Programming. The reason is as far as I can ascertain in the Clojure world, they go hand-in-hand. Imperative Programming in Clojure is really fighting the language. So one could say I really bit off way more than maybe I should have. On one hand I decided to learn something about Functional Programming and on the other decided Clojure would facilitate that because you are virtually forced to immerse yourself in FP from the start. Or maybe it was the other way around. Maybe it was Clojure making me discover FP.

Either way I took on two totally unknowns and for two months now I have paid a mental price for it. It has been really frustrating. The simplest imperative programming solutions don’t even give me a hint, a clue, as to an approach of how to solve the same problem in Clojure/FP without a lot of mutable state. Take a simple problem we use at my employer to get a small perspective on possible new hires and how they approach solving problems in Java: This problem is to take a vector of intra-day stock prices for some particular stock (numbers) and determine the maximum profit you could have made buying that day and selling that day. Simple stuff in an imperative language. You simply keep track of the minimum value, the profit, and replace the minimum whenever the next value is less the current minimum.

Well my dear readers… I could not for the life of me figure that out in Clojure/FP. Now remember I’ve not written ANY Clojure or done any FP. Just read about it. This was my ‘do something real’, sort of  a Hello World.

“What’s the big deal?”, all you expert FP’ers (or even novice) FP’ers are saying. Yeah thats the thing, I have no idea what the big deal is. Just could not find the approach. Or the correct higher order function(s) to use. Just could not figure out how to keep track of two things (minimum and profit) without a mutable variable someplace. Frankly, maybe at age 51 you can not teach an old dog of 30 years, this new trick.

To this day I did not really figure it out on my own. Instead I casually mentioned the challenge to my local friendly Scala enthusiast. He took up the cause and came up with something like this

A.foldLeft((Math.MAX_INT,0))((s,curr)=>(Math.min(curr,s._1),Math.max(curr-s._1,s._2)))._2

Now you begin to see how stupid I was feeling. Not only did he solve the problem, he solved it in a line of code that fit in a tweet on Twitter with room to spare.

The tuple was what gave me the idea of using a structure. And I guess I did solve it. It’s not pretty, and it’s way more than one line, but it appears to pass my tests.

(def closing-prices '(23 24 21 23 45 33 22 1 33 4 2 23)
;;(def no-profit-closing-prices '(23 21 20 19 18 17 16 15 14 13 1 0))
;;(def closing-prices '(23 24 20 19 18 17 16 15 14 13 1 0))

(defstruct values :min :profit)

(defn calc-profit [current-values next-val]
    ;;(println (current-values :min) (current-values :profit) next-val)
    (let [mini (min (:min current-values) next-val)
          profit (max (:profit current-values) (- next-val mini))]
        (assoc current-values :min mini :profit profit)))

(defn find-profit [coll]
    (let [v (struct-map values :min (Integer/MAX_VALUE) :profit -1)]
        (reduce calc-profit v coll)))

(:profit (find-profit closing-prices))

Of course my greatest fear is you’ll post a comment showing me a single line of Clojure that does the trick. 😉

So why subject myself to this self-inflicted torture?

1. I believe Rich Hickey is right. More cores not higher clock frequencies are the near future. FP is really a boon in concurrent coding as you can more easily reason about the code’s behavior with multiple threads.
2. I almost never see a really great object hierarchy in OO languages. They start out small and clean and grow to crap and cruft. Not in 20 years of OOP have a seen a really great object hierarchy (yes a logical fallacy but still true in my case). Indeed most of us now use composition, aggregation, and delegation, not inheritance. In FP you have to use composition. It is all about composing functions from other functions.
3. Its a challenge. Different way of thinking. And maybe secretly I want to join that elite illuminati of Lisp engineers…

But Groovy is soooo much easier! Will Clojure win my heart…

Feeling Groovy: Quick Summary of a Jersey WADL

I have a Jersey based REST service and just needed a quick API summary. Groovy to the rescue! After grabbing the WADL (http://(whatever your context is)/application.wadl) I simply fed it to this little script. I am sure it could be improved to get the WADL itself, or read from stdin (which would be more Unix like) etc…


#!/usr/bin/env groovy
if (!this.args) {
System.err.println "usage summarize_wadl <wadl file> "
return -1
}

def s = [];
def nodes = new XmlParser().parseText(new File(this.args[0]).getText())
nodes.resources.resource.resource.each {
s << "${it.method.@id} => ${it.method.@name} => ${it.@path}"
}
s.sort().each { println it };
return 0

Got a better way? Faster way? More Groovy-o-matic (idiomatic) way? Teach us all! Leave a comment. I'll credit and incorporate ideas if allowed.

Feeling Groovy: Numbers List to Values List

So I have a bunch of strings being sent to me like 1,3,4,5,6

Those numbers are selections like foods I like. Maybe 1 means gnocchi, 3 means lasagna, 4 means creme brulee, etc. I needed to output these readable values not the numbers to a quick and dirty web page. Actually I need to output quite a few of them. Groovy to the rescue.

def map = [1:"apples", 2:"oranges",3:"grapes",4:"cherries"]
def text = "1,2"
def result = "apples, oranges"
assert result == text.split(',').collect {value -> map[value.toInteger()]}.join(", ")
// Thanks to Noah Sloan for an even groovier way!
assert result == map.subMap(text.split(',')*.toInteger()).values().join(", ")

text = "4"
assert "cherries" == text.split(',').collect {value -> map[value.toInteger()]}.join(", ")
// Thanks to Noah Sloan for an even groovier way!
assert "cherries" == map.subMap(text.split(',')*.toInteger()).values().join(", ")
text = ""
try {
assert "" == map.subMap(text.split(',')*.toInteger()).values().join(", ")
fail() // NumberFormatException - Validate your imputs - or use StringUtils
} catch(NumberFormatException nfex) {
}


Groovy is even smart enough to deal with a string of one value (no comma). You do have to be careful about that toInteger() call but then in real code you check your preconditions right?

Got a better way? Fast way? More Groovy-o-matic (idiomatic) way? Teach us all! Leave a comment.

SpringOne/2GX – My Groovy and Grails Experience

Some Thoughts on the springOne 2GX Conference

It’s been a very long time indeed since I went to any conference. For quite some time I simply did not have the wherewithal to advance my employers that amount of cash. Hard to believe perhaps but true nonetheless. So this was my first conference in quite some time and therefore my first 2GX. Yes yes i know it’s springOne 2GX. It would not surprise me if Rod Johnson and SpringSource at some point drops the 2GX name. To me it was mostly about Groovy and Grails. Spring is not a paradigm shift. Groovy and Grails is in my opinion.

Let me get this out of the way, I am opinionated. If you are easily offended, then you should move on to another blog. However I am also quite capable of stating I was wrong. So feel free to comment. I get the feeling SpringSource would prefer springOne 2GX be more about Enterprise’ish Spring. I love Spring… kind of… like most Java/XML/Annotations gorging technologies, it is a love/hate relationship.

The City

Whoever decided to hold 2GX in New Orleans ought get a raise! Keep those ideas coming! Fabulous city. My wife and I were very lucky to spend 6 non-conference days as well as the four conference days there. We barely scratched the surface of places to see and thing to do  We loved the architecture, the fabulous food (we are foodies), and the great party atmosphere. Oh my goodness did we adore the restaurants!

First Impressions of the Conference

Well can I be frank? Or Jeff? Here comes the first controversial comment… Was putting Rod Johnson on the T-shirts a good idea? Really? Come on. Maybe it was supposed to be funny, idiomatic, but to me,  it’s a egotistical misstep – even if it was not Rod Johnson’s idea. And the Rod Johnson bobble-head? Give me a break. Again, maybe it’s an inside ‘SpringSource’ joke, but if so, I don’t get it. Maybe someone will explain it?

So I have a collector’s item t-shirt cause I am not wearing Rod Johnson on my chest just like I would not wear Steve Jobs or Larry Ellison on my chest. A shirt with Groovy on it would have been better. 🙂

The SWAG

I don’t go to conferences for the SWAG of course. However the presenters got nice tasteful springOne 2Gx golf shirts which would have been a nice memento. We got ‘Rod Johnson’ t-shirts and bobble-heads. 😛 And the stated backpacks were no shows. Oh well. The portfolio is ok. All in all, I’ve seen others come back from conferences with better. 🙂

Conference Food

Excellent. ‘nuff said. Breakfast, lunch, and dinner were included. That was awesome. I am not sure if that is how all conferences go, but given the conference cost, it was quite nice. I will admit though I did not partake of dinner as I’d be damned if I was going to eat a buffet (albeit a good one) when there was world class food within walking distance!! And of course my wife was with me. My one critique was the last day there was no more coffee for the final sessions and was unceremoniously told that by one female attendant.

Organization

Excellent. Kudos to the organizers. I thought it all went off splendidly. Almost transparent to the attendees. Things just seemed to work. Bravo!

Keynote

Well I don’t remember much other than thinking, “Note to keynote speakers: Do not do demos in your speech!!!!!” It is the fastest way to bore your audience. When you lose your audience you lose your impact. Speaking 101. Leave the demos for sessions or ‘technical keynotes’. I ducked out after the second demo. It was getting late my wife and I needed to get to dinner before things closed (which they do on Mondays). It would have been a non-issue except the speech went longer than stated in part because of the demos! Also, though I left early, I do not think Guillaume Laforge was called upon. In my opinion, if you call on Graeme Rocher to talk, which is great, you should call on the Groovy Project Lead as Groovy is the great enabler. And far more important I think, than a chat program written in FLEX. 😀 It’s not that I am putting down FLEX… okay… lied… I am… 😛

The Speakers

I can’t say enough about how good the speakers were. I do know how hard it is to prepare… which brings me to several points, I suppose true of most conferences, but this was my first in a long time…

There were obviously a hierarchy of presenters.

1. Those that prepared
2. Those that did not

The first group fell into two classes.

• Those that rehearsed or had given their presentations beforehand

These speakers were fluid and never dwell too long on any one slide. They knew the pace of their presentation and what it would take to get through it. I assume some used timers on their phones or screens. Well done!! (In fairness one presenter was honest about not being able to get through every slide as the presentation was meant to be more than 1.5 hours.)

• Those that prepared but did not rehearse.

These presenters knew what they want to say, had prepared slides, but had not thought about how long it would take to get through them. They would dwell upon items too long and too early in the presentation. This caused them to only get through a third to two thirds of their presentation. This resulted in the final third being whizzed through in the last 5 to 10 minutes. Also, please don’t read off the slides. 🙂 Have notes in front of you that explain the points.

The second group fell into two classes

• Those that did not prepare because they are completely natural speakers. They have such a rich background in the subject matter that they can ad-lib their way through and be totally coherent.  Several of the Groovy Rockstars obviously were in this realm.

• Those that did not prepare, thought they knew what they were going to say and type, thought they were natural speakers, thought they could ad-lib, spent lots of time looking at their screens, and lost their audience. There were only one or two of these. They were rare.

Take away… should I ever be a presenter I will :

• Have slides because in my opinion, it’s the right thing to do.
• If I am going to live code, make sure I have working and tested code in another project as a ‘backup’ and test it again just before my talk just in case ‘something goes wrong’. Don’t depend on network connectivity. 😉
• Make sure I rehearse back at my company and get the timing of the slides right.
• And, I won’t read off the slides!!! 🙂
  

Oh and don’t spend a lot of time talking off topic unless you put in the conference summary you tend to go off topic. It’s not fair to the attendees.

The Content

I have a few comments.

1. I’d love to see more depth. Who wouldn’t want to be taught by the domain experts?! Now I know people make good livings consulting, (or at least make a living).  I am not asking anyone to give away that which they could get paid for, but I’d love to have seen some 3 or 4 hour deep dives for those that were willing to forgo ‘overviews’.  I called this the “more depth, less breadth track’.
2. There was almost no session I did not learn something in. That is after all why one goes.
3. Some sessions should be marked, “Must already know Groovy (or GRAILS)”. Why? Because it’s 10 or 15 minutes that could be spent digging in.

I must say many of the presenters gave outstanding content and are obviously domain experts. Paul King, Burt Beckwith, Guillaume Laforge, Scott Brown, to name a few were marvelous. I learned a tasty morsel or more in each presentation just as I had hoped.

Those were people I knew coming to the conference I wanted to listen to. However there were others I had never heard of before. Chris Richardson‘s talk on Amazon Web Services IMO was a real eye opener. In someways his talk was THE wave of the future for scalable systems. Not the cloud. But the concepts you need to wrap your head around to code in a high availability replicated environment.

Then comes the Terminator of Talkers: Venkat Subramaniam! He is an expert and a showman. That makes a powerful and memorable impression! His patterns presentation really was nothing tricky that you could not learn on your own – given time and experience. However his masterful presentation and the way he grabs his audience at the start and never lets go means I had a lot of “‘Ah ha!” light bulb turning on in my head moments.

Bottom line, mostly very good, some good, some could use a tweak or two.

Most Hype

Roo. I did not attend the Roo sessions but my colleagues did. They were impressed to say the least! I heard more about Roo than almost anything else. Though Roo and Grails are siblings, real siblings fight occasionally. I wonder where this will lead…

The Community

I was nearly the oldest person in the rooms if not the oldest. That felt weird. It is sobering to say the least. 🙂 I was pleasantly surprised that most of the Groovy and Grails rockstars are VERY approachable and gracious. You all know who you are (if you are reading this). In any community there are always a few that take themselves too seriously. And there are those perhaps that are generally uncomfortable with too much attention and shun the limelight. I think there were a few of those as well. No one was downright rude to me! LOL! And nobody (yet) called me ‘pops’. 🙂

Happiness and Wishes

When I get out and talk to other developers I am always really thankful for the job I have. Many of those I talk with are locked into technologies because of their corporate policies or their client’s policies. Also some I meet can’t (or worse won’t) explored new items. To paraphrase a Venkat quip, “Does your old technology know you are seeing new technology?”

I am lucky. If I can make it perform (see next paragraph) I can make it to production. Use the best tool (balanced with productivity) for the job.

For all the interesting things I heard at 2GX the issues of performance were the least understood and the least covered. Perhaps that is off topic for 2GX. I think there was maybe a session or so about terracotta. And of course Burt Beckworth’s talk on clustering. However: I work on a site that gets 100’s of thousands of logins a day. 30+ million page views a day. 100’s of millions of service calls. Where does Groovy and Grails fit there? Or don’t they? Where are papers on sites bigger than a big blogging app? Or an internal CRM app? Anything I use either has to be fast out of the box or easily scaled horizontally. That’s why I liked Scott Brown’s talk on GRAILS without a UI. I am all over that, just have to crank up a Faban test and benchmark / load test it.

Finally, should this community lead the charge away from RDBM’s? GRAILS makes CRUD easy. But CRUD is still the bottleneck any high traffic site will get to. And look at all the under the covers cruft CRUD requires. That’s what was cool about Chris Richardson’s session. He was pointing the way. You don’t need real time CRUD. You can design for eventually consistency. And you often don’t need to normalize data anymore. Is that not the next simplify our apps direction? Key Value stores, HBASE, no SQL…

That’s where I am headed. GRAILS in the service layer with multi-threaded, simultaneous calls to DBs (until I can ditch them), distributed read-repair caches (like Amazon Dynamo), and other services. Maybe on to NoSQL. Should be fun. I’ll keep you posted here. Who knows, maybe I’ll be I’ll get my own  cool springOne 2GX golf shirt next year!!

---

The Hotel

You can stop right here as the rest of this is probably even less interesting than the stuff above.

This is going to come as a shock to some of you, but as a 4 star hotel the Roosevelt left much to be desired. As a conference venue it was fine. But as a 4 star hotel the staff has a long way to go.

• After booking me on one floor, they made my wife and I move because the floor was taken by a football team (the Giants if you must know). However when we checked in we were told it was no problem and we should not have to move. So we completely unpacked. After all we were staying 10 days. When we checked in at the front desk surely someone knew the Giants were coming that weekend oui?
• They made us move to a smaller room with broken items, doors that did not shut, and a shower and toilet that were coupled such that using one backed up the other.
• They mis-charged my credit card when the room was already paid for and thus for a day maxed out my credit card. Thus we were cash and carry for a day. Good thing we had enough. Oh and at the time, they were completely ‘unapologetic’ insisting they had done nothing wrong.
• We were charge $31 for the minibar because my wife placed her meds on top of some items where they would stay cool. We used nothing, but they don’t check. They just charge. That got reversed. Cute aside: The front desk told me “The minibar does not get cold.” Precious. It took 2 refrigerators and 3 calls to get a one that worked.
• At first they knew nothing about free internet in the room. Those charges also eventually got reversed.
• Even after that, several times the room went unmade up and took several calls to get someone up to do it. When it was made up sometime things like bathroom tissue were not replenished, dirty glasses left, etc.
• After some rather frank exchanges with my assistant back at the office and their management we were moved to a suite, comp’d a night, and our bill reduced
• So while I kept hearing all this gushing from attendees about how great the hotel was, I could only think, “Well, maybe most conference hotels are generally a lot worse than this?” However this is supposed to be a four star hotel where service is king. Otherwise it’s just a luxurious and expensive Motel 6. I guess people have a different expectation of what a luxury hotel should be than me… or maybe I’ve stayed in a few in my day… 😉

groovy gpars – trials and tribulations Part Duex

When we left off in part one of this sojourn I had given up for the night hoping someone might see my tweet a take a look. Well getting the authors of gpars to take a look is a good thing! 😉 Thank you both.

A quick update to my part one, I see the download link for gpars is live on CodeHaus.

First I tried to join the gpars mailing list. I tried to join the google code mailing list first because that had recent activity. But I am awaiting the normal confirmation. It occurred to me, being the sharp guy I am :P, that when the move was made from Google Code to CodeHaus, the mailing list moved. So I went over to CodeHaus and tried to join a list over there. And I  am awaiting a confirmation approval. ;P

In the mean time I thought I’d browse the archive there. However the list of mailing lists showed no gpars mailinglist. So maybe the google group is the only group for now.

While this may all seem annoying, I assure you it is not. It is all completely understandable when you realize the authors are moving to a new home, and they probably have real work, jobs, and familiy to attend to. So patience should be extended to them. In time everything will straighten itself out. I did not know Dierk König was on the project! Mr “Groovy in Action!”, very cool! And I believe Vaclav works on Intellij? Is that right. I love IDEA and have used it since 2005.

Ok back to the grind….

So I am back to using the MacBook Pro since using Windows XP gave essentially the same results. First thing I did is look in lib directory of gpars. Duh!!! Right there was the correct version of the jsr166y jar. Who knew? 🙂 Okay so that was not in my classpath and needed to be. I copied that JAR to the groovy 1.6.5 library directory and deleted the one I had downloaded elsewhere Voilà! Now the DemoParallelizer and Enhancer works.

Vaclav was correct that the integration page of the CodeHaus site would have tipped me off to the correct version, but I was impatient and did not want to work with maven (ick), or gradle, or grape, or.. <insert favorite build system here>. etc.

However the Actor demos, like the DemoDiningPhilosophers were still not compiling at all. So I was perplexed at the comment about using Groovy 1.6.5 since that is what I am using. But then a light bulb (more like flash blub) went off in my head. Maybe what I had to do is compile the gpars project with Groovy 1.6.5… and do so with the source of gparallelizer 0.8.4… so now I had to find the 0.8.4 source of gpars.

I have no knowledge at all of gradle or git except for a very interesting interview with the author of gradle on the Groovy and Grails Podcast. You DO listen to that right? No? Go subscribe… I wait… go on… I’ll wait… Back? Excellent.

What I know and dislike for the most part is maven (that should be another blog post). I know Subversion, and I am learning bazaar. But git and gradle were not on the radar. For this ‘spike‘ I was just using the demonstration gpars source. No build system. Just the jars, the source, and good ole Groovy 1.6.5. I had no intention of building from source.

Best I could figure, on CodeHaus I was able to find a tag that said “Updated to groovy 1.6.5” That sounded promising so I clicked on ‘snapshot‘ which downloaded a tar file.

Again being a baby in the cradle of gradle all I could do is poke about. Like a –help switch. 🙂 Ah ha! A  –gui switch produced a nice GUI (imagine that!!) and I tried a build. Once again the building process got to :compileTests and hung. About 30 seconds later it ran out of heap space. Well I’d been down this road before (BTW: in Intellij, using the project ipr provided, the tests run just fine… weird huh?). For some reason the unit tests will not compile using gradlew. I tried using JAVA_OPTS set to  -Xmx2048m and still no worky. 🙂

So I clicked on the assemble gradle target which (apparently) skips the tests. (Yes yes I know that is not good, but this is a code spike – not production.) Hmmm… that produced a .0.9-SNAPSHOT jar, where I’d been expecting 0.8.4… no matter, that is where the SCM tag indicated the switch to groovy 1.6.5 took place. I copied that jar to groovy lib directory and ran some of the demos that had given me problems.

Success is Nearly Ours!

The jar I built ran most of the demos. Not all of them, the DemoMashupWithMethods in the Dataflow demos still, on the Mac, just starts the downloads and exits. But virtually all the Actor demos ran. Whether they are correct, of course I can not know without the unit tests working. I will bring the project into Intellij and see if the unit tests pass. They did yesterday. The difference is Intellij is compiling and running the tests, not Gradle. Not that I know that is the problem. I don’t.

I believe I have enough working now to study what types of object in gpars might map to my use case and how. Then to try some ideas of my own with gpars in the service tier of my application. For now it’ll just be ‘playing around’. At least until gpars settles down in its new home and everything runs ‘out of the box’.

More to come… comments are very welcome. Hope to see some of you (and talk) at SpringOne 2GX

Quick update: For those still curious, I did build the IntelliJ project and IntelliJ ran all the test except one. An exception test. This could be just a result of getting an intermediate build from the GIT repository instead of an official release.

I see the author of gpars did an interview with the Groovy and Grails Podcast. It will be interesting to listen. As I mentioned, Glen Smith and Sven Haiges‘ podcast is definitely worth a listen if you are in any way involved with Groovy, Grails, Gradle, or Griffon development.

gpars – trials and tribulations

I am trying to get gpars (formerly known as gparallelizer) to simply run it’s demos. I thought that would be a great introduction to gpars and the use cases it can solve. Needless to say, either I am not so swift, or its harder that I imagined it would be. So far I failed miserably on Mac OS-X with Java 1.6 (from Apple of course) and Groovy 1.6.5 so I thought I would start from scratch on a native Windows XP box.

So…

• I downloaded and installed Java 1.6. Here is what I see at the command prompt

java version "1.6.0_16"
Java(TM) SE Runtime Environment (build 1.6.0_16-b01)
Java HotSpot(TM) Client VM (build 14.2-b01, mixed mode, sharing)

• I downloaded and installed Groovy 1.6.5. Just took all the defaults.

C:\Dev\gparallelizer\samples>groovy --version
Groovy Version: 1.6.5 JVM: 1.6.0_16

Ok! So far so good. So now I go to the gpars site. Now I hit my first snag. This snag is common to Mac and Windows. The download link is not. Not a link that is. Drats. I don’t want to build from source. So I head to the old Google Code Site because there I can get the gparallelizer 0.8.4 jar. Which I do. I install it into the groovy lib directory… this might be a mistake, but with no other guidance, I took a chance.

While at the old Google Code Site I also grab version 0.8.4 of the samples. Now I should be stylin’

I tried this:

C:\Dev\gparallelizer\samples>groovy DemoParallelEnhancer.groovy
Caught: java.lang.NoClassDefFoundError: jsr166y/forkjoin/ForkJoinTask
at org.gparallelizer.ParallelEnhancer.class$(ParallelEnhancer.groovy)
at org.gparallelizer.ParallelEnhancer.$get$$class$org$gparallelizer$actors$pooledActors$FJPool(ParallelEnhancer.groovy)
at org.gparallelizer.ParallelEnhancer.<clinit>(ParallelEnhancer.groovy:40)
at org.gparallelizer.samples.DemoParallelEnhancer.class$(DemoParallelEnhancer.groovy)
at org.gparallelizer.samples.DemoParallelEnhancer.$get$$class$org$gparallelizer$ParallelEnhancer(DemoParallelEnhancer.groovy)
at org.gparallelizer.samples.DemoParallelEnhancer.run(DemoParallelEnhancer.groovy:28)

Ok so things are still complicated. 😀 So what’s JSR-166 I ask? Concurrency Utilities. I sorta gathered something like that from the ForkJoin thing. Well I downloaded from here. Ah well no… see that’s packaged as

C:\Dev\gparallelizer\samples>jar tvf "\Program Files\Groovy\Groovy-1.6.5\lib\jsr166y.jar"
 0 Tue Oct 06 15:36:08 PDT 2009 META-INF/
 102 Tue Oct 06 15:36:06 PDT 2009 META-INF/MANIFEST.MF
 0 Tue Oct 06 15:36:06 PDT 2009 jsr166y/
 1096 Tue Oct 06 15:36:06 PDT 2009 jsr166y/ForkJoinPool$1.class
 912 Tue Oct 06 15:36:06 PDT 2009 jsr166y/ForkJoinPool$DefaultForkJoinWorkerThreadFactory.class
 305 Tue Oct 06 15:36:06 PDT 2009 jsr166y/ForkJoinPool$ForkJoinWorkerThreadFactory.class
 1101 Tue Oct 06 15:36:06 PDT 2009 jsr166y/ForkJoinPool$InvokeAll.class
 306 Tue Oct 06 15:36:06 PDT 2009 jsr166y/ForkJoinPool$ManagedBlocker.class
 1308 Tue Oct 06 15:36:06 PDT 2009 jsr166y/ForkJoinPool$WaitQueueNode.class
 28273 Tue Oct 06 15:36:06 PDT 2009 jsr166y/ForkJoinPool.class
 1096 Tue Oct 06 15:36:06 PDT 2009 jsr166y/ForkJoinTask$1.class

See it’s jsr166y/ForkJoinTask not jsr166y/forkjoin/ForkJoinTask

So I figured, the gparallelizer maybe was compiled against a different version. Two of the four top level demos ran, lets try Actors.

C:\Dev\gparallelizer\samples\actors>groovy DemoLoadBalancer.groovy
org.codehaus.groovy.control.MultipleCompilationErrorsException: startup failed,
General error during semantic analysis: Type ActorMessage not present

So that’s where I am right now.

Some Mac OS-X notes as well.

Most of the time I work on a MacBook Pro. All day I tinkered with gpars. On the Mac I did a GIT on src and tried to use the gradlew program. But the process hung at :compileTests and eventually (no matter how much heap I gave it with JAVA_OPTS and -XmxNNNNm) it would alway run out of heap space. I could get the assemble goal to run as it skips the tests.  Then I wound up with a 0.9-SNAPSHOT jar. But it acted strangely on the demos as well. For example the DemoMashupsWithMethods.groovy would printout “Starting download from XXXXX” and exit. Nothing else. Just exit. The Mac is running Apple’s latest 1.6 JVM and Groovy 1.6.5.

Now I know this is a pretty new item so I am not too worried. I hope maybe I can talk to folks about this at SpringOne 2GX. My use case is to break N sequential SOAP web service calls into N parallel calls joining the results. I also want to introduce groovy into production and this is finally a great use case if I can make it work. Simple, Groovy, and not introducing a whole new language which could meet with a lot more resistance (like Scala).  The point is to make more efficient use of the service tier.