Infinity: Easier Than You Think. Infinitely.

Spaghettified musings on software

  • June 2010
    S M T W T F S
    « May    
     12345
    6789101112
    13141516171819
    20212223242526
    27282930  

Article Link About Batch Importing with GRAILS

Posted by gdjsky01 on 2010/06/06

Ted Naleid has a nice write-up on his optimizations of a database batch loading process using GRAILS. It’s a few months old but if I can help spread the information, so much the better. I do tend to agree with the comment that GRAILS might not be the correct solution for bulk loading of records into a database. Still, I see Ted’s solution as quick, easy, and works just fine for the size of the import he has. So why not?

I certainly would not have the same GRAILS application instance that is doing the bulk upload also taking normal traffic. That seem like a recipe for problems. Otherwise it is a good read with a decent solution. Enough so I thought to write a short post here. 😉

Were I to go beyond Ted’s volume, I’d look at a dedicated GRAILS app using the same domain objects or perhaps using GORM outside of GRAILS (if that is still possible). In some ways I see this as a good justification (or rationalization) for keeping Hibernate mappings separate, versioned, and in (gasp) XML. You can then use them in GRAILS or in any other code that can link to Hibernate.  Most of my domains predate GRAILS and GORM so mapping them in GORM is not straight forward. That costs me some of the GORM goodness to begin with. Of course I have the XML mappings already so the choice is easy to make. That’s the cool thing about GRAILS, you can use your legacy domain objects and/or mappings. For validations there is always Hibernate Validator.

Reference: http://naleid.com/blog/2009/10/01/batch-import-performance-with-grails-and-mysql/

Advertisements

One Response to “Article Link About Batch Importing with GRAILS”

  1. Ted Naleid said

    Thanks for pointing people to my post, I’m glad you found it useful.

    I agree with you about not wanting this to be in your main grails application that’s also experiencing additional traffic. I do something similar to what you propose but keep it in grails by using plugins.

    I have a regular transactional application that handles normal traffic and a second batch application that has quartz jobs and can process small to medium batch files (up to a few hundred thousand records). They both share a “domain” plugin that contains just my GORM domain model (plus one or two other shared things).

    Using the domain plugin as a local plugin (see “Specifying Plugin Locations” in the grails docs) you get all the same behavior as if the domain classes were right in each of the applications while coding it.

    In your case, you’ve already got XML mappings, but you could likely share them in a similar manner across applications if you put them in a local plugin.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

 
%d bloggers like this: