atom feed9 messages in org.codehaus.gradle.dev[gradle-dev] Improving startup time
FromSent OnAttachments
Adam MurdochJul 25, 2010 10:51 pm 
Tomek KaczanowskiJul 25, 2010 11:36 pm 
Hans DockterJul 26, 2010 12:36 am 
Hans DockterJul 26, 2010 12:50 am 
Hans DockterJul 26, 2010 1:15 am 
Adam MurdochJul 26, 2010 2:55 am 
Pfau, MatthiasJul 29, 2010 1:51 am 
Philip CrotwellAug 18, 2010 12:01 pm 
Jason PorterAug 18, 2010 12:12 pm 
Subject:[gradle-dev] Improving startup time
From:Adam Murdoch (ad@gradle.biz)
Date:Jul 25, 2010 10:51:22 pm
List:org.codehaus.gradle.dev

Hi,

I've been doing some quick experiments with an approach to improve the startup time of Gradle. The approach involves having the 'gradle' command fork a build daemon process, which then does the actual build. The daemon process is reused by subsequent invocations of 'gradle'. This way, we avoid the startup time of Groovy, Gradle (and Scala, too, if you're using it), loading the script classes and so on. Plus we get the hotspot compiler kicking in.

It seems like this approach might have some potential. Here are some numbers comparing the time to run gradle -t for Gradle's build: * gradle daemon 1.22s * gradle trunk 4.8s

I thought I'd also see how we compare with Gradle 0.8, Ant 1.8.0, Maven 3.0-beta1, maven 2.2.1 and maven shell 0.10:

1. Using our single project benchmark build: - 1000 source and test classes - Test reports disabled, no parallel test execution

to do a clean build (or clean package), from fastest to slowest: * gradle daemon 4.12s * mvnsh 6.18s * ant 6.96s * mvn2 8.08s * mvn3 8.87s * gradle0.8 9.93s * gradle trunk 10.54s

to do a build (or package) when everything has already been built, from fastest to slowest: * gradle daemon 0.82s * mvnsh 2.52s * gradle trunk 3.78s * ant 3.97s * mvn2 4.27s * mvn3 5.17s * gradle0.8 7.74

2. Using our multi-project benchmark build - 25 projects - 100 source and test classes - Test reports disabled, no parallel test execution

clean build, fastest to slowest: * ant 14.77s * gradle0.8 28.58s * gradle daemon 30.24s * gradle trunk 35.33s * mvnsh 40.15s * mvn3 41.99s

build when everything up-to-date, fastest to slowest: * gradle daemon 2.5s * gradle trunk 5.84s * ant 10.12s * mvnsh 14.68s * mvn3 22.85s * gradle0.8 24.68s

There are also a lot of interesting things we can do in the future if we have a long running process to do them in. Here are some initial possibilities: - check for updates to snapshots and download in the background, so that they are ready for the next build (even better if the repository managers could push changes to us). - configure the project model, build buildSrc, compile scripts, etc in the background - garbage collect all the various caches we leave lying around on the file system - use file system change notifications (java 7 and/or native notifiers) to watch for out-of-date source files

It's also a step towards providing remote and/or distributed builds and testing.

I've added a prototype to the 'build_daemon' branch. Please try it out if you want. You just run the 'gradle' command as per normal, and the daemon is automatically started as needed. There are a couple of command-line options if you want to control this:

gradle --stop stops the daemon gradle --nodaemon runs the build without using the daemon gradle --foreground runs the daemon, in the foreground. For debugging.

It generally works fine. There are a few broken things: - exit code from 'gradle' is always 0 - error handling is pretty rough - the console stuff (test counters, etc) don't work - stderr and stdout are merged

Once 0.9 is out, I'd like to merge this into a 0.9.1 release.