How JVM Settings Affect the Flex Compiler Performance

September 29, 2009 at 4:48 pm · Filed under Flex Compiler API, HellFire Compiler, Technology, hfcd ·Tagged , , , , , · Edit

Many of you who build medium to large-sized Flex projects with Flex Builder know that you need to modify the memory settings of the JVM in FlexBuilder.ini. For example, my JVM settings in FlexBuilder.ini look like this:

-Xms512m -Xmx1024m -XX:MaxPermSize=256m -XX:PermSize=64m

Other than having bigger -Xms and -Xmx values, it’s exactly the same as the default setting. The Flex Builder team did not arbitrarily set these values. They did not set the values for the compiler only. They set the values so that all the features in FB achieve optimal performance.

One of the arguments of using HFCD is that if you run the compiler out-of-process, you get a chance to configure the JVM for the compiler only. The HFCD installation only puts in the following default setting:

-Xms512m -Xmx1024m -XX:+UseParNewGC

I take the minimalist approach here because I want to give the HotSpot VM maximum flexibility to do its own tuning at runtime. So, does this HFCD out-of-process-compilation argument make sense? Well, let’s take a look at using Flex Builder to compile the Flex framework (a 800-file project).

My machine is a MacBookPro with dual-core 2.4GHz and 2GB of memory. The default Java setting is Java 6 64-bit.

Flex Builder spends consistently (in several full builds) about 15 seconds to compile the Flex framework in-memory. When Flex Builder uses HFCD to compile the Flex framework out-of-process (throw away the first warm-up build), HFCD initially compiles in 15 seconds and then gradually improves to about 11 seconds. That’s 25% faster! The following is my HFCD log. Note that the time is in milliseconds and build time = compile() + link()…

[Tue Sep 29 15:08:53 EDT 2009] compile() result=1 time=14314
[Tue Sep 29 15:09:07 EDT 2009] link() size=1515458 time=543
[Tue Sep 29 15:09:26 EDT 2009] compile() result=1 time=12243
[Tue Sep 29 15:09:38 EDT 2009] link() size=1515449 time=490
[Tue Sep 29 15:09:46 EDT 2009] compile() result=1 time=11215
[Tue Sep 29 15:09:57 EDT 2009] link() size=1515458 time=587
[Tue Sep 29 15:10:12 EDT 2009] compile() result=1 time=11694
[Tue Sep 29 15:10:24 EDT 2009] link() size=1515446 time=441
[Tue Sep 29 15:10:50 EDT 2009] compile() result=1 time=10842
[Tue Sep 29 15:11:01 EDT 2009] link() size=1515453 time=522

In my opinion, the performance improvement is not framework-specific, but likely for mid-to-large-sized projects in general.

HFCD Installation and Configuration

September 29, 2009 at 8:19 am

UPDATE: The instructions here also applies to hfcd_4.zip. You can skip the instructions here if you simply use the installer (currently available for Mac OS X Leopard and up).

As promised, I’m going to talk about how to install and configure the HellFire Compiler Daemon, a.k.a. HFCD. Again, if you’ve never heard of HFCD before, please check out the following posts:

  1. The Flex Compiler API… RPC Style
  2. A New Version of HFCD
  3. HFCD Beta Is Now Available

Before you download HFCD, make sure you meet the following system requirements:

  • Java 5 installed (Java 6 highly recommended). BTW, Flex Builder ships with JRE 5. It’s okay to use that, but the point of using HFCD is build Flex apps faster… and Java 6 is *faster* than Java 5…
  • Apache Ant installed. Mac OS X ships with Ant. If you use Windows, go download it at http://ant.apache.org.

Now, go to http://stopcoding.org/. Register and download hfcd_3.zip. The installation consists of 2 parts:

  1. Install the HFCD server and client SDK. In this step, you need hfcd_3.zip.
  2. Install the HFCD for Flex Builder plugin. In this step, you install the plugin via the Eclipse/Flex Builder Update Manager.

It’s highly recommended that you install the HFCD server and client SDK first. Once you’ve downloaded hfcd_3.zip. Unzip it in a directory called hfcd_3. Run ant.

> mkdir hfcd_3
> cp hfcd_3.zip hfcd_3
> cd hfcd_3
> unzip hfcd_3.zip

> ant server client

The ant tasks ’server’ and ‘client’ install the HFCD server and client SDK in the hfcd_3 directory. The tasks download the Flex 3.3.0 SDK automatically from Adobe (For hfcd_4, the tasks copy the Flex 4.0.0 SDK from the Flash Builder home directory). Because of that, the tasks may take a while to finish.

If you intend to run the HFCD server on a second machine, you only need to run ‘ant client’ on your Flex Builder machine. Then, download hfcd_3.zip to your second machine and run ‘ant server’ there. Alternatively, you run 'ant client server' on your Flex Builder machine and then simply copy the server directory over to your second machine.

After the ant tasks, please check out the JVM settings for HFCD: Open the ‘hfcd’ script (Mac OS X/Linux) or ‘jvm.config’ (Windows). The ant tasks modify the settings to something like this:

-XX:+UseParNewGC -Xms512m -Xmx1024m

You may change the -Xms and -Xmx settings. You may also change the parallel Young Generation GC to the parallel GC (-XX:+UseParallelGC). I would not add more VM settings until I see it in action. More on tuning later. If you’re happy with the default setting, go ahead and start hfcd. The HFCD server, by default, listen on port 50207 and 50208. Please make sure your firewall setting is properly configured for this.

> cd hfcd_3/server/bin
> hfcd
HellFire Compiler Daemon (hfcd) version 3

Now, you’re ready to install the HFCD for Flex Builder plugin. Note that you do NOT need to install the plugin twice for both HFCD 3 and HFCD 4.

  • Launch Flex Builder.
  • If you have a previous version of HFCD installed, click ‘Help’ –> ‘Software Updates’ –> ‘Manage Configuration’ to uninstall.
  • If you’ve never used HFCD before, click ‘Help’ –> ‘Software Updates’ –> ‘Find and Install…’.
  • Click ‘Search for new features to install’.
  • Click ‘New Remote Site…’.
  • Enter http://stopcoding.org/hfcd/3/download (or http://stopcoding.org/hfcd/4/download for HFCD 4) as the update site. You may use the URL as the name.
  • Click ‘Next’.
  • Choose the ‘HellFire Compiler 3.4.1′ feature.
  • Accept the license agreement.
  • Click ‘Next’, ‘Finish’ or whatever comes out. Click ‘Install All’.
  • Restart Flex Builder.

Now, it’s time to configure Flex Builder to use HFCD. There are three pieces in the configuration:

  1. Configure Flex Builder to use the HFCD client SDK.
  2. Change the HFCD hostname.
  3. Sync up source files with HFCD.

To configure Flex Builder to use the HFCD client SDK:

  • Click ‘Preferences…’ –> ‘Flex’ –> ‘Installed Flex SDKs’.
  • Add and choose the hfcd_3/client directory and set this SDK as the default SDK for Flex Builder.

To change the HFCD hostname that Flex Builder connects to:

  • Click ‘Preferences…’ –> ‘HellFire Compiler’.
  • If you intend to run HFCD locally, do nothing. Otherwise, change the hostname setting. You can use either hostname or IP address.
  • Restart Flex Builder.

To sync up your source files with HFCD:

  • Close your Flex projects in your Flex Builder workspace.
  • Reopen them.
  • Clean and Build All.

If everything is fine, you should see output from HFCD.

In the next few posts, I’m going to talk more about HFCD… about what you should expect from using HFCD.

HFCD Beta Is Now Available

September 28, 2009 at 2:54 pm

I am pleased to announce that HFCD beta is now available for download. As I mentioned in my previous post, this is a much improved version, with a few key features added to make it a valuable tool for Flex developers. I am pretty excited about this but I need your help. If you are a Flex Builder user, please download HFCD and give it a try. I'd like to get as many bugs fixed as possible in the following weeks. This beta is good until the end of October, but I will likely have weekly updates until it's stabilized.

Before you download HFCD, make sure you know what it is. HFCD allows for Flex Builder to compile Flex applications out-of-process. This usually results in shorter build time. You can run HFCD and Flex Builder on the same machine or you can run HFCD on a second machine. Running HFCD on a second machine (as long as it's faster than the Flex Builder machine) is highly recommended.

Be sure to install the HFCD client and server SDK before installing the Flex Builder plugin. There is a short README in the zip archive and I promise to get the installation instructions online as soon as possible.

When you register, please provide accurate contact info. I will give away some licenses when this goes final. Also, if you like HFCD, please tell your coworkers and your friends. I believe all Flex Builder users should give this a try.

Thanks and happy coding!

HFCD 3: http://stopcoding.org/hfcd/3/download

HFCD 4: http://stopcoding.org/hfcd/4/download

A New Version of HFCD

September 25, 2009 at 2:40 pm

Recently I picked up the hfcd project again and finally added features that I promised last year. Yes, it’s a new version of hfcd. This morning, I’ve finished testing what I got so far. It’s in beta quality, but I think I should release it so you folks can give it a spin and give me some feedback.

Let me give a brief intro of what hfcd is. HellFire Compiler Daemon is a RPC-style Flex Compiler server. Clients use the Flex Compiler API to communicate with hfcd. Basically, client processes use hfcd to compile Flex applications. The most obvious client is Flex Builder 3. FB3, by using the hfcd client SDK, can offload compilations to hfcd so that its memory and CPU resources can be reserved for other FB3 popular tasks: design view, code model, profiling, just to name a few. The result is dramatic. hfcd compiles Flex applications much faster in most cases than compiling apps in-memory with FB3.

The hfcd version I released last year was an proof-of-concept attempt. One limitation was that it had to run locally with FB3 (i.e. on the same machine). Now, with the new features recently added to hfcd, it’s able to run on a remote machine. This implies that if the remote machine has more memory and faster processors, the build time could be dramatically reduced. This is certainly one option of improving the compiler performance.

Furthermore, I added the following features:

  1. hfcd supports a virtual file system that holds source files in memory. It means your hfcd server machine will never be littered with codes from many different sources.
  2. hfcd supports background compilations, both full and incremental. This is a game changer. Why? Developers usually edit multiple files before they build (Ctrl-B) and test. As long as developers save changes continuously (instead of keeping all the changes to the one-and-only Ctrl-S), the background compilation could kick in and compile some changes while developers are making the rest of the changes. The result, developers wait much shorter incremental build time after the last Ctrl-S and Ctrl-B.
  3. hfcd takes advantage of multi-processor-core technology wherever possible. It attempts to build applications and libraries that have no mutual dependencies simultaneously. For example, if you have 2 SWC projects and 3 SWF projects. The SWC projects don’t depend on each other. With FB3, your build time would be: T(SWC1 + SWC2 + SWF1 + SWF2 + SWF3). With FB3 + hfcd, your build time would be roughly: T(max(SWC1, SWC2) + max(SWF1, SWF2, SWF3))! Another game changer!

The currently supported SDK version is 3.3.0.4852. I hope to add support for 3.4.0 in the near term.

In the next few weeks, I need some developers to test my first beta. If you are interested, please email hfcd@stopcoding.org. This weekend, I hope to get a web page ready so you folks can register and download there.