How to increase memory allocation on Win Server 2012?

I have YouTrack 5.0.5 running as a service on Windows Server 2012, and I'm trying to increase the memory allocation. I followed the directions here:

<http://confluence.jetbrains.com/display/YTD5/Windows.+Installing+and+Running+YouTrack+as+a+Service>

I set the max memory pool value to 8192, but the service fails to start, and these entries are recorded in the jakarta_service log:

[2014-02-07 12:06:34] [info] Invalid maximum heap size: -Xmx8192m
[2014-02-07 12:06:34] [info] The specified size exceeds the maximum representable size.

In trying to find the max value by trying successively lower values, I eventually found that at values around 1100, it usually works but sometimes fails with these entries in the log:

[2014-02-07 12:44:10] [info] Error occurred during initialization of VM
[2014-02-07 12:44:10] [info] Could not reserve enough space for object heap

I managed to get it to start at 1500, but I really want to go to about 8 GiB. The machine has 12 GiB of RAM, of which only 1.7 GiB are being used, according to Task Manager.

How do I get YouTrack to start up properly with the larger allocation?