$GLASSFISH_HOME/domains/domain1/config/domain.xml, search for XML element name
jvm-options. For example,
<java-config suffix="..."> <jvm-options>-Xmx512m</jvm-options> <jvm-options>-XX:NewRatio=2</jvm-options> <jvm-options>-XX:MaxPermSize=128m</jvm-options> ...</java-config>
You can also change these settings in the web-based admin console, typically at http://localhost:4848/, or https://localhost:4848/. Go to Application Server near the top of the left panel, and then on the right panel, click JVM Settings | JVM Options, and you will see a list of existing JVM options. You can add new ones and modify existing ones there.
Yet another option is to use its Command Line Interface (CLI) tool command, such as
./asadmin help create-jvm-options ./asadmin help delete-jvm-options
They may be a bit hard to use manually, but are well suited for automated scripts.