xgdsplot program allows you to define in the Configuration menu the amount of RAM to use for loading the GDSII file and for the rasterizing process. The maximum value the user should put is 2000(MB). When putting a value bigger than that, you will get an error message during the rasterizing process which will look like this:
Flattening -8999%

In that case, you should stop the process and lower the RAM settings in the Configuration menu to 2000 or less.