This seems similiar to another bug some time ago. Back then the problem was AOBD. Some mod handels fluids the wrong way but it only crashes in combination with GT.
Thank you for the reply.
I tried disabling AOBD. Now at world creation, it didn't crash, but it did throw me back to the main menu.
Quitting the game didn't shutdown Java so it was still eating memory.
So I tried disabling DenseOres as I have read that in this thread as well, that it might help, and GT's own ore generation makes DenseOres superfluous.
With both AOBD and DenseOres disabled the world generation was successful and everything seems to work and playable.
Thank you very much for the help!
Time to mine.