Add a `max_heap_size` configuration option

[hansl]

Can be u64::MAX by default. If malloc() fails I expect the process to be aborted, but maybe if we reach the heap size we can throw instead (not sure what the spec says).