Clearer error message when LMDB has oom error (fix #517) #519

Merged
lx merged 1 commits from lmdb-oom-message into main 2023-03-06 10:49:04 +00:00
Owner
No description provided.
lx added 1 commit 2023-03-06 10:39:34 +00:00
continuous-integration/drone/pr Build is passing Details
continuous-integration/drone/push Build is passing Details
b8123fb6cd
Clearer error message when LMDB has oom error (fix #517)
lx merged commit 0d0906b066 into main 2023-03-06 10:49:04 +00:00
lx deleted branch lmdb-oom-message 2023-03-06 10:49:05 +00:00
First-time contributor

I think that error is still not sufficient. It should tell the user how much is the minium it has to allocate for it to function properly.

If it's not known how much it needs, there is no way to actually make proper limits that would allow it.

$ ulimit -v
67108864
$ ulimit -v unlimited
-bash: ulimit: virtual memory: cannot modify limit: Operation not permitted

If that virtual memory is not enough, then it would probably need optimization.

I think that error is still not sufficient. It should tell the user how much is the minium it has to allocate for it to function properly. If it's not known how much it needs, there is no way to actually make proper limits that would allow it. ``` $ ulimit -v 67108864 $ ulimit -v unlimited -bash: ulimit: virtual memory: cannot modify limit: Operation not permitted ``` If that virtual memory is not enough, then it would probably need optimization.
Author
Owner

If that virtual memory is not enough, then it would probably need optimization.

That's not how it works. LMDB needs to be able to map in RAM the entire file of the biggest possible database we want to support; concievably someone will someday use Garage with a metadata db bigger than 67GB. Currently the limit is set to 1TB (on 64-bit systems), we might raise it if needed but definitely not make it smaller. On 32-bit systems Garage tries to use a 1GB map size but it's wanky and LMDB should better be avoided on 32-bit systems.

> If that virtual memory is not enough, then it would probably need optimization. That's not how it works. LMDB needs to be able to map in RAM the entire file of the biggest possible database we want to support; concievably someone will someday use Garage with a metadata db bigger than 67GB. Currently the limit is set to 1TB (on 64-bit systems), we might raise it if needed but definitely not make it smaller. On 32-bit systems Garage tries to use a 1GB map size but it's wanky and LMDB should better be avoided on 32-bit systems.
Sign in to join this conversation.
No description provided.