Manage overall memory usage from libscope

[iapaddler]

We want to enable users to manage the amount of memory utilized by libscope. The largest memory consumer are the ring buffers. We may want to start there.

Issue #93 requests control over buffers used for console data. We will close that issue in place of this as the concern is management of overall memory and not just console data.

[michalbiesek]

A couple of my thoughts:

• to control overall memory usage we must know how much memory we actually used by the libscope and distinguish it from the memory used by the scoped application
• for heap - we can count allocated memory - an additional layer on malloc,realloc,calloc,free,strdup,mmap,munmap... called on the libscope side, where we have a counter of currently allocated memory
• for bss and data segment we know this after compilation e.g. size libscope.so
• getrusage ??
• read /proc/<pid>/smaps ?
• What action can we do to ensure that we don't hit the limit:
• limit the size of circular buffer - which is defined at the constructor time
• evict/flush the buffer ?

[iapaddler]

After a detailed design discussion, it becomes clear that the overall goal of enabling a user to describe a memory limit requires a bit of research. We need to establish a viable understanding of how to measure the memory used by the library and validate what we currently think the response should look like.

Therefore, we propose:

1. as a first step, we will merge PR 689 to provide user config of the number of queues used for metrics and events
2. we will close this issue
3. we will create a new issue that represent a spike enabling us to obtain the detail needed (ref issue Spike: Determine how to measure viable memory usage and effective steps to take #715)
4. once the spike is complete we will create an issue to define the steps needed to enable memory usage controls

[jrcheli]

I'm closing this ticket per @iapaddler's notes above. PR #689 (actually issue #93) has been added to our project board, and #715 has been created to run a spike to figure out what's next.