Game++. Cooking vectors (part 2)
Hybrid vector
The hybrid_vector is essentially a static_vector that uses internal storage when element counts stay below a threshold, only allocating dynamically when that limit is exceeded.
The concept draws an analogy to SSO (Short String Optimization), attributed to Alexandrescu. The idea involves reusing class member variables (pointer, size, capacity) as an inline byte buffer for short strings — "on a 64-bit system, a pointer alone can hold up to 8 characters without allocating memory."
Consider an NPC weapons example: most NPCs carry ≤2 weapon types, but assault units may carry a third. Using a hybrid_vector means 90% of NPCs store data internally with no heap allocation, while the remaining 10% get a dynamic block. The tradeoff is some unused capacity in the common case, offset by faster access and fewer allocations.
When hybrid_vector doesn't help
- No reasonable small N — bloating class data has its own cost
- Typical size much smaller than N — wastes internal buffer space, though monitoring allocations is easier than with standard containers since "you can always attach custom metrics or counters"
- Large element sizes — data locality benefits are diluted by increased memory footprint, yielding "speeds comparable to using heap-allocated arrays"
Hardware matters: on older mobile devices/consoles, stack-local gains were noticeable up to ~512 bytes, degrading around 16 KB. Modern processors show less difference as long as stack size suffices.
C++17 alternative with std::pmr
C++17 introduced std::pmr as a way to emulate a hybrid_vector:
char buffer[64] = {}; // a small buffer on the stack
std::pmr::monotonic_buffer_resource pool{std::data(buffer), std::size(buffer)};
std::pmr::vector<char> vec{ &pool };
There's a conceptual spectrum from vector to array where "performance improves at the cost of reduced flexibility." The vector is often used as a "convenient but thoughtless default" and "not all developers even understand how std::vector works under the hood."