Dynamic Buffers

This section introduces dynamic buffers—growable storage that adapts to data flow between producers and consumers.

Prerequisites

The Producer/Consumer Model

Dynamic buffers serve as intermediate storage between a producer (typically network I/O) and a consumer (your application code).

The flow:

  1. Producer writes data into the buffer

  2. Buffer accommodates data within its capacity

  3. Consumer reads and processes data

  4. Buffer releases consumed data

This model decouples production rate from consumption rate—the buffer absorbs variations.

The DynamicBuffer Concept

template<typename T>
concept DynamicBuffer = requires(T& t, T const& ct, std::size_t n) {
    typename T::const_buffers_type;
    typename T::mutable_buffers_type;

    // Producer side
    { t.prepare(n) } -> std::same_as<typename T::mutable_buffers_type>;
    { t.commit(n) };

    // Consumer side
    { ct.data() } -> std::same_as<typename T::const_buffers_type>;
    { t.consume(n) };

    // Capacity
    { ct.size() } -> std::convertible_to<std::size_t>;
    { ct.max_size() } -> std::convertible_to<std::size_t>;
    { ct.capacity() } -> std::convertible_to<std::size_t>;
};

A dynamic buffer, within its capacity provides a potentially empty readable region and a potentially empty writeable region. Sizes of these regions change as operations are invoked on the dynamic buffer.

Producer Interface

prepare(n)

Returns a mutable buffer sequence mb for writing up to n bytes:

capy::MutableBufferSequence auto buffers = dynamic_buf.prepare(1024);  // Space for up to 1024 bytes

Effects: If n exceeds the available capacity, throws. Otherwise, ensures that a writeable region of size at least n exists and returns an object of type mutable_buffers_type representing this region.

Throws: The concrete adapters throw std::invalid_argument when n exceeds the available capacity; circular_dynamic_buffer throws std::length_error. std::bad_alloc may also be thrown if an allocation is performed and fails.

Postcondition: buffer_size(buffers) >= n.

commit(n)

Transfers n bytes of from the beginning of the writeable storage to the end of the readable storage:

// After writing data:
dynamic_buf.commit(bytes_written);
// Data is now visible via data()

Let n1 be the smaller number of n and the size of the writeable region.

Effects: Removes n1 bytes from the front of the writeable region and adds them at the back of the readable region.

Notes: n can be smaller than the writeable region size.

Typical Producer Pattern

task<> read_into_buffer(Stream& stream, DynamicBuffer auto& buffer)
{
    // Prepare space
    auto space = buffer.prepare(1024);

    // Read into prepared space
    auto [ec, n] = co_await stream.read_some(space);

    buffer.commit(n);  // Make data readable
}

Consumer Interface

data()

Returns a const buffer sequence representing the readable region.

capy::ConstBufferSequence auto readable = dynamic_buf.data();
// Process readable bytes

Postcondition: capy::buffer_size(readable) == dynamic_buf.size().

consume(n)

Removes n bytes from the front of readable data:

dynamic_buf.consume(processed_bytes);
// Those bytes are no longer in data()

Let n1 be the smaller of n and size().

Effects: Removes n1 bytes from the front of the readable region. Sets the size of the writeable rgion to zero. Invalidates all buffer sequences previously obtained via calls to data() or `prepare().

Typical Consumer Pattern

void process_buffer(DynamicBuffer auto& buffer)
{
    auto data = buffer.data();

    while (buffer_size(data) >= message_header_size)
    {
        auto msg_size = parse_header(data);
        if (buffer_size(data) < msg_size)
            break;  // Need more data

        process_message(data, msg_size);
        buffer.consume(msg_size);
        data = buffer.data();  // Refresh after consume
    }
}

Capacity Management

size()

Returns: The size of the readable region.

max_size()

Returns: The maximum possible sum o sizes of the readable region and the writeable region.

capacity()

Returns: Current allocated capacity.

Class invariant

size() <= capacity().

capacity() <= max_size().

DynamicBufferParam

When passing dynamic buffers to coroutines, use DynamicBufferParam for safe parameter handling:

template<typename DB>
concept DynamicBufferParam = DynamicBuffer<std::remove_reference_t<DB>>;

template<DynamicBufferParam Buf>
task<std::size_t> read_until(Stream& stream, Buf&& buffer, char delimiter);

This concept ensures proper handling of lvalues and rvalues, preventing dangling references across suspension points.

Provided Implementations

flat_dynamic_buffer

A fixed-capacity adapter over caller-owned contiguous storage, with single-buffer sequences. It never reallocates: the capacity is fixed at construction, and a default-constructed buffer has zero capacity (so prepare would throw).

#include <boost/capy/buffers/flat_dynamic_buffer.hpp>

char storage[1024];
flat_dynamic_buffer buffer(storage, sizeof(storage));
auto space = buffer.prepare(256);
// ... write data ...
buffer.commit(n);

// data() returns a single const_buffer

Advantages:

  • Contiguous memory—good for parsing that needs contiguous data

  • Cache-friendly

Disadvantages:

  • Fixed capacity; prepare throws when more space is requested than remains

circular_dynamic_buffer

Ring buffer implementation:

#include <boost/capy/buffers/circular_dynamic_buffer.hpp>

char storage[1024];
circular_dynamic_buffer buffer(storage, sizeof(storage));  // Fixed capacity

Advantages:

  • No copying on wrap—head/tail pointers move

  • Fixed memory footprint

Disadvantages:

  • data() may return two buffers (wrapped around end)

  • Fixed capacity

vector_dynamic_buffer

Backed by a caller-owned std::vector of byte-sized elements (vector_dynamic_buffer itself uses unsigned char). Use the dynamic_buffer factory, which deduces the element type:

#include <boost/capy/buffers/vector_dynamic_buffer.hpp>

std::vector<unsigned char> storage;
auto buffer = dynamic_buffer(storage);

Adapts an existing vector for use as a dynamic buffer. The vector must outlive the adapter.

string_dynamic_buffer

Backed by std::string:

#include <boost/capy/buffers/string_dynamic_buffer.hpp>

std::string storage;
auto buffer = dynamic_buffer(storage);

Useful when you want the final data as a string. The dynamic_buffer factory wraps the string (you may also construct string_dynamic_buffer(&storage) directly). The string must outlive the adapter.

Example: Line-Based Protocol

task<std::string> read_line(Stream& stream)
{
    char storage[4096];
    flat_dynamic_buffer buffer(storage, sizeof(storage));

    while (true)
    {
        // Prepare space and read
        auto space = buffer.prepare(256);
        auto [ec, n] = co_await stream.read_some(space);
        buffer.commit(n);
        if (ec)
            throw std::system_error(ec);

        // Search for newline in readable data
        auto data = buffer.data();
        std::string_view sv(
            static_cast<char const*>(data.data()), data.size());

        auto pos = sv.find('\n');
        if (pos != std::string_view::npos)
        {
            std::string line(sv.substr(0, pos));
            buffer.consume(pos + 1);  // Include newline
            co_return line;
        }
    }
}

Reference

Header Description

<boost/capy/concept/dynamic_buffer.hpp>

DynamicBuffer concept definition

<boost/capy/buffers/flat_dynamic_buffer.hpp>

Linear dynamic buffer

<boost/capy/buffers/circular_dynamic_buffer.hpp>

Ring buffer implementation

<boost/capy/buffers/vector_dynamic_buffer.hpp>

Vector-backed adapter

<boost/capy/buffers/string_dynamic_buffer.hpp>

String-backed adapter

You have now learned about dynamic buffers for producer/consumer patterns. This completes the Buffer Sequences section. Continue to Stream Concepts to learn about Capy’s stream abstractions.

Edit this Page