1.. _module-pw_blob_store: 2 3============= 4pw_blob_store 5============= 6.. pigweed-module:: 7 :name: pw_blob_store 8 9``pw_blob_store`` is a storage container library for storing a single blob of 10data. ``BlobStore`` is a flash-backed persistent storage system with integrated 11data integrity checking that serves as a lightweight alternative to a file 12system. 13 14----- 15Usage 16----- 17Most operations on a ``BlobStore`` are done using ``BlobReader`` and 18``BlobWriter`` objects that have been constructed using a ``BlobStore``. Though 19a ``BlobStore`` may have multiple open ``BlobReader`` objects, no other 20readers/writers may be active if a ``BlobWriter`` is opened on a blob store. 21 22The data state of a blob can be checked using the ``HasData()`` method. 23The method returns true if the blob is currenty valid and has at least one data 24byte. This allows checking if a blob has stored data without needing to 25instantiate and open a reader or writer. 26 27Write buffer 28============ 29 30BlobStore uses a write buffer to allow writes smaller than and/or unaligned to 31the flash write aligment. BlobStore also supports using the write buffer for 32deferred writes that can be enqueued and written to flash at a later time or by 33a different thread/context. 34 35BlobStore can be used with a zero-size write buffer to reduce memory 36requirements. When using zero-size write buffer, the user is required to write 37maintain write sizes that are a multiple of the flash write size the blob is 38configured for. 39 40If a non-zero sized write buffer is used, the write buffer size must be a 41multiple of the flash write size. 42 43Writing to a BlobStore 44---------------------- 45``BlobWriter`` objects are ``pw::stream::Writer`` compatible, but do not support 46reading any of the blob's contents. Opening a ``BlobWriter`` on a ``BlobStore`` 47that contains data will discard any existing data if ``Discard()``, ``Write 48()``, or ``Erase()`` are called. Partially written blobs can be resumed using 49``Resume()``. There is currently no mechanism to allow appending to completed 50data write. 51 52.. code-block:: cpp 53 54 BlobStore::BlobWriterWithBuffer writer(my_blob_store); 55 writer.Open(); 56 writer.Write(my_data); 57 58 // ... 59 60 // A close is implied when a BlobWriter is destroyed. Manually closing a 61 // BlobWriter enables error handling on Close() failure. 62 writer.Close(); 63 64Resuming a BlobStore write 65-------------------------- 66``BlobWriter::Resume()`` supports resuming writes for blobs that have not been 67completed (``writer.Close()``). Supported resume situations are after ``Abandon()`` 68has been called on a write or after a crash/reboot with a fresh ``BlobStore`` 69instance. 70 71``Resume()`` opens the ``BlobWriter`` at the most recent safe resume point. 72``Resume()`` finds the furthest written point in the flash partition and then backs 73up by erasing any partially written sector plus a full sector. This backing up is to 74try to avoid any corrupted or otherwise wrong data that might have resulted from the 75previous write failing. ``Resume()`` returns the current number of valid bytes in the 76resumed write. 77 78If the blob is using a ChecksumAlgorithm, the checksum of the resumed blob write instance 79calculated from the content of the already written data. If it is desired to check the 80integrity of the already written data, ``BlobWriter::CurrentChecksum()`` can be used to 81check against the incoming data. 82 83Once ``Resume()`` has successfully completed, the writer is ready to continue writing 84as normal. 85 86Erasing a BlobStore 87=================== 88There are two distinctly different mechanisms to "erase" the contents of a BlobStore: 89 90#. ``Discard()``: Discards any ongoing writes and ensures ``BlobReader`` objects 91 see the ``BlobStore`` as empty. This is the fastest way to logically erase a 92 ``BlobStore``. 93#. ``Erase()``: Performs an explicit flash erase of the ``BlobStore``'s 94 underlying partition. This is useful for manually controlling when a flash 95 erase is performed before a ``BlobWriter`` starts to write data (as flash 96 erase operations may be time-consuming). 97 98Naming a BlobStore's contents 99============================= 100Data in a ``BlobStore`` May be named similarly to a file. This enables 101identification of a BlobStore's contents in cases where different data may be 102stored to a shared blob store. This requires an additional RAM buffer that can 103be used to encode the BlobStore's KVS metadata entry. Calling 104``MaxFileNameLength()`` on a ``BlobWriter`` will provide the max file name 105length based on the ``BlobWriter``'s metadata encode buffer size. 106 107``SetFileName()`` performs a copy of the provided file name, meaning it's safe 108for the ``std::string_view`` to be invalidated after the function returns. 109 110.. code-block:: cpp 111 112 constexpr size_t kMaxFileNameLength = 48; 113 BlobStore::BlobWriterWithBuffer<kMaxFileNameLength> writer(my_blob_store); 114 writer.Open(); 115 writer.SetFileName("stonks.jpg"); 116 writer.Write(my_data); 117 // ... 118 writer.Close(); 119 120Reading from a BlobStore 121------------------------ 122A ``BlobStore`` may have multiple open ``BlobReader`` objects. No other 123readers/writers may be open/active if a ``BlobWriter`` is opened on a blob 124store. 125 1260) Create BlobReader instance 1271) BlobReader::Open() 1282) Read data using BlobReader::Read() or 129 BlobReader::GetMemoryMappedBlob(). BlobReader is seekable. Use 130 BlobReader::Seek() to read from a desired offset. 1313) BlobReader::Close() 132 133-------------------------- 134FileSystem RPC integration 135-------------------------- 136``pw_blob_store`` provides an optional ``FileSystemEntry`` implementation for 137use with ``pw_file``'s ``FlatFileSystemService``. This simplifies the process of 138enumerating ``BlobStore`` objects as files via ``pw_file``'s ``FileSystem`` RPC 139service. 140 141----------- 142Size report 143----------- 144The following size report showcases the memory usage of the blob store. 145 146.. include:: blob_size 147 148.. note:: 149 The documentation for this module is currently incomplete. 150
