StreamCompression.h File Reference

Convenience wrappers to using Blosc and reading and writing of Paged data. More...

#include <openvdb/io/io.h>
#include <openvdb/util/Assert.h>
#include <tbb/spin_mutex.h>
#include <memory>
#include <string>

Go to the source code of this file.

Classes
class	Page
	Stores a variable-size, compressed, delayed-load Page of data that is loaded into memory when accessed. Access to the Page is thread-safe as loading and decompressing the data is protected by a mutex. More...
class	PageHandle
	A PageHandle holds a unique ptr to a Page and a specific stream pointer to a point within the decompressed Page buffer. More...
class	PagedInputStream
	A Paging wrapper to std::istream that is responsible for reading from a given input stream and creating Page objects and PageHandles that reference those pages for delayed reading. More...
class	PagedOutputStream
	A Paging wrapper to std::ostream that is responsible for writing from a given output stream at intervals set by the PageSize. As Pages are variable in size, they are flushed to disk as soon as sufficiently large. More...

Namespaces
	openvdb
	openvdb::v12_0
	openvdb::v12_0::compression

Functions
OPENVDB_API bool	bloscCanCompress ()
	Returns true if compression is available. More...
OPENVDB_API size_t	bloscUncompressedSize (const char *buffer)
	Retrieves the uncompressed size of buffer when uncompressed. More...
OPENVDB_API void	bloscCompress (char *compressedBuffer, size_t &compressedBytes, const size_t bufferBytes, const char *uncompressedBuffer, const size_t uncompressedBytes)
	Compress into the supplied buffer. More...
OPENVDB_API std::unique_ptr< char[]>	bloscCompress (const char *buffer, const size_t uncompressedBytes, size_t &compressedBytes, const bool resize=true)
	Compress and return the heap-allocated compressed buffer. More...
OPENVDB_API size_t	bloscCompressedSize (const char *buffer, const size_t uncompressedBytes)
	Convenience wrapper to retrieve the compressed size of buffer when compressed. More...
OPENVDB_API void	bloscDecompress (char *uncompressedBuffer, const size_t expectedBytes, const size_t bufferBytes, const char *compressedBuffer)
	Decompress into the supplied buffer. Will throw if decompression fails or uncompressed buffer has insufficient space in which to decompress. More...
OPENVDB_API std::unique_ptr< char[]>	bloscDecompress (const char *buffer, const size_t expectedBytes, const bool resize=true)
	Decompress and return the the heap-allocated uncompressed buffer. More...

Variables
static const int	BLOSC_MINIMUM_BYTES = 48
static const int	BLOSC_PAD_BYTES = 128
static const int	PageSize = 1024 * 1024

Detailed Description

Convenience wrappers to using Blosc and reading and writing of Paged data.

Author

Dan Bailey

Blosc is most effective with large (> ~256KB) blocks of data. Writing the entire data block contiguously would provide the most optimal compression, however would limit the ability to use delayed-loading as the whole block would be required to be loaded from disk at once. To balance these two competing factors, Paging is used to write out blocks of data that are a reasonable size for Blosc. These Pages are loaded lazily, tracking the input stream pointers and creating Handles that reference portions of the buffer. When the Page buffer is accessed, the data will be read from the stream.