Lines Matching full:compression
76 zstd, short for Zstandard, is a fast lossless compression algorithm, targeting
77 real-time compression scenarios at zlib-level and better compression ratios.
78 The zstd compression library provides in-memory compression and decompression
81 The library supports regular compression levels from 1 up to ZSTD_maxCLevel(),
84 compression levels, which extend the range of speed vs. ratio preferences.
85 The lower the level, the faster the speed (at the cost of compression).
87 Compression can be done in:
90 - unbounded multiple steps (described as Streaming compression)
92 The compression ratio achievable on small data can be highly improved using
93 a dictionary. Dictionary compression can be performed in:
182 …* note 3 : decompressed size is always present when compression is completed using single-pass f…
217 * When invoking `ZSTD_compress()` or any other one-pass compression function,
241 ZSTDLIB_API int ZSTD_minCLevel(void); /*!< minimum negative compression level…
242 ZSTDLIB_API int ZSTD_maxCLevel(void); /*!< maximum compression level availabl…
243 ZSTDLIB_API int ZSTD_defaultCLevel(void); /*!< default compression level, specifi…
249 /*= Compression context
252 * and reuse it for each successive compression operation.
255 * It doesn't change the compression ratio, which remains identical.
266 * this function compresses at the requested compression level,
279 * and reuse it for each successive compression operation.
297 * Advanced compression API (Requires v1.4.0+)
314 /* Compression strategies, listed from fastest to strongest */
330 /* compression parameters
334 … ZSTD_c_compressionLevel=100, /* Set compression parameters according to pre-defined cLevel table.
335 * Note that exact compression parameters are dynamically determined,
336 * depending on both compression level and srcSize (when known).
339 * Note 1 : it's possible to pass a negative compression level.
340 … * Note 2 : setting a level does not automatically set all other compression parameters
341 … * to default. Setting this will however eventually dynamically impact the compression
344 /* Advanced compression parameters :
345 * It's possible to pin down compression parameters to some specific values.
358 * Larger tables improve compression ratio of strategies <= dFast,
364 * Larger tables result in better and slower compression.
370 * More attempts result in better and slower compression.
376 … * Larger values increase compression and decompression speed, but decrease ratio.
384 * Larger values make compression stronger, and slower.
387 * Larger values make compression faster, and weaker.
391 * resulting in stronger and slower compression.
408 * This parameter is designed to improve compression ratio
414 … * compression strategy >= ZSTD_btopt (== compression level 16+) */
416 * Larger values increase memory usage and compression ratio,
417 * but decrease compression speed.
422 * Larger/too small values usually decrease compression ratio.
426 … * Larger values improve collision resolution but decrease compression speed.
432 * Larger values improve compression speed.
433 … * Deviating far from default value will likely result in a compression ratio decrease.
438 * Content size must be known at the beginning of compression.
453 * while compression is performed in parallel, within worker thread(s).
458 … * compression is performed inside Caller's thread, and all invocations are blocking */
459 …ZSTD_c_jobSize=401, /* Size of a compression job. This value is enforced only when nbWorkers …
460 …* Each compression job is completed in parallel, so this value can indirectly impact the nb of act…
461 … * 0 means default, which is dynamically determined based on compression parameters.
466 … * It helps preserve compression ratio, while each job is compressed in parallel.
468 * Larger values increase compression ratio, but decrease speed.
537 * Set one compression parameter, selected by enum ZSTD_cParameter.
540 …g a parameter is generally only possible during frame initialization (before starting compression).
542 * the following parameters can be updated _during_ compression (within same frame):
575 * Useful after an error, or to interrupt any ongoing compression.
577 * Compression parameters and dictionary remain unchanged.
582 …* Parameters can only be changed between 2 sessions (i.e. no compression is curre…
589 …* Behave the same as ZSTD_compressCCtx(), but compression parameters are set using the advanced A…
590 * (note that this entry point doesn't even expose a compression level parameter).
593 …* - Compression parameters are pushed into CCtx before starting compression, using ZSTD_CCtx_set*…
594 * - The function is always blocking, returns when compression is completed.
657 * Set one compression parameter, selected by enum ZSTD_dParameter.
693 * Streaming compression - HowTo
697 * ZSTD_CStream objects can be reused multiple times on consecutive compression operations.
704 * Parameters are sticky : when starting a new compression on the same context,
705 * it will reuse the same sticky parameters as previous compression session.
759 /*===== Streaming compression functions =====*/
761 …llect more data, encoder decides when to output compressed result, for optimal compression ratio */
764 …ll continue: any future data can still reference previously compressed data, improving compression.
765 … * note : multithreaded compression will block to flush as much output as possible. */
770 … : note : multithreaded compression will block to flush as much output as possible. */
775 …* - Compression parameters are pushed into CCtx before starting compression, using ZSTD_CCtx_set*…
776 …* - Compression parameters cannot be changed once compression is started (save a list of exceptio…
792 * Before starting a new compression job, or changing compression parameters,
798 …* or is sometimes implied by methods starting a new compression job (ZSTD_initCStream(), …
815 …* It's not rare that performance ends being spent more into the interface, rather than compression…
929 * Compression at an explicit compression level using a Dictionary.
943 * Dictionary must be identical to the one used during compression.
962 …* The resulting state can be used for future compression operations with very limited startup cos…
979 * Compression using a digested Dictionary.
981 * Note : compression level is _decided at dictionary creation time_,
1068 * Tables are dependent on compression parameters, and for this reason,
1069 * compression parameters can no longer be changed after loading a dictionary.
1083 * Note that compression parameters are enforced from within CDict,
1084 * and supersede any compression parameter previously set within CCtx.
1100 …erforms much faster, especially during decompression (compression speed is tunable with compressio…
1104 * Note 1 : Prefix buffer is referenced. It **must** outlive compression.
1105 * Its content must remain unmodified during compression.
1109 …* Note 3 : Referencing a prefix involves building tables, which are dependent on compression para…
1156 * and must use the same prefix as the one used during compression.
1225 /* compression parameter bounds */
1316 …unsigned windowLog; /**< largest match distance : larger == more compression, more memory ne…
1317 …unsigned chainLog; /**< fully searched segment : larger == more compression, slower, more m…
1319 unsigned searchLog; /**< nb of searches : larger == more compression, slower */
1320 …; /**< match length searched : larger == faster decompression, sometimes less compression */
1321 …th; /**< acceptable match size for optimal parser (only) : larger == more compression, slower */
1328 … 1: no dictID will be saved into frame header (dictID is only useful for dictionary compression) */
1375 * means that the compression can search both the dictionary and input
1377 * the compression faster per-byte of input. However, the initial copy of
1379 * compression. For small compressions (< 8 KB), that copy can dominate
1380 * the cost of the compression.
1382 * - The CDict's tables can be used in-place. In this model, compression is
1395 * at the beginning of a compression. However, if experimentation shows that
1406 …lcm_auto = 0, /**< Automatically determine the compression mode based on the compression …
1407 … * Negative compression levels will be uncompressed, and positive compression
1409 …ZSTD_lcm_huffman = 1, /**< Always attempt Huffman compression. Uncompressed literals will st…
1410 * emitted if Huffman compression is not profitable. */
1439 * note 2 : decompressed size is always present when compression is done with ZSTD_compress()
1565 * @param zc The compression context to be used for ZSTD_compress2(). Set any
1566 * compression parameters you need on this context.
1609 * The compression behavior changes based on cctx params. In particular:
1623 …ionLevel accordingly adjusts the strength of the entropy coder, as it would in typical compression.
1686 * to compress data of any size using one-shot compression ZSTD_compressCCtx() or ZSTD_compress2()
1687 * associated with any compression level up to max specified one.
1691 * Note that the size estimation is specific for one-shot compression,
1701 …* Both can be used to estimate memory using custom compression parameters and arbitrary srcSize l…
1703 * Note : only single-threaded compression is supported.
1712 * ZSTD_estimateCStreamSize() will provide a memory budget large enough for streaming compression
1713 * using any compression level up to the max specified one.
1717 …ed in tandem with ZSTD_CCtxParams_setParameter(). Only single-threaded compression is supported. T…
1718 * Note : CStream size estimation is only correct for single-threaded compression.
1739 …* ZSTD_estimateCDictSize_advanced() makes it possible to control compression parameters precisely…
1760 * Note 3 : cParams : use ZSTD_getCParams() to convert a compression level
1813 * These prototypes make it possible to share a thread pool among multiple compression contexts.
1815 * a threaded compression mode (via ZSTD_c_nbWorkers parameter).
1846 * Advanced compression functions
1850 * Create a digested dictionary for compression
1858 …* @return ZSTD_compressionParameters structure for a selected compression level and estimated srcS…
1882 * Note : if modifying parameters during compression (MT mode only),
1953 * it will result in compression ratio degradation.
1958 * NOTE 3: Rsyncable mode limits maximum compression speed to ~400 MB/s.
1959 * If the selected compression level is already running significantly slower,
1964 /* Select a compression format.
1986 * literals compression based on the compression parameters - specifically,
1987 * negative compression levels do not use literal compression.
1994 * but compression ratio may regress significantly if guess considerably underestimates */
2005 * construction (via ZSTD_createCDict_advanced2()). A compression will then
2017 * To an approximation, a compression using a dictionary can then use those
2019 * compression where the simulated compression of the dictionary left off.
2026 * written as the compression goes along. This means we can choose a search
2033 * highly recommended that you simply set a compression level in the CCtxParams
2047 * In general, you should expect compression to be faster--sometimes very much
2064 * and compression will fail if they are not respected.
2066 * MUST not be modified during compression or it will result in data corruption.
2076 * memory. However, compression WILL fail if conditions are not respected.
2079 * not be modified during compression or it will result in data corruption.
2109 * For use with sequence compression API: ZSTD_compressSequences().
2120 * For use with sequence compression API: ZSTD_compressSequences().
2141 * block splitting based on the compression parameters.
2153 * Note that this only pertains to compression strategies: greedy, lazy, and lazy2
2160 * Zstd produces different results for prefix compression when the prefix is
2169 * like when doing delta compression, you should select this option. It comes
2182 * In such situations, compression speed is seriously impacted when CDict tables are
2207 * follow any other cParam settings, such as compression level, the same as in a
2208 * normal (fully-internal) compression operation.
2239 * reduces sequence compression overhead by about 25% while sacrificing some
2240 * compression ratio.
2243 * based on compression level.
2251 * Get the requested compression parameter value, selected by enum ZSTD_cParameter,
2269 * - ZSTD_compressStream2() : Do compression using the CCtx.
2273 * for static allocation of CCtx for single-threaded compression.
2284 * Initializes the compression parameters of cctxParams according to
2285 * compression level. All other parameters are reset to their default values.
2290 * Initializes the compression and frame parameters of cctxParams according to
2297 * Set one compression parameter, selected by enum ZSTD_cParameter.
2307 * Get the requested value of one compression parameter, selected by enum ZSTD_cParameter.
2313 * Apply a set of ZSTD_CCtx_params to the compression context.
2314 * This can be done even after compression is started,
2315 * if nbWorkers==0, this will have no impact until a new compression is started.
2390 * allowing selection between ZSTD_format_e input compression formats
2430 * of whether checksumming was specified during compression. This offers some
2516 /*===== Advanced Streaming compression functions =====*/
2578 * note : cdict will just be referenced, and must outlive compression session
2706 * user application continues to invoke the same compression functions
2736 * - compressionLevel: a signed integer representing the zstd compression level
2738 * to use this information to change its compression strategy and speed/ratio
2739 * tradeoff. Note: the compression level does not reflect zstd parameters set
2766 * zstd will only validate these conditions (and fail compression if they do not
2771 * to an internal sequence producer or fail the compression operation. The user can
2773 * cParam. Fallback compression will follow any other cParam settings, such as
2774 * compression level, the same as in a normal compression operation.
2788 * This API is compatible with all zstd compression APIs which respect advanced parameters.
2792 * COMPRESSION WILL FAIL if it is enabled and the user tries to compress with a block-level
2805 * - Dictionaries are not currently supported. Compression will *not* fail if the user
2807 * - Stream history is not currently supported. All advanced compression APIs, including
2811 * Third, multi-threading within a single compression is not currently supported. In other words,
2812 * COMPRESSION WILL FAIL if ZSTD_c_nbWorkers > 0 and an external sequence producer is registered.
2835 * compressions. It will remain set until the user explicitly resets compression
2839 * part of the "advanced API". This means it will only have an effect on compression
2841 * Older compression APIs such as compressCCtx(), which predate the introduction of
2877 * It allows streaming (de)compression with user allocated buffers.
2888 Buffer-less streaming compression (synchronous mode)
2892 ZSTD_CCtx object can be reused multiple times within successive compression operations.
2895 Use ZSTD_compressBegin(), or ZSTD_compressBegin_usingDict() for dictionary compression.
2903 ZSTD_compressContinue() doesn't guarantee recover after a failed compression.
2916 /*===== Buffer-less streaming compression functions =====*/
2939 …rameters const fParams, unsigned long long const pledgedSrcSize); /* compression parameters are …
2980 Finally, if you control the compression process, you can also ignore all buffer size rules,
3039 This API is deprecated in favor of the regular compression API.
3058 + compression : any ZSTD_compressBegin*() variant, including with dictionary
3076 ZSTD_DEPRECATED("The block API is deprecated in favor of the normal compression API. See docs.")
3078 ZSTD_DEPRECATED("The block API is deprecated in favor of the normal compression API. See docs.")
3080 ZSTD_DEPRECATED("The block API is deprecated in favor of the normal compression API. See docs.")
3082 ZSTD_DEPRECATED("The block API is deprecated in favor of the normal compression API. See docs.")