1 //! # Parsing Partial Input
2 //!
3 //! Typically, the input being parsed is all in-memory, or is complete. Some data sources are too
4 //! large to fit into memory, only allowing parsing an incomplete or [`Partial`] subset of the
5 //! data, requiring incrementally parsing.
6 //!
7 //! By wrapping a stream, like `&[u8]`, with [`Partial`], parsers will report when the data is
8 //! [`Incomplete`] and more input is [`Needed`], allowing the caller to stream-in additional data
9 //! to be parsed. The data is then parsed a chunk at a time.
10 //!
11 //! Chunks are typically defined by either:
12 //! - A header reporting the number of bytes, like with [`length_and_then`]
13 //!   - [`Partial`] can explicitly be changed to being complete once the specified bytes are
14 //!     acquired via [`StreamIsPartial::complete`].
15 //! - A delimiter, like with [ndjson](http://ndjson.org/)
16 //!   - You can parse up-to the delimiter or do a `take_until(0.., delim).and_then(parser)`
17 //!
18 //! If the chunks are not homogeneous, a state machine will be needed to track what the expected
19 //! parser is for the next chunk.
20 //!
21 //! Caveats:
22 //! - `winnow` takes the approach of re-parsing from scratch. Chunks should be relatively small to
23 //!   prevent the re-parsing overhead from dominating.
24 //! - Parsers like [`repeat`] do not know when an `eof` is from insufficient data or the end of the
25 //!   stream, causing them to always report [`Incomplete`].
26 //!
27 //! # Example
28 //!
29 //! `main.rs`:
30 //! ```rust,ignore
31 #![doc = include_str!("../../examples/ndjson/main.rs")]
32 //! ```
33 //!
34 //! `parser.rs`:
35 //! ```rust,ignore
36 #![doc = include_str!("../../examples/ndjson/parser.rs")]
37 //! ```
38 
39 #![allow(unused_imports)] // Used for intra-doc links
40 
41 use crate::binary::length_and_then;
42 use crate::combinator::repeat;
43 use crate::error::ErrMode::Incomplete;
44 use crate::error::Needed;
45 use crate::stream::Partial;
46 use crate::stream::StreamIsPartial;
47