Compare commits

...

3 commits

Author SHA1 Message Date
Alex abc27adea4
Remove nested structs and add list syntax 2022-12-15 14:45:27 +01:00
Alex 08c221558f
A list is now called a sequence; works the same 2022-12-15 13:32:38 +01:00
Alex a43b51211a
Old changes 2022-12-15 13:23:01 +01:00
10 changed files with 496 additions and 308 deletions

View file

@ -2,7 +2,7 @@
name = "nettext" name = "nettext"
description = "A text-based data format for cryptographic network protocols" description = "A text-based data format for cryptographic network protocols"
authors = ["Alex Auvolat <alex@adnab.me>"] authors = ["Alex Auvolat <alex@adnab.me>"]
version = "0.2.3" version = "0.3.0"
edition = "2021" edition = "2021"
license = "AGPL-3.0" license = "AGPL-3.0"
readme = "README.md" readme = "README.md"

View file

@ -15,10 +15,8 @@ A term can be of any of the following kinds:
- a string, which may contain only ASCII alphanumeric terms and `.-_*?` - a string, which may contain only ASCII alphanumeric terms and `.-_*?`
- a dict, which maps strings (as defined above) to any term type - a dict, which maps strings (as defined above) to any term type
- a list, which is a consecutive sequence of at least 2 strings or dicts (can be mixed), simply separated by whitespace - a list, which may contain any number of any kind of terms (can be mixed)
- a sequence, consistuted of at least two of the above (can be mixed), simply separated by whitespace; sequences cannot be nested
Nested lists can be represented using a special dictionnary with a single key, `.`,
for instance `TEST a { . = 0 4 2 1 9 7 0 } c`.
Dicts are represented as follows: Dicts are represented as follows:
@ -29,7 +27,13 @@ Dicts are represented as follows:
} }
``` ```
Lists are represented as follows: Dicts are represented as follows:
```
[ term1, term2 ]
```
Sequences are represented as follows:
``` ```
term1 term2 term3 term1 term2 term3
@ -38,21 +42,26 @@ term1 term2 term3
As a consequence, complex data structures can be defined as follows: As a consequence, complex data structures can be defined as follows:
``` ```
SENDTO alex { SEND MESSAGE {
topic = blah, topic = blah,
to = [
TOPIC hello,
USER john
],
body = blah blah body = blah blah
} }
``` ```
The raw representation of a parsed dict or list is retained for hashing purposes. The raw representation of a parsed dict or sequence is retained for hashing purposes.
It in the sequence of bytes, in the encoded string, trimmed from whitespace at extremities, It in the sequence of bytes, in the encoded string, trimmed from whitespace at extremities,
that represents the encoded dict or list in that string. that represents the encoded dict or sequence in that string.
In the complex stance example above, here are the lists and dicts and their raw representation: In the complex stance example above, here are the sequence and dicts and their raw representation:
- the toplevel term is a list, whose raw representation is the entire encoded string (assuming no whitespace at beginning or end) - the toplevel term is a sequence, whose raw representation is the entire encoded string (assuming no whitespace at beginning or end)
- the third term of the list is a dict, whose raw representation starts at `{` and ends at `}` - the third term of the sequence is a dict, whose raw representation starts at `{` and ends at `}`
- the second mapping of the dict is a list, whose raw representation is exactly `blah blah`. - the second mapping of the dict is a list, whose raw representation starts at `[` and ends at `]`
- the third mapping of the dict is a sequence, whose raw representation is exactly `blah blah`.
Since strings cannot contain whitespace, they are always equivalent to their raw representation. Since strings cannot contain whitespace, they are always equivalent to their raw representation.
@ -61,13 +70,13 @@ Since strings cannot contain whitespace, they are always equivalent to their raw
Terms can be interpreted in a number of different ways, depending on the context: Terms can be interpreted in a number of different ways, depending on the context:
- RAW: the term is interpreted as its raw encoding (see above) - RAW: the term is interpreted as its raw encoding (see above)
- STRING: if the term is a string or a list composed exclusively of strings, the term is interpreted as its raw encoding - STRING: if the term is a string or a sequence composed exclusively of strings, the term is interpreted as its raw encoding
- VARIANT: if the term is a list whose first item is a string, it is interpreted as a variant with the following properties: - VARIANT: if the term is a sequence whose first item is a string, it is interpreted as a variant with the following properties:
- a discriminator (the first item) - a discriminator (the first item)
- a value, which is either the second item in case there are only two items, or the list composed of all items starting from the second if there are more than two - a value, which is either the second item in case there are only two items, or the sequence composed of all items starting from the second if there are more than two
- DICT: if the term is a dict, interpret it as such - DICT: if the term is a dict, interpret it as such
- LIST: if the term is a string or a dict, interpret it as a list composed of that single term. Otherwise, the term is a list, interpret it as a list of terms. - LIST: if the term is a list, interpret it as such
- NESTED: if the term is a dict with a single key `.`, interpret it as the term associated to that key - SEQ: if the term is a string, a list, or a dict, interpret it as a sequence composed of that single term. Otherwise, the term is a sequence, interpret it as a sequence of terms.
## Data mappings ## Data mappings

View file

@ -8,8 +8,11 @@ use nom::{
IResult, InputLength, IResult, InputLength,
}; };
use crate::dec::{AnyTerm, DecodeError, NonListTerm, Term}; use crate::dec::{AnyTerm, DecodeError, NonSeqTerm, Term};
use crate::{is_string_char, is_whitespace, DICT_ASSIGN, DICT_CLOSE, DICT_DELIM, DICT_OPEN}; use crate::{
is_string_char, is_whitespace, DICT_ASSIGN, DICT_CLOSE, DICT_DELIM, DICT_OPEN, LIST_CLOSE,
LIST_DELIM, LIST_OPEN,
};
// ---- // ----
@ -25,21 +28,22 @@ pub fn decode(input: &[u8]) -> std::result::Result<Term<'_, '_>, DecodeError<'_>
fn decode_term(input: &[u8]) -> IResult<&'_ [u8], AnyTerm<'_, '_>> { fn decode_term(input: &[u8]) -> IResult<&'_ [u8], AnyTerm<'_, '_>> {
let (start, _) = take_while(is_whitespace)(input)?; let (start, _) = take_while(is_whitespace)(input)?;
let (rest, list) = separated_list1(take_while1(is_whitespace), decode_nonlist_term)(start)?; let (rest, seq) = separated_list1(take_while1(is_whitespace), decode_nonseq_term)(start)?;
if list.len() == 1 { if seq.len() == 1 {
Ok((rest, list.into_iter().next().unwrap().into())) Ok((rest, seq.into_iter().next().unwrap().into()))
} else { } else {
let raw_len = start.input_len() - rest.input_len(); let raw_len = start.input_len() - rest.input_len();
let list_raw = &start[..raw_len]; let seq_raw = &start[..raw_len];
Ok((rest, AnyTerm::List(list_raw, list))) Ok((rest, AnyTerm::Seq(seq_raw, seq)))
} }
} }
fn decode_nonlist_term(input: &[u8]) -> IResult<&'_ [u8], NonListTerm<'_, '_>> { fn decode_nonseq_term(input: &[u8]) -> IResult<&'_ [u8], NonSeqTerm<'_, '_>> {
let (rest, term) = alt(( let (rest, term) = alt((
map(decode_str, NonListTerm::Str), map(decode_str, NonSeqTerm::Str),
map(decode_dict, |(raw, d)| NonListTerm::Dict(raw, d)), map(decode_dict, |(raw, d)| NonSeqTerm::Dict(raw, d)),
map(decode_list, |(raw, l)| NonSeqTerm::List(raw, l)),
))(input)?; ))(input)?;
Ok((rest, term)) Ok((rest, term))
} }
@ -81,6 +85,27 @@ fn decode_dict_item(d: &[u8]) -> IResult<&'_ [u8], (&'_ [u8], AnyTerm<'_, '_>)>
Ok((d, (key, value))) Ok((d, (key, value)))
} }
type ListType<'a> = (&'a [u8], Vec<AnyTerm<'a, 'a>>);
fn decode_list(list_begin: &[u8]) -> IResult<&'_ [u8], ListType<'_>> {
let (d, _) = tag(&[LIST_OPEN][..])(list_begin)?;
let (d, list) = separated_list0(list_separator, decode_term)(d)?;
let (d, _) = opt(list_separator)(d)?;
let (d, _) = take_while(is_whitespace)(d)?;
let (list_end, _) = tag(&[LIST_CLOSE][..])(d)?;
let raw_len = list_begin.input_len() - list_end.input_len();
let list_raw = &list_begin[..raw_len];
Ok((list_end, (list_raw, list)))
}
fn list_separator(d: &[u8]) -> IResult<&'_ [u8], ()> {
let (d, _) = take_while(is_whitespace)(d)?;
let (d, _) = tag(&[LIST_DELIM][..])(d)?;
Ok((d, ()))
}
// ---- // ----
#[cfg(test)] #[cfg(test)]
@ -94,17 +119,17 @@ mod tests {
} }
#[test] #[test]
fn list_of_str_str() { fn seq_of_str_str() {
let bytes = b" plop plap plip ploup "; let bytes = b" plop plap plip ploup ";
assert_eq!( assert_eq!(
decode(bytes), decode(bytes),
Ok(AnyTerm::List( Ok(AnyTerm::Seq(
b"plop plap plip ploup", b"plop plap plip ploup",
vec![ vec![
NonListTerm::Str(b"plop"), NonSeqTerm::Str(b"plop"),
NonListTerm::Str(b"plap"), NonSeqTerm::Str(b"plap"),
NonListTerm::Str(b"plip"), NonSeqTerm::Str(b"plip"),
NonListTerm::Str(b"ploup"), NonSeqTerm::Str(b"ploup"),
] ]
) )
.into()) .into())
@ -122,9 +147,9 @@ mod tests {
(&b"aze"[..], AnyTerm::Str(b"hello")), (&b"aze"[..], AnyTerm::Str(b"hello")),
( (
&b"by"[..], &b"by"[..],
AnyTerm::List( AnyTerm::Seq(
b"bojzkz pipo", b"bojzkz pipo",
vec![NonListTerm::Str(b"bojzkz"), NonListTerm::Str(b"pipo")] vec![NonSeqTerm::Str(b"bojzkz"), NonSeqTerm::Str(b"pipo")]
) )
), ),
(&b"ccde"[..], AnyTerm::Str(b"ke")), (&b"ccde"[..], AnyTerm::Str(b"ke")),
@ -147,9 +172,9 @@ mod tests {
(&b"aze"[..], AnyTerm::Str(b"hello")), (&b"aze"[..], AnyTerm::Str(b"hello")),
( (
&b"by"[..], &b"by"[..],
AnyTerm::List( AnyTerm::Seq(
b"bojzkz pipo", b"bojzkz pipo",
vec![NonListTerm::Str(b"bojzkz"), NonListTerm::Str(b"pipo")] vec![NonSeqTerm::Str(b"bojzkz"), NonSeqTerm::Str(b"pipo")]
) )
), ),
(&b"ccde"[..], AnyTerm::Str(b"ke")), (&b"ccde"[..], AnyTerm::Str(b"ke")),
@ -161,14 +186,56 @@ mod tests {
); );
} }
#[test]
fn simple_list() {
let bytes = b" [ hello, bojzkz pipo, ke ] ";
assert_eq!(
decode(bytes),
Ok(AnyTerm::List(
b"[ hello, bojzkz pipo, ke ]",
[
AnyTerm::Str(b"hello"),
AnyTerm::Seq(
b"bojzkz pipo",
vec![NonSeqTerm::Str(b"bojzkz"), NonSeqTerm::Str(b"pipo")]
),
AnyTerm::Str(b"ke"),
]
.to_vec()
)
.into())
);
}
#[test]
fn simple_list_2() {
let bytes = b" [ hello, bojzkz pipo , ke , ] ";
assert_eq!(
decode(bytes),
Ok(AnyTerm::List(
b"[ hello, bojzkz pipo , ke , ]",
[
AnyTerm::Str(b"hello"),
AnyTerm::Seq(
b"bojzkz pipo",
vec![NonSeqTerm::Str(b"bojzkz"), NonSeqTerm::Str(b"pipo")]
),
AnyTerm::Str(b"ke"),
]
.to_vec()
)
.into())
);
}
#[test] #[test]
fn real_world_1() { fn real_world_1() {
let bytes = b"HEAD alexpubkey"; let bytes = b"HEAD alexpubkey";
assert_eq!( assert_eq!(
decode(bytes), decode(bytes),
Ok(AnyTerm::List( Ok(AnyTerm::Seq(
b"HEAD alexpubkey", b"HEAD alexpubkey",
vec![NonListTerm::Str(b"HEAD"), NonListTerm::Str(b"alexpubkey")] vec![NonSeqTerm::Str(b"HEAD"), NonSeqTerm::Str(b"alexpubkey")]
) )
.into()), .into()),
); );
@ -179,22 +246,22 @@ mod tests {
let bytes = b"STANCE sthash stsign { author = alexpubkey, height = 12, parent = parenthash, data = MESSAGE { text = hello } }"; let bytes = b"STANCE sthash stsign { author = alexpubkey, height = 12, parent = parenthash, data = MESSAGE { text = hello } }";
assert_eq!( assert_eq!(
decode(bytes), decode(bytes),
Ok(AnyTerm::List( Ok(AnyTerm::Seq(
&bytes[..], &bytes[..],
vec![ vec![
NonListTerm::Str(b"STANCE"), NonSeqTerm::Str(b"STANCE"),
NonListTerm::Str(b"sthash"), NonSeqTerm::Str(b"sthash"),
NonListTerm::Str(b"stsign"), NonSeqTerm::Str(b"stsign"),
NonListTerm::Dict(b"{ author = alexpubkey, height = 12, parent = parenthash, data = MESSAGE { text = hello } }", NonSeqTerm::Dict(b"{ author = alexpubkey, height = 12, parent = parenthash, data = MESSAGE { text = hello } }",
[ [
(&b"author"[..], AnyTerm::Str(b"alexpubkey")), (&b"author"[..], AnyTerm::Str(b"alexpubkey")),
(&b"height"[..], AnyTerm::Str(b"12")), (&b"height"[..], AnyTerm::Str(b"12")),
(&b"parent"[..], AnyTerm::Str(b"parenthash")), (&b"parent"[..], AnyTerm::Str(b"parenthash")),
(&b"data"[..], AnyTerm::List( (&b"data"[..], AnyTerm::Seq(
b"MESSAGE { text = hello }", b"MESSAGE { text = hello }",
vec![ vec![
NonListTerm::Str(b"MESSAGE"), NonSeqTerm::Str(b"MESSAGE"),
NonListTerm::Dict( NonSeqTerm::Dict(
b"{ text = hello }", b"{ text = hello }",
[ [
(&b"text"[..], AnyTerm::Str(b"hello")), (&b"text"[..], AnyTerm::Str(b"hello")),
@ -209,4 +276,51 @@ mod tests {
]).into(), ]).into(),
)); ));
} }
#[test]
fn real_world_3() {
let bytes = b"[ USER john, USER luke, GROUP strategy { owner = USER john, members = [ USER john, USER luke ] } ]";
let user_john = AnyTerm::Seq(
b"USER john",
vec![NonSeqTerm::Str(b"USER"), NonSeqTerm::Str(b"john")],
);
let user_luke = AnyTerm::Seq(
b"USER luke",
vec![NonSeqTerm::Str(b"USER"), NonSeqTerm::Str(b"luke")],
);
assert_eq!(
decode(bytes),
Ok(AnyTerm::List(
&bytes[..],
vec![
user_john.clone(),
user_luke.clone(),
AnyTerm::Seq(
b"GROUP strategy { owner = USER john, members = [ USER john, USER luke ] }",
vec![
NonSeqTerm::Str(b"GROUP"),
NonSeqTerm::Str(b"strategy"),
NonSeqTerm::Dict(
b"{ owner = USER john, members = [ USER john, USER luke ] }",
[
(&b"owner"[..], user_john.clone()),
(
&b"members"[..],
AnyTerm::List(
b"[ USER john, USER luke ]",
vec![user_john, user_luke,]
)
)
]
.into_iter()
.collect()
)
]
),
]
)
.into())
);
}
} }

View file

@ -8,6 +8,8 @@ use std::collections::HashMap;
#[cfg(any(feature = "dryoc"))] #[cfg(any(feature = "dryoc"))]
use crate::crypto; use crate::crypto;
use crate::debug;
pub use decode::*; pub use decode::*;
pub use error::*; pub use error::*;
@ -26,34 +28,42 @@ pub(crate) enum AnyTerm<'a, 'b> {
Str(&'a [u8]), Str(&'a [u8]),
Dict(&'a [u8], HashMap<&'a [u8], AnyTerm<'a, 'b>>), Dict(&'a [u8], HashMap<&'a [u8], AnyTerm<'a, 'b>>),
DictRef(&'a [u8], &'b HashMap<&'a [u8], AnyTerm<'a, 'b>>), DictRef(&'a [u8], &'b HashMap<&'a [u8], AnyTerm<'a, 'b>>),
List(&'a [u8], Vec<NonListTerm<'a, 'b>>), List(&'a [u8], Vec<AnyTerm<'a, 'b>>),
ListRef(&'a [u8], &'b [NonListTerm<'a, 'b>]), ListRef(&'a [u8], &'b [AnyTerm<'a, 'b>]),
Seq(&'a [u8], Vec<NonSeqTerm<'a, 'b>>),
SeqRef(&'a [u8], &'b [NonSeqTerm<'a, 'b>]),
} }
#[derive(Eq, PartialEq, Clone)] #[derive(Eq, PartialEq, Clone)]
pub(crate) enum NonListTerm<'a, 'b> { pub(crate) enum NonSeqTerm<'a, 'b> {
Str(&'a [u8]), Str(&'a [u8]),
Dict(&'a [u8], HashMap<&'a [u8], AnyTerm<'a, 'b>>), Dict(&'a [u8], HashMap<&'a [u8], AnyTerm<'a, 'b>>),
DictRef(&'a [u8], &'b HashMap<&'a [u8], AnyTerm<'a, 'b>>), DictRef(&'a [u8], &'b HashMap<&'a [u8], AnyTerm<'a, 'b>>),
List(&'a [u8], Vec<AnyTerm<'a, 'b>>),
ListRef(&'a [u8], &'b [AnyTerm<'a, 'b>]),
} }
impl<'a, 'b> From<NonListTerm<'a, 'b>> for AnyTerm<'a, 'b> { impl<'a, 'b> From<NonSeqTerm<'a, 'b>> for AnyTerm<'a, 'b> {
fn from(x: NonListTerm<'a, 'b>) -> AnyTerm<'a, 'b> { fn from(x: NonSeqTerm<'a, 'b>) -> AnyTerm<'a, 'b> {
match x { match x {
NonListTerm::Str(s) => AnyTerm::Str(s), NonSeqTerm::Str(s) => AnyTerm::Str(s),
NonListTerm::Dict(raw, d) => AnyTerm::Dict(raw, d), NonSeqTerm::Dict(raw, d) => AnyTerm::Dict(raw, d),
NonListTerm::DictRef(raw, d) => AnyTerm::DictRef(raw, d), NonSeqTerm::DictRef(raw, d) => AnyTerm::DictRef(raw, d),
NonSeqTerm::List(raw, l) => AnyTerm::List(raw, l),
NonSeqTerm::ListRef(raw, l) => AnyTerm::ListRef(raw, l),
} }
} }
} }
impl<'a, 'b> TryFrom<AnyTerm<'a, 'b>> for NonListTerm<'a, 'b> { impl<'a, 'b> TryFrom<AnyTerm<'a, 'b>> for NonSeqTerm<'a, 'b> {
type Error = (); type Error = ();
fn try_from(x: AnyTerm<'a, 'b>) -> Result<NonListTerm<'a, 'b>, ()> { fn try_from(x: AnyTerm<'a, 'b>) -> Result<NonSeqTerm<'a, 'b>, ()> {
match x { match x {
AnyTerm::Str(s) => Ok(NonListTerm::Str(s)), AnyTerm::Str(s) => Ok(NonSeqTerm::Str(s)),
AnyTerm::Dict(raw, d) => Ok(NonListTerm::Dict(raw, d)), AnyTerm::Dict(raw, d) => Ok(NonSeqTerm::Dict(raw, d)),
AnyTerm::DictRef(raw, d) => Ok(NonListTerm::DictRef(raw, d)), AnyTerm::DictRef(raw, d) => Ok(NonSeqTerm::DictRef(raw, d)),
AnyTerm::List(raw, l) => Ok(NonSeqTerm::List(raw, l)),
AnyTerm::ListRef(raw, l) => Ok(NonSeqTerm::ListRef(raw, l)),
_ => Err(()), _ => Err(()),
} }
} }
@ -84,6 +94,20 @@ impl<'a, 'b> Term<'a, 'b> {
self.0.raw() self.0.raw()
} }
/// Get the term's raw representation as an str
///
/// Example:
///
/// ```
/// use nettext::dec::decode;
///
/// let term = decode(b"hello { a = x, b = y }").unwrap();
/// assert_eq!(term.raw_str().unwrap(), "hello { a = x, b = y }");
/// ```
pub fn raw_str(&self) -> Result<&'a str, TypeError> {
Ok(std::str::from_utf8(self.0.raw())?)
}
/// If the term is a single string, get that string /// If the term is a single string, get that string
/// ///
/// Example: /// Example:
@ -104,7 +128,7 @@ impl<'a, 'b> Term<'a, 'b> {
} }
} }
/// If the term is a single string, or a list containing only strings, /// If the term is a single string, or a sequence containing only strings,
/// get its raw representation /// get its raw representation
/// ///
/// Example: /// Example:
@ -121,16 +145,16 @@ impl<'a, 'b> Term<'a, 'b> {
pub fn string(&self) -> Result<&'a str, TypeError> { pub fn string(&self) -> Result<&'a str, TypeError> {
match &self.0 { match &self.0 {
AnyTerm::Str(s) => Ok(std::str::from_utf8(s)?), AnyTerm::Str(s) => Ok(std::str::from_utf8(s)?),
AnyTerm::List(r, l) if l.iter().all(|x| matches!(x, NonListTerm::Str(_))) => { AnyTerm::Seq(r, l) if l.iter().all(|x| matches!(x, NonSeqTerm::Str(_))) => {
Ok(std::str::from_utf8(r)?) Ok(std::str::from_utf8(r)?)
} }
_ => Err(TypeError::WrongType("STRING")), _ => Err(TypeError::WrongType("STRING")),
} }
} }
/// Return a list of terms made from this term. /// Return a sequence of terms made from this term.
/// If it is a str or a dict, returns a list of a single term. /// If it is a str or a dict, returns a seq of a single term.
/// If it is a list, that's the list of terms we return. /// If it is a sequence, that's the seq of terms we return.
/// ///
/// Example: /// Example:
/// ///
@ -138,26 +162,26 @@ impl<'a, 'b> Term<'a, 'b> {
/// use nettext::dec::decode; /// use nettext::dec::decode;
/// ///
/// let term1 = decode(b"hello").unwrap(); /// let term1 = decode(b"hello").unwrap();
/// let list1 = term1.list(); /// let seq1 = term1.seq();
/// assert_eq!(list1.len(), 1); /// assert_eq!(seq1.len(), 1);
/// assert_eq!(list1[0].str().unwrap(), "hello"); /// assert_eq!(seq1[0].str().unwrap(), "hello");
/// ///
/// let term2 = decode(b"hello world").unwrap(); /// let term2 = decode(b"hello world").unwrap();
/// let list2 = term2.list(); /// let seq2 = term2.seq();
/// assert_eq!(list2.len(), 2); /// assert_eq!(seq2.len(), 2);
/// assert_eq!(list2[0].str().unwrap(), "hello"); /// assert_eq!(seq2[0].str().unwrap(), "hello");
/// assert_eq!(list2[1].str().unwrap(), "world"); /// assert_eq!(seq2[1].str().unwrap(), "world");
/// ``` /// ```
pub fn list(&self) -> Vec<Term<'a, '_>> { pub fn seq(&self) -> Vec<Term<'a, '_>> {
match self.0.mkref() { match self.0.mkref() {
AnyTerm::ListRef(_r, l) => l.iter().map(|x| Term(x.mkref().into())).collect::<Vec<_>>(), AnyTerm::SeqRef(_r, l) => l.iter().map(|x| Term(x.mkref().into())).collect::<Vec<_>>(),
x => vec![Term(x)], x => vec![Term(x)],
} }
} }
/// Same as `.list()`, but deconstructs it in a const length array, /// Same as `.seq()`, but deconstructs it in a const length array,
/// dynamically checking if there are the correct number of items. /// dynamically checking if there are the correct number of items.
/// This allows to directly bind the resulting list into discrete variables. /// This allows to directly bind the resulting seq into discrete variables.
/// ///
/// Example: /// Example:
/// ///
@ -165,24 +189,24 @@ impl<'a, 'b> Term<'a, 'b> {
/// use nettext::dec::decode; /// use nettext::dec::decode;
/// ///
/// let term1 = decode(b"hello").unwrap(); /// let term1 = decode(b"hello").unwrap();
/// let [s1] = term1.list_of().unwrap(); /// let [s1] = term1.seq_of().unwrap();
/// assert_eq!(s1.str().unwrap(), "hello"); /// assert_eq!(s1.str().unwrap(), "hello");
/// ///
/// let term2 = decode(b"hello world").unwrap(); /// let term2 = decode(b"hello world").unwrap();
/// let [s2a, s2b] = term2.list_of().unwrap(); /// let [s2a, s2b] = term2.seq_of().unwrap();
/// assert_eq!(s2a.str().unwrap(), "hello"); /// assert_eq!(s2a.str().unwrap(), "hello");
/// assert_eq!(s2b.str().unwrap(), "world"); /// assert_eq!(s2b.str().unwrap(), "world");
/// ``` /// ```
pub fn list_of<const N: usize>(&self) -> Result<[Term<'a, '_>; N], TypeError> { pub fn seq_of<const N: usize>(&self) -> Result<[Term<'a, '_>; N], TypeError> {
let list = self.list(); let seq = self.seq();
let list_len = list.len(); let seq_len = seq.len();
list.try_into() seq.try_into()
.map_err(|_| TypeError::WrongLength(list_len, N)) .map_err(|_| TypeError::WrongLength(seq_len, N))
} }
/// Same as `.list_of()`, but only binds the first N-1 terms. /// Same as `.seq_of()`, but only binds the first N-1 terms.
/// If there are exactly N terms, the last one is bound to the Nth return variable. /// If there are exactly N terms, the last one is bound to the Nth return variable.
/// If there are more then N terms, the remaining terms are bound to a new list term /// If there are more then N terms, the remaining terms are bound to a new seq term
/// that is returned as the Nth return variable. /// that is returned as the Nth return variable.
/// ///
/// Example: /// Example:
@ -191,21 +215,21 @@ impl<'a, 'b> Term<'a, 'b> {
/// use nettext::dec::decode; /// use nettext::dec::decode;
/// ///
/// let term1 = decode(b"hello world").unwrap(); /// let term1 = decode(b"hello world").unwrap();
/// let [s1a, s1b] = term1.list_of_first().unwrap(); /// let [s1a, s1b] = term1.seq_of_first().unwrap();
/// assert_eq!(s1a.str().unwrap(), "hello"); /// assert_eq!(s1a.str().unwrap(), "hello");
/// assert_eq!(s1b.str().unwrap(), "world"); /// assert_eq!(s1b.str().unwrap(), "world");
/// ///
/// let term2 = decode(b"hello mighty world").unwrap(); /// let term2 = decode(b"hello mighty world").unwrap();
/// let [s2a, s2b] = term2.list_of_first().unwrap(); /// let [s2a, s2b] = term2.seq_of_first().unwrap();
/// assert_eq!(s2a.str().unwrap(), "hello"); /// assert_eq!(s2a.str().unwrap(), "hello");
/// assert_eq!(s2b.list().len(), 2); /// assert_eq!(s2b.seq().len(), 2);
/// assert_eq!(s2b.raw(), b"mighty world"); /// assert_eq!(s2b.raw(), b"mighty world");
/// ``` /// ```
pub fn list_of_first<const N: usize>(&self) -> Result<[Term<'a, '_>; N], TypeError> { pub fn seq_of_first<const N: usize>(&self) -> Result<[Term<'a, '_>; N], TypeError> {
match self.0.mkref() { match self.0.mkref() {
AnyTerm::ListRef(raw, list) => match list.len().cmp(&N) { AnyTerm::SeqRef(raw, seq) => match seq.len().cmp(&N) {
std::cmp::Ordering::Less => Err(TypeError::WrongLength(list.len(), N)), std::cmp::Ordering::Less => Err(TypeError::WrongLength(seq.len(), N)),
std::cmp::Ordering::Equal => Ok(list std::cmp::Ordering::Equal => Ok(seq
.iter() .iter()
.map(|x| Term(x.mkref().into())) .map(|x| Term(x.mkref().into()))
.collect::<Vec<_>>() .collect::<Vec<_>>()
@ -213,15 +237,15 @@ impl<'a, 'b> Term<'a, 'b> {
.unwrap()), .unwrap()),
std::cmp::Ordering::Greater => { std::cmp::Ordering::Greater => {
let mut ret = Vec::with_capacity(N); let mut ret = Vec::with_capacity(N);
for item in list[0..N - 1].iter() { for item in seq[0..N - 1].iter() {
ret.push(Term(item.mkref().into())); ret.push(Term(item.mkref().into()));
} }
let remaining_begin = list[N - 1].raw().as_ptr() as usize; let remaining_begin = seq[N - 1].raw().as_ptr() as usize;
let remaining_offset = remaining_begin - raw.as_ptr() as usize; let remaining_offset = remaining_begin - raw.as_ptr() as usize;
let remaining_raw = &raw[remaining_offset..]; let remaining_raw = &raw[remaining_offset..];
ret.push(Term(AnyTerm::ListRef(remaining_raw, &list[N - 1..]))); ret.push(Term(AnyTerm::SeqRef(remaining_raw, &seq[N - 1..])));
Ok(ret.try_into().unwrap()) Ok(ret.try_into().unwrap())
} }
@ -261,7 +285,7 @@ impl<'a, 'b> Term<'a, 'b> {
} }
/// Checks term is a dictionnary whose keys are exactly those supplied, /// Checks term is a dictionnary whose keys are exactly those supplied,
/// and returns the associated values as a list. /// and returns the associated values as a seq.
/// ///
/// Example: /// Example:
/// ///
@ -303,7 +327,7 @@ impl<'a, 'b> Term<'a, 'b> {
} }
/// Checks term is a dictionnary whose keys are included in those supplied, /// Checks term is a dictionnary whose keys are included in those supplied,
/// and returns the associated values as a list of options. /// and returns the associated values as a seq of options.
/// ///
/// Example: /// Example:
/// ///
@ -339,28 +363,23 @@ impl<'a, 'b> Term<'a, 'b> {
} }
} }
/// Checks term is a dictionary with a single key `.`, /// Checks if the term is a list, and if so, return its elements in a vec.
/// and returns the associated value.
/// ///
/// Example: /// Example:
/// ///
/// ``` /// ```
/// use nettext::dec::decode; /// use nettext::dec::decode;
/// ///
/// let term = decode(b"{ . = a b c d e }").unwrap(); /// let term2 = decode(b"[ hello, world ]").unwrap();
/// assert_eq!(term.nested().unwrap().raw(), b"a b c d e"); /// let seq2 = term2.list().unwrap();
/// assert_eq!(seq2.len(), 2);
/// assert_eq!(seq2[0].str().unwrap(), "hello");
/// assert_eq!(seq2[1].str().unwrap(), "world");
/// ``` /// ```
pub fn nested(&self) -> Result<Term<'a, '_>, TypeError> { pub fn list(&self) -> Result<Vec<Term<'a, '_>>, TypeError> {
match self.0.mkref() { match self.0.mkref() {
AnyTerm::DictRef(_, d) if d.len() == 1 => { AnyTerm::ListRef(_r, l) => Ok(l.iter().map(|x| Term(x.mkref().into())).collect::<Vec<_>>()),
let (k, v) = d.iter().next().unwrap(); _ => Err(TypeError::WrongType("LIST")),
if k != b"." {
Err(TypeError::WrongType("NESTED"))
} else {
Ok(Term(v.mkref()))
}
}
_ => Err(TypeError::WrongType("NESTED")),
} }
} }
@ -405,10 +424,10 @@ impl<'a, 'b> Term<'a, 'b> {
decode(encoded) decode(encoded)
} }
} }
AnyTerm::ListRef(_, list) => { AnyTerm::SeqRef(_, seq) => {
let mut ret = Vec::with_capacity(128); let mut ret = Vec::with_capacity(128);
for term in list.iter() { for term in seq.iter() {
if let NonListTerm::Str(encoded) = term { if let NonSeqTerm::Str(encoded) = term {
ret.extend(decode(encoded)?) ret.extend(decode(encoded)?)
} else { } else {
return Err(TypeError::WrongType("BYTES")); return Err(TypeError::WrongType("BYTES"));
@ -495,7 +514,9 @@ impl<'a, 'b> AnyTerm<'a, 'b> {
AnyTerm::Dict(r, _) AnyTerm::Dict(r, _)
| AnyTerm::DictRef(r, _) | AnyTerm::DictRef(r, _)
| AnyTerm::List(r, _) | AnyTerm::List(r, _)
| AnyTerm::ListRef(r, _) => r, | AnyTerm::ListRef(r, _)
| AnyTerm::Seq(r, _)
| AnyTerm::SeqRef(r, _) => r,
} }
} }
@ -504,25 +525,30 @@ impl<'a, 'b> AnyTerm<'a, 'b> {
AnyTerm::Str(s) => AnyTerm::Str(s), AnyTerm::Str(s) => AnyTerm::Str(s),
AnyTerm::Dict(r, d) => AnyTerm::DictRef(r, d), AnyTerm::Dict(r, d) => AnyTerm::DictRef(r, d),
AnyTerm::DictRef(r, d) => AnyTerm::DictRef(r, d), AnyTerm::DictRef(r, d) => AnyTerm::DictRef(r, d),
AnyTerm::List(r, l) => AnyTerm::ListRef(r, &l[..]), AnyTerm::List(r, l) => AnyTerm::ListRef(r, l),
AnyTerm::ListRef(r, l) => AnyTerm::ListRef(r, l), AnyTerm::ListRef(r, l) => AnyTerm::ListRef(r, l),
AnyTerm::Seq(r, l) => AnyTerm::SeqRef(r, &l[..]),
AnyTerm::SeqRef(r, l) => AnyTerm::SeqRef(r, l),
} }
} }
} }
impl<'a, 'b> NonListTerm<'a, 'b> { impl<'a, 'b> NonSeqTerm<'a, 'b> {
fn raw(&self) -> &'a [u8] { fn raw(&self) -> &'a [u8] {
match &self { match &self {
NonListTerm::Str(s) => s, NonSeqTerm::Str(s) => s,
NonListTerm::Dict(r, _) | NonListTerm::DictRef(r, _) => r, NonSeqTerm::Dict(r, _) | NonSeqTerm::DictRef(r, _) => r,
NonSeqTerm::List(r, _) | NonSeqTerm::ListRef(r, _) => r,
} }
} }
fn mkref(&self) -> NonListTerm<'a, '_> { fn mkref(&self) -> NonSeqTerm<'a, '_> {
match &self { match &self {
NonListTerm::Str(s) => NonListTerm::Str(s), NonSeqTerm::Str(s) => NonSeqTerm::Str(s),
NonListTerm::Dict(r, d) => NonListTerm::DictRef(r, d), NonSeqTerm::Dict(r, d) => NonSeqTerm::DictRef(r, d),
NonListTerm::DictRef(r, d) => NonListTerm::DictRef(r, d), NonSeqTerm::DictRef(r, d) => NonSeqTerm::DictRef(r, d),
NonSeqTerm::List(r, l) => NonSeqTerm::ListRef(r, l),
NonSeqTerm::ListRef(r, l) => NonSeqTerm::ListRef(r, l),
} }
} }
} }
@ -547,10 +573,6 @@ impl<'a, 'b> std::fmt::Display for Term<'a, 'b> {
// ---- DEBUG REPR ---- // ---- DEBUG REPR ----
pub(crate) fn debug(x: &[u8]) -> &str {
std::str::from_utf8(x).unwrap_or("<invalid ascii>")
}
impl<'a, 'b> std::fmt::Debug for AnyTerm<'a, 'b> { impl<'a, 'b> std::fmt::Debug for AnyTerm<'a, 'b> {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::result::Result<(), std::fmt::Error> { fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::result::Result<(), std::fmt::Error> {
match self.mkref() { match self.mkref() {
@ -569,12 +591,19 @@ impl<'a, 'b> std::fmt::Debug for AnyTerm<'a, 'b> {
} }
write!(f, "]") write!(f, "]")
} }
AnyTerm::SeqRef(raw, l) => {
write!(f, "Seq[`{}`", debug(raw))?;
for i in l.iter() {
write!(f, "\n {:?}", i)?;
}
write!(f, "]")
}
_ => unreachable!(), _ => unreachable!(),
} }
} }
} }
impl<'a, 'b> std::fmt::Debug for NonListTerm<'a, 'b> { impl<'a, 'b> std::fmt::Debug for NonSeqTerm<'a, 'b> {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::result::Result<(), std::fmt::Error> { fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::result::Result<(), std::fmt::Error> {
AnyTerm::from(self.mkref()).fmt(f) AnyTerm::from(self.mkref()).fmt(f)
} }

View file

@ -12,7 +12,7 @@ pub enum Error {
#[error(display = "Duplicate key: {}", _0)] #[error(display = "Duplicate key: {}", _0)]
DuplicateKey(String), DuplicateKey(String),
#[error( #[error(
display = "Refusing to build nested lists with list(), use either list_flatten() or list_nested()" display = "Refusing to build nested sequencess with seq(), use either seq_flatten() or seq_nested()"
)] )]
ListInList, SeqInSeq,
} }

View file

@ -5,7 +5,7 @@
//! ``` //! ```
//! use nettext::enc::*; //! use nettext::enc::*;
//! //!
//! let nettext_encoding = list([ //! let nettext_encoding = seq([
//! string("CALL").unwrap(), //! string("CALL").unwrap(),
//! string("myfunction").unwrap(), //! string("myfunction").unwrap(),
//! dict([ //! dict([
@ -35,6 +35,7 @@ enum T<'a> {
OwnedStr(Vec<u8>), OwnedStr(Vec<u8>),
Dict(HashMap<Cow<'a, [u8]>, T<'a>>), Dict(HashMap<Cow<'a, [u8]>, T<'a>>),
List(Vec<T<'a>>), List(Vec<T<'a>>),
Seq(Vec<T<'a>>),
} }
/// The result type for trying to encode something as nettext /// The result type for trying to encode something as nettext
@ -95,7 +96,40 @@ pub fn raw(bytes: &[u8]) -> Result<'_> {
Ok(Term(T::Str(bytes))) Ok(Term(T::Str(bytes)))
} }
/// Term corresponding to a list of terms /// Term corresponding to a sequence of terms. Subsequences are banned and will raise an error.
///
/// ```
/// use nettext::enc::*;
///
/// assert_eq!(seq([
/// string("Hello").unwrap(),
/// string("world").unwrap()
/// ]).unwrap().encode(), b"Hello world");
/// ```
pub fn seq<'a, I: IntoIterator<Item = Term<'a>>>(terms: I) -> Result<'a> {
let mut tmp = Vec::with_capacity(8);
for t in terms {
match t.0 {
T::Seq(_) => return Err(Error::SeqInSeq),
x => tmp.push(x),
}
}
Ok(Term(T::Seq(tmp)))
}
/// Term corresponding to a sequence of terms. Sub-sequences are flattenned.
pub fn seq_flatten<'a, I: IntoIterator<Item = Term<'a>>>(terms: I) -> Term<'a> {
let mut tmp = Vec::with_capacity(8);
for t in terms {
match t.0 {
T::Seq(t) => tmp.extend(t),
x => tmp.push(x),
}
}
Term(T::Seq(tmp))
}
/// Term corresponding to a list of terms.
/// ///
/// ``` /// ```
/// use nettext::enc::*; /// use nettext::enc::*;
@ -103,41 +137,11 @@ pub fn raw(bytes: &[u8]) -> Result<'_> {
/// assert_eq!(list([ /// assert_eq!(list([
/// string("Hello").unwrap(), /// string("Hello").unwrap(),
/// string("world").unwrap() /// string("world").unwrap()
/// ]).unwrap().encode(), b"Hello world"); /// ]).encode(), b"[\n Hello,\n world,\n]");
/// ``` /// ```
pub fn list<'a, I: IntoIterator<Item = Term<'a>>>(terms: I) -> Result<'a> { pub fn list<'a, I: IntoIterator<Item = Term<'a>>>(terms: I) -> Term<'a> {
let mut tmp = Vec::with_capacity(8); let terms = terms.into_iter().map(|x| x.0).collect::<Vec<_>>();
for t in terms { Term(T::List(terms))
match t.0 {
T::List(_) => return Err(Error::ListInList),
x => tmp.push(x),
}
}
Ok(Term(T::List(tmp)))
}
/// Term corresponding to a list of terms. Sub-lists are flattenned.
pub fn list_flatten<'a, I: IntoIterator<Item = Term<'a>>>(terms: I) -> Result<'a> {
let mut tmp = Vec::with_capacity(8);
for t in terms {
match t.0 {
T::List(t) => tmp.extend(t),
x => tmp.push(x),
}
}
Ok(Term(T::List(tmp)))
}
/// Term corresponding to a list of terms. Sub-lists are represented as NESTED: `{.= sub list items }`.
pub fn list_nested<'a, I: IntoIterator<Item = Term<'a>>>(terms: I) -> Result<'a> {
let mut tmp = Vec::with_capacity(8);
for t in terms {
match t.0 {
T::List(t) => tmp.push(Term(T::List(t)).nested().0),
x => tmp.push(x),
}
}
Ok(Term(T::List(tmp)))
} }
/// Term corresponding to a dictionnary of items /// Term corresponding to a dictionnary of items
@ -177,7 +181,7 @@ pub fn bytes(bytes: &[u8]) -> Term<'static> {
} }
/// Same as `bytes()`, but splits the byte slice in 48-byte chunks /// Same as `bytes()`, but splits the byte slice in 48-byte chunks
/// and encodes each chunk separately, putting them in a list of terms. /// and encodes each chunk separately, putting them in a sequence of terms.
/// Usefull for long byte slices to have cleaner representations, /// Usefull for long byte slices to have cleaner representations,
/// mainly usefull for dictionnary keys. /// mainly usefull for dictionnary keys.
pub fn bytes_split(bytes: &[u8]) -> Term<'static> { pub fn bytes_split(bytes: &[u8]) -> Term<'static> {
@ -186,7 +190,7 @@ pub fn bytes_split(bytes: &[u8]) -> Term<'static> {
.map(|b| T::OwnedStr(base64::encode_config(b, base64::URL_SAFE_NO_PAD).into_bytes())) .map(|b| T::OwnedStr(base64::encode_config(b, base64::URL_SAFE_NO_PAD).into_bytes()))
.collect::<Vec<_>>(); .collect::<Vec<_>>();
if chunks.len() > 1 { if chunks.len() > 1 {
Term(T::List(chunks)) Term(T::Seq(chunks))
} else { } else {
Term(chunks.into_iter().next().unwrap_or(T::Str(b"."))) Term(chunks.into_iter().next().unwrap_or(T::Str(b".")))
} }
@ -194,15 +198,15 @@ pub fn bytes_split(bytes: &[u8]) -> Term<'static> {
impl<'a> Term<'a> { impl<'a> Term<'a> {
/// Append a term to an existing term. /// Append a term to an existing term.
/// Transforms the initial term into a list if necessary. /// Transforms the initial term into a seq if necessary.
#[must_use] #[must_use]
pub fn append(self, t: Term<'a>) -> Term<'a> { pub fn append(self, t: Term<'a>) -> Term<'a> {
match self.0 { match self.0 {
T::List(mut v) => { T::Seq(mut v) => {
v.push(t.0); v.push(t.0);
Term(T::List(v)) Term(T::Seq(v))
} }
x => Term(T::List(vec![x, t.0])), x => Term(T::Seq(vec![x, t.0])),
} }
} }
@ -219,21 +223,6 @@ impl<'a> Term<'a> {
_ => Err(Error::NotADictionnary), _ => Err(Error::NotADictionnary),
} }
} }
/// Makes a NESTED term of this term, by putting it in a dict
/// with a single key `.`.
///
/// Example:
///
/// ```
/// use nettext::enc::*;
///
/// assert_eq!(list([string("hello").unwrap(), string("world").unwrap()]).unwrap().nested().encode(), b"{.= hello world }");
/// ```
#[must_use]
pub fn nested(self) -> Term<'a> {
dict([(".", self)]).unwrap()
}
} }
// ---- additional internal functions for serde module ---- // ---- additional internal functions for serde module ----
@ -268,6 +257,11 @@ impl<'a> Term<'a> {
self.0.encode_aux(&mut buf, 0, true); self.0.encode_aux(&mut buf, 0, true);
buf buf
} }
/// Generate the nettext representation of a term, as a String
pub fn encode_string(self) -> String {
unsafe { String::from_utf8_unchecked(self.encode()) }
}
} }
impl<'a> T<'a> { impl<'a> T<'a> {
@ -280,13 +274,9 @@ impl<'a> T<'a> {
buf.extend_from_slice(b"{}"); buf.extend_from_slice(b"{}");
} else if d.len() == 1 { } else if d.len() == 1 {
let (k, v) = d.into_iter().next().unwrap(); let (k, v) = d.into_iter().next().unwrap();
if k.as_ref() == b"." { buf.extend_from_slice(b"{ ");
buf.extend_from_slice(b"{.= "); buf.extend_from_slice(k.borrow());
} else { buf.extend_from_slice(b" = ");
buf.extend_from_slice(b"{ ");
buf.extend_from_slice(k.borrow());
buf.extend_from_slice(b" = ");
}
v.encode_aux(buf, indent + 2, false); v.encode_aux(buf, indent + 2, false);
buf.extend_from_slice(b" }"); buf.extend_from_slice(b" }");
} else { } else {
@ -310,7 +300,30 @@ impl<'a> T<'a> {
buf.push(b'}'); buf.push(b'}');
} }
} }
T::List(l) => { T::List(l) => {
if l.len() == 0 {
buf.extend_from_slice(b"[]");
} else if l.len() == 1 {
buf.extend_from_slice(b"[ ");
l.into_iter().next().unwrap().encode_aux(buf, indent + 2, false);
buf.extend_from_slice(b" ]");
} else {
let indent2 = indent + 2;
buf.extend_from_slice(b"[\n");
for item in l {
for _ in 0..indent2 {
buf.push(b' ');
}
item.encode_aux(buf, indent2, false);
buf.extend_from_slice(b",\n");
}
for _ in 0..indent {
buf.push(b' ');
}
buf.push(b']');
}
}
T::Seq(l) => {
let indent2 = indent + 2; let indent2 = indent + 2;
for (i, v) in l.into_iter().enumerate() { for (i, v) in l.into_iter().enumerate() {
if !is_toplevel && buf.iter().rev().take_while(|c| **c != b'\n').count() >= 70 { if !is_toplevel && buf.iter().rev().take_while(|c| **c != b'\n').count() >= 70 {
@ -330,13 +343,18 @@ impl<'a> T<'a> {
#[cfg(test)] #[cfg(test)]
mod tests { mod tests {
use crate::debug;
use super::*; use super::*;
#[test] #[test]
fn complex1() { fn complex1() {
let input = list([ let input = seq([
string("HELLO").unwrap(), string("HELLO").unwrap(),
string("alexhelloworld").unwrap(), string("alexhelloworld").unwrap(),
list([
string("dude").unwrap(),
string("why").unwrap(),
]),
dict([ dict([
("from", string("jxx").unwrap()), ("from", string("jxx").unwrap()),
("subject", string("hello").unwrap()), ("subject", string("hello").unwrap()),
@ -345,37 +363,17 @@ mod tests {
.unwrap(), .unwrap(),
]) ])
.unwrap(); .unwrap();
let expected = b"HELLO alexhelloworld { let expected = "HELLO alexhelloworld [
dude,
why,
] {
data = { f1 = plop, f2 = kuko }, data = { f1 = plop, f2 = kuko },
from = jxx, from = jxx,
subject = hello, subject = hello,
}"; }";
let enc = input.encode(); let enc = input.encode();
eprintln!("{}", std::str::from_utf8(&enc).unwrap()); eprintln!("{}", debug(&enc));
eprintln!("{}", std::str::from_utf8(&expected[..]).unwrap()); eprintln!("{}", expected);
assert_eq!(&enc, &expected[..]); assert_eq!(debug(&enc), expected);
}
#[test]
fn nested() {
assert!(list([
string("a").unwrap(),
string("b").unwrap(),
list([string("c").unwrap(), string("d").unwrap()]).unwrap()
])
.is_err());
assert_eq!(
list([
string("a").unwrap(),
string("b").unwrap(),
list([string("c").unwrap(), string("d").unwrap()])
.unwrap()
.nested()
])
.unwrap()
.encode(),
b"a b {.= c d }"
);
} }
} }

View file

@ -9,7 +9,7 @@
//! let keypair = SigningKeyPair::gen_with_defaults(); //! let keypair = SigningKeyPair::gen_with_defaults();
//! //!
//! // Encode a fist object that represents a payload that will be hashed and signed //! // Encode a fist object that represents a payload that will be hashed and signed
//! let signed_payload = list([ //! let signed_payload = seq([
//! string("CALL").unwrap(), //! string("CALL").unwrap(),
//! string("myfunction").unwrap(), //! string("myfunction").unwrap(),
//! dict([ //! dict([
@ -44,7 +44,7 @@
//! //!
//! let object2 = decode(payload.raw()).unwrap(); //! let object2 = decode(payload.raw()).unwrap();
//! //!
//! let [verb, arg1, arg2, pubkey] = object2.list_of().unwrap(); //! let [verb, arg1, arg2, pubkey] = object2.seq_of().unwrap();
//! let pubkey = pubkey.public_key().unwrap(); //! let pubkey = pubkey.public_key().unwrap();
//! assert!(verify_signature(&signature, payload.raw(), &pubkey)); //! assert!(verify_signature(&signature, payload.raw(), &pubkey));
//! //!
@ -100,6 +100,9 @@ pub(crate) const DICT_OPEN: u8 = b'{';
pub(crate) const DICT_CLOSE: u8 = b'}'; pub(crate) const DICT_CLOSE: u8 = b'}';
pub(crate) const DICT_ASSIGN: u8 = b'='; pub(crate) const DICT_ASSIGN: u8 = b'=';
pub(crate) const DICT_DELIM: u8 = b','; pub(crate) const DICT_DELIM: u8 = b',';
pub(crate) const LIST_OPEN: u8 = b'[';
pub(crate) const LIST_CLOSE: u8 = b']';
pub(crate) const LIST_DELIM: u8 = b',';
pub(crate) const STR_EXTRA_CHARS: &[u8] = b"._-+*?@:"; pub(crate) const STR_EXTRA_CHARS: &[u8] = b"._-+*?@:";
pub(crate) fn is_string_char(c: u8) -> bool { pub(crate) fn is_string_char(c: u8) -> bool {
@ -109,3 +112,7 @@ pub(crate) fn is_string_char(c: u8) -> bool {
pub(crate) fn is_whitespace(c: u8) -> bool { pub(crate) fn is_whitespace(c: u8) -> bool {
c.is_ascii_whitespace() c.is_ascii_whitespace()
} }
pub(crate) fn debug(x: &[u8]) -> &str {
std::str::from_utf8(x).unwrap_or("<invalid ascii>")
}

View file

@ -6,7 +6,7 @@ use serde::de::{
}; };
use serde::Deserialize; use serde::Deserialize;
use crate::dec::debug as fmtdebug; use crate::debug as fmtdebug;
use crate::dec::*; use crate::dec::*;
use crate::serde::error::{Error, Result}; use crate::serde::error::{Error, Result};
@ -15,11 +15,7 @@ pub struct Deserializer<'de, 'a>(Term<'de, 'a>);
impl<'de, 'a> Deserializer<'de, 'a> { impl<'de, 'a> Deserializer<'de, 'a> {
fn from_term(input: &'a Term<'de, 'a>) -> Deserializer<'de, 'a> { fn from_term(input: &'a Term<'de, 'a>) -> Deserializer<'de, 'a> {
if let Ok(nested) = input.nested() { Deserializer(Term(input.0.mkref()))
Deserializer(nested)
} else {
Deserializer(Term(input.0.mkref()))
}
} }
} }
@ -228,7 +224,7 @@ impl<'de, 'a> de::Deserializer<'de> for &'a mut Deserializer<'de, 'a> {
where where
V: Visitor<'de>, V: Visitor<'de>,
{ {
let [variant, args] = self.0.list_of_first()?; let [variant, args] = self.0.seq_of_first()?;
if variant.string()? == name { if variant.string()? == name {
visitor.visit_newtype_struct(&mut Deserializer(args)) visitor.visit_newtype_struct(&mut Deserializer(args))
} else { } else {
@ -244,14 +240,14 @@ impl<'de, 'a> de::Deserializer<'de> for &'a mut Deserializer<'de, 'a> {
where where
V: Visitor<'de>, V: Visitor<'de>,
{ {
visitor.visit_seq(&mut Seq(&self.0.list())) visitor.visit_seq(&mut Seq(&self.0.list()?))
} }
fn deserialize_tuple<V>(self, _len: usize, visitor: V) -> Result<V::Value> fn deserialize_tuple<V>(self, _len: usize, visitor: V) -> Result<V::Value>
where where
V: Visitor<'de>, V: Visitor<'de>,
{ {
visitor.visit_seq(&mut Seq(&self.0.list())) visitor.visit_seq(&mut Seq(&self.0.seq()))
} }
fn deserialize_tuple_struct<V>( fn deserialize_tuple_struct<V>(
@ -263,9 +259,9 @@ impl<'de, 'a> de::Deserializer<'de> for &'a mut Deserializer<'de, 'a> {
where where
V: Visitor<'de>, V: Visitor<'de>,
{ {
let [variant, args] = self.0.list_of_first()?; let [variant, args] = self.0.seq_of_first()?;
if variant.string()? == name { if variant.string()? == name {
visitor.visit_seq(&mut Seq(&args.list())) visitor.visit_seq(&mut Seq(&args.seq()))
} else { } else {
Err(Error::custom(format!( Err(Error::custom(format!(
"Expected {}, got: `{}`", "Expected {}, got: `{}`",
@ -291,7 +287,7 @@ impl<'de, 'a> de::Deserializer<'de> for &'a mut Deserializer<'de, 'a> {
where where
V: Visitor<'de>, V: Visitor<'de>,
{ {
let [variant, data] = self.0.list_of()?; let [variant, data] = self.0.seq_of()?;
if variant.string()? != name { if variant.string()? != name {
return Err(Error::custom(format!( return Err(Error::custom(format!(
"Expected {}, got: `{}`", "Expected {}, got: `{}`",
@ -414,7 +410,7 @@ impl<'de, 'a> EnumAccess<'de> for Enum<'de, 'a> {
where where
V: DeserializeSeed<'de>, V: DeserializeSeed<'de>,
{ {
let variant = &self.0.list()[0]; let variant = &self.0.seq()[0];
let variant = seed.deserialize(&mut Deserializer(Term(variant.0.mkref())))?; let variant = seed.deserialize(&mut Deserializer(Term(variant.0.mkref())))?;
Ok((variant, self)) Ok((variant, self))
} }
@ -424,7 +420,7 @@ impl<'de, 'a> VariantAccess<'de> for Enum<'de, 'a> {
type Error = Error; type Error = Error;
fn unit_variant(self) -> Result<()> { fn unit_variant(self) -> Result<()> {
if self.0.list().len() > 1 { if self.0.seq().len() > 1 {
Err(Error::custom("Spurrious data in unit variant")) Err(Error::custom("Spurrious data in unit variant"))
} else { } else {
Ok(()) Ok(())
@ -435,7 +431,7 @@ impl<'de, 'a> VariantAccess<'de> for Enum<'de, 'a> {
where where
T: DeserializeSeed<'de>, T: DeserializeSeed<'de>,
{ {
let [_, rest] = self.0.list_of_first()?; let [_, rest] = self.0.seq_of_first()?;
seed.deserialize(&mut Deserializer(rest)) seed.deserialize(&mut Deserializer(rest))
} }
@ -443,15 +439,15 @@ impl<'de, 'a> VariantAccess<'de> for Enum<'de, 'a> {
where where
V: Visitor<'de>, V: Visitor<'de>,
{ {
let [_, rest] = self.0.list_of_first()?; let [_, rest] = self.0.seq_of_first()?;
visitor.visit_seq(&mut Seq(&rest.list())) visitor.visit_seq(&mut Seq(&rest.seq()))
} }
fn struct_variant<V>(self, _fields: &'static [&'static str], visitor: V) -> Result<V::Value> fn struct_variant<V>(self, _fields: &'static [&'static str], visitor: V) -> Result<V::Value>
where where
V: Visitor<'de>, V: Visitor<'de>,
{ {
let [_, rest] = self.0.list_of_first()?; let [_, rest] = self.0.seq_of_first()?;
visitor.visit_map(&mut Dict::from_term(&rest)?) visitor.visit_map(&mut Dict::from_term(&rest)?)
} }
} }

View file

@ -13,15 +13,18 @@ mod tests {
use super::*; use super::*;
use serde::{Deserialize, Serialize}; use serde::{Deserialize, Serialize};
use std::collections::HashMap; use std::collections::HashMap;
use crate::debug;
fn test_bidir<T: Serialize + for<'de> Deserialize<'de> + PartialEq + std::fmt::Debug>( fn test_bidir<T: Serialize + for<'de> Deserialize<'de> + PartialEq + std::fmt::Debug>(
input: T, input: T,
expected: &[u8], expected: &str,
) { ) {
eprintln!("Expecting: {}", expected);
let ser = to_bytes(&input).unwrap(); let ser = to_bytes(&input).unwrap();
eprintln!("Serialized: {}", std::str::from_utf8(&ser).unwrap()); let ser = debug(&ser);
assert_eq!(&ser, expected); eprintln!("Serialized: {}", ser);
assert_eq!(&from_bytes::<T>(&ser).unwrap(), &input); assert_eq!(ser, expected);
assert_eq!(from_bytes::<T>(ser.as_bytes()).unwrap(), input);
} }
#[test] #[test]
@ -36,9 +39,12 @@ mod tests {
int: 1, int: 1,
seq: vec!["a".to_string(), "b".to_string()], seq: vec!["a".to_string(), "b".to_string()],
}; };
let expected = br#"Test { let expected = r#"Test {
int = 1, int = 1,
seq = YQ Yg, seq = [
YQ,
Yg,
],
}"#; }"#;
test_bidir(input, expected); test_bidir(input, expected);
@ -52,13 +58,22 @@ mod tests {
seq: vec!["c".to_string(), "d".to_string()], seq: vec!["c".to_string(), "d".to_string()],
}, },
]; ];
let expected = br#"{.= Test { let expected = r#"[
int = 1, Test {
seq = YQ Yg, int = 1,
} } {.= Test { seq = [
int = 2, YQ,
seq = Yw ZA, Yg,
} }"#; ],
},
Test {
int = 2,
seq = [
Yw,
ZA,
],
},
]"#;
test_bidir(input, expected); test_bidir(input, expected);
} }
@ -73,19 +88,19 @@ mod tests {
} }
let input = E::Unit; let input = E::Unit;
let expected = br#"E.Unit"#; let expected = r#"E.Unit"#;
test_bidir(input, expected); test_bidir(input, expected);
let input = E::Newtype(1); let input = E::Newtype(1);
let expected = br#"E.Newtype 1"#; let expected = r#"E.Newtype 1"#;
test_bidir(input, expected); test_bidir(input, expected);
let input = E::Tuple(1, 2); let input = E::Tuple(1, 2);
let expected = br#"E.Tuple 1 2"#; let expected = r#"E.Tuple 1 2"#;
test_bidir(input, expected); test_bidir(input, expected);
let input = E::Struct { a: 1 }; let input = E::Struct { a: 1 };
let expected = br#"E.Struct { a = 1 }"#; let expected = r#"E.Struct { a = 1 }"#;
test_bidir(input, expected); test_bidir(input, expected);
let input = vec![ let input = vec![
@ -97,36 +112,45 @@ mod tests {
E::Tuple(3, 2), E::Tuple(3, 2),
]; ];
let expected = let expected =
br#"E.Unit E.Unit {.= E.Newtype 1 } {.= E.Tuple 1 2 } {.= E.Struct { a = 1 } } {.= r#"[
E.Tuple 3 2 }"#; E.Unit,
E.Unit,
E.Newtype 1,
E.Tuple 1 2,
E.Struct { a = 1 },
E.Tuple 3 2,
]"#;
test_bidir(input, expected); test_bidir(input, expected);
} }
#[test] #[test]
fn test_seq1() { fn test_seq1() {
let input = (1, 2, 3, 4); let input = (1, 2, 3, 4);
let expected = br#"1 2 3 4"#; let expected = r#"1 2 3 4"#;
test_bidir(input, expected); test_bidir(input, expected);
} }
#[test] #[test]
fn test_seq2() { fn test_list() {
let input = (1, 2, (2, 3, 4), 5, 6); let input = vec![1, 2, 3, 4];
let expected = br#"1 2 {.= 2 3 4 } 5 6"#; let expected = r#"[
1,
2,
3,
4,
]"#;
test_bidir(input, expected); test_bidir(input, expected);
} }
#[test] #[test]
fn test_seq3() { fn test_seqlist() {
let input = [1, 2, 3, 4]; let input = vec![(1, 2), (2, 3), (3, 4), (5, 6)];
let expected = br#"1 2 3 4"#; let expected = r#"[
test_bidir(input, expected); 1 2,
} 2 3,
3 4,
#[test] 5 6,
fn test_seq4() { ]"#;
let input = [[1, 2], [2, 3], [3, 4]];
let expected = br#"{.= 1 2 } {.= 2 3 } {.= 3 4 }"#;
test_bidir(input, expected); test_bidir(input, expected);
} }
@ -135,7 +159,7 @@ mod tests {
let mut input = HashMap::new(); let mut input = HashMap::new();
input.insert("hello".to_string(), "world".to_string()); input.insert("hello".to_string(), "world".to_string());
input.insert("dont".to_string(), "panic".to_string()); input.insert("dont".to_string(), "panic".to_string());
let expected = br#"{ let expected = r#"{
ZG9udA = cGFuaWM, ZG9udA = cGFuaWM,
aGVsbG8 = d29ybGQ, aGVsbG8 = d29ybGQ,
}"#; }"#;
@ -144,9 +168,16 @@ mod tests {
let mut input = HashMap::new(); let mut input = HashMap::new();
input.insert(12, vec![42, 125]); input.insert(12, vec![42, 125]);
input.insert(33, vec![19, 22, 21]); input.insert(33, vec![19, 22, 21]);
let expected = br#"{ let expected = r#"{
12 = 42 125, 12 = [
33 = 19 22 21, 42,
125,
],
33 = [
19,
22,
21,
],
}"#; }"#;
test_bidir(input, expected); test_bidir(input, expected);
} }

View file

@ -28,7 +28,7 @@ impl<'a> ser::Serializer for &'a mut Serializer {
type Error = Error; type Error = Error;
type SerializeSeq = SeqSerializer; type SerializeSeq = ListSerializer;
type SerializeTuple = SeqSerializer; type SerializeTuple = SeqSerializer;
type SerializeTupleStruct = SeqSerializer; type SerializeTupleStruct = SeqSerializer;
type SerializeTupleVariant = SeqSerializer; type SerializeTupleVariant = SeqSerializer;
@ -104,7 +104,7 @@ impl<'a> ser::Serializer for &'a mut Serializer {
where where
T: ?Sized + Serialize, T: ?Sized + Serialize,
{ {
Ok(value.serialize(self)?.nested()) Ok(value.serialize(self)?)
} }
fn serialize_unit(self) -> Result<Self::Ok> { fn serialize_unit(self) -> Result<Self::Ok> {
@ -128,7 +128,7 @@ impl<'a> ser::Serializer for &'a mut Serializer {
where where
T: ?Sized + Serialize, T: ?Sized + Serialize,
{ {
Ok(list_flatten([string(name)?, value.serialize(self)?])?) Ok(seq_flatten([string(name)?, value.serialize(self)?]))
} }
fn serialize_newtype_variant<T>( fn serialize_newtype_variant<T>(
@ -141,14 +141,14 @@ impl<'a> ser::Serializer for &'a mut Serializer {
where where
T: ?Sized + Serialize, T: ?Sized + Serialize,
{ {
Ok(list_flatten([ Ok(seq_flatten([
string_owned(format!("{}.{}", name, variant))?, string_owned(format!("{}.{}", name, variant))?,
value.serialize(self)?, value.serialize(self)?,
])?) ]))
} }
fn serialize_seq(self, _len: Option<usize>) -> Result<Self::SerializeSeq> { fn serialize_seq(self, _len: Option<usize>) -> Result<Self::SerializeSeq> {
Ok(SeqSerializer { items: vec![] }) Ok(ListSerializer { items: vec![] })
} }
fn serialize_tuple(self, len: usize) -> Result<Self::SerializeTuple> { fn serialize_tuple(self, len: usize) -> Result<Self::SerializeTuple> {
@ -213,22 +213,6 @@ impl<'a> ser::Serializer for &'a mut Serializer {
pub struct SeqSerializer { pub struct SeqSerializer {
items: Vec<Term<'static>>, items: Vec<Term<'static>>,
} }
impl ser::SerializeSeq for SeqSerializer {
type Ok = Term<'static>;
type Error = Error;
fn serialize_element<T>(&mut self, value: &T) -> Result<()>
where
T: ?Sized + Serialize,
{
self.items.push(value.serialize(&mut Serializer)?);
Ok(())
}
fn end(self) -> Result<Self::Ok> {
Ok(list_nested(self.items.into_iter())?)
}
}
impl ser::SerializeTuple for SeqSerializer { impl ser::SerializeTuple for SeqSerializer {
type Ok = Term<'static>; type Ok = Term<'static>;
@ -243,7 +227,7 @@ impl ser::SerializeTuple for SeqSerializer {
} }
fn end(self) -> Result<Self::Ok> { fn end(self) -> Result<Self::Ok> {
Ok(list_nested(self.items.into_iter())?) Ok(seq(self.items.into_iter())?)
} }
} }
@ -260,7 +244,7 @@ impl ser::SerializeTupleStruct for SeqSerializer {
} }
fn end(self) -> Result<Self::Ok> { fn end(self) -> Result<Self::Ok> {
Ok(list_nested(self.items.into_iter())?) Ok(seq(self.items.into_iter())?)
} }
} }
@ -277,7 +261,27 @@ impl ser::SerializeTupleVariant for SeqSerializer {
} }
fn end(self) -> Result<Self::Ok> { fn end(self) -> Result<Self::Ok> {
Ok(list_nested(self.items.into_iter())?) Ok(seq(self.items.into_iter())?)
}
}
pub struct ListSerializer {
items: Vec<Term<'static>>,
}
impl ser::SerializeSeq for ListSerializer {
type Ok = Term<'static>;
type Error = Error;
fn serialize_element<T>(&mut self, value: &T) -> Result<()>
where
T: ?Sized + Serialize,
{
self.items.push(value.serialize(&mut Serializer)?);
Ok(())
}
fn end(self) -> Result<Self::Ok> {
Ok(list(self.items.into_iter()))
} }
} }
@ -334,7 +338,7 @@ impl ser::SerializeStruct for StructSerializer {
} }
fn end(self) -> Result<Term<'static>> { fn end(self) -> Result<Term<'static>> {
Ok(list([string(self.name)?, dict(self.fields.into_iter())?])?) Ok(seq([string(self.name)?, dict(self.fields.into_iter())?])?)
} }
} }
@ -357,7 +361,7 @@ impl ser::SerializeStructVariant for StructVariantSerializer {
} }
fn end(self) -> Result<Term<'static>> { fn end(self) -> Result<Term<'static>> {
Ok(list([ Ok(seq([
string_owned(format!("{}.{}", self.name, self.variant))?, string_owned(format!("{}.{}", self.name, self.variant))?,
dict(self.fields.into_iter())?, dict(self.fields.into_iter())?,
])?) ])?)