Binary Format

From Ripple Wiki
Jump to: navigation, search

Contents

Binary Format

Introduction

The binary format will consist of a set of fields, each of which has a type and a name. The list of names will be specific for each type.

Encoding

The first byte is split in half. The four high-order bits are the type and the four low-order bits are the field. If the type or field is 1 through 15 inclusive, that is the type or field value. If the type or field value is zero, then a subsequent byte encodes the type or field value.

The next byte will encode the type if the type in the first byte was 0. In that case, it's an uncommon type and the next byte encodes which uncommon type it is.

The next byte will encode the value if the value in the first byte was 0. In that case, it's an uncommon value for that type, and the next byte encodes which uncommon value it is.

If the type is common and the field is common, then only a single byte is needed to encode the type and field. Code changes will be needed to support new types. Supporting new fields of a known type will only require adding a new name/number mapping.

Unless we ever add uncommon fields for uncommon types, the worst case encoding will be two bytes. It seems we'd never need this because it would have to be an uncommon field that nevertheless had so many types that they couldn't all be represented as common. In that unlikely case, three bytes would be used.

Sorting

For signing or hashing, transactions and ledgers must be sorted into canonical order. Canonical order consists of sorting numerically first by type and then by name. Multiple entries of the same name and type are only permitted inside an array which preserves order internally.

Type Encoding

Note: The most current type table is found in the src/ripple_data/protocol/SerializeDeclarations.h file.

Common Types:

  • 1: 16-bit unsigned integer
  • 2: 32-bit unsigned integer
  • 3: 64-bit unsigned integer
  • 4: 128-bit hash
  • 5: 256-bit hash
  • 6: Currency Amount
  • 7: Variable length data
  • 8: Account
  • 9-13: Reserved
  • 14: Inner object (An object consisting of multiple fields)
  • 15: Array of objects (An ordered array of objects)

Uncommon Types:

  • 16: 8-bit unsigned integer
  • 17: 160-bit unsigned integer
  • 18: Path set
  • 19: Vector of 256-bit values

Field Name Encodings

Note: The most current encodings are found in the ./src/ripple_data/protocol/SerializedDeclarations.h file. This list is not complete (in particular, it doesn't list any of the uncommon fields).

8-bit unsigned common:

  • 1: Ledger Close Time Resolution
  • 2: Template Entry Type
  • 3: Transaction Result (for metadata)

16-bit unsigned common:

  • 1: Ledger Entry Type
  • 2: Transaction Type

32-bit unsigned common:

  • 2: Flags
  • 3: Source tag
  • 4: Sequence Number
  • 5: Previous Transaction Ledger Sequence
  • 6: Ledger Sequence
  • 7: Ledger Close Time
  • 8: Parent Ledger Close Time
  • 9: Signing Time
  • 10: Expiration Time
  • 11: Transfer Rate
  • 12: Wallet Size
  • 13: Owner Count
  • 14: Destination Tag

Variable Length common:

  • 1: Public Key
  • 2: Message Key
  • 3: Signing Public Key
  • 4: Transaction Signature
  • 5: Generator
  • 6: Signature
  • 7: Domain
  • 8: Fund Script
  • 9: Remove Script
  • 10: Expire Script
  • 11: Create Script
  • 12: Memo Type
  • 13: Memo Data

Account common:

  • 1: Account
  • 2: Owner
  • 3: Destination
  • 4: Issuer
  • 7: Target
  • 8: Authorized/Regular Key

Amount common:

  • 1: Amount
  • 2: Balance
  • 3: Limit
  • 4: Taker Pays
  • 5: Taker Gets
  • 6: Low Limit
  • 7: High Limit
  • 8: Fee
  • 9: Send Maximum

128-bit common:

  • 1: Email Hash

256-bit common:

  • 1: Ledger Hash
  • 2: Parent Ledger Hash
  • 3: Transaction Tree Hash
  • 4: State Tree Hash
  • 5: Previous Transaction ID
  • 6: Ledger Index (for metadata)
  • 7: Wallet Locator
  • 8: Root Index
  • 9: Account Transaction ID

Data Encoding

8-bit integer
A single byte containing the value.
16-bit integer
Two bytes containing the value in big-endian order.
32-bit integer
Four bytes containing the value in big-endian order.
64-bit integer
Eight bytes containing the value in big-endian order.
128-bit hash
16 bytes containing the value in big-endian order.
160-bit hash
20 bytes containing the value in big-endian order.
256-bit hash
32 bytes containing the value in big-endian order.
Amount
See below.
Variable length data
See below.
Account
Same binary format as variable length data, but currently always contains a 160-bit account ID.
Object
Each inner object in its normal binary format. The last inner object should be followed by an "end of object" marker. The "end of object marker" consists of a single byte that would encode a type of "object" and a field name of "1".
Array
Each inner object in its normal binary format. The last array entry should be followed by an "end of array" marker. The "end of array" marker" consists of a single byte that would encode a type of "array" and a field name of "1".
Path Set
See below.
Vector of 256-bit hashes
Same binary format as variable length data. Contains 0 or more 256-bit hashes.

Amount Encoding

Native Currency

Native amounts are indicated by the most-significant bit (0x8000000000000000) being clear. The remaining 63 bits represent a sign-and-magnitude integer. Positive amounts are encoded with the second highest bit (0x4000000000000000) set. The lower 62 bits denote the absolute value.

Examples:

  0  0x4000000000000000
  1  0x4000000000000001
 -1  0x0000000000000001
256  0x4000000000000100

Ripple/IOU Currencies

Amounts of non-native currencies are indicated by the most-significant bit (0x8000000000000000) being set. They are encoded as a 64-bit raw amount followed by a 160-bit currency identifier followed by a 160-bit issuer. The issuer is always present, even if zero to indicate any issuer is acceptable.

The 64-bit raw amount is encoded with the most-significant bit set and the second most significant bit set if the raw amount is greater than zero. If the raw amount is zero, the remaining bits are zero. Otherwise, the remaining bits encode the mantissa (between 10^15 and 10^16-1) and exponent.

See Currency Format for more details. Caution: That information is partially out of date.

Variable Length Data Encoding

A variable-length type contains a length indicator followed by the data. The length of the data is determined based on the value of the first byte as follows:

0 - 192
The length is 0 to 192, occupies 1-byte, and is the value of the first byte
193 - 240
The length 193 to 12,480, occupies 2-bytes, and is computed: 193 + (b1-193)*256 + b2
241 - 254
The length is 12,481 to 918,744, occupies 3-bytes, and is computed: 12481 + (b1-241)*65536 + b2*256 + b3
255
Reserved

Path Set Encoding

A path set contains one or more paths, separated by a boundary byte (0xFF) and terminated by an end byte (0x00). Each path contains zero or more path entries.

So a path set that looked like this, "Entry, Boundary, Entry, Entry, End" would consist of two paths, one with one entry and one with two entries. Note that a zero-entry path is perfectly legal.

The smallest path set consists of just an "End" marker, a single 0x00 byte. This would indicate one path, an empty path. This is semantically meaningful because path sets are typically used in cases where the source and destination is specified elsewhere and only the intermediate points are specified.

Path Entry

Each path entry begins with a type byte that can contain the following bits:

Account Bit
0x01 - indicates going through an account rather than an offer
Redeem Bit (proposed)
0x02 - indicates redeeming rather than issuing
Currency Bit
0x10 - indicates a change of currency
Issuer Bit
0x20 - indicate an issuer is specified

Next, a 160-bit account follows if the account bit is set. Then, a 160-bit currency follows if the currency bit is set. Lastly, a 160-bit issuer follows if the issuer bit is set.

Ledger Entries

A ledger entry is stored at a particular 256-bit index in the state tree. Because the tree stores this index separately, they are not part of the ledger entry's binary encoding. When presented as JSON, an additional 'index' virtual field (containing the ledger index) is added.

Transactions

The signing hash of a transaction is the prefix hash (using the STX\0 prefix) of the transaction in binary form. Obviously, any signature fields must first be removed before computing this hash.

Transactions are identified by a 256-bit transaction ID. This ID is the prefix hash (using the TXN\0 prefix) of the transaction with its signature.

A transaction in JSON can be presented with or without a hash field containing the transaction ID. This field should never be presented in binary (and has no binary encoding).

See also: Transaction Malleability

Templates

A template is a set of rules that an object must match. Templates will be primarily used for m-of-n transactions. Signers will approve a template and then that template can be used to allow a transaction.

CAUTION: A final decision whether or not to implement templates has not been made yet. The contract mechanism may include this functionality another way.

Overall Structure

A template is an array. The type name for a template is 'Template'. The array consists of arrays of type either 'Necessary' or 'Sufficient'. Each 'Necessary' of 'Sufficient' array contains entries of three types: 'Necessary', 'Sufficient', or a field match specification.

Necessary and Sufficient

For a template to match an object, all of its 'Necessary' entries must match (if there are any) or at least one 'Sufficient' entry must match. Typically, a template would consist of either all 'Necessary' entries and act as an AND, or all 'Sufficient' entries and act as an OR. By rule, an empty template always matches.

Compare Types

Inside 'Necessary' and 'Sufficient' entries can be other 'Necessary' and 'Sufficient' entries and field match specifications. A field match specification is an array consisting of two members, a 'TemplateEntryType' and an arbitrary field. A field match specification matches if the rules for that 'TemplateEntryType' are satisfied. The basic types are:

  • 1: Must be present: A field of this type must be present. The value of the field in both the object matched and the template entry is ignored.
  • 2: Must be absent: A field of this type must be absent. The value of the field in both the object matched and the template entry is ignored.
  • 3: Must be equal to: The field must be present in the object and must match this entry.
  • 4: Must be equal to if present: The field may be absent, but if present must match this entry.
  • 5: Must be less than: The field may be absent, but if present must be less than this entry.
  • 6: Must be less than if present: The field may be absent, but if present must be less than this entry.
  • 7: Must be less than or equal to: The field must be present in the object and must be less than or equal to this entry.
  • 8: Must be less than or equal to if present: The field may be absent, but if present must be less than or equal to this entry.
  • 9: Must be greater than: The field must be present in the object and must be greater than this entry.
  • 10: Must be greater than if present: The field may be absent, but if present must be greater than this entry.
  • 11: Must be greater than or equal to: The field must be present in the object and must be greater than or equal to this entry.
  • 12: Must be greater than or equal to if present: The field may be absent, but if present must be greater than or equal to this entry.

Note that relative comparison types (5-12) are only defined for types that have a numeric value such as integer and amount types. They all evaluate to false if they are applied to incomparable amount types.

Special Compare Types:

  • 128: Field not in object must be less than
  • 129: Field not in object must be less than or equal to
  • 130: Field not in object must be greater than
  • 131: Field not in object must be greater than or equal to

These are defined for comparisons on transactions only. They can be applied to fields like 'LedgerSequence' and 'ParentCloseTime' that apply to the transaction execution context. These permit templates to expire.

Personal tools
Namespaces

Variants
Actions
Navigation
Tools