Hello World
The original hello.c program was first published in "THE C PROGRAMMING LANGUAGE" in 1978. The authors wrote in the "Getting Started" section:
The only way to learn a new programming language is by writing programs in it.
Similarly, we believe that in order to learn a new protocol you have to write specifications (schemas) and programs that build and parse the messages in the schemas. You may have access to an environment that generates the encoders and decoders for you. Otherwise, you will have to code them by hand.
A simple Blink schema:
Hello/1 ->
string Greeting
A message in Blink tagged text format:
@Hello|Greeting=Hello world
In Blink Compact binary format:
0d 01 0b 48 65 6c 6c 6f 20 77 6f 72 6c 64
In Blink Compact binary format with comments:
0d // MsgSize: 13 01 // MsgType: 1 (Hello) 0b // Greeting: 11 bytes 48 65 6c 6c 6f 20 77 6f // "Hello " 72 6c 64 // "world"
In Blink Native binary format:
1c 00 00 00 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0b 48 65 6c 6c 6f 20 77 6f 72 6c 64
In Blink Native binary format with comments:
1c 00 00 00 // MsgSize: 28 01 00 00 00 00 00 00 00 // MsgType: 1 (Hello) 00 00 00 00 // Extension offset: N/A 00 00 00 00 // Greeting: offset // -- data area -- 0b // Greeting: 11 bytes 48 65 6c 6c 6f 20 77 6f // "Hello " 72 6c 64 // "world"
The schema annotated with a maximum size for the Greeting field:
Hello/1 ->
string (12) Greeting
In Blink Native binary format (with schema annotation):
19 00 00 00 01 00 00 00 00 00 00 00 00 00 00 00 0b 48 65 6c 6c 6f 20 77 6f 72 6c 64 00
In Blink Native binary format with comments:
19 00 00 00 // MsgSize: 25 01 00 00 00 00 00 00 00 // MsgType: 1 (Hello) 00 00 00 00 // Extension offset: N/A 0b // Greeting: 11 bytes 48 65 6c 6c 6f 20 77 6f // "Hello " 72 6c 64 00 // "world" (1 pad byte)
A Slightly Longer Example
A Blink schema may have a namespace specified.
namespace Tutorial
The following schema defines a few messages that allow a client to insert and cancel orders, and a server to acknowledge that an order has been inserted or canceled.
InsertOrder/1 -> string Symbol, string OrderId, u32 Price, u32 Quantity CancelOrder/2 -> string OrderId OrderInserted/3 -> string OrderId OrderCanceled/4 -> string OrderId
This schema has all information a Blink codec needs to encode any of the four messages defined.
An InsertOrder message in Blink tagged text format:
@Tutorial:InsertOrder|Symbol=IBM|OrderId=ABC123|Price=125|Quantity=1000
In Blink Compact binary format:
0f 01 03 49 42 4d 06 41 42 43 31 32 33 7d a8 0f
In Blink Compact binary format with comments:
0f // MsgSize: 15 bytes 01 // MsgType: 1 (InsertOrder) 03 // Symbol: 3 bytes 49 42 4d // "IBM" 06 // OrderId: 6 bytes 41 42 43 31 32 33 // "ABC123" 7d // Price: 125 a8 0f // Quantity: 1000
An OrderInserted message in Blink tagged text format:
@Tutorial:OrderInserted|OrderId=ABC123
In Blink Compact binary format:
08 03 06 41 42 43 31 32 33
In Blink Compact binary format with comments:
08 // MsgSize: 8 bytes 03 // MsgType: 3 (OrderInserted) 06 // OrderId: 6 bytes 41 42 43 31 32 33 // "ABC123"
An InsertOrder With Overlong Price
Some applications use ILP (Integrated Layer Processing) to increase performance. For example: a response message may be created ahead of time and one or a few fields are then updated to match the exact scenario. In the case of an InsertOrder it could be used to send a hedge order to the market in response to a received execution message.
Overlong fields are legal in Blink Compact binary format so it is easy to prepare a message and just fill in one field to minimize the encoding overhead in the critical path.
The value 0 (zero) can be represented in an overlong form:
c4 00 00 00 00
Using the InsertOrder example above, a prepared encoding could be:
13 01 03 49 42 4d 06 41 42 43 31 32 33 c4 00 00 00 00 a8 0f
In Blink Compact binary format with comments:
13 // MsgSize: 19 bytes 01 // MsgType: 1 (InsertOrder) 03 // Symbol: 3 bytes 49 42 4d // "IBM" 06 // OrderId: 6 bytes 41 42 43 31 32 33 // "ABC123" c4 00 00 00 00 // Price: 0 (overlong) a8 0f // Quantity: 1000
The application can then replace the overlong zero price with another overlong price at byte offset 13 instead of having to rebuild the entire message.
An InsertOrder With Optional Fields
Let's extend the schema definition of InsertOrder to include two optional fields at the end of the message.
InsertOrder/1 -> string Symbol, string OrderId, u32 Price, u32 Quantity, string ExecInst?, string ClearingAccount?
An InsertOrder message with absent optional fields:
@Tutorial:InsertOrder|Symbol=IBM|OrderId=ABC123|Price=125|Quantity=1000
In Blink Compact binary format:
0f 01 03 49 42 4d 06 41 42 43 31 32 33 7d a8 0f c0 c0
In Blink Compact binary format with comments:
11 // MsgSize: 17 bytes 01 // MsgType: 1 (InsertOrder) 03 // Symbol: 3 bytes 49 42 4d // "IBM" 06 // OrderId: 6 bytes 41 42 43 31 32 33 // "ABC123" 7d // Price: 125 a8 0f // Quantity: 1000 c0 // ExecInst: NULL (absent) c0 // ClearingAccount: NULL (absent)
The Blink encoding rules allow trailing absent optional fields to be optimized away.
In Blink Compact binary format:
0f 01 03 49 42 4d 06 41 42 43 31 32 33 7d a8 0f
In blink Compact binary format with comments:
0f // MsgSize: 15 bytes 01 // MsgType: 1 (InsertOrder) 03 // Symbol: 3 bytes 49 42 4d // "IBM" 06 // OrderId: 6 bytes 41 42 43 31 32 33 // "ABC123" 7d // Price: 125 a8 0f // Quantity: 1000
If you compare this InsertOrder example with the first example given further above, you will find that the two absent fields at the end of the message do not result in any extra overhead.
Absent optional fields that are not located at the end of a message will obviously have to be encoded with a NULL.
A Schema Exchange Example
Blink schemas can be encoded in Blink format. The Blink schema for Blink schemas is defined in the Blink Schema Exchange Specification, and can also be downloaded separately here.
Given the schema for InsertOrder:
InsertOrder/1 -> string Symbol, string OrderId, u32 Price, u32 Quantity
The schema in Blink Compact binary format:
4f // Msg size=79 81 fa // Msg Blink:GroupDef, id=16001 c0 // Annotations: NULL // Name: -> c0 // Ns: NULL 0b // Name: 11 bytes 49 6e 73 65 72 74 4f 72 // "InsertOrder" 64 65 72 // ... 01 // Id: 1 04 // Fields: [4] // 0 -> c0 // Annotations: NULL 06 // Name: 6 bytes 53 79 6d 62 6f 6c // "Symbol" c0 // Id: NULL // Type: -> 04 // DynGrp size=4 86 fa // DynGrp Blink:String, id=16006 c0 // Annotations: NULL c0 // MaxSize: NULL 00 // Optional: false // 1 -> c0 // Annotations: NULL 07 // Name: 7 bytes 4f 72 64 65 72 49 64 // "OrderId" c0 // Id: NULL // Type: -> 04 // DynGrp size=4 86 fa // DynGrp Blink:String, id=16006 c0 // Annotations: NULL c0 // MaxSize: NULL 00 // Optional: false // 2 -> c0 // Annotations: NULL 05 // Name: 5 bytes 50 72 69 63 65 // "Price" c0 // Id: NULL // Type: -> 03 // DynGrp size=3 8e fa // DynGrp Blink:U32, id=16014 c0 // Annotations: NULL 00 // Optional: false // 3 -> c0 // Annotations: NULL 08 // Name: 8 bytes 51 75 61 6e 74 69 74 79 // "Quantity" c0 // Id: NULL // Type: -> 03 // DynGrp size=3 8e fa // DynGrp Blink:U32, id=16014 c0 // Annotations: NULL 00 // Optional: false
An InsertOrder encoded in Blink Native format
We can reuse the the schema definition of InsertOrder without any additional annotations.
InsertOrder/1 -> string Symbol, string OrderId, u32 Price, u32 Quantity, string ExecInst?, string ClearingAccount?
An InsertOrder message in Blink tagged text format:
@Tutorial:InsertOrder|Symbol=IBM|OrderId=ABC123|Price=125|Quantity=1000
In Blink Native binary format:
3b 00 00 00 01 00 00 00 00 00 00 00 00 00 00 00 18 00 00 00 19 00 00 00 7d 00 00 00 e8 03 00 00 00 00 00 00 00 00 00 00 00 00 03 00 00 00 49 42 4d 06 00 00 00 41 42 43 31 32 33
In Blink Native binary format with comments:
3b 00 00 00 // MsgSize: 59 bytes 01 00 00 00 00 00 00 00 // MsgType: 1 (InsertOrder) 00 00 00 00 // Extension offset: N/A 1a 00 00 00 // Symbol offset: 26 1d 00 00 00 // OrderId offset: 29 7d 00 00 00 // Price: 125 e8 03 00 00 // Quantity: 1000 00 // ExecInst: absent 00 00 00 00 // ExecInst offset: N/A 00 // ClearingAccount: absent 00 00 00 00 // ClearingAccount offset: N/A // -- data area -- 03 00 00 00 // Symbol: 3 bytes 49 42 4d // "IBM" 06 00 00 00 // OrderId: 6 bytes 41 42 43 31 32 33 // "ABC123"
Another InsertOrder encoded in Blink Native format
We can specify a fixed capacity in the schema for strings in InsertOrder.
InsertOrder/1 -> string (8) Symbol, string (12) OrderId, u32 Price, u32 Quantity, string ExecInst?, string ClearingAccount?
An InsertOrder message in Blink tagged text format:
@Tutorial:InsertOrder|Symbol=IBM|OrderId=ABC123|Price=125|Quantity=1000
In Blink Native binary format:
38 00 00 00 01 00 00 00 00 00 00 00 00 00 00 00 03 49 42 4d 00 00 00 00 00 06 41 42 43 31 32 33 00 00 00 00 00 00 7d 00 00 00 e8 03 00 00 00 00 00 00 00 00 00 00 00 00
In Blink Native binary format with comments:
38 00 00 00 // MsgSize: 56 bytes 01 00 00 00 00 00 00 00 // MsgType: 1 (InsertOrder) 00 00 00 00 // Extension offset: N/A 03 // Symbol: 3 bytes 49 42 4d 00 00 00 00 00 // "IBM" 06 // OrderId: 6 bytes 41 42 43 31 32 33 00 00 // "ABC123" 00 00 00 00 // (6 pad bytes) 7d 00 00 00 // Price: 125 e8 03 00 00 // Quantity: 1000 00 // ExecInst: absent 00 00 00 00 // ExecInst offset: N/A 00 // ClearingAccount: absent 00 00 00 00 // ClearingAccount offset: N/A // -- data area --