MessageML is a markup for representing messages including formatting (bold, italic, numbered and unnumbered lists etc.) which is used by the existing Symphony Agent API. Complex Financial Objects will be represented as entities in MessageML markup. This page describes both the current PresentationML and MessageML formats as well as the proposed EntityML extensions to support the representation of complex financial objects.

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",    "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119.

MessageML messages are transmitted over the Symphony platform and may be composed either by the integral Symphony Rich Text Editor or by an installed application.

When received these messages will be rendered either by the integral Symphony rendering engine or by an installed application.

The sending and receiving users may have a different set of applications installed (or no applications installed) and may have different preferences of which installed application to use to compose and render messages of any particular type. It is therefore necessary that different applications, developed independently of each other MUST be capable of generating and interpreting MessageML in a consistent and predictable way.

When a message author expresses formatting for a message this should be done in a semantic rather than a presentational way. For example, the message author may say that there is a paragraph break in a message but it is up to the renderer to decide precisely how that should be rendered, e.g. whether the space between the end of one paragraph and the next should be 2 pixels or 15 pixels. In an ideal world we would want to mark up messages to indicate emphasis in a presentationally neutral way for example by referring to a word being "emphasized" as opposed to being "bold" or "italic".

However, the existing rich text editor provides buttons labelled as bold and italic and users are familiar with these terms as ways of editing a message. We will therefore treat bold and italic as semantic formatting, even though these terms express a presentational concept.

Actors in the Construction and Presentation of a Message

Messages in the Symphony messaging system are created by one user and received by another. The Symphony platform can be extended and augmented by the installation of Applications (Apps) which may add the capability to compose and / or render various complex structured objects. APIs provide mechanisms for computer systems to act as Symphony users and to originate and consume messages.

There are several actors in the lifecycle of a single Symphony message sent to a single user:

1. The Originating User composes the message and decides who to send it to.
2. One or more Apps may be invoked to provide UI support to compose structured complex objects included in the message. These are known as Composing Apps, and they are responsible for constructing messages as defined by this document.
3. The Originating User Agent takes the constructed message and submits it for delivery to the intended user. The User Agent is also responsible for validating messages and may reject them if they are invalid, and for encrypting messages on behalf of the Originating User. Examples of User Agents are the Symphony Web UI, the Symphony iOS Client App, and the API Agent process.
4. The Symphony Service is responsible for the physical storage, transmission and delivery to the Receiving User Agent.
5. The Receiving User Agent is responsible for validating received messages and informing the Receiving User of any errors in the structure of received messages, and for selecting appropriate Apps to render complex structured objects included in received messages.
6. One or more Apps may be invoked to enrich complex objects contained in the message. The enrichment Apps may add to, but may not delete or change, the representation of the complex entities.
7. One or more Apps may be invoked to render included complex structured objects in the message. These are known as Rendering Apps and they are responsible for rendering complex structured objects in several formats.

There is no guarantee (and it may well be improbable that) the originator and recipient of a message containing a complex structured entity have the same set of Apps installed, and it must therefore be assumed that the Composing and Rendering Apps for any particular message will not be the same.

It is a design intent of the markup that receivers who are not able to interpret the full detail of a received entity should be able to fall back gracefully and should be able to interpret parts of embedded entities which they can understand so as to provide the best possible experience for their user.

Because messages in the Symphony platform are encrypted and accessible only to the originator and recipients, it is difficult to systematically ensure that badly formed messages do not enter the system. It is probable that badly formed messages will be received by Receiving User Agents which must therefore be able to deal with such messages.

It is a requirement that all User Agents reject badly formed messages when acting as originators and normalize badly formed messages when acting as receivers.

MessageML in general, and EntityML in particular, are defined with the intention that all messages, including badly formed ones, will be rendered predictably and consistently by any compliant rendering application. For this reason, this document specifies some constraints which are mandatory for all message originators and also defines rules for how messages should be interpreted if these constraints are not met.

Receiving User Agents MUST apply these rules so that Rendering Apps are not faced with badly formed messages and that faulty Composing Apps are not able to inject badly formed messages into the system.

Versioning

It is inevitable that over time the need to make changes to the definitions of entities will arise. Complex entities include a version identifier which MUST contain a two level semantic version number of the form Major.Minor where major and minor are decimal integer values. At initialization time each App can register a list of entity type IDs and versions which it is able to render. When registering a type ID the App MUST be capable of displaying entities of the specified type up to and including the specified version and all prior versions.

When selecting an App to render a particular complex entity the system will consider all Apps which have registered the type identifier of the entity and whose version capability has a major version number which is greater than or equal to the version of the received entity. For example if an entity of version 2.2 is received and App which registers its ability to handle that type at version 2.1 would be eligible to handle the entity but one which registers itself at version 1.9 would not.

Where optional attributes are added to an existing entity definition and all existing instances of such entities would continue to a) be valid and b) have the same meaning when interpreted under the expanded definition, then the minor version number of the entity definition should be incremented. In other cases the major version number of the entity definition must be incremented.

The major version number 0 may be used in cases where an entity definition is under initial development. At the point where an entity definition is intended for production use it should be assigned a major version of at least 1.

Character Encoding

All PresentationML documents MUST be encoded in UTF-8.

PresentationML

PresentationML is a markup language to express semantic presentation of messages in a domain agnostic way. PresentationML allows the message author to describe how the message should be presented in a semantic way by marking up text to be presented as

• Bold
• Italic
• Paragraph Break
• Line Break
• Numbered List
• Unnumbered list
• Simple html style tables.

No specific guarantee will be made about the graphical presentation of these elements which may vary from device to device and from user to user.

All valid PresentationML will be valid HTML and valid XML without modification. This means that API consumers will be able to parse PresentationML content with an XML parser and the UI will be able to display it as HTML with a CSS style sheet, although validation of received markup is necessary to ensure that malicious HTML has not been inserted by an untrustworthy message originator.

EntityML is a markup to allow for the representation of arbitrary embedded entities. All valid EntityML is valid XML.

MessageML is the combination of PresentationML and EntityML. All valid MessageML is also valid XML which means that API consumers can treat MessageML content as XML and parse it with an XML parser. Messages transmitted over Symphony APIs will be in MessageML format.

All EntityML is required to provide a default rendering expressed either as PresentationML (or plain text, which is itself valid PresentationML). In the event that no renderer for a particular EntityML expression can be found then this default rendering will be used. A renderer to present PresentationML as plain text will be provided for use by content export and other consumers requiring unformatted messages.

The Symphony platform currently stores messages in a different format, but conversion functions exist to map messages in the current storage format to MessageML and vice versa. In the future we envisage that messages will be stored directly in MessageML.

The following tags are defined for PresentationML, tag names are case sensitive:

Named Character References

PresentationML special characters can be escaped using HTML named character references. &lt; &gt; and &amp; should be used to escape < > and &. The set of supported entities is as defined in http://www.w3.org/TR/html5/syntax.html#named-character-references

EntityML

EntityML is a markup for describing Complex Financial Objects and other complex structured entities (Entities) within a message. Entities are composed and rendered by Apps within the Symphony platform, and may be constructed from multiple sub-entities and attributes by composition.

Overall Structure

Entities are built up from a number of sub-Entities and Attributes[1]. Entities may contain structured content whereas attributes contain only a single value, and an entity may exist directly in a message, whereas an attribute may only exist as a child of an entity. Entities and Attributes are defined to have a name and a type. In the event that the name is not specified it has the same value as the type. In cases where an Entity definition has only one sub-Entity or Attribute of a given type, it may be unnamed (taking the type as its name).

When a name is specified in an Entity definition, the name MUST NOT contain any period characters (‘.’). Because all format identifiers contain period marks (see below), this guarantees that a defined name will not collide with an inferred one.

Where two or more Attributes within a single Entity have the same name, Composing Apps MUST:

1. Attach an XML-attribute named “priority” with an integer value in the range 0 to 127 to each Attribute.
2. Ensure that exactly one such Attribute has the priority value 0.

Composing Apps SHOULD ensure that the priority value for each Attribute within a single parent Entity is unique amongst Attributes with the same name in the same parent Entity. There is no specific intention to limit the maximum number of Attributes with a given name for a single entity to 256 although it is unclear if there is any real use case for such an example. Originators MAY therefore specify multiple id elements with the same, non-zero priority.

Composing Apps MUST NOT generate multiple sub-Entities within a single entity with the same name.

Originating User Agents MUST reject all badly formed messages, including those which fail to comply with the restrictions on Attributes and sub-Entities defined above.

Receiving User Agents who receive an Entity with multiple sub-entities having the same name and type MUST interpret these as if the multiple sub-entities were combined.

Receiving User Agents who receive an Entity with multiple sub-Entities having the same name and different types MUST ignore the second and subsequent occurrences of that named sub-Entity.

Receiving User Agents who receive an entity with multiple attributes having the same name and type MUST ignore the second and subsequent occurrences of that named attribute.

Receiving User Agents who receive an entity with multiple attributes having the same name which do not have priority values defined MUST assign the priority value zero to the first such attribute (unless there is another such Attribute which has the priority value zero specified) and assign the priority value 127 to all other such attributes.

Receiving User Agents who receive an entity with multiple attributes having the same name and the priority value zero MUST assign the priority value one to the second and subsequent such attributes.

For example the following badly formed entity

<entity type=”org.symphonyoss.fin.security.price” version="1.0"/>
	<entity type=”org.symphonyoss.fin.security” version="1.1"/>
        <attribute name=”id” type=”org.symphonyoss.fin.security.id.isin” priority=0 value=”US0378331005”/>
        <attribute name=”id” type=”org.symphonyoss.fin.security.id.cusip” priority=1 value=”037833100”/>
	</entity>
	<entity type=”org.symphonyoss.fin.security” version="1.2"/>
		<attribute name=”id” type=”org.symphonyoss.fin.security.id.cusip” priority=1 value=”459200-10-1”/>
        <attribute name=”id” type=”org.symphonyoss.fin.security.id.openfigi” priority=2 value=”BBG000B9XRY4”/>
	</entity>
</entity>

MUST be interpreted by Receiving User Agents as if it was

<entity type=”org.symphonyoss.fin.security.price” version="1.0"/>
	<entity type=”org.symphonyoss.fin.security” version="1.1"/>
		<attribute name=”id” type=”org.symphonyoss.fin.security.id.isin” priority=0 value=”US0378331005”/>
        <attribute name=”id” type=”org.symphonyoss.fin.security.id.cusip” priority=1 value=”037833100”/>
        <attribute name=”id” type=”org.symphonyoss.fin.security.id.openfigi” priority=2 value=”BBG000B9XRY4”/>
	</entity>
</entity>

Type Identifiers

Type identifiers follow Java package naming conventions, and are derived from DNS domain names. Only the organization which owns a domain in the DNS may define types with identifiers beginning with that name. The namespace org.symphonyoss is reserved by the Symphony Foundation for agreed standard type definitions. The namespace org.symphonyoss.fin is reserved for standard complex financial objects to be defined and agreed by the Complex Financial Objects Working Group.

Until the establishment of other Working Groups to cover other parts of the overall org.symphonyoss namespace, the Complex Financial Objects Working Group will define appropriate general formats in that namespace, subject to approval by the Symphony Foundation Engineering Steering Committee (ESCo).

Attribute Priorities

The lack of globally recognized identifiers for some entities such as financial securities, and the existence of multiple widely understood identifiers for the same entity (such as ISIN, CUSIP and OpenFIGI) means that it is necessary to be able to express multiple identifiers for the same entity. The intention is that in cases where more than one identifier is given that the id values stated are equivalent and that they refer to the same entity. It would be an error, for example, to specify an entity with a CUSIP for Apple and an ISIN for Oracle.

Where multiple well known identifiers exist for a particular Entity type, the Entity format definition SHOULD permit the presence of multiple attributes with the same name to hold the potentially multiple identifiers for the same Entity.

Originators SHOULD generate id elements for all identifiers they are aware of for the entity described, for the benefit of receivers who may be unable to interpret some identifier types.

Where an Attribute has a priority, it MUST have an integer value in the range 0 to 127. The priority describes the order of preference or reliability of the given value in the eyes of the message originator, with lower values indicating a preference over higher values, an identifier with a priority of 0 is the most preferred or reliable and one with a priority of 127 is the least preferred.

Within a single entity element where there are id elements present, there MUST be exactly one id element with a priority value of zero. This SHOULD be the identifier that the originator uses as the primary key for the entity being described.

Each id element SHOULD have a priority value which is unique among id elements in the same entity element. There is no intention to limit the total number of identifiers provided to 127 or any other number, this is why duplicate non-zero priorities are permitted.

Rendering Apps SHOULD use the identifier with the lowest priority which they are able to interpret. In the event that there are multiple identifiers with the same priority value which are the best available option for a rendering app it SHOULD use the identifier which appears first in the markup.

Composing Apps MUST NOT generate entities with multiple identifiers which refer to different entities.

Attribute Enrichment

Attributes and the default presentation may be added or updated by an enrichment application after receipt and before being rendered. This enrichment may be used in the routing logic to decide which App is selected to render the entity and may also be used by the rendering App to produce the final on-screen representation of the entity. Enriching Apps are not permitted to remove any of the content of the received entity but may replace attributes or the default presentation by providing an addition element with the same name and ID and with an XML attribute called enrichment which contains a sequence number starting at 1. In the event that there are multiple enrichment applications and more than one enrichment App wishes to update a single attribute then the attributes added by later Apps will have higher enrichment values than those added by earlier Apps and these will be taken in preference by the rendering App when it is eventually called.

For example the following Entity could be received:

<entity type=”org.symphonyoss.fin.security” version="1.0"/>
	<attribute name=”id” type=”org.symphonyoss.fin.security.id.isin” priority=0 value=”US0378331005”/>
	<presentationML>Apple</presentationML>
</entity>

After enrichment an additional identifier and two additional unnamed attributes might be added. These additions are all identified with the XML attribute enrichment="1":

<entity type=”org.symphonyoss.fin.security” version="1.0"/>
	<attribute name=”id” type=”org.symphonyoss.fin.security.id.isin” priority=0 value=”US0378331005”/>
    <attribute name=”id” type=”org.symphonyoss.fin.security.id.cusip” enrichment="1" value=”037833100”/>
	<attribute type="org.symphonyoss.fin.region" enrichment="1" value="NA"/>
    <attribute type="org.symphonyoss.type.country" enrichment="1" value="US"/>
	<presentationML>Apple</presentationML>
	<presentationML enrichment="1">Apple common stock</presentationML>
</entity>

After enrichment by a second enrichment App a further identifier and two further additional unnamed attributes might be added. These additions are all identified with the XML attribute enrichment="2":

<entity type=”org.symphonyoss.fin.security” version="1.0"/>
	<attribute name=”id” type=”org.symphonyoss.fin.security.id.isin” priority=0 value=”US0378331005”/>
    <attribute name=”id” type=”org.symphonyoss.fin.security.id.cusip” enrichment="1" value=”037833100”/>
    <attribute name=”id” type=”org.symphonyoss.fin.security.id.openfigi” enrichment="2" value=”BBG000B9XRY4”/>
	<attribute type="org.symphonyoss.fin.region" enrichment="1" value="NA"/>
    <attribute type="org.symphonyoss.type.country" enrichment="1" value="US"/>
    <attribute type="com.example.fin.tier" enrichment="2" value="BlueChip"/>
    <attribute type="com.example.fin.sector" enrichment="2" value="Tech"/>
	<presentationML>Apple</presentationML>
	<presentationML enrichment="1">Apple common stock</presentationML>
	<presentationML enrichment="2"><b>Apple</b> common stock, traded on NYSE</presentationML>
</entity>

Note that in this case the enrichment Apps have both added extra attributes named id, which if present in the original entity would have required priority attributes to order them. The enrichment added attributes are all considered to come after all attributes of the same name in the original entity, and the enrichment added attributes are then taken in descending order or enrichment value. In the example above the openfigi ID would be treaded as if it had a priority of 2 and the cusip would be treated as if it had a priority of 3.

Default Representation

Any Entity element MAY contain a single presentationML element which contains a default representation of the described Entity in PresentationML markup. This rendering is a default presentation for use by receivers who are unable to understand the structured data in the entity markup and MUST NOT contain any information which is not included in, or derivable from data included in the structured data in the entity. Where present, the presentationML element of an entity MUST contain a representation of the entity which is consistent with the structured data contained within the entity.

Any entity element which is not contained within another entity element MUST contain a presentationML entity containing a default representation of the element for the benefit of receivers who are unable to decode the structured data in the entity.

Receivers MAY ignore the presentationML element of any entity and SHOULD do so if they are able to fully interpret the structured data content of the element.

EntityML Tags

EntityML contains the following tags: (additional syntax is described in Rendering Pipeline which is not yet included here, TBD)