Table of Contents

Date

27 Oct 2016

Attendees

Actions items from previous meeting

• Matthew Gardner: Review internally with Blackrock team re interest in early access to renderer framework

Agenda

Meeting notes

• Roll call
• Review previous action items
  
  • Skipped as Matthew Gardner is not present
• Bruce: screen share of Market Data Demo app:
  • 
  • This is running in the Symphony LLC production pod, which is running sprint 43 code (we deploy to our own pod first, then to customers).  It's due to hit customer pods November 12.  This renderer has not been certified for production use, but the code will be available.  If customers want to turn this on in their test pods, that would be ok.  There's no documentation or customer support yet for writing renderer apps, though ad-hoc support will be available.  You can see that this app is ticking data in real time - the graph is growing over here:
  • 
  • The data is coming from Yahoo finance - it's delayed, and not particularly good data.  It's not a very beautiful renderer, but nonetheless this is a complex object with a custom renderer.
  • I also wanted to show this JIRA integration:
  • 
  • Different environment, but same code line and also coming in November (sprint 43). This is real data from the LLC's JIRA instance.  These messages are being injected directly into this chatroom based on changes in JIRA tickets - our engineers are already using this in anger. This integration is also implemented on top of the rendering framework. If you try to submit a messageML message containing an entity right now (sprint 42 and earlier), you will get a rejection with the error "entities are not supported".  Once spint 43 code arrives, by default the agent process will support those messages.  If you're running an on-premises agent then you will have to deploy the new sprint 43 version of the agent to get support for entities in messageML messages.  JIRA integration is an example of a more beautiful renderer.  Hopefully that's interesting for people.  
  • Afsheen: both are really cool.  How can I play with it?
  • Bruce: the JIRA integration is not certified for production use.  Each sprint is prioritised driven by customer demand - our product management people don't see demand for this feature to be completed.  So if you try this out and want to put it into production and be supported, you should formally request it through your sales contact.
  • Afsheen: that's not going to stop us from playing with it though, right?  How do we play with it?
  • Bruce: when sprint 43 hits your test pod (November 12, I expect), this code will be available to you and I can informally provide to you the source code for this sample app.  And you can start from there.
  • Afsheen: slightly embarrasing question: who owns the test pod at JP Morgan?
  • Bruce: I don't know, but I can find out and let you know.
  • Peter M: the JIRA integration will be open sourced via the Foundation in the next month or two as well, so it'll be available to everyone.
  • Nick: could you go through the messageML driving these two renderers?
  • Bruce shows this messageML:
  • 
  • Bruce: this is the message being used to render the stock ticker graph.  This is the relevant entity snippet:
  • 
  • Bruce: let me remove the app to demonstrate the fallback behaviour to vanilla messageML.
  • Bruce: the ability to resize the message (open twizzler shown in JIRA integration) is magic right now - apps can't do this right now.
  • Nick: so I understand the workflow to be that you can pass off the message to a chain of apps that each have renderers.  Is that correct - that more than one app can deal with rendering entities?  That there can be multiple visualisations for a single message?
  • Bruce: no - there's a process of selection to pick which single renderer will be used to render the message.  There's a mediation process that allows the user to control which app gets the message.  This isn't an issue for JIRA since it's an integral part of Symphony.  The default is that the most recently installed app gets the message.  Sending different messages to different apps based on soophisticated criteria has not been implemented at this stage.
  • Nick: I recall you also mentioned enrichers?
  • Bruce: none of that has been done.  What has happened here is that we've talked about this in various meetings in the past I've asked for participants actually interested in exchanging messages.  We haven't had traction for that or demand for that through the product management process so it's not making it into the list of things that will be done in this sprint.   A requirement to provide specific integrations like IJRA has come along, and so a subset of the rich functionality has been implemented to get the JIRA integration out the door.  Now I would obviously prefer if we had a complete integration of all of that functionality, but we're a customer driven company and customers aren't asking for that right now.
  • Nick: what's the interface look like for the app for the app to render the chart inside the message?  Does the app have access to the DOM?
  • Bruce: the app is given an iframe, loaded from a URL specified by the app.  You can paint whatever you want inside that iframe.  The HTML and JS responsible for rendering the iframe is entirely up to the app.
  • Nick: so that's a cross-domain iframe?
  • Bruce: I don't understand.  It's an iframe and you specify the URL.
  • Nick: the URL of the app is different from the URL of Symphony.
  • Bruce: correct - you must host some kind of server.  This example is running a little node.js app on my laptop, for example.  Because this is an untrusted app, when loading the page we see this warning:
  • 
  • Bruce: When an app is released and installed in the Symphony app store, that warning won't appear.  This just appears because this app is in development mode.
  • Nick: this helps a lot.
  • Bruce: there's no way to pipe a message from the editor to include an editor.  The message must be sent through the agent API from another process.  Something the Factset team have done is write a bot that listens to messages in a room and when it sees a cashtag it looks up information on that cashtag and posts a structured object back.  So you could implement something where the user just types $AAPL and the bot then posts back to the chat.
  • Peter M: it's like a poor man's enrichment.
  • Johan: we have a bot that not just sends back a financial object, but a rich object that renders in a specific way with data from Factset's backend service.  We have very much tried it and it works.
  • Peter M: how is the MessageML passed into the iframe?
  • Bruce: the app gets handed a JSON blob I think?  I'm not a front-end engineer so I don't know.
  • Nick: how is messaging done between the Symphony container and the app in the iframe?
  • Bruce: the app has to be given the information from the entity.  I believe it's passed as a JSON object, rather than a MessageML string.
  • Nick: but is there an interface at runtime between the app and the container?
  • Bruce: I don't really know.
  • Nick: it'd be interesting to hear about that at some point.
  • Bruce: I can provide source code for the ticker app, which will reveal all.
  • Nick: that would be good to see.
  • Johan S: our engineers were very concerned about reloading the iframe all the time.  So somehow they're loading a skeleton and all the information that's needed is passed in the message then the skeleton is filled with information so we don't have to go to the server more than once.  As messages come in, we pick out the values from the objects to build the nice looking view.
  • Nick: that's interesting.  I was wondering about performance consequence of lots of iframes, and loading constantly.  It'd be interesting to see this diagrammed out.  It sounds like there's a single controller that can populate all of the app domain messages?
  • Johan S: yes I think for the next session we do a demo of our more production ready app, and aI can get the engineer who built it on the call to discuss these issues.  It was something they brought up - we don't want to constantly reload the iframe.
  • Nick: yeah it'd be great to see a demo of that.
  • Johan: is the JIRA object documented somewhere?  If we wanted to send these updates.  There's also a JIRA hackathon project in the Foundation.  We'd like to know what to send to turn it into a JIRA object?
  • Bruce: I don't have that at hand, but I will add that information for you.  It's obviously part of the next release.  I don't know that we've prepared the documentation on that.  I would hope that you don't need to write a bot - you just need to configure your JIRA.
  • Peter M: how are new types registered?  Is there a registry?
  • Bruce: there is no registry per se - entityML uses Java package names as a convention.  The financial types we registered originally in this WG are all namespaced under symphonyosf.fin.security (or something like that) - so if you own foo.com then you can define new entity types under com.foo.whateveryouwant.  It's intended that that's no barrier to entry for new entity types - you simply register whichever type ids you want and when a message arrives matching your type id, your app will be triggered.  The confluence site for this WG contains definitions for the MessageML and also the financial object types this WG has defined.  That's where I would propose that those things be documented.  Not a registry in the sense that they have to be there to be used, but so that 
  • Peter M: suggestion: how about we use Github instead of Confluence for managing these type definitions.  Github allows for better workflow (pull requests, etc.) than Confluence.  Perhaps a WG member vote could be used to decide this?
  • Bruce: is anyone else expecting to be creating new entity type definitions?
  • Peter M: I expect to, yes, though I don't count (as a non-member).  <pause> Doesn't sound like anyone else has a need yet, so let's make a decision if/when we cross that bridge.
  • Johan S: we've spoken to Anjana about some of the issues we've found with the renderers.  We want to send cashtags in the MessageML if the user doesn't have the renderer installed.  But the messageML comes up as plain text.  Any thoughts?
  • Bruce: because there hasn't been official customer demand for entities, this has not been prioritised.  In order to support entities correctly, we will have to change the internal storage format of messageML messages from markdown to XML.  This has a big impact across the platform - it's a very broad change across the codebase.  That piece of work therefore has not been prioritised and as a consequence we can't really properly support the capabilities of messageML with messages containing entities - they have to be converted into Markdown + JSON internally, and the nested structures you're describing have no representation in the current internal format.  It's a more difficult thing to implement than you might imagine, so that capability will not come until we do the full implementation of messageML.
  • Johan S: that makes sense, but I was hoping / assuming that this conversion had already been done.
  • Bruce: what's going to production right now is the JIRA integration, which just happens to include some of the capabilities of entities.  I've demonstrated what's possible to do with the version of the code that'll be in production November 12, and it allows you to send a messageML message containing entities and they're shoehorned into the current internal format.  Some things aren't supported, like nested objects (cashtags, hashtags, links).  A rather nasty side effect is that if you send a message containing an entity via the API, then read it back out, you won't see the entity because of an internal implementation detail.  That's one of the reason we're saying this isn't certified for production yet.  It works well enough that it can be used for interesting thing, but I hope that the organisations on this call will start saying "yes this is an important thing we want this and this next". 
• AOB:
  • None

Action items

• Former user (Deleted) to find out who at JP Morgan manages the JP Morgan test post, and let Afsheen know
• Former user (Deleted) to provide group with information on how the entity ML is passed into the renderer's iframe
• Former user (Deleted) to provide group with documentation on the JIRA entity ML structure
  
  • Ideally this should be added to the type "registry" on Confluence
• Johan Sandersson schedule engineer for next meeting (2 weeks time) to discuss some of the technical details of how Factset's renderer works
• All: review capability and provide any feedback to their sales rep to ensure product management can properly prioritise enhancements to this feature