Table of Contents

Date

11 May 2017

Attendees

Actions items from previous meetings

• Language Bindings
  
  • Former user (Deleted): create API definitions based on SJC abstracted binding layer and generate a YAML Swagger specification from it
  • Former user (Deleted): create a CONTRIB to transfer the repository with the REST API YAML files to the Foundation
  • Look into how to enable limited cross-pod capabilities on the Foundation's test pod and others to make integration testing possible.
  • Former user (Deleted), Former user (Deleted), Matt Joyce (Deactivated):
    
    • Start a list of the basic functionality bindings should have test for before being considered active. (Possibly Former user (Deleted) too.)
    • Draft list of API endpoints that should be supported by active language bindings.
• REST API Guidelines and Best Practices
  • Former user (Deleted): clear BNY REST API guidelines for external publication.
• Use Cases
  • Former user (Deleted): Progress conversation within Symphony and with Johan Sandersson about how to structure use cases.
  • All: contribute example use cases in the wiki.
• Consuming the Symphony API: 
  • All: Tell Former user (Deleted) which errors/exceptions you're struggling with most so Symphony can prioritize improvement of error handling.
  • All: If you are aware of an open source project that provides a good API for creating bots, tell Former user (Deleted).
  • Former user (Deleted): Talk to Former user (Deleted) about bot deployment work, possibly contributing to Foundation.

Agenda

Meeting notes

Anthony Fabriccino: First we’ll go through previous action items and a couple of updates. First is an update on API versioning from Paul, and Johan, if you don’t mind going over your updates to the use cases and what you’d like to see done with them?

Johan Sandersson: Absolutely.

Anthony Fabriccino: Let’s defer the action items to the end and go straight in the discussion around versioning. Paul?

Paul Teyssier: IT’s really to get input from the group. The versioning strategy we’ve had till now is that any change to the API interface definition requires a new version number for the endpoint. We’ve realized it’s a bit too rigorous for most of our customers, so we’re planning in the next release to, for new endpoints, we won’t bump the version number if we add optional fields. So we still have version numbers that are different per-endpoint. We’ll gradually introduce this endpoint-by-endpoint as we have reason to add optional parameters. We realize it’s a bit different from other API platforms out there, but wanted to get your input.

Jon Freedman: If you’re not breaking it, you don’t need to change the version. So if you’re adding a new field to the JSON body and it has a simple obvious value, don’t roll the version. If it doesn’t have one, do.

Johan Sandersson: I tend to agree with this approach, it sounds practical. The only thing I’d say is some of the big firms may think certain versions should not be changed as it could introduce bugs or break things.

Jon Freedman: So you’re basically saying you want a new version every time anything changes. Just because the parameters have changed doesn’t mean it will break your code.

PL: The problem we’re trying to address is to be judicious in when to change the version. You’re either

Peter Monks: I shared some information from previous initiative where we had this same problem, and our thought process and some factors to consider. You, Paul, and Vincent might want to look at that information too. What we figured out was that: there’s a communication problem, how do you communicate changes to your customers, usually that’s through a version change. The second is, what constitutes a breaking change and what doesn’t? The current approach is very conservative and lots of work.

Paul Teyssier: I want to help unpack that – it’s not just work for the LLC, effectively every API version work on their own functional silos. When we make improvements, those typically only apply to the latest versions. That’s what we’ve heard most from our customers: we don’t want to change versions all the time to get the benefit of better performance if there’s no functional difference in the API.

Peter Monks: Good point. The point I wanted to make is that the two problems I mentioned are intimately coupled. What we realized is that backwards compatible changes are essentially invisible to the developer and shouldn’t trigger a version change because they don’t need to be communicated to the developer. And there’s good information on our thought process in the information I sent Dan.

Anthony Fabriccino: We ought to have a section in the wiki on versioning. Maybe you could put what you sent Dan into the wiki, and I can get from our engineers information on how we approach this.

Peter Monks: This isn’t a decision, just information about the thought process.

Anthony Fabriccino: I understand, but it’s useful information about this issue.

Peter Monks: Ok, I’ll need to write it up, but I’m happy to.

Anthony Fabriccino: Yeah, and I’ll do the same, and we can put this into the “best practices” portion of the wiki.

Peter Monks: A couple other things: the version numbers are tightly coupled to the pod version.

Paul Teyssier: There’s no notion of an API version at this stage, because the endpoints are separately versioned. So it’s the set of all endpoint versions that matter. And we’ve asked whether to version the whole API like most platforms, or stick with versioning the endpoints. And as we develop the java client, that’s something we’ll have to continue thinking about.

Peter Monks: You’re right, the Symphony APIs have a little of both – platform and endpoint versioning.

Paul Teyssier: The YAML are versioned on Symphony releases, but that’s about all. We’re still figuring out whether that’s helpful or confusing to customers. But as systems decouple a little more, the YAML file will have its own release cadence. We still think they should be versioned separately given how it ties into customer workflows.

Peter Monks: But that does violate the principle of least surprise, since on most platforms it works differently. I get there’s a grandfathering process, but I think it would be best for Symphony to evolve to suite versioning.

PL: I agree with that. Even decoupling pod version from API/endpoint versioning is a good step, but I agree that we should move in that direction.

Paul Teyssier: We are doing a lot of changes regularly, considering all the demand in the ecosystem, there’s going to be a lot of new things coming up, so accounting for that without breaking the expectations of customers is important. So I hear a lot of agreement with PM’s statement that it would be good to have suite-level API versioning. Is that right?

Anthony Fabriccino: We certainly support that.

Leslie: To the extent you can upgrade without our having to change our code, that’s a huge win. If there was a way to discover the endpoint version number programmatically, that would be useful. Get version, get context, etc. That way if I wanted to hard-code to a version, we could do that on the client side.

Paul Teyssier: That’s a good point, thanks.

Peter Monks: It would also be good to have the versions that a pod exposes discoverable. One of the API frameworks I helped define required everything to be discoverable by the client. And versions could fit into that discovery thing. Having meta-APIs that are always available is something to consider.

Paul Teyssier: That’s a good point. We’ve build API suites like that, and eventually it becomes important. If we find ourselves needing to significantly evolve the versioning scheme, we’ll want to do that.

Peter Monks: One thing you can’t version is backward-compatibility breaking changes to the versioning scheme itself. Just an observation—versioning needs to be got right early and left unchanged or you break things.

Paul Teyssier: Yeah, that’s sort of the whole point of having a versioning scheme.

Johan Sandersson: Right now the URL of the endpoint contains the version number. How would that work if we did suite-level versioning? Would the endpoint stay the same but you’d define higher-up what version to use?

Paul Teyssier: The spirit is, in the current policy, we increase the version number even on optional fields, meaning customers have to upgrade their code often. Particularly when you’re working on pre-release products, you have quality changes internally. When you move to suite-level versioning, then you define rules around backward compatibility and breaking changes, so that the only time you need to change the version is if you change something that causes … Most platforms that use this system only change the version once a year, at most.

Peter Monks: At what point do you deprecate a version entirely? Most platforms have a never-deprecate policy, but that can be a big engineering lift. I’ve worked on platforms where we said we’d keep API versions for a minimum of two years.

LS: I put something chat about the OBO use case that we’re most interested in, could you put that in the wiki?

Johan Sandersson: Got it.

Paul Teyssier: The current deprecation policy is the same for on-prem components, and the endpoints themselves. From a contractual standpoint, the policy is 6 months. So after a given API is released, any version that is 6 months older than that one could be deprecated. But we haven’t so far deprecated anything. In the next release, we’ll mark a couple endpoints as deprecated in the YAML file, and they won’t break at the time, but they will in a future release. It’s not what you’d get from a very mature suite, but that’s where we are right now.

Anthony Fabriccino: Good discussion. We should capture some of this in the wiki.

Paul Teyssier: One follow-up. It feels that it’s tightly related to the standards discussion. We’ve talked about the members meeting as an opportunity for in-person problem solving. This would be a good thing to work on there if we have critical mass. This is going to have a big impact on users, so we should try to agree collectively on what we should do. In particular, I’m thinking heavy API users that attended in earlier meetings. So maybe being clear on what we’re talking about would bring them in.

Anthony Fabriccino: Johan, want to talk about your use case example?

Johan Sandersson: One two and three are the FactSet use cases that Leslie also required us to spend some time on. But I also saw that Jon had added four and five about the productivity bot, and that lined up with things Frank has been thinking about. Six, seven, and eight are some of Frank’s ideas that I’ve hopefully captured correctly. Functionality that listens & posts in a chatroom. He says this shouldn’t be a bot-based solution, but an app-based solution. So when you go into a chatroom, the app tells you that the user is in X, Y, and Z chatroom, and the app can change the context of what’s going on in the chatroom. I wanted to get Jon and Frank to speak about how they want to use chatrooms. Jon, you said you have somethings in production. Could you give a brief overview?

Jon Freedman: We don’t have hundreds of users, so what we do is have various developer systems (Jenkins, Prometheus, etc.) running and they send notifications via webhooks to a bot that posts messages to the room it’s mapped onto.

Johan Sandersson: And do they also listen to what the user’s typing? Are there user commands?

Jon Freedman: Yeah, we have some scripts like that, we’ve been doing DB releases through Symphony for a few months now, including the approval process for deployment.

Johan Sandersson: Are you using hubot for all those things?

Jon Freedman: Yeah.

Johan Sandersson: So it’s a connection between the Symphony API and hubot scripts?

Jon Freedman: Yeah, the Jenkins script is 20 lines of codes, and the DB language, which is quite complicated, is about 250.

Johan Sandersson: So we have two use cases or more, but where clients want to be in the chatroom in Symphony and get information from external apps and also send it via the room and have external apps listening to that…

Jon Freedman: Yeah, from an API perspective, it doesn’t matter how the decision to listen to a message comes in, it’s simple posting versus listening n to the chat stream.

Johan Sandersson: Anything you feel you’re missing on the API today that would make your use cases easier to address?

Jon Freedman: There’s something fundamentally wrong, which is the permission model: you’re either an admin or not. There are things you’d like to do as a bot that require admin capabilities, but you don’t want the bot do be able to do everything an admin could.

Paul Teyssier: What would the right level of granularity be?

Jon Freedman: The simplest thing would be a third type of user. Maybe the conversation about OBO is relevant as well—if you permit a bot to do something on your behalf, the permissioning should be very fine-grained. Say I want to write code to allow people to opt in to Gravitar. In order to set an image for a user, I need to be an admin. So I need to create an account that can delete all users, which is a very bad idea.

Peter Monks: Likewise if you want to list all the users that the bot has visibility into, the bot can’t tell which rooms it’s invited into. That struck me as strange. One possible solution would be to do what scopes do in other platforms, like on Facebook. You get a pop-up saying, this app is trying to do X, Y, Z. Which is a matrix of capabilities and permissions: read or write, etc. So you take the matrix and those are all the capabilities. Read room details, write room list, etc.

Paul Teyssier: The OBO behavior will address this…

Peter Monks: I’m talking about the bot running as a service account. Bot needs to be able to say I need these capabilities and the admin should be able to say yes or no.

Paul Teyssier: One question is, what is the overall permission system? The other is, is something different about bots? For the former, we’re planning to add a lot more granularity, including scoping for users or rooms. On the bot v. human side, there’s another distinction: on the user level you want to have coarse-grained roles: what can you do against what object, so it’s human manageable. So it’s role-first, with exceptions. On the bot side, it should be the reverse: define fine-grained entitlements down to the object level, and that’s how you permission a bot when it’s created. The underlying entitlements are the same. It’s just a lot of work to address both. There is an endpoint that gives you all the rooms you’re entitled to as a bot.

Peter Monks: But I think it’s stream-specific.

Paul Teyssier: No, it’s all of it. Send me your example and I’ll take a look. The endpoints for users and bots for this behave very differently.

Johan Sandersson: Excellent. Obviously something I should add as another use case – if you want a bot to admin something, we have an issue. I’ll add that in. What I mainly meant though, was that we clearly have a lot of ideas about how bots interact in a chatroom. With these basic use cases, we should be able to go into more detailed scenarios. I created spaces in the wiki to add more information on the more detailed use cases. But I’ll offer to chase you down occasionally to work on those pages and scope what bots should be able to do in the chatrooms via the APIs so those are documented. Does that sound ok?

Jon Freedman: Yeah, sure.

Anthony Fabriccino: At BNY we’re doing some work on bots, and I’m sure we can add some information from our perspective as well.

Johan Sandersson: Great, the more the merrier.

Paul Teyssier: There are two angles that would be useful: get as precise as possible on your short-term projects, so we can focus on what will make a difference for you. And if more than one firm is interested in a particular use case, capturing the interest of the group would be very helpful to us.

Johan Sandersson: And that’s kind of the pattern with the others I added as well. And the last bit is the CRM thing, which is what Dov is working on.

Aaron Williamson: Can anyone propose agenda items and action items for the next meeting?

Johan Sandersson: If people could let me know if these are relevant use cases for their organization, or if there’s anything they’re doing that’s not captured here, email me or put it on the list, so we can get an idea of those use cases.

Anthony Fabriccino: This is good work and I appreciate the effort behind it.

Paul Teyssier: Maybe if you could send an email to everyone on the list so people could add to it.

Action items

• All: review the use cases in the wiki and:
  • Tell Johan Sandersson if they're relevant to your organization
  • Add use cases that aren't captured
  • Indicate in Confluence or to Former user (Deleted)which are the highest priority for your organization
• Former user (Deleted): work with Johan Sandersson to create detailed use cases to define the capabilities that bots should have access to
• Peter Monks: send Former user (Deleted) information about your issue getting a list of chatrooms bots have access to
• Peter Monks: put information in Confluence about API versioning considerations
• Former user (Deleted): work with BNY team to put API versioning best practice info into Confluence
• Johan Sandersson and Leslie Spiro: put Leslie's OBO use case into the wiki
• All: add agenda items to the agenda for the next meeting