2017-04-18 Meeting notes

Owned by Peter Monks
Last updated: Nov 15, 2017
17 min read

Table of Contents

Date

Attendees

NameESCo RoleOrganisationPresent / Absent

Michael Harmon

Project LeadSymphony LLCAbsent

john.stecher@barclays.com

Project LeadGoldman SachsAbsent

Lawrence Miller (Deactivated)

Project LeadSymphony LLCPresent

Former user (Deleted)

Project LeadIHS MarkitPresent

Former user (Deleted)

Project LeadCredit SuissePresent
Former user (Deleted)AdvisorCitiPresent
Nicholas KolbaAdvisorOpenFinPresent

Peter Monks

SecretarySymphony Software FoundationPresent

Actions items from previous meetings

Agenda

TimeItemWhoNotes
5 minConvene & roll call

Peter Monks


5 minReview action items from previous meetings
Peter Monks


See above

50 minOpen source compliance policy review


5 minAOB & adjourn



Meeting notes

  • Quorum not achieved (3/5 project leads present)
  • Action items:
    • Peter: good progress from Frank and James
    • Lawrence's action items:
      • Would like to align with Aaron
      • I have not sent out LLC contribution policy yet.  I don't have a problem doing that.  My comment in the previous meeting is that I'm happy to share the LLC's policy, but it's going to be much more conducive to look at policies of financial institutions, because our policy (like IHM's) is likely more liberal than the big banks.
      • Peter: is there any reason not to share it?
      • Lawrence: not really, if there's interest from ESCo in sharing it.
      • Frank: is it based on something, or did you create it yourself?
      • Lawrence: extensively modified version of Blackrock policy.
      • Frank: and that was crafted by Blackrock?
      • Lawrence: extensively modified  (wink)
      • Frank: it would be valuable if you took the IHS Markit policy and walked through the diagram, and discussed the differences - what's equivalent, what's different.  Then we can talk about what banks are dealing with that's different.  We have to be realistic that as vendors the bars we have to meet might be lower.  As we're going through the discussions it would be good to understand differences, and if they're substantial then I'd want to see the document.
      • Lawrence: the biggest difference is that the symphony process is designed around a policy where most LLC contributions are intended to go into the Symphony Foundation, whereas a bank policy is unlikely to do that (though the foundation may want that!  (wink)).  I'll dig it up, but I don't think it's going to be very useful.
  • Open source policy discussion:
    • James: looking at your policy Frank, how does it work practically?  When you or your team contribute?  Once you've been through some of this do you do continuous contribution, or does each contribution have to be evaluated by an independent 3rd party?
    • Frank: we have exec level + legal approval for initial contribution, the idea being that at an IP level are we giving away our assets that generate revenue, or expose us in ways we don't want?  That's an exec level decision, with an understanding that the contribution will be managed by a review board that's close to the development activity.  That board includes the dev lead and legal counsel, and has access to the actual code to evaluate what's being contributed.  That process has turned into email trail, post-contribution.  The initial contribution focuses on trying to make sure the contributions are mirrored to a local repository (local git and remote GitHub).  Updates are to the distribution list (peer review group) to inform them that that's happening.  On top of that there has to be some level of peer review of the code that's being contributed on each commit.  Realistically speaking we review approximately monthly.  That's basically it - roughly speaking that's the process we follow.
    • James: and you don't have nay tooling to support that?  Just email & manual?
    • Frank: yes, though we also use GitLab, which helps a bit with that.  We get notifications etc.
    • Aaron: is the open source review board one or many?
    • Frank: it's N for both the exec group and project review board.
    • Aaron: so it's project specific?
    • Frank: yes - we don't want those groups to be divorced from the project.  We don't want some central group that's decoupled from the work.
    • Aaron: where do you find your process gets hung up most often?
    • Frank: that people aren't aware of it within the company.  We're focusing on this right now, and remastering the document to a wider audience within the company.  To get an exec and legal to look at an initial contribution and ensure it's not a liability - that's also very important.  The other difficulty is having contributors following standard practices we've learnt through the Symphony Foundation (licensing in the source, clean source code, etc.).  You can go to the exec board to get approval, but you have to go to the review board to sign off the legal ramifications of what you're contributing.  You have to have someone on the review board working with you to understand the requirements.  So two difficulties:
      1. Senior management sign off.
      2. Getting the right people in the review board, to care to work with the contributor (who may not have contributed to open source before) and make the appropriate adjustments.
    • Aaron: do you offer any training for contributors?
    • Frank: no, and that's one of the difficulties.  Within a bank scenario they'd likely say you can't contribute until you're trained and have that established process.  That's a problem we have to solve within the company, both in terms of use of open source and contribution.  This has been raised to senior tech sponsor (Randy Harvey) regarding open source use and contribution - but it's something we have to work on.  And this is similar to most companies - this really requires direct engagement between development leads and senior technical management.
    • James: looking at the diagram there's one thing that says "developer publishes contribution" with blocks above and below.  I'm assuming that they can't publish themselves?
    • Frank: they can publish right now and there's some language above about the "proxy contributor".  It's possible to define who can contribute, based on the policy, and a "proxy contributor" can be defined.  There are forms for this that describe how you're going to deliver your code into open source - individual contributor vs proxy contributor.  Right now it's mostly been individual contributor, not proxy.
    • Lawrence: I sent to ESCo a PDF of our policy.  When you're done Frank I can talk about ours.
    • James: Is this process mainly for where you're publishing Markit projects?  What about contributing to others' projects?
    • Frank: it should cover both, and this is something we've been discussing internally.  It's not clarified well right now, thought the latest version talks about it.  If it is a project that is already existing in open source, we will not republish it as a personal project - we'd contribute it to the sponsoring Foundation.  If we're contributing to our Markit open source repository, it will be specific to a Markit product - not a patch for some email system somewhere.  Anything published to the Markit repository would be specific to our products.  Anything that is existing project work outside Markit would be contributed to that repository, ensuring licensing requirements etc.  We also have an open source use policy, which prevents copyleft technologies from being used.
    • James: so if you're patching something it goes through the same process, but will land in a different place?
    • Frank: yes.  That's our staging git repo and that's where source gets put even if it's unrelated to Markit products.  If it's related to external code it would still go there.
    • James: we're interested in this precise thing and have been talking about it for some time.
    • Frank: any other questions?
    • Nick: is the staging repo private?
    • Frank: correct - it's on our GitLab.  We "stole" this from the Linux open source compliance site.  Tonnes of material there - templates etc.  We didn't create this on our own.
    • James: as I've said before, the process theoretically is very similar to what we've defined, but our big stumbling block has been process & tooling.  We don't have this idea of a staging area, we'd have to set that up.  And then setting up rules around how to use the staging area, and publish from it is hard.
    • Frank: right - it's very manual right now.
    • James: indeed, and ours probably would be too.  We also have practical difficulties around firewalls etc. - doing this manually would be almost impossible.
    • Lawrence: does everyone have the PDF I sent out?
    • <confirmation from several attendees>
    • Lawrence: first up you'll notice our policy is only a 3rd the length of Frank's, but in seriousness the first comment I'd make is that this document is labeled as a process.  In our framework we also have something called a policy, but the policy itself says practically nothing - just some general principle statements.  That's based off of how Blackrock InfoSec policies are defined.  Part of the rationale for that is that if someone is doing an audit, and they're only interested in the policy document, you hand them the policy document and it doesn't have much in it so you're less likely to cause confusion by providing unnecessary information.  The beef of our "policy" is in the process document, which is the next level down.  Our process document is most similar to the Markit document.  This is not based on Apache or other established procedural document - it's specified to Symphony.  There are two types of open source activities this document is intended to cover:
      1. Open source coming into Symphony - what's the process for managing the addition of open source to the Symphony code base.  The primary consideration is: is the license compatible with Apache 2.0?  Because if we open source something with the Foundation, it has to be Apache 2.0 compliant (so that we can contribute it).  Second part is ensuring we don't violate the terms of the incoming open source license - e.g. the Affero GPL license which can have unintended viral requirements.  It's about giving the developer a flowchart so that if a developer is bringing in open source, they can simply check whether the software is in the license whitelist.
      2. Contributing to open source.  What you'll see here is that this document mostly talks about contributing to a Foundation open source project, because that's most of the open source contributing we do.  We also have some language about projects that aren't contributed to the Foundation.  But most of the process focuses on contributing to the Foundation, where we already have a contributor license agreement in place.  On page 2 is a flowchart regarding new projects vs existing projects.  If it's a new project it has to go to a review board (Thomas Kiessling and myself) before being contributed.  For existing projects, we've simplified the review process - originally we had a process that was similar to the Markit process - every single commit would theoretically need to go through review.  The procedure we wanted was a model where we have that level of control - no contributions that haven't been approved.  But it's not practical to review every single commit to an open source code base.  So we introduced a concept of "pre-approved contributions".  It allows Thomas and I to say "we are reviewing and pre-approving every change to this project", meaning Thomas and I don't have to review every single commit.  It ensures Thomas and I understand the boundaries of contributions, and can approve changes (e.g. for a given sprint) in advance.
    • Lawrence: The rest of this document goes through procedurally what you do as a developer.  We've tried to minimise the amount of information that a developer needed to provide so that a developer isn't filling out a huge amount of paperwork to do something - we've found developers are really resistant to that.  Thomas and myself probably already know, given the size of our organisation, what the developer is proposing.
    • James: do you review code?
    • Lawrence: yes - for existing code we require developers to provide a link to the code.
    • James: so you're trusting the developer to report that?
    • Lawrence: let's say we look at the upcoming two sprints of Symphony, and those sprints have a known list of functional changes defined by product management.  We trust that the developer's changes don't go beyond the scope of what's been proposed by the product manager.  On the other hand, if we had a change that wasn't within the scope of a pre-reviewed change series, we would need an individual approval request, and as part of that we'd get a description from the developer about what the change is, and (if necessary) do a code review.  That's the philosophy of it.
    • James: philosophically that works, but in our organisation that paranoia of losing, say, a trading algorithm via a pre-approval, comes in.
    • Lawrence: if you go to the review criteria - these are the criteria we look at to determine if something is high risk or low risk, and therefore how much scrutiny it needs.  If we're looking at any change that falls outside a standing approval, these are the criteria Thomas and I would apply to determine how much scrutiny to give to the contribution.  These are risk factors, not a checklist, but if it looks like a duck and quacks like a duck...  And this isn't to say that something that falls into the high risk category wouldn't be approved - just that it needs more scrutiny.
    • James: how do you implement this?  Do you use GitHub?
    • Lawrence: yep.  If there were lots and lots of lines of code, that's a high risk thing.
    • James: can you practically stop the developers from contributing?  Or is it a trust thing?
    • Lawrence: I think it's true in every organisation that you can't stop your developers from doing stuff.
    • James: we've made it really hard for our developers to do anything.  (wink)
    • Nick: that's a different problem!  (wink)
    • Lawrence: my experience has been if you make it too hard, your developers will just do something else.
    • James: we've made it very hard to develop on our developer's desktops...
    • Lawrence: I have an interesting story about this.  Back when Blackrock acquired Merrill Lynch, we discovered that Merrill Lynch had locked down their developer's environment - no C: drive access, no registry access, no software installation allowed, etc.  It was almost impossible to do any development on these machines.  And what the developers ended up doing is developing on personal laptops!
    • James: we've had to shoot a few developers who did that...
    • Lawrence: yet those were the only Merrill Lynch developers who got anything done!
    • Aaron: based on my conversation with Goldman, their policy is very similar to the LLC's - they have the ability to approve these "standing commits".  James to your point, the idea is that once you give that authorisation, you're approving not only the project and its risk, but you're also scoping the authorisation so that it can't be gamed by the employee.
    • James: yes I agree with that, but it depends on the paranoia of your organisation.  Our policy, which I can share, says little more than "you need approval", and in practical terms that means they'll get approval from senior management and legal, and on an ongoing basis they only need approval from line management.  But practically it's so difficult that people won't do it - there's no tooling or repo to support review - it's very painful and manual.
    • Aaron: Goldman have been through several iterations and have worked that out.  I've heard from more than one member organisation that their procedures require their developers to contribute from personal laptops and personal GitHub accounts.
    • Frank: in Thomson Reuters the development network was completely partitioned from the corporate network.
    • Nick: yeah - completely different worlds.  It was easy to use open source, but I didn't find a policy regarding contribution.  To my mind the real question is "what can the Foundation do to help these processes get adopted in the banks?".
    • Aaron: there are two approaches I've been attempting or considering to take:
      1. A requirements gathering process to find out what issues are holding them up.  While the policies are similar across banks, the cultures and policies vary wildly.  I've had limited success finding time to sit down with folks.
      2. I'd also be more than happy to give presentations to the people involved in legal and on the business side, to the extent that there's education that needs to be done on the basics of open source, or how to go about developing an open source policy.  That's something I'd gladly offer if it was of interest.  The tricky part is getting together the right people and putting it on their calendars.
    • Nick: to James' point, there's a big difference between having a policy and having an effective process i.e not forcing developers off the grid.
    • Aaron: sure, but practically speaking we have to start somewhere and we have some members who don't have a policy at all yet.
    • James: definitely a couple of different things:
      • Getting people to a place where they want to contribute / explaining the benefits of doing so.
      • The legal aspects of contributing - fear of patent trolling, being sued, etc.
      • The intellectual property problem.
    • Lawrence: are there concerns that the patent and IP claims in Apache are inadequate?
    • James: I think it's angst, rather than legitimate concern.
    • Aaron: that sounds like an education issue.
    • Nick: in my experience it's more an issue of fear of the GPL.
    • Lawrence: GPL is more an issue of hygiene around incoming use of open source.
    • Nick: and Black Duck takes care of that, but I think that's where the angst from the legal side comes from.
    • Laurence: in thinking about JPMC policies in the late 90s, there was so much angst about open source that I had to go get commercial licenses of open source software to be able to use it.
    • James: as a consumer, what I get from my team is "I'm using open source and it's fine - if there's a problem I'll fix it".  But I keep asking them: if you can't build the code internally, how are you going to patch it?
    • Lawrence: and then you have $10K open source software!  (wink)  The point I'm making is that I think along with a lot of these things it's a question of companies nowadays not widely having requirements about bringing open source into the environment - it's expected you'll use open source in your applications and compile from source, that you're not going to be required to have a commercial support license for every library.  I think there's a similar sort of maturation around liabilities and risks and levels of risk and trade-offs - a company should be seeing value in contributing.  Otherwise it's a one-way street - consume only, no contribution.  The rest of the industry have identified that value, and financial services is starting to see it too.
    • Frank: we're spending hundreds of thousands of dollars on licenses for commercial open source.
    • Lawrence: you probably don't have a policy requirement to do so though?  It's more of a business decision?
    • Frank: correct.  Companies should realise that if they concentrate on contributing back, then they can save a lot of money in consuming some open source because of a deeper level of understanding in-house.  That's how we're looking at it, and how a lot of companies are looking at it: "why am I spending so much money on this software I'm consuming if I can build expertise in-house and not need commercial support?"
    • Lawrence: you probably only have commercial licenses with 2% of the open source software you're using.
    • Frank: sure, but they're the most important 2%.
    • James: I can see that though - "I'm using Linux and don't have deep OS experience - I want to be able to call someone up."
    • Frank: absolutely.  To Lawrence's point, that's probably no more than 5% - the rest we're just using it and the liability is extremely low.  It tends to be things like utility libraries which have low risk.
    • Lawrence: if we're making the case for further liberalisation of open source policies in financial services, there's a legal question of how to explain "it's not risky".  Just as it wasn't risky to use open source - you weren't going to get sued by SCO, or get into trouble because you don't have a contract for the Apache web server.
    • James: it's confidence by press release, right?  If we put together policies that address certain valid concerns and promote them.
    • Lawrence: and outweighed by the benefits - developers being able to participate in open source communities.  If banks are thinking of themselves as technology companies, you'd expect your developers to be working in communities.
    • James: how do you do it though?  Onesy-twoesy, or get enough of the banks together to get them to talk together?  The latter is bigger bang for the buck, since other banks will just follow their lead.
    • Lawrence: one thing we could do is to try to define a reference policy implementation that we can then put together as an industry standard for financial services.  If we can get enough of the Symphony Foundation members, especially the big banks who are interested in changing their policies, to agree to implement their policies based off the reference policy, then you have the PR you're talking about.  The Foundation can roll out the "reference open source policy for financial services and banks".  This would also make regulators happy.  On top of the PR, there would be a statement that certain large banks have reformulated their policies based on the reference policy.  That would result in it cascading into many other places.  As soon as the OCC or other regulatory bodies hear there's an open source standard, they're going to start asking if people are using the Foundation's reference policy.
    • James: I think we should kick this around at the board meeting tomorrow.
    • Aaron: great idea.  I think it would be good to convene an initial working meeting of stakeholders from as many of the member banks as possible, to sit down and talk about the high level challenges, and how a reference policy like this can handle the wide variety of procedural and bureaucratic variances at the various organisations.
    • James: and is it a policy or a process document?
    • Lawrence: and here's a flowchart of the process my firm follows, and it's liberal enough that developers will follow it.  I think that flipping that dynamic around to make the policy an enabler for developers is important.  The people we want in the room are those who are open to, and capable of, implementing the policy.  There's no point in having that meeting if no one present can execute on it.
    • Aaron: to some extent I hope people would self select on that basis, but do you have suggestions on how to enforce or guarantee that?

    • James: in most organisations I'm aware of there's a person who's the open source policy owner.  They may not be able to dictate anything, but at least that person can say whether it's "adoptable" within their firm.   That person will typically never be able to mandate the policy themselves though - policies are typically a senior management thing that requires a lot of review.  So it's not a slam dunk, but at least that person is willing to change their policy.

    • Venkat: I think it's about talking to those people to get the policy to the right people.  As an example, Ali and I have been trying to find our policy maker within Citi and we're getting the run around.  I think maybe in tomorrow's board meeting if there's one decision that could be made, it's that if firms have such a person, they identify them to us.
    • Lawrence: we should also go in with everyone having the approach of "we'll show you ours, if you show us yours".  No one should be in that group if they're not willing to share.
    • James: and if there's something super specific in a firm's policy, it can always be redacted.
    • Frank: I would limit the group if we could - just my opinion, but it may be easier to start with one or two banks to start . These collectives tend to get watered down.
    • Lawrence: 5-ish.
    • James: I think it'll be self-selecting to some degree - I bet if you asked at the Foundation board you'd only get 5 or so volunteers anyway.
    • Frank: we're over time, and I have another meeting to run to.
  • AOB:
    • Frank: I'll present the ESCo slides tomorrow.  Sorry I missed your working group in the slides, James.
    • James: no worries we can bring it up at an appropriate moment.
    • Lawrence: the people participating should:
      • be the open source policy maker
      • be able to adopt or promote the template policy
      • be willing to share their current policy to the sub-group
      • Should be 5-ish at most
    • Frank: should we present this tomorrow?
      • Lawrence: I'm happy to do it, but before we go into that meeting, rule #1 of parliamentary procedure is never go to a vote unless you already know the result.  So would Citi, or CS, or Goldman, ore Markit be willing to adopt the reference policy?  Would you be willing to join the group under those conditions?  If there isn't a volunteer in the meeting, I wouldn't want to ask the question.
      • James: we can't guarantee that we'll adopt it, but I'm here exactly to try to drive this.
      • Frank: we would incorporate this into an existing policy
      • Venkat: I'm willing and Ali is willing - if there's a policy, we can take it and push it.  I don't see any problem participating and using it to push our internal people to adopt one.
      • Lawrence: so to stage this, I'll present this and then ask the question, and if you guys could speak up in response to the question and express your interest that would help trigger other organisations to participate.
      • James: if we can start the herd, that'll be good.
  • Adjourned

Action items

  • Lawrence Miller (Deactivated): raise topic of standard open source policy template in tomorrow's board meeting, and ask for volunteers to participate

Need help? Email help@finos.org we'll get back to you.

Content on this page is licensed under the CC BY 4.0 license.
Code on this page is licensed under the Apache 2.0 license.