Board Reporting and Program Health Checks

At a minimum, each Program's PMC is required to report to the Board at each quarterly board meeting, though the Board may pro-actively request a status update at any time.

Board meeting status updates should be provided using this presentation template (see slide 7) one week before each board meeting (to allow time for distribution to the Board of Directors). 

Each Program's PMC is also required to be able to produce a full roster of the program's active participants, across all Projects and Working Groups, at any time.

Program Health Check Criteria - Version 1.1

Program Health Check Rubric


Target (Expected) OperationsAt RiskIntervention RequiredMetrics

Diversity & Viability

  • Program PMC is composed of individuals from 3+ organizations, ideally of 2+ org types, and including 1 bank
  • Program leadership is actively seeking new participants and contributions across different channel, forums, and organizations
  • Contributed projects are “seconded” - additional organizations other than the contributor commit to contributing to the code base
  • Women represent 50%+ of active participants
  • Program PMC is composed of individuals from just two organizations
  • Less than 35% of the program’s active participants are women
  • Less than 50% of the program’s active participants are members of traditionally underrepresented demographic groups
  • Program PMC is composed of individuals all from the same organization
  • PMC membership itself is in transition, incomplete, or in doubt
  • Less than 25% of the program’s active participants are members of traditionally underrepresented demographic groups
  • # of Orgs represented on PMC
  • % of Women Active Participants (APs)
  • # or projects / working groups w/o a chair/lead

Roadmap & Resources

  • Programs and projects work to a public roadmap; programs consider the releases from all component workgroups and projects and how they fit together
  • Programs are actively supported with appropriate resources needed as they move through project lifecycle, e.g. from incubating to operating.
  • Program has committed resources from participating firms who themselves have permission to work on the program as part of their day job
  • No requirement exists for developers to create a separate “work” /corporate Github ID
  • 2019 roadmap exists but is not published on the wiki and/or is incomplete (e.g., does consider work activities of all projects and work groups)
  • Roadmap milestones were missed in the previous quarter
  • Projects are staying in incubating status for >6 months
  • Participants in the program largely do so “on the side”, fitting time for FINOS and other OSS work amongst their official job duties on which their performance review and incentive comp plan depends
  • No 2019 roadmap exists
  • No roadmap was used in the last quarter of 2018
  • Projects are staying in incubating status for >12 months
  • Participants in the program do so from home, on home networks using their personal computer; OSS work is not an official or even officially permitted part of most participants’ “day jobs”
  • One or more projects or working groups are missing a lead / chair, and have been for > 1 month
  • No plausible path to a production ready, publicly available, product/release can be identified amongst the program’s projects and working groups
  • 2019 Roadmap Published On Wiki (Y/N)
  • # of FTEs (excl. FINOS staff) committed to program activity
  • # of projects in incubating status > 6 months

Hygiene & Operations

  • Meeting agenda developed and shared via the mailing list >24 hours in advance
  • Minutes are kept including all action items, resolutions and attendees and where/if a policy exists the policy is followed; minutes are distributed within a week of the meeting
  • PMC meets at least once a month
  • Questions/Inquiries to the general list are answered intermittently
  • Minutes are kept for most but not all meetings and/or minutes are sometimes incomplete
  • PMC members, especially PMC leads have difficultly using collaboration infrastructure (e.g., Google Groups)
  • Questions/Inquiries to the general list often go unanswered
  • Communication is ad hoc, infrequently, uses program mailing lists, and is largely dependent on 1:1 phone calls and emails (and thus opaque to the larger community)
  • PMC and/or general meetings are not happening
  • # of PMC Meetings in last quarter
  • Unactioned PRs
  • # of cancelled meetings w less than 48 hour notice

Quality & Security

  • Code is well written, demonstrably proven to be secure, and follows best practices
  • Code in program projects is written consistent with foundation legal and licensing requirements
  • Projects build test cases as they code, integrating test cases into the build process
  • Use of CI/CD tooling is common but not consistent across all code producing projects
  • Code of programs projects’ is uses differing coding standards, idioms, and design patters
  • Testing of program’s projects’ codebases is intermittent and haphazard
  • Coding standards are idiosyncratic
  • One or more projects introduce a dependency that has a non-compliant license
  • Use of CI/CD tools is not consistent and dependent solely in personal preferences of individual developers
  • License inspection of code is not occurring or haphazard
  • # of projects in program using ODP CI tooling
  • Repositories with or pulling from upstream repos with non-standard licenses (e.g., licenses difficult to reconcile with Apache 2.0)

Growth & Adoption

  • There is QoQ growth in active participants
  • The program has projects or working groups that have released code or other work products that are being used in production, and that usage is growing
  • How the program’s projects’ released code and work products create/will create business value can be quantified and well explained
  • Active participation is flat
  • The potential/expected business value to be derived from projects is generally understood through not known for sure
  • Active participants are declining QoQ
  • The connection between what projects and working groups are working on and how that work could create value for users/consumers of project/WG work products is not understood
  • Growth in Active Participants (quarter ending vs. two quarters ago)
  • Number of Pull Requests across all projects (last 3 months)
  • Number of Commits (last 3 months)
  • Number of Organizations Using Program Released Code, Software, Standards, Etc

Where to Find Program Measures and Data for Quarterly Health Checks

Measure

Where does Master Data Reside

Available in metrics.finos.org?Provided in Monthly PMC Report Email (sent on first of month)?Board Reporting Guidance for PMC Leads and PMCs

Program Health Check (Board Reporting Measures)

Diversity & Viability

# of Orgs (Companies) represented on PMCFINOS metadataYes, on Program Health Dashboard(via link in monthly PMC report email to Program Health Dashboard in metrics.finos.org)Provide number from dashboard; Ideally no program should have >1 company on the PMC
% of Women Active Participants (APs)(Not currently tracked in a FINOS system)NoNoFor now, estimate percentage of people in the program who identify as women
# or projects / working groups w/o a chair/leadFINOS metadata / GithubNoYes
Roadmap & Resources
2019 Roadmap Published On Wiki (Y/N)WikiNoNoIndicate whether roadmap has been published, and where (wiki, GH pages, both)
# of FTEs (excl. FINOS staff) committed to program activityn/aNoNoPMCs will need to discuss and manually estimate what resourcing commitments look like across the program contributor community. 
# of projects in incubating status > 6 months

FINOS metadata

NoYesPMC Email Reports
Hygiene & Operations

# of PMC Meetings in last quarter

Wiki; Google CalendarYesNoValidate number in metrics.finos.org
# of Un-actioned PRs (Pull Requests)Github


# of cancelled meetings w less than 48 hour noticeFINOS tracks this manuallyNoNoFINOS will provide this to PMCs
Quality & Security
# of projects in program using ODP CI tooling



Repositories with, or pulling from upstream repos, with non-standard licenses (e.g., licenses difficult to reconcile with Apache 2.0)



Growth & Adoption
Growth in Active Participants (quarter ending vs. two quarters ago)Github; WebEx; Google GroupsList of active participants is available in the Program Health Dashboard(via link in monthly PMC report email to Program Health Dashboard)Filter in metrics.finos.org by time period to do time period comparison
Number of Pull Requests across all projects (last 3 months)Github / Gitlab


Number of Commits (last 3 months)Github / Gitlab


Number of Organizations Using Program Released Code, Software, Standards, Etcn/aNoNoPMCs must track manually for now
Program Master Data 
PMC LeadFINOS MetadataNoYesNot required
PMC MembersFINOS MetadataNoYesNot required
List of Working Groups (and Stage/Status)FINOS MetadataNoYesNot required
List of Projects (and Stage/Status)FINOS MetadataNoYesNot required
List of Active Participants
Yes(via link in monthly PMC report email to Program Health Dashboard in metrics.finos.org)Not required





Program Steering (For More Information on thee following measures See "PMC Report" section of PMC Monthly Report Overview)
Activities (Projects/Workring Groups) without a Lead/ChairFINOS MetadataNoYes, if there is oneNot required
Inactive ActivitiesFINOS MetadataNoYes, if there is oneNot required
Activities with Unactioned PRsFINOS MetadataNoYes, if there is oneNot required
Activities with Unactioned IssuesFINOS MetadataNoYes, if there is oneNot required
Activities with GitHub Repositories with Non-standard LicensesFINOS MetadataNoYes, if there is oneNot required
Archived Activities that Aren't Archived in GitHubFINOS MetadataNoYes, if there is oneNot required
Activities that Have GitHub Repositories Without Issue Tracking EnabledFINOS MetadataNo

Yes, if there is oneNot required
FINOS OKR Related Measures
% of PRs from financial institutions (Specific KR: ">20% of PRs come from financial institutions")Github; FINOS MetadataNoNoNot required

Version History

VersionDateChanges
1.02019.1.23Initial version presented to the board
1.0.12019.1.31Minor grammatical edits and clarifications
1.22019.2.7Added metrics column; Changed "% of code contributors using ODP CI/CD" to "# of projects in program using ODP CI tooling"; clarified licensing measure; Changed Number of Accepted Pull Requests to Number of Pull Requests
1.32019.3.27Added "Where to Find Key Program Health Check Metrics" Table (WIP)