2019-11-18 kdb+ Project Meeting Minutes

Meeting minutes status: Approved (at 2019-12-02 meeting)

Table of Contents

Date/Time

  • November 18, 2019. 11am ET

Attendees

NameOrganisationGithub ID 
Alvin ShihMorgan Stanleyalvin-c-shih
FINOSbrooklynrob
Jonny PressAquaQ
Peter GyorokMorgan Stanleygyorokpeter
Timofey BeletskiyJP Morgan

Action Items

Tasks Identified and Assigned

  • Alok to reach out to BAML and GS
  • Alvin Shih to send out conventions that get used for qualifying names.

Task Report from the Last Meeting

(In the task report select the meetings notes from the LAST meeting in the 'Space(s) and page(s)' field to auto-populate this table– remove this note before publishing)

DescriptionDue dateAssigneeTask appears on
  • Asking about license for developers
2018-09-05 kdb+ WG Minutes
  • Decision - How to make use of the Kx/Kdb naming space in a consistent way across firms
2018-09-05 kdb+ WG Minutes
  • Share sample tests, configurations, in part to start to create guidelines for naming (on-going task as working group stands up)
2018-09-05 kdb+ WG Minutes

Decisions Made


Agenda

TimeItemWhoNotes from the Meeting
5 minConvene & roll call.

All


5 minReview and approve minutes from last meeting.All
  • Approved by acclamation.
10 min

HDB Well-Formedness.md document is live under "technical-documentation".

Aaron to go over what it is and what it isn't.  Hopefully opens the door to shared development of HDB utils.

All
  • Aaron unable to make the call.
  • Good discussions about extensions that go beyond Kx functionality.
    • Ideas that have been implemented at Morgan Stanley:
      • Sparse HDB
        • If new table is introduced, backfilling many years of empty table segments is time consuming and makes schema changes more expensive.
      • Staging Directory
        • When a table segment is only partially written due to I/O speed limitations, the HDB as a whole is technically not well-formed.
        • Staging a table on the same mount point and moving it atomically when writes complete could avoid problems with intra-day HDB reloads.
      • Partial HDB Reload
        • Intraday writedown (w.q) often implies intraday reload.  Loading of HDBs is I/O intensive.  A means of loading for a target date range could help tremendously.
    • JP Morgan interested in extensions that support archiving in a standard way.
      • Perhaps recent data run on more performant storage, whereas older data would be hosted on slower, cheaper drives.
  • Perhaps worth creating an rfc/ directory in the repository to get the details out in a shareable format.
5 minGitHub repository now has reviewers.All
  • Should be able to turn reviews / approvals around more quickly.
  • Hope to add one or two more based on strong contributors.
5 min

Managed to merge a couple of branches to main (with the help of Magit):

Hoping it makes it easier for others to improve on them.

All
  • timer pull request to be merged soon.
5 minUpdate on AquaQ's effort to FINOS-ize memusage.q.All
  • memusage pull request is up.
  • Question arose about how documentation should be maintained.
    • AquaQ is familiar with GitHub microsites as they use those already.
    • Suggestion that they talk to Eric Griswald to see what (if anything) is different with microsites under FINOS compared to what AquaQ is doing already.
5 minAOBAll

Action Items

  • Jason Foster-Bey to start follow-up thread re:structlog
  • Aaron Davies to send out HDB consistency doc.
  • Peter Gyorok to put "include" on GitHub.
  • Alvin Shih to put together qstructlog PoC.
  • Aaron Davies to get Symphony access to discuss authz_ro with Jason Foster-Bey.
  • Aaron Davies to get approvals for pushing kdb+ WG-related work to GitHub.

Need help? Email help@finos.org we'll get back to you.

Content on this page is licensed under the CC BY 4.0 license.
Code on this page is licensed under the Apache 2.0 license.