2019-10-07 kdb+ Project Meeting Minutes

Owned by Alvin Shih
Last updated: Dec 02, 2019
3 min read

Meeting minutes status: Approved (at 2019-11-18 meeting)

Table of Contents

Date/Time

  • October  7, 2019. 11am ET

Attendees

NameOrganisationGithub ID 
Alvin ShihMorgan Stanleyalvin-c-shih
Timofey BeletskiyJP Morgan
Aaron DaviesMorgan Stanleyadavies42
Aitana MyohlFINOSaitana16
Alok DuttCiti
Jonny PressAquaQ
Gary DaviesAquaQ
James McLeod (Unlicensed)FINOSmcleo-d
Kai ZengUBS

Action Items

Tasks Identified and Assigned

  • Alok to reach out to BAML and GS
  • Alvin Shih to send out conventions that get used for qualifying names.

Task Report from the Last Meeting

(In the task report select the meetings notes from the LAST meeting in the 'Space(s) and page(s)' field to auto-populate this table– remove this note before publishing)

DescriptionDue dateAssigneeTask appears on
  • Asking about license for developers
 2018-09-05 kdb+ WG Minutes
  • Decision - How to make use of the Kx/Kdb naming space in a consistent way across firms
 2018-09-05 kdb+ WG Minutes
  • Share sample tests, configurations, in part to start to create guidelines for naming (on-going task as working group stands up)
 2018-09-05 kdb+ WG Minutes

Decisions Made


Agenda

TimeItemWhoNotes from the Meeting
5 minConvene & roll call.

All


5 minReview and approve minutes from last meeting.All
20 min

Overview of TorQ and 4 components that AquaQ could break out as contributions to FINOS.


All
  • AquaQ offered to FINOS-ify 4 components as contributions.
    • Agreement from AquaQ and folks on the call that it'd be better to start with a few components rather than trying to bring in TorQ all at once.
  • In terms of Enterprise kdb+ sites using TorQ, a team in JPMC is starting to use it.
  • How can FINOS be useful to AquaQ / TorQ?
    • AquaQ developed TorQ to avoid rewriting utilities / framework for every new client.
    • Goals aligned with those of FINOS kdb+ Project.
    • FINOS allows Enterprise users to contribute back and / or provide a channel to discuss requirements.
25 minLook at AquaQ's memusage.q in more detail.All
  • Attendees decided to take a closer look at memusage.q first.
  • For compound columns longer than a specified threshold, sampling is used to estimate the space consumed by the subvectors.
    • Can increase the threshold to avoid sampling.
    • Might be useful to return an indication of whether sampling was employed.
  • memusage.q also tries to account for memory utilization due to attributes.
  • Seems like it could be useful during development.
  • AquaQ said it's useful for diagnosing memory issues on live systems as well.
    • Caller from MS said it would be useful in cases where users are stashing results on a shared q process.
  • A wrapper to find large values could be included.
    • Need to take care to handle \d / namespaces properly.
    • Some "by name" functions in q are inconsistent in how namespaces are handled.
      • Example was "set" vs "::".
  • How to FINOS-ify?
    • Suggested putting the functions in .finos.objutil since memory consumption is just one facet of an object that a developer might care about.
    • Others might include recursive traversal (related to finding large objects) and unpacking projections.
    • Mentioned that global variable like "threshold" after which statistical sampling is employed should have getter / setter to support namespace aliasing.
  • Caller from Citi seemed enthusiastic about trying it out.
5 minAOBAll
  • James McLeod introduced himself as new FINOS Director of Community based in London.
    • Might facilitate meeting up with AquaQ representatives.

Action Items

  • Jason Foster-Bey to start follow-up thread re:structlog
  • Aaron Davies to send out HDB consistency doc.
  • Peter Gyorok to put "include" on GitHub.
  • Alvin Shih to put together qstructlog PoC.
  • Aaron Davies to get Symphony access to discuss authz_ro with Jason Foster-Bey.
  • Aaron Davies to get approvals for pushing kdb+ WG-related work to GitHub.

Need help? Email help@finos.org we'll get back to you.

Content on this page is licensed under the CC BY 4.0 license.
Code on this page is licensed under the Apache 2.0 license.