2019-12-02 kdb+ Project Meeting Minutes

Meeting minutes status: Approved (at 2020-01-06 meeting)

Table of Contents

Date/Time

  • December 2, 2019. 11am ET

Attendees

NameOrganisationGithub ID 
Alvin ShihMorgan Stanleyalvin-c-shih
Aitana MyohlFINOSaitana16
James McLeod (Unlicensed)FINOSmcleo-d
Aaron DaviesMorgan Stanleyadavies42
Alok DuttCiti
Gary DaviesAquaQ
Kai ZengUBS
Peter GyorokMorgan Stanleygyorokpeter
Tim BeletskiyJP Morgan

Action Items

Tasks Identified and Assigned

  • Alok to reach out to BAML and GS
  • Alvin Shih to send out conventions that get used for qualifying names.

Task Report from the Last Meeting

(In the task report select the meetings notes from the LAST meeting in the 'Space(s) and page(s)' field to auto-populate this table– remove this note before publishing)

DescriptionDue dateAssigneeTask appears on
  • Asking about license for developers
2018-09-05 kdb+ WG Minutes
  • Decision - How to make use of the Kx/Kdb naming space in a consistent way across firms
2018-09-05 kdb+ WG Minutes
  • Share sample tests, configurations, in part to start to create guidelines for naming (on-going task as working group stands up)
2018-09-05 kdb+ WG Minutes

Decisions Made


Agenda

TimeItemWhoNotes from the Meeting
5 minConvene & roll call.

All


5 minReview and approve minutes from last meeting.All
  • Approved by acclamation.
20 min

memusage.q and timer merged to master.

Any news about microsite for documentation?

All
20 min

Best practices for extending HDB functionality?

Some ideas from last time:

  • Sparse HDB / Partial HDB reload
    • Push old functions to be callable from somewhere?
    • Only hook entry points and reimplement everything under .finos?
    • Detect changes in implementation of upstream functions using a hash?
  • Staging Directory / Archive Directories
    • How to specify?
    • finos_hdb_config/ under HDB root?
    • *.q file to verify correct extensions are loaded?
All
  • Some discussion of the HDB well-formedness doc first.
    • Aaron said that the information contained therein exists in various places.
    • The main benefit is seeing it all in one place.
      • Could use more detail on AnyMap.
    • Could start writing unit tests for HDB functionality based on the doc.
      • Aaron might have a few tests lying around.
    • Good to have before embarking on any HDB extensions.
  • Extending HDB functionality?
    • Significant interest in the ability to load multiple HDBs.
      • Some have heard that Kx is looking into it.
      • Questions as to any restrictions on joins between HDBs.
    • Many of the interesting extensions are not backwards compatible.
      • Need a *.q file in the HDB root to verify that correct extensions are loaded.
      • MD5 hashes of .Q.ps to detect stock vs. custom HDB logic.
    • .Q.l extensions
      • Adds staging directories as extra argument?
      • Or maybe looks for an alternative to par.txt since there might have to be one staging directory per segment for performance.
  • How might FINOS and Kx collaborate?
    • Boost C++ libraries are something of a testing ground.
    • Those which get wide usage are then used as the basis for additions to the C++ Standard Library.
      • Sometimes the API receives significant changes before being incorporated into the standard.
    • Analogously, Kx could see what might be worth incorporating directly in to kdb.
      • Those features that they don't want to support directly, they could add hooks to facilitate extension.
      • For example, .z.ph implementation was changed to call .h.val to allow authorization handlers to validate requests in HTML mode.
  • Hiding files related to HDB extensions?
    • Hide files using trailing '$'?
      • Seems like a nonzero risk of being overwritten.
    • html/ directory can contain anything.
      • Risk of unintentionally serving configuration information / code over HTTP?
    • Maybe a single directory ending in '$', like "finos$/"?
    • Ignore .Q.l completely?
      • .finos.hdb.load[...]
  • Backwards compatible enhancements?
    • .Q.chk with peach support.
    • Support alternate partitionings.
      • Currently limited to: date, month, year, or int.
      • Desirable partition types include: long, timestamp.
10 minDemo of Docusaurus.
  • James gave a quick demo on his MacBook.
    • cd website/
    • npm install
    • npm start
      • Starts the microsite locally for preview.
  • Much more understandable after seeing it in action.
5 minAOBAll

Action Items

  • Jason Foster-Bey to start follow-up thread re:structlog
  • Aaron Davies to send out HDB consistency doc.
  • Peter Gyorok to put "include" on GitHub.
  • Alvin Shih to put together qstructlog PoC.
  • Aaron Davies to get Symphony access to discuss authz_ro with Jason Foster-Bey.
  • Aaron Davies to get approvals for pushing kdb+ WG-related work to GitHub.

Need help? Email help@finos.org we'll get back to you.

Content on this page is licensed under the CC BY 4.0 license.
Code on this page is licensed under the Apache 2.0 license.