2019-06-03 kdb+ WG Minutes

Table of Contents

Date/Time

  • June 3, 2019. 11am ET

Attendees

NameOrganisationGithub ID 
Alvin ShihMorgan Stanleyalvin-c-shih
Rob Underwood (Deactivated)FINOSbrooklynrob
Aitana MyohlFINOS
Aaron DaviesMorgan Stanleyadavies42
Peter GyorokMorgan Stanleygyorokpeter
Michal PawlowskiUBS
Kai ZengUBS
Jason Foster-BeyJP Morgan

Action Items

Tasks Identified and Assigned

  • Alok to reach out to BAML and GS
  • Alvin Shih to send out conventions that get used for qualifying names.

Task Report from the Last Meeting

(In the task report select the meetings notes from the LAST meeting in the 'Space(s) and page(s)' field to auto-populate this table– remove this note before publishing)

DescriptionDue dateAssigneeTask appears on
  • Asking about license for developers
2018-09-05 kdb+ WG Minutes
  • Decision - How to make use of the Kx/Kdb naming space in a consistent way across firms
2018-09-05 kdb+ WG Minutes
  • Share sample tests, configurations, in part to start to create guidelines for naming (on-going task as working group stands up)
2018-09-05 kdb+ WG Minutes

Decisions Made


Agenda

TimeItemWhoNotes from the Meeting
5 minConvene & roll call.

All


5 minReview and approve minutes from last meeting.AllApproved by acclamation.
5 minKx engagement?  How much do we seek?  Ideas for increasing involvement.AllKx has two new Technical Evangelists in NYC.  They did not seem aware of the kdb+ Working Group.  Passed along a few links.
5 min

structlog revisited.  Discuss proposed q API.  Who can make time to work on pieces of it?

All

A PoC will be put into the incubator with minimum required pipeline stages.  Intention is for others to contribute additional pipes / filters based on their own use cases.

For the most part, the pipes will be manipulating a q dictionary.  The standard keys would be: `timestamp, `level, `event.

Would be good to start standardizing names for keys.  Possibilities include: `hostname, `username, `workspace_statistics .

To ensure interoperability of logging producers and consumers, a pipeline stage might check for nonstandard tags and throw an error (in development mode, anyway).

Further discussion can take place on the mail group.

2 minDefault security handler API update.AllPoC should be on GitHub incubation branch before the next kdb+ WG call.
10 minPrioritize proposed APIs: Timer, Debug utilities, Include, Inithook.All

Start with "Include".  Still needs some iterating; basic version would be easy.

"InitHook" creates a framework which later modules can build on, so would be nice if that went out on the earlier side.

Debug utilities which can get the arity of lambdas / projections are also available, but probably lower priority.

5schema.q best practices?  Y2.1k hacks?All

kdb used to infer the schema based on the earliest date partition.  Later on, this changed to latest date partition.  This can result in an inconsistent view if accessed while tables are being written for latest date.  Some hack around this by writing a partition at an arbitrarily chosen date in the future.  A schema.q standard might avoid this.

A document of what is considered a "consistent HDB" is something kdb+ WG can work on.  There are ways to modify kdb to ignore empty table segments.  This might evolve into a kdb+ WG best practices document or even code.

5 minAOBAllNew joiners to the call can view the mailing list archives on Google Groups via the link on the kdb+ Working Group Wiki home page.

Action Items

  • Jason Foster-Bey to start follow-up thread re:structlog
  • Aaron Davies to send out HDB consistency doc.
  • Peter Gyorok to put "include" on GitHub.
  • Alvin Shih to put together qstructlog PoC.

Need help? Email help@finos.org we'll get back to you.

Content on this page is licensed under the CC BY 4.0 license.
Code on this page is licensed under the Apache 2.0 license.