@Alvin Shih

Morgan Stanley

alvin-c-shih

@Rob Underwood (Deactivated)

FINOS

brooklynrob

Aaron Davies

Morgan Stanley

adavies42

Alok Dutt

Citi

Kai Zeng

UBS

Peter Gyorok

Morgan Stanley

gyorokpeter

Pinhus Dashevsky

UBS

Jason Foster-Bey

JPMC

5 min

Convene & roll call.

All

5 min

Review and approve minutes from last meeting.

All

Approved by acclamation.

5 min

Review "Enterprise kdb+ vs. Small Team kdb+" linked from 2019 Roadmap for kdb+ Working Group.

All

Discussion

• How do we move from styles of code that are more exploratory (in JupyterQ for example) to something that fits "enterprise" constraints?

  • There is a difference between a quant writing code and a professional software developer.

  • One idea is for code to pass through approval stages before being deployed in Production.

• This is related to managing name collisions.

  • For example, how does GS manage its SLANG code?

  • Also does JPMC Athena provide any models?
    
    
    

• Perhaps coming up with use cases will point us in the right direction.

5 min

Rethink internal documentation convention.  JavaDoc-style comment as first comment in function?  Start with /// .

All

• JavaDoc-style comments put where Python DocStrings typically appear would require qdoc to be hacked on.

  • Maybe a simple Perl5 preprocessing script to lift the /// comment block out of the function body would allow qdoc to be salvaged.

• Possible integration with help.q from Kx?

• Functionality to lift comment out of function from a static *.q file would require an implementation of the q script loading logic.  Maybe possible to reverse engineer, but MS has a request to Kx to expose the functionality to make things easier for everyone.

• Kx Analyst allows generation of Markdown files.

  • Need to get more information on how they expect comments to be formatted to take advantage of that.

  • Side comment that Kx Analyst wraps calls in try-catch blocks, thereby eating diagnostic information.

    • Wrapping the function with one's own error trap function and using .Q.sbt could allow one to capture the call stack information.

    • Use a global variable to turn this off and on.

• q-studio purportedly has a way of generating HTML files.

  • Not sure it has enough market share to worry about.

5 min

structlog configuration complexiity.  Does not have nice formatting out of the box.  (Not even log level!)

All

• Those with experience with Python structlog felt it wasn't ideal.

• Would prefer something that accumulates logs in-memory or appends to disk in binary format to avoid having to parse / scrape them.

• One concern raised that support personnel might not be familiar with how to work with the binary files.

• ELK centralized logging suggested as a model to emulate.

  • Correlated processes on different machines.

• To avoid getting too bogged down, need to articulate short-term vs. long-term requirements.

• Revisit logging API with pluggable back end.

  • Formatting hooks.

  • Storage hooks.

5 min

AOB

All