The roadmap below has been set out with the needs of "enterprise kdb+ users" in mind. For more background, see:
We'd like a working group small enough to work quickly, but diverse enough to capture a majority of use cases. We are targeting 8-12 active participants with the assumption that not all members will be willing / able to participate in all discussions. Participants should have experience in organizations facing the challenges of "enterprise" deployment of kdb+.
Working Group members should actively recruit known members of the kdb+ community likely to add significant value.
Should the working group exceed these participation targets, we may choose to institute policies, workflows, and/or committees to increase overall effectiveness.
The working group will begin by developing a common coding style, documentation format, and namespace standards.
Adherence to these standards will be required for all FINOS-hosted projects and evangelized through working group members and external forums (such as mailing lists, meetups, conferences, etc.).
We seek to identify the greatest needs of the enterprise kdb+ community and document them as a "wish list" to onboard to The Foundation.
This will express the features required for enterprise users as well as how projects can interoperate. The goal is to be implementation-agnostic and focus on APIs that can remain stable.
It is expected that APIs could take significant time to reach a stable state due to:
Users invoke standalone utilities through a command shell which is less expressive than q. Therefore, it should be easier to reach agreement on the interface to standalone utilities and release them to the community.
Projects that are not fully ratified can live in "incubator" branches for comment / refinement. Policy and procedures around such projects will be formalized and posted.
Kx Systems has released q-Python cross-interpreter calling capability as part of their "Fusion APIs":
For many enterprise kdb+ licensees, projects are springing up with a Data Science pipelines which combine:
We hope to find ways to support this by ensuring that our standards align with Python best practices where it is beneficial to do so. However, many of the best kdb developers have been deeply-steeped in Kx culture for a decade or more and may be new to Python. Therefore, some ramp up time is required unless we can recruit Python experts into the kdb+ Working Group.
Preloaded components have less complex interactions with the user and other components. Therefore, it is likely that such components will be released before module-loadable components for the same reasons as those cited for "Standalone Utilities", Examples of "preloaded components" under consideration:
By this point, the expectation is that the Working Group will have gathered sufficient information about:
With that information in mind, it should be quick progress to agree on a module loading standard and PoC.
Once the PoC is available, an ecosystem of components should build from there.
The ultimate goal is to foster a community where the best project "wins" through adoption and contributions - not through a blessing of the Working Group.
The WG is envisioned as a marketplace for projects where we identify common areas of difficulty and offer solutions to those problems that may benefit the wider Data Technologies Program.