Kdb+

kdb+
Original author(s) Arthur Whitney
Developer(s) Kx Systems
Initial release 2003
Written in Q

kdb+ is a column-based relational database with in-memory capabilities, developed and marketed by Kx Systems.[1] The database is commonly used in high frequency trading to store, analyze, process and retrieve large data sets at high speed.[2] kdb+ has the ability to handle billions of records and analyzes data within the database.[3] The database is available in 32 and 64-bit versions over a variety of operating systems.[4] Financial institutions use kdb+ to analyze time series data such as stock or commodity exchange data.[5] The database has also been used for other time-sensitive data applications including energy trading, telecommunications, sensor data, log data, and machine and network usage monitoring.[6]

Overview

kdb+ is a high-performance column-store database that was designed for processing and storing large amounts of data. Commonly accessed data is pushed into memory, which is faster to access than data stored on disk. Created with financial institutions in mind, the database was developed as a central repository to store time series data that supports real-time analysis of billions of records.[7][1][8] kdb+ has the ability to analyze data over time and responds to SQL-like queries.[2]

Columnar databases return answers to queries in a more efficient way than row-based database management systems.[4][5] kdb+ dictionaries, tables and nanosecond time stamps are native data types and are used to store time series data.[5][9]

At the core of kdb+ is the built-in programming language, q. q is an expressive query language.[7] The concise array language can manipulate streaming, real-time and historical data.[2] kdb+ uses q to aggregate and analyze data, perform statistical functions, and join data sets and supports SQL queries[10] The vector language q was built for speed and expressiveness and mostly eliminates the need for looping structures.[5] kdb+ includes interfaces in C/C++, Java, C# and Python.[4][5]

History

In 1998, Kx Systems released kdb, a database built on the language K written by Arthur Whitney. In 2003, kdb+ was released as a 64-bit version of kdb. In 2004, the kdb+ tick market database framework was released along with kdb+ taq, a loader for the NYSE taq data. kdb+ was created by Arthur Whitney built on his previous work with array languages.[5]

In April 2007, Kx Systems announced that it was releasing a Mac OS X-compatible version of the kdb+ database software. At that time, kdb+ was also available on the Linux, Windows and Solaris operating systems.[11]

Version 3.0 was released in September 2012 which was optimized for Intel's upgraded processors with support for WebSockets, GUIDs and UUIDs. Intel's Advanced Vector Extensions (AVX) and SSE4.2 on the Sandy Bridge processors of the time allowed for enhanced support of the kdb+ system.[3] Version 3.1 was released in June 2013. Benchmarks for the release were up to 8 times faster than benchmarks on previous versions.[12]

References

  1. 1 2 "Kdb+ and the Internet of Things/Big Data". Inside Big Data. January 19, 2015. Retrieved June 8, 2016.
  2. 1 2 3 "Kx Systems and High Frequency Trading". The Trading Mesh. June 10, 2011. Retrieved June 8, 2016.
  3. 1 2 Tom Groenfeldt (September 24, 2012). "At Kx Systems Big Data Is Old News". Forbes. Retrieved June 8, 2016.
  4. 1 2 3 Luca Sbardella (October 2, 2009). "A first look at kdb+". Quantmind. Retrieved June 8, 2016.
  5. 1 2 3 4 5 6 Douglas Eadline (September 9, 2014). "Working Down the Column: The Kdb+ Community". Cluster Monkey. Retrieved June 8, 2016.
  6. ↑ Jason Stamper (September 4, 2014). "Kx Systems readies faster database in push beyond financial services". 451 Research.
  7. 1 2 Victor Golovtchenko (January 1, 2016). "First Derivatives Boosts its KDB+ Software Business with New Acquisition". Finance Magnates. Retrieved June 8, 2016.
  8. ↑ "Kdb+". Itl Group. Retrieved June 8, 2016.
  9. ↑ Penny Crosman (September 14, 2009). "Kx Offers Nanosecond Timestamps in Updated Database". Information Week. Retrieved June 8, 2016.
  10. ↑ Steve Marlin (June 19, 2014). "Wall Street Firms Extend 'Big Data; Capabilities". Markets Media. Retrieved June 8, 2016.
  11. ↑ Peter Cohen (April 17, 2007). "Kdb+ financial database comes to Mac OS X". Macworld. Retrieved June 8, 2016.
  12. ↑ "Kx's kdb+ claims up to 8x faster than any previous STAC M3 benchmarks". Automated Trader. June 11, 2013. Retrieved June 8, 2016.
This article is issued from Wikipedia - version of the 9/17/2016. The text is available under the Creative Commons Attribution/Share Alike but additional terms may apply for the media files.