Fakultät für Informatik TU München - Fakultät für Informatik
Lehrstuhl III: Datenbanksysteme
Technische Universität München
Sie befinden sich auf einer Seite unserer alten Webpräsenz. Die aktuelle Lehrstuhls-Website finden Sie unter http://www3.informatik.tu-muenchen.de/.

HyPer: Hybrid OLTP&OLAP High-Performance Database System

Summary

The HyPer prototype demonstrates that it is indeed possible to build a main-memory database system that achieves world-record transaction processing throughput and best-of-breed OLAP query response times in one system in parallel on the same database state. The two workloads of online transaction processing (OLTP) and online analytical processing (OLAP) present different challenges for database architectures. Currently, users with high rates of mission-critical transactions have split their data into two separate systems, one database for OLTP and one so-called data warehouse for OLAP. While allowing for decent transaction rates, this separation has many disadvantages including data freshness issues due to the delay caused by only periodically initiating the Extract Transform Load-data staging and excessive resource consumption due to maintaining two separate information systems. We present an efficient hybrid system, called HyPer, that can handle both OLTP and OLAP simultaneously by using hardware-assisted replication mechanisms to maintain consistent snapshots of the transactional data (see the figure on the right). HyPer is a main-memory database system that guarantees the full ACID properties for OLTP transactions and executes OLAP query sessions (multiple queries) on arbitrarily current and consistent snapshots. The utilization of the processor-inherent support for virtual memory management (address translation, caching, copy-on-write) yields both at the same time: unprecedentedly high transaction rates as high as 100000 per second and very fast OLAP query response times on a single system executing both workloads in parallel. The performance analysis is based on a combined TPC-C and TPC-H benchmark.

We have developed the novel hybrid OLTP&OLAP database system HyPer that is based on snapshotting transactional data via the virtual memory management of the operating ... more

Research topics

Plenty - contact us if you are interested in a Thesis, student job or even a PhD position!

Documents

HyPer – Hybrid OLTP&OLAP High Performance Database System
Alfons Kemper and Thomas Neumann, Technical Report, TUM-I1010, May, 19, 2010. (pdf)
HyPer – A Hybrid OLTP&OLAP Main Memory Database System Based on Virtual Memory Snapshots.
Alfons Kemper and Thomas Neumann, ICDE 2011
Benchmarking Hybrid OLTP&OLAP Database Systems
Florian Funke and Alfons Kemper and Thomas Neumann, BTW 2011
HyPer: Die effiziente Reinkarnation des Schattenspeichers in einem Hauptspeicher-DBMS
Florian Funke and Alfons Kemper and Henrik Muehe and Thomas Neumann, Datenbank Spektrum, Springer-Verlag, 2011
How to Efficiently Snapshot Transactional Data: Hardware or Software Controlled?
Henrik Muehe and Alfons Kemper and Thomas Neumann, DaMoN 2011
The mixed workload CH-benCHmark
Dagstuhl "Robust Query Processing" Breakout Group "Workload Management", DBTest 2011
Efficiently Compiling Efficient Query Plans for Modern Hardware
Thomas Neumann, VLDB 2011 (pdf)
HyPer-sonic Combined Transaction AND Query Processing (project demo)
Florian Funke and Alfons Kemper and Thomas Neumann, VLDB 2011
The Mainframe Strikes Back: Multi Tenancy in the Main Memory Database HyPer on a TB-Server
Henrik Muehe, Alfons Kemper and Thomas Neumann, GI Workshop zum Thema Innovative Unternehmensanwendungen mit In-Memory Data Management. (website), December 2011
The Mainframe Strikes Back: Elastic Multi-Tenancy Using Main Memory Database Systems On A Many-Core Server
Henrik Muehe, Alfons Kemper and Thomas Neumann, EDBT 2012, March 2012
Massively Parallel Sort-Merge Joins in Main Memory Multi-Core Database Systems
Martina-Cezara Albutiu, Alfons Kemper and Thomas Neumann, Technical Report, TUM-I121, March, 16, 2012. (pdf,pptx)
Excerpt published in VLDB 2012 (pdf)
HyPer: Adapting Columnar Main -Memory Data Management for Transactional AND Query Processing
A. Kemper, T. Neumann, F. Funke, V. Leis, H. Muehe, Bulletin of the Technical Committee on Data Engineering, March 2012, Vol 35, No 1, pages 46-51.
Compacting Transactional Data in Hybrid OLTP&OLAP Databases
F. Funke, A. Kemper, T. Neumann, VLDB 2012 (pdf)
The Adaptive Radix Tree: ARTful Indexing for Main-Memory Databases
Viktor Leis and Alfons Kemper and Thomas Neumann, ICDE 2013 (pdf)
CPU and Cache Efficient Management of Memory-Resident Databases
Holger Pirk, Florian Funke, Martin Grund, Thomas Neumann, Ulf Leser, Stefan Manegold, Alfons Kemper, Martin Kersten, ICDE 2013
Executing Long-Running Transactions in Synchronization-Free Main Memory Database Systems
Henrik Mühe and Alfons Kemper and Thomas Neumann, CIDR 2013 (pdf)
ScyPer: A Hybrid OLTP&OLAP Distributed Main Memory Database System for Scalable Real-Time Analytics
T. Mühlbauer, W. Rödiger, A. Reiser, A. Kemper, T. Neumann, BTW 2013
Extending the MPSM Join
Martina-Cezara Albutiu, Alfons Kemper and Thomas Neumann, BTW 2013
DeltaNI: An Efficient Labeling Scheme for Versioned Hierarchical Data
J. Finis, R. Brunel, A. Kemper, T. Neumann, F. Faerber, N. May, ACM SIGMOD Conference on Management of Data 2013
ScyPer: Elastic OLAP Throughput on Transactional Data
T. Mühlbauer, W. Rödiger, A. Reiser, A. Kemper, T. Neumann, ACM SIGMOD Workshop on Data analytics in the Cloud 2013
Transaction Processing in the Hybrid OLTP&OLAP Main-Memory Database System HyPer
A. Kemper, T. Neumann, J. Finis, F. Funke, V. Leis, H. Mühe, T. Mühlbauer, W. Rödiger, IEEE Computer Society Data Engineering Bulletin, Special Issue on "Main Memory Databases", 2013

Presentations / Systemdemos

  • Colloquium of the Chair of Database Systems (May 21, 2010)
  • "Grundlagen von Datenbanken " Workshop (GvDB, Bad Helmstedt, May 26, 2010)
  • IBM Böblingen (June 26, 2010)
  • Inaugural Lecture ("Antrittsvorlesung" T. Neumann, July 22, 2010)
  • IBM Almaden Research (Aug 13, 2010)
  • HP Labs Palo Alto (Aug 24, 2010)
  • SAP Labs Palo Alto (Aug 30, 2010)
  • Greenplum (Sep 1, 2010. See Florian Waas' Blog about the presentation)
  • Oracle Redwood Shores (Sep 3, 2010)
  • Keynote at the VLDB BIRTE Workshop (Sep 13, 2010)
  • IBM DB2 Community Meeting, Böblingen (Sep 30, 2010)
  • SAP Walldorf (Oct 1,2010)
  • BTW (Mar 3, 2011. Benchmark-Presentation)
  • ICDE (April 12, 2011. Poster)
  • Humboldt Univ. Berlin (May 30, 2011).
  • "Grundlagen von Datenbanken " Workshop (Tirol, Austria, June 2011)
  • HyPer-sonic Combined Transaction AND Query Processing at HIPERFIT Workshop, Kopenhagen (02.12.11)
  • Skalierbarkeit ODER Virtualisierung at FGDB Herbsttreffen, Potsdam (18.11.11)
  • Oracle Labs Research - Tea Time Talk (June 13, 2012)
  • HyPer and its Scale-Out at Software AG (June 20, 2012)
  • IBM DB2 Community Meeting, Böblingen (Oct 11, 2012)
  • GI FG-DB Workshop Scalable Analytics (Nov 2, 2012)
  • Join Processing and Indexing in Multi-Core Main-Memory Databases, Oracle Labs (Jan 4, 2013)
  • ScyPer: A Hybrid OLTP&OLAP Distributed Main Memory Database System for Scalable Real-Time Analytics (Demo Poster), BTW (March 11-15, 2013)
  • The Adaptive Radix Tree, University of Sydney (April 5, 2013)

Lehrstuhl für Datenbanksysteme
Letzte Änderung: 19.07.2013 um 14:18:40