Sun. Nov 17th, 2019

Apache Kylin v2.6.3 released, Open source distributed analytics engine

2 min read

Apache Kylin is an open source Distributed Analytics Engine designed to provide SQL interface and multi-dimensional analysis (OLAP) on Hadoop supporting extremely large datasets, originally contributed from eBay Inc.

Apache Kylin lets you query massive dataset at sub-second latency in 3 steps.

  1. Identify a Star Schema on Hadoop.
  2. Build Cube from the identified tables.
  3. Query with ANSI-SQL and get results in sub-second, via ODBC, JDBC or RESTful API.

WHAT IS KYLIN?

– Extremely Fast OLAP Engine at Scale: 

Kylin is designed to reduce query latency on Hadoop for 10+ billions of rows of data

– ANSI SQL Interface on Hadoop: 

Kylin offers ANSI SQL on Hadoop and supports most ANSI SQL query functions

– Interactive Query Capability: 

Users can interact with Hadoop data via Kylin at sub-second latency, better than Hive queries for the same dataset

– MOLAP Cube:

User can define a data model and pre-build in Kylin with more than 10+ billions of raw data records

– Seamless Integration with BI Tools:

Kylin currently offers integration capability with BI Tools like Tableau, PowerBI, and Excel. Integration with Microstrategy is coming soon

– Other Highlights: 

– Job Management and Monitoring
– Compression and Encoding Support
– Incremental Refresh of Cubes
– Leverage HBase Coprocessor for query latency
– Both approximate and precise Query Capabilities for Distinct Count
– Approximate Top-N Query Capability
– Easy Web interface to manage, build, monitor and query cubes
– Security capability to set ACL at Cube/Project Level
– Support LDAP and SAML Integration

Apache Kylin v2.6.3 was released.

Changelog

Improvement

  • [KYLIN-4024] – Support pushdown to Presto
  • [KYLIN-3977] – Avoid mistaken deleting dicts by storage cleanup while building jobs are running
  • [KYLIN-4023] – Convert to local time for column with timestamp or date type of queries by Jdbc
  • [KYLIN-3893] – Cube build failed for wrong row key column description
  • [KYLIN-4020] – Add check for the input of fix_length rowkey encoding
  • [KYLIN-3998] – Make “bpus-min-benefit-ratio” configurable in cube planner phase 1
  • [KYLIN-4025] – Add detail exception in kylin http response

Bug Fix

  • [KYLIN-4015] – Fix build cube error at the “Build UHC Dictionary” step
  • [KYLIN-4022] – Error with message “Unrecognized column type: DECIMAL(xx,xx)” happens when do query pushdown
  • [KYLIN-3994] – Storage cleanup job may delete data of newly built segment because of cube cache in CubeManager
  • [KYLIN-2620] – Result of sum measure may by replaced by the TopN measure with the same column during query
  • [KYLIN-4041] – Concat function with null value does not work correctly
  • [KYLIN-3978] – InternalErrorException happens for queries with precise count distinct measures
  • [KYLIN-3845] – Streaming cube build error if the Kafka data source lacks selected dimensions or metrics
  • [KYLIN-3980] – Cube planner cuboid id error there are too many dimensions
  • [KYLIN-4049] – Storage should be cleaned after cube refreshing with config kylin.storage.clean-after-delete-operation set to true

Download