Release 1.6.0

Key Features

AreaFeature
Star TreeSupport update cube command to allow admin to easily update an existing cube when the underlying data changes
Bloom IndexHindex-Optimize Bloom Index Size-Reduce bloom index size by 10X+ times
Task Recovery1. Improve failure detection time: It need take 300s to determine a task is failed and resume after that. Improving this would improve the resume & also the overall query time
2. snapshotting speed & size: When sql execute takes a snapshot, now use direct Java serialization which is slow and also takes more size. Using kryo serialization would reduce size and also increase speed there by increasing the overall throughput
Spill to Disk1. Spill to disk speed & size improvement: When spill happens during HashAggregation & GroupBy, the data serialized to disk is slow and also size is more. It can improve the overall performance by reducing size and also improving the writing speed. Using kryo serialization improves both speed and reduces size
2. Support spilling to hdfs: Currently data can spill to multiple disks, now support spill to hdfs to improve throughput
3. Async spill/unspill: When revocable memory crosses threshold and spill is triggered, it blocks accepting the data from the downstream operators. Accepting this and adding to the existing spill would help to complete the pipeline faster
4. Enable spill for right outer & full join for spilling: It don’t spill the build side data when the join type is right outer or full join as it needs the entire data in memory for lookup. This leads to out of memory when the data size is more. Instead by enable spill and create a Bloom Filter to identify the data spilled and use it during join with probe side
Connector EnhancementSupport data update and delete operator for PostgreSQL and openGauss

Known Issues

CategoryDescriptionGitee issue
Task RecoveryWhen a snapshot is enabled and a CTAS with transaction is executed, an error is reported in the SQL statement.I502KF
An error occurs occasionally when snapshot is enabled and exchange.is-timeout-failure-detection-enabled is disabled.I4Y3TQ
Star TreeIn the memory connector, after the star tree is enabled, data inconsistency occurs during query.I4QQUB
When the reload cube command is executed for 10 different cubes at the same time, some cubes fail to be reloaded.I4VSVJ

Obtaining the Document

For details, see https://gitee.com/openlookeng/hetu-core/tree/1.6.0/hetu-docs/en

有奖捉虫

“有虫”文档片段

0/500

存在的问题

文档存在风险与错误

● 拼写,格式,无效链接等错误;

● 技术原理、功能、规格等描述和软件不一致,存在错误;

● 原理图、架构图等存在错误;

● 版本号不匹配:文档版本或内容描述和实际软件不一致;

● 对重要数据或系统存在风险的操作,缺少安全提示;

● 排版不美观,影响阅读;

内容描述不清晰

● 描述存在歧义;

● 图形、表格、文字等晦涩难懂;

● 逻辑不清晰,该分类、分项、分步骤的没有给出;

内容获取有困难

● 很难通过搜索引擎,openLooKeng官网,相关博客找到所需内容;

示例代码有错误

● 命令、命令参数等错误;

● 命令无法执行或无法完成对应功能;

内容有缺失

● 关键步骤错误或缺失,无法指导用户完成任务,比如安装、配置、部署等;

● 逻辑不清晰,该分类、分项、分步骤的没有给出

● 图形、表格、文字等晦涩难懂

● 缺少必要的前提条件、注意事项等;

● 描述存在歧义

0/500

您对文档的总体满意度

非常不满意
非常满意

请问是什么原因让您参与到这个问题中

您的邮箱

创Issue赢奖品
根据您的反馈,会自动生成issue模板。您只需点击按钮,创建issue即可。
有奖捉虫