https://ad-bean.github.io/categories/database/ Adbean's Blog Hugo 0.166.0 & FixIt v0.4.5 zh-CN adbeanx@outlook.com (Adbean) adbeanx@outlook.com (Adbean) Sat, 16 Nov 2024 16:08:33 -0500 https://ad-bean.github.io/posts/paper-apiary/ Sat, 16 Nov 2024 16:08:33 -0500adbeanx@outlook.com (Adbean) https://ad-bean.github.io/posts/paper-apiary/ Database <h2 class="heading-element" id="apiary-a-dbms-backed-transactional-function-as-a-service-framework"><span>Apiary: A DBMS-Backed Transactional Function-as-a-Service Framework</span> <a href="#apiary-a-dbms-backed-transactional-function-as-a-service-framework" class="heading-mark"> <svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path d="m7.775 3.275 1.25-1.25a3.5 3.5 0 1 1 4.95 4.95l-2.5 2.5a3.5 3.5 0 0 1-4.95 0 .751.751 0 0 1 .018-1.042.751.751 0 0 1 1.042-.018 1.998 1.998 0 0 0 2.83 0l2.5-2.5a2.002 2.002 0 0 0-2.83-2.83l-1.25 1.25a.751.751 0 0 1-1.042-.018.751.751 0 0 1-.018-1.042Zm-4.69 9.64a1.998 1.998 0 0 0 2.83 0l1.25-1.25a.751.751 0 0 1 1.042.018.751.751 0 0 1 .018 1.042l-1.25 1.25a3.5 3.5 0 1 1-4.95-4.95l2.5-2.5a3.5 3.5 0 0 1 4.95 0 .751.751 0 0 1-.018 1.042.751.751 0 0 1-1.042.018 1.998 1.998 0 0 0-2.83 0l-2.5 2.5a1.998 1.998 0 0 0 0 2.83Z"></path></svg> </a> </h2><p>来自 DBOS 的一个很有意思的项目，DBOS 这个项目去年关注过，txn + serverless 很有趣的思路</p> https://ad-bean.github.io/posts/paper-delta-lake/ Mon, 11 Nov 2024 11:24:58 -0500adbeanx@outlook.com (Adbean) https://ad-bean.github.io/posts/paper-delta-lake/ Database <h2 class="heading-element" id="delta-lake-high-performance-acid-table-storage-over-cloud-object-stores"><span>Delta Lake: High-Performance ACID Table Storage over Cloud Object Stores</span> <a href="#delta-lake-high-performance-acid-table-storage-over-cloud-object-stores" class="heading-mark"> <svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path d="m7.775 3.275 1.25-1.25a3.5 3.5 0 1 1 4.95 4.95l-2.5 2.5a3.5 3.5 0 0 1-4.95 0 .751.751 0 0 1 .018-1.042.751.751 0 0 1 1.042-.018 1.998 1.998 0 0 0 2.83 0l2.5-2.5a2.002 2.002 0 0 0-2.83-2.83l-1.25 1.25a.751.751 0 0 1-1.042-.018.751.751 0 0 1-.018-1.042Zm-4.69 9.64a1.998 1.998 0 0 0 2.83 0l1.25-1.25a.751.751 0 0 1 1.042.018.751.751 0 0 1 .018 1.042l-1.25 1.25a3.5 3.5 0 1 1-4.95-4.95l2.5-2.5a3.5 3.5 0 0 1 4.95 0 .751.751 0 0 1-.018 1.042.751.751 0 0 1-1.042.018 1.998 1.998 0 0 0-2.83 0l-2.5 2.5a1.998 1.998 0 0 0 0 2.83Z"></path></svg> </a> </h2><p><a href="https://github.com/delta-io/delta" target="_blank" rel="external nofollow noopener noreferrer">https://github.com/delta-io/delta</a></p> https://ad-bean.github.io/posts/minilsm-3/ Sat, 27 Jul 2024 22:10:57 +0800adbeanx@outlook.com (Adbean) https://ad-bean.github.io/posts/minilsm-3/ Database <h2 class="heading-element" id="mini-lsm-week-1-day3"><span>Mini-LSM Week 1 Day3</span> <a href="#mini-lsm-week-1-day3" class="heading-mark"> <svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path d="m7.775 3.275 1.25-1.25a3.5 3.5 0 1 1 4.95 4.95l-2.5 2.5a3.5 3.5 0 0 1-4.95 0 .751.751 0 0 1 .018-1.042.751.751 0 0 1 1.042-.018 1.998 1.998 0 0 0 2.83 0l2.5-2.5a2.002 2.002 0 0 0-2.83-2.83l-1.25 1.25a.751.751 0 0 1-1.042-.018.751.751 0 0 1-.018-1.042Zm-4.69 9.64a1.998 1.998 0 0 0 2.83 0l1.25-1.25a.751.751 0 0 1 1.042.018.751.751 0 0 1 .018 1.042l-1.25 1.25a3.5 3.5 0 1 1-4.95-4.95l2.5-2.5a3.5 3.5 0 0 1 4.95 0 .751.751 0 0 1-.018 1.042.751.751 0 0 1-1.042.018 1.998 1.998 0 0 0-2.83 0l-2.5 2.5a1.998 1.998 0 0 0 0 2.83Z"></path></svg> </a> </h2><p>Week1 Day3 的内容，实现 SST block 编解码和 iterator</p> https://ad-bean.github.io/posts/duckdb-2019/ Fri, 07 Jun 2024 15:37:41 +0800adbeanx@outlook.com (Adbean) https://ad-bean.github.io/posts/duckdb-2019/ Database <h2 class="heading-element" id="duckdb-an-embeddable-analytical-database"><span>DuckDB: an Embeddable Analytical Database</span> <a href="#duckdb-an-embeddable-analytical-database" class="heading-mark"> <svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path d="m7.775 3.275 1.25-1.25a3.5 3.5 0 1 1 4.95 4.95l-2.5 2.5a3.5 3.5 0 0 1-4.95 0 .751.751 0 0 1 .018-1.042.751.751 0 0 1 1.042-.018 1.998 1.998 0 0 0 2.83 0l2.5-2.5a2.002 2.002 0 0 0-2.83-2.83l-1.25 1.25a.751.751 0 0 1-1.042-.018.751.751 0 0 1-.018-1.042Zm-4.69 9.64a1.998 1.998 0 0 0 2.83 0l1.25-1.25a.751.751 0 0 1 1.042.018.751.751 0 0 1 .018 1.042l-1.25 1.25a3.5 3.5 0 1 1-4.95-4.95l2.5-2.5a3.5 3.5 0 0 1 4.95 0 .751.751 0 0 1-.018 1.042.751.751 0 0 1-1.042.018 1.998 1.998 0 0 0-2.83 0l-2.5 2.5a1.998 1.998 0 0 0 0 2.83Z"></path></svg> </a> </h2><p>读一下 DuckDB 的论文，对 OLAP 没什么了解，但论文很短也可以看看。</p> https://ad-bean.github.io/posts/minilsm-2/ Sat, 23 Mar 2024 16:51:28 -0400adbeanx@outlook.com (Adbean) https://ad-bean.github.io/posts/minilsm-2/ Database <h2 class="heading-element" id="mini-lsm-week-1-day2"><span>Mini-LSM Week 1 Day2</span> <a href="#mini-lsm-week-1-day2" class="heading-mark"> <svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path d="m7.775 3.275 1.25-1.25a3.5 3.5 0 1 1 4.95 4.95l-2.5 2.5a3.5 3.5 0 0 1-4.95 0 .751.751 0 0 1 .018-1.042.751.751 0 0 1 1.042-.018 1.998 1.998 0 0 0 2.83 0l2.5-2.5a2.002 2.002 0 0 0-2.83-2.83l-1.25 1.25a.751.751 0 0 1-1.042-.018.751.751 0 0 1-.018-1.042Zm-4.69 9.64a1.998 1.998 0 0 0 2.83 0l1.25-1.25a.751.751 0 0 1 1.042.018.751.751 0 0 1 .018 1.042l-1.25 1.25a3.5 3.5 0 1 1-4.95-4.95l2.5-2.5a3.5 3.5 0 0 1 4.95 0 .751.751 0 0 1-.018 1.042.751.751 0 0 1-1.042.018 1.998 1.998 0 0 0-2.83 0l-2.5 2.5a1.998 1.998 0 0 0 0 2.83Z"></path></svg> </a> </h2><p>Week1 Day2 的内容，实现 Merge Iterator</p> https://ad-bean.github.io/posts/minilsm-1/ Fri, 22 Mar 2024 14:04:33 -0400adbeanx@outlook.com (Adbean) https://ad-bean.github.io/posts/minilsm-1/ Database <h2 class="heading-element" id="mini-lsm-week-1-day1"><span>Mini-LSM Week 1 Day1</span> <a href="#mini-lsm-week-1-day1" class="heading-mark"> <svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path d="m7.775 3.275 1.25-1.25a3.5 3.5 0 1 1 4.95 4.95l-2.5 2.5a3.5 3.5 0 0 1-4.95 0 .751.751 0 0 1 .018-1.042.751.751 0 0 1 1.042-.018 1.998 1.998 0 0 0 2.83 0l2.5-2.5a2.002 2.002 0 0 0-2.83-2.83l-1.25 1.25a.751.751 0 0 1-1.042-.018.751.751 0 0 1-.018-1.042Zm-4.69 9.64a1.998 1.998 0 0 0 2.83 0l1.25-1.25a.751.751 0 0 1 1.042.018.751.751 0 0 1 .018 1.042l-1.25 1.25a3.5 3.5 0 1 1-4.95-4.95l2.5-2.5a3.5 3.5 0 0 1 4.95 0 .751.751 0 0 1-.018 1.042.751.751 0 0 1-1.042.018 1.998 1.998 0 0 0-2.83 0l-2.5 2.5a1.998 1.998 0 0 0 0 2.83Z"></path></svg> </a> </h2><p>记录下 LSM 的学习过程，感谢迟先生的教程 <a href="https://skyzh.github.io/mini-lsm/" target="_blank" rel="external nofollow noopener noreferrer">https://skyzh.github.io/mini-lsm/</a></p> https://ad-bean.github.io/posts/analyticdb-paper/ Sun, 25 Feb 2024 13:00:46 -0500adbeanx@outlook.com (Adbean) https://ad-bean.github.io/posts/analyticdb-paper/ Database <h2 class="heading-element" id="analyticdb-v-a-hybrid-analytical-engine-towards-query-fusion-for-structured-and-unstructured-data"><span>AnalyticDB-V: A Hybrid Analytical Engine Towards Query Fusion for Structured and Unstructured Data</span> <a href="#analyticdb-v-a-hybrid-analytical-engine-towards-query-fusion-for-structured-and-unstructured-data" class="heading-mark"> <svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path d="m7.775 3.275 1.25-1.25a3.5 3.5 0 1 1 4.95 4.95l-2.5 2.5a3.5 3.5 0 0 1-4.95 0 .751.751 0 0 1 .018-1.042.751.751 0 0 1 1.042-.018 1.998 1.998 0 0 0 2.83 0l2.5-2.5a2.002 2.002 0 0 0-2.83-2.83l-1.25 1.25a.751.751 0 0 1-1.042-.018.751.751 0 0 1-.018-1.042Zm-4.69 9.64a1.998 1.998 0 0 0 2.83 0l1.25-1.25a.751.751 0 0 1 1.042.018.751.751 0 0 1 .018 1.042l-1.25 1.25a3.5 3.5 0 1 1-4.95-4.95l2.5-2.5a3.5 3.5 0 0 1 4.95 0 .751.751 0 0 1-.018 1.042.751.751 0 0 1-1.042.018 1.998 1.998 0 0 0-2.83 0l-2.5 2.5a1.998 1.998 0 0 0 0 2.83Z"></path></svg> </a> </h2><h2 class="heading-element" id="abstract"><span>ABSTRACT</span> <a href="#abstract" class="heading-mark"> <svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path d="m7.775 3.275 1.25-1.25a3.5 3.5 0 1 1 4.95 4.95l-2.5 2.5a3.5 3.5 0 0 1-4.95 0 .751.751 0 0 1 .018-1.042.751.751 0 0 1 1.042-.018 1.998 1.998 0 0 0 2.83 0l2.5-2.5a2.002 2.002 0 0 0-2.83-2.83l-1.25 1.25a.751.751 0 0 1-1.042-.018.751.751 0 0 1-.018-1.042Zm-4.69 9.64a1.998 1.998 0 0 0 2.83 0l1.25-1.25a.751.751 0 0 1 1.042.018.751.751 0 0 1 .018 1.042l-1.25 1.25a3.5 3.5 0 1 1-4.95-4.95l2.5-2.5a3.5 3.5 0 0 1 4.95 0 .751.751 0 0 1-.018 1.042.751.751 0 0 1-1.042.018 1.998 1.998 0 0 0-2.83 0l-2.5 2.5a1.998 1.998 0 0 0 0 2.83Z"></path></svg> </a> </h2><p>随着<strong>非结构化数据</strong>的爆炸性增长（例如图像，视频和音频），非结构化数据分析在真实世界应用的丰富脉络中广泛存在。许多数据库系统开始合并非结构化数据分析以满足此类需求。但是，在大多数系统中，对<strong>非结构化和结构化数据</strong>的查询通常被视为 disjoint tasks，在大多数系统中，<strong>混合查询</strong>（即涉及两种数据类型）尚未得到充分支持。</p> https://ad-bean.github.io/posts/milvus-paper/ Sun, 25 Feb 2024 10:08:12 -0500adbeanx@outlook.com (Adbean) https://ad-bean.github.io/posts/milvus-paper/ Database <h2 class="heading-element" id="milvusa-purpose-built-vector-data-management-system"><span>Milvus：A Purpose-Built Vector Data Management System</span> <a href="#milvusa-purpose-built-vector-data-management-system" class="heading-mark"> <svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path d="m7.775 3.275 1.25-1.25a3.5 3.5 0 1 1 4.95 4.95l-2.5 2.5a3.5 3.5 0 0 1-4.95 0 .751.751 0 0 1 .018-1.042.751.751 0 0 1 1.042-.018 1.998 1.998 0 0 0 2.83 0l2.5-2.5a2.002 2.002 0 0 0-2.83-2.83l-1.25 1.25a.751.751 0 0 1-1.042-.018.751.751 0 0 1-.018-1.042Zm-4.69 9.64a1.998 1.998 0 0 0 2.83 0l1.25-1.25a.751.751 0 0 1 1.042.018.751.751 0 0 1 .018 1.042l-1.25 1.25a3.5 3.5 0 1 1-4.95-4.95l2.5-2.5a3.5 3.5 0 0 1 4.95 0 .751.751 0 0 1-.018 1.042.751.751 0 0 1-1.042.018 1.998 1.998 0 0 0-2.83 0l-2.5 2.5a1.998 1.998 0 0 0 0 2.83Z"></path></svg> </a> </h2><h2 class="heading-element" id="abstract"><span>ABSTRACT</span> <a href="#abstract" class="heading-mark"> <svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path d="m7.775 3.275 1.25-1.25a3.5 3.5 0 1 1 4.95 4.95l-2.5 2.5a3.5 3.5 0 0 1-4.95 0 .751.751 0 0 1 .018-1.042.751.751 0 0 1 1.042-.018 1.998 1.998 0 0 0 2.83 0l2.5-2.5a2.002 2.002 0 0 0-2.83-2.83l-1.25 1.25a.751.751 0 0 1-1.042-.018.751.751 0 0 1-.018-1.042Zm-4.69 9.64a1.998 1.998 0 0 0 2.83 0l1.25-1.25a.751.751 0 0 1 1.042.018.751.751 0 0 1 .018 1.042l-1.25 1.25a3.5 3.5 0 1 1-4.95-4.95l2.5-2.5a3.5 3.5 0 0 1 4.95 0 .751.751 0 0 1-.018 1.042.751.751 0 0 1-1.042.018 1.998 1.998 0 0 0-2.83 0l-2.5 2.5a1.998 1.998 0 0 0 0 2.83Z"></path></svg> </a> </h2><p>最近，数据科学和人工智能应用中迫切需要管理<strong>高维向量数据</strong>。 非结构化数据和机器学习 (ML) 的激增推动了这一趋势，其中 ML 模型通常将非结构化数据转换为特征向量以进行数据分析，例如产品推荐。 现有的向量数据管理系统和算法有两个局限性：（1）在处理<strong>大规模动态向量</strong>数据时会出现严重的<strong>性能问题</strong>； (2)它们提供的<strong>功能有限</strong>，不能满足多功能应用的要求。 本文介绍了 Milvus，这是一个专门构建的数据管理系统，用于有效管理大规模向量数据。 Milvus 支持易于使用的应用程序接口（包括 SDK 和 RESTful API）； <strong>针对具有现代 CPU 和 GPU 的异构计算平台进行优化</strong>； 实现超越简单向量<strong>相似性搜索</strong>的高级查询处理； 处理动态数据以实现快速更新，同时确保高效的查询处理； 并将数据分布到多个节点以实现可扩展性和可用性。 我们首先描述 Milvus 的设计和实现。 然后我们演示 Milvus 支持的实际用例。 特别是，我们在 Milvus 之上构建了一系列 10 个应用程序（例如图像/视频搜索、化学结构分析、COVID-19 数据集搜索、个性化推荐、生物多因素认证、智能问答）。 最后，我们使用广泛的系统对 Milvus 进行实验评估，包括两个开源系统（Veach 和 Microsoft SPTAG）和三个商业系统。 实验表明，Milvus 比竞争对手快两个数量级，同时提供更多功能。 现在 Milvus 已被全球数百家组织部署，也被认定为 LF AI &amp; Data Foundation 的孵化阶段项目。 Milvus 开源于 <a href="https://github.com/milvus-io/milvus" target="_blank" rel="external nofollow noopener noreferrer">https://github.com/milvus-io/milvus</a>。</p> https://ad-bean.github.io/posts/vbase-paper/ Sat, 17 Feb 2024 11:18:45 -0500adbeanx@outlook.com (Adbean) https://ad-bean.github.io/posts/vbase-paper/ Database <h2 class="heading-element" id="vbase-unifying-online-vector-similarity-search-and-relational-queries-via-relaxed-monotonicity"><span>VBase: Unifying Online Vector Similarity Search and Relational Queries via Relaxed Monotonicity</span> <a href="#vbase-unifying-online-vector-similarity-search-and-relational-queries-via-relaxed-monotonicity" class="heading-mark"> <svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path d="m7.775 3.275 1.25-1.25a3.5 3.5 0 1 1 4.95 4.95l-2.5 2.5a3.5 3.5 0 0 1-4.95 0 .751.751 0 0 1 .018-1.042.751.751 0 0 1 1.042-.018 1.998 1.998 0 0 0 2.83 0l2.5-2.5a2.002 2.002 0 0 0-2.83-2.83l-1.25 1.25a.751.751 0 0 1-1.042-.018.751.751 0 0 1-.018-1.042Zm-4.69 9.64a1.998 1.998 0 0 0 2.83 0l1.25-1.25a.751.751 0 0 1 1.042.018.751.751 0 0 1 .018 1.042l-1.25 1.25a3.5 3.5 0 1 1-4.95-4.95l2.5-2.5a3.5 3.5 0 0 1 4.95 0 .751.751 0 0 1-.018 1.042.751.751 0 0 1-1.042.018 1.998 1.998 0 0 0-2.83 0l-2.5 2.5a1.998 1.998 0 0 0 0 2.83Z"></path></svg> </a> </h2><p>VBase: 通过 Relaxed Monotonicity (松弛单调性) 统一在线矢量相似性搜索和关系查询</p> https://ad-bean.github.io/posts/ddia-2/ Sat, 10 Feb 2024 18:09:40 -0500adbeanx@outlook.com (Adbean) https://ad-bean.github.io/posts/ddia-2/ Database <h2 class="heading-element" id="第二章数据模型与查询语言"><span>第二章：数据模型与查询语言</span> <a href="#%e7%ac%ac%e4%ba%8c%e7%ab%a0%e6%95%b0%e6%8d%ae%e6%a8%a1%e5%9e%8b%e4%b8%8e%e6%9f%a5%e8%af%a2%e8%af%ad%e8%a8%80" class="heading-mark"> <svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path d="m7.775 3.275 1.25-1.25a3.5 3.5 0 1 1 4.95 4.95l-2.5 2.5a3.5 3.5 0 0 1-4.95 0 .751.751 0 0 1 .018-1.042.751.751 0 0 1 1.042-.018 1.998 1.998 0 0 0 2.83 0l2.5-2.5a2.002 2.002 0 0 0-2.83-2.83l-1.25 1.25a.751.751 0 0 1-1.042-.018.751.751 0 0 1-.018-1.042Zm-4.69 9.64a1.998 1.998 0 0 0 2.83 0l1.25-1.25a.751.751 0 0 1 1.042.018.751.751 0 0 1 .018 1.042l-1.25 1.25a3.5 3.5 0 1 1-4.95-4.95l2.5-2.5a3.5 3.5 0 0 1 4.95 0 .751.751 0 0 1-.018 1.042.751.751 0 0 1-1.042.018 1.998 1.998 0 0 0-2.83 0l-2.5 2.5a1.998 1.998 0 0 0 0 2.83Z"></path></svg> </a> </h2><p><strong>数据模型</strong>可能是软件开发中最重要的部分了，因为它们的影响如此深远：不仅仅影响着<strong>软件的编写方式</strong>，而且影响着我们的 <strong>解题思路</strong>。</p> https://ad-bean.github.io/posts/ddia-1/ Fri, 05 Jan 2024 14:16:41 -0500adbeanx@outlook.com (Adbean) https://ad-bean.github.io/posts/ddia-1/ Database <h2 class="heading-element" id="ddia-序章"><span>DDIA 序章</span> <a href="#ddia-%e5%ba%8f%e7%ab%a0" class="heading-mark"> <svg class="octicon octicon-link" viewBox="0 0 16 16" version="1.1" width="16" height="16" aria-hidden="true"><path d="m7.775 3.275 1.25-1.25a3.5 3.5 0 1 1 4.95 4.95l-2.5 2.5a3.5 3.5 0 0 1-4.95 0 .751.751 0 0 1 .018-1.042.751.751 0 0 1 1.042-.018 1.998 1.998 0 0 0 2.83 0l2.5-2.5a2.002 2.002 0 0 0-2.83-2.83l-1.25 1.25a.751.751 0 0 1-1.042-.018.751.751 0 0 1-.018-1.042Zm-4.69 9.64a1.998 1.998 0 0 0 2.83 0l1.25-1.25a.751.751 0 0 1 1.042.018.751.751 0 0 1 .018 1.042l-1.25 1.25a3.5 3.5 0 1 1-4.95-4.95l2.5-2.5a3.5 3.5 0 0 1 4.95 0 .751.751 0 0 1-.018 1.042.751.751 0 0 1-1.042.018 1.998 1.998 0 0 0-2.83 0l-2.5 2.5a1.998 1.998 0 0 0 0 2.83Z"></path></svg> </a> </h2><blockquote> <p><a href="http://ddia.vonng.com/#/ch1" target="_blank" rel="external nofollow noopener noreferrer">http://ddia.vonng.com/#/ch1</a></p>