Java学习者

作者：微信小助手

<section style="margin-right: 8px;margin-bottom: 15px;margin-left: 8px;max-width: 100%;font-family: -apple-system-font, BlinkMacSystemFont, "Helvetica Neue", "PingFang SC", "Hiragino Sans GB", "Microsoft YaHei UI", "Microsoft YaHei", Arial, sans-serif;letter-spacing: 0.544px;white-space: normal;background-color: rgb(255, 255, 255);text-align: center;line-height: 1.75em;box-sizing: border-box !important;overflow-wrap: break-word !important;"> 线程 </section> <section style="margin-right: 8px;margin-bottom: 15px;margin-left: 8px;max-width: 100%;font-family: -apple-system-font, BlinkMacSystemFont, "Helvetica Neue", "PingFang SC", "Hiragino Sans GB", "Microsoft YaHei UI", "Microsoft YaHei", Arial, sans-serif;white-space: normal;background-color: rgb(255, 255, 255);letter-spacing: 1px;line-height: 1.75em;box-sizing: border-box !important;overflow-wrap: break-word !important;"> 线程的概念，百度是这样解释的： </section> <section style="margin-right: 8px;margin-bottom: 15px;margin-left: 8px;max-width: 100%;font-family: -apple-system-font, BlinkMacSystemFont, "Helvetica Neue", "PingFang SC", "Hiragino Sans GB", "Microsoft YaHei UI", "Microsoft YaHei", Arial, sans-serif;white-space: normal;background-color: rgb(255, 255, 255);letter-spacing: 1px;line-height: 1.75em;box-sizing: border-box !important;overflow-wrap: break-word !important;"> 线程（英语：Thread）是操作系统能够进行运算调度的最小单位。它被包含在进程之中，是进程中的实际运作单位。一条线程指的是进程中一个单一顺序的控制流，一个进程中可以并发多个线程，每条线程并行执行不同的任务。在Unix System V及SunOS中也被称为轻量进程（Lightweight Processes），但轻量进程更多指内核线程（Kernel Thread），而把用户线程（User Thread）称为线程。 </section> <section style="margin-right: 8px;margin-bottom: 15px;margin-left: 8px;max-width: 100%;font-family: -apple-system-font, BlinkMacSystemFont, "Helvetica Neue", "PingFang SC", "Hiragino Sans GB", "Microsoft YaHei UI", "Microsoft YaHei", Arial, sans-serif;white-space: normal;background-color: rgb(255, 255, 255);letter-spacing: 1px;line-height: 1.75em;box-sizing: border-box !important;overflow-wrap: break-word !important;"> 1.1 线程与进程的区别 </section> <section style="margin-right: 8px;margin-bottom: 15px;margin-left: 8px;max-width: 100%;font-family: -apple-system-font, BlinkMacSystemFont, "Helvetica Neue", "PingFang SC", "Hiragino Sans GB", "Microsoft YaHei UI", "Microsoft YaHei", Arial, sans-serif;letter-spacing: 0.544px;white-space: normal;background-color: rgb(255, 255, 255);text-align: center;line-height: 1.75em;box-sizing: border-box !important;overflow-wrap: break-word !important;"> <img class="rich_pages" data-ratio="0.5880952380952381" data-s="300,640" data-type="png" data-w="840" src="/upload/4a9d58f051ead1c52d18b62b6b1e93f2.png" style="box-sizing: border-box !important;overflow-wrap: break-word !important;width: 677px !important;visibility: visible !important;"> </section> <section style="margin-right: 8px;margin-bottom: 15px;margin-left: 8px;max-width: 100%;font-family: -apple-system-font, BlinkMacSystemFont, "Helvetica Neue", "PingFang SC", "Hiragino Sans GB", "Microsoft YaHei UI", "Microsoft YaHei", Arial, sans-serif;white-space: normal;background-color: rgb(255, 255, 255);letter-spacing: 1px;line-height: 1.75em;box-sizing: border-box !important;overflow-wrap: break-word !important;"> 进程： 指在系统中正在运行 的一个应 用程序；程 序一旦运行就是进程；进程——资源分配的最小单位。 </section> <section style="margin-right: 8px;margin-bottom: 15px;margin-left: 8px;max-width: 100%;font-family: -apple-system-font, BlinkMacSystemFont, "Helvetica Neue", "PingFang SC", "Hiragino Sans GB", "Microsoft YaHei UI", "Microsoft YaHei", Arial, sans-serif;white-space: normal;background-color: rgb(255, 255, 255);letter-spacing: 1px;line-height: 1.75em;box-sizing: border-box !important;overflow-wrap: break-word !important;"> 线程：系统分配处理器时间资源的基本单元，或者说进程之内独立执行的一个单元执行流。线程——程序执行的最小单位。 </section> <section style="margin-right: 8px;margin-bottom: 15px;margin-left: 8px;max-width: 100%;font-family: -apple-system-font, BlinkMacSystemFont, "Helvetica Neue", "PingFang SC", "Hiragino Sans GB", "Microsoft YaHei UI", "Microsoft YaHei", Arial, sans-serif;white-space: normal;background-color: rgb(255, 255, 255);letter-spacing: 1px;line-height: 1.75em;box-sizing: border-box !important;overflow-wrap: break-word !important;"> 也就是，进程可以包含多个线程，而线程是程序执行的最小单位。 </section> <section style="margin-right: 8px;margin-bottom: 15px;margin-left: 8px;max-width: 100%;font-family: -apple-system-font, BlinkMacSystemFont, "Helvetica Neue", "PingFang SC", "Hiragino Sans GB", "Microsoft YaHei UI", "Microsoft YaHei", Arial, sans-serif;white-space: normal;background-color: rgb(255, 255, 255);letter-spacing: 1px;line-height: 1.75em;box-sizing: border-box !important;overflow-wrap: break-word !important;"> 1.2 线程的状态 </section> <section style="margin-right: 8px;margin-bottom: 15px;margin-left: 8px;max-width: 100%;font-family: -apple-system-font, BlinkMacSystemFont, "Helvetica Neue", "PingFang SC", "Hiragino Sans GB", "Microsoft YaHei UI", "Microsoft YaHei", Arial, sans-serif;letter-spacing: 0.544px;white-space: normal;background-color: rgb(255, 255, 255);text-align: center;line-height: 1.75em;box-sizing: border-box !important;overflow-wrap: break-word !important;"> <img class="rich_pages" data-ratio="0.6671122994652406" data-s="300,640" data-type="jpeg" data-w="748" src="/upload/6d6efe112ce19043d1cbb47a34a8d549.jpg" style="box-sizing: border-box !important;overflow-wrap: break-word !important;width: 677px !important;visibility: visible !important;"> </section> <ul class="list-paddingleft-2" style="margin-right: 8px;margin-left: 8px;max-width: 100%;font-family: -apple-system-font, BlinkMacSystemFont, "Helvetica Neue", "PingFang SC", "Hiragino Sans GB", "Microsoft YaHei UI", "Microsoft YaHei", Arial, sans-serif;letter-spacing: 0.544px;white-space: normal;background-color: rgb(255, 255, 255);overflow-wrap: break-word !important;"> <li style="max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;"> <section style="margin-bottom: 15px;max-width: 100%;line-height: 1.75em;box-sizing: border-box !important;overflow-wrap: break-word !important;"> NEW： 线程刚创建 </section></li> <li style="max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;"> <section style="margin-bottom: 15px;max-width: 100%;line-height: 1.75em;box-sizing: border-box !important;overflow-wrap: break-word !important;"> RUNNABLE: 在JVM中正在运行的线程，其中运行状态可以有运行中RUNNING和READY两种状态，由系统调度进行状态改变。 </section></li> <li style="max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;"> <section style="margin-bottom: 15px;max-width: 100%;line-height: 1.75em;box-sizing: border-box !important;overflow-wrap: break-word !important;"> BLOCKED：线程处于阻塞状态，等待监视锁，可以重新进行同步代码块中执行 </section></li> <li style="max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;"> <section style="margin-bottom: 15px;max-width: 100%;line-height: 1.75em;box-sizing: border-box !important;overflow-wrap: break-word !important;"> WAITING : 等待状态 </section></li> <li style="max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;"> <section style="margin-bottom: 15px;max-width: 100%;line-height: 1.75em;box-sizing: border-box !important;overflow-wrap: break-word !important;"> TIMED_WAITING: 调用sleep() join() wait()方法可能导致线程处于等待状态 </section></li> <li style="max-width: 100%;box-sizing: border-box !important;overflow-wrap: break-word !important;"> <section style="margin-bottom: 15px;max-width: 100%;line-height: 1.75em;box-sizing: border-box !important;overflow-wrap: break-word !important;"> TERMINATED: 线程执行完毕，已经退出 </section></li> </ul> <section style="margin-right: 8px;margin-bottom: 15px;margin-left: 8px;max-width: 100%;font-family: -apple-system-font, BlinkMacSystemFont, "Helvetica Neue", "PingFang SC", "Hiragino Sans GB", "Microsoft YaHei UI", "Microsoft YaHei", Arial, sans-serif;white-space: normal;background-color: rgb(255, 255, 255);letter-spacing: 1px;line-height: 1.75em;box-sizing: border-box !important;overflow-wrap: break-word !important;"> 1.3 Notify和Wait ： </section> <section style="margin-right: 8px;margin-bottom: 15px;margin-left: 8px;max-width: 100%;font-family: -apple-system-font, BlinkMacSystemFont, "Helvetica Neue", "PingFang SC", "Hiragino Sans GB", "Microsoft YaHei UI", "Microsoft YaHei", Arial, sans-serif;white-space: normal;background-color: rgb(255, 255, 255);letter-spacing: 1px;line-height: 1.75em;box-sizing: border-box !important;overflow-wrap: break-word !important;"> Notify和Wait 的作用 </section> <section style="margin-right: 8px;margin-bottom: 15px;margin-left: 8px;max-width: 100%;font-family: -apple-system-font, BlinkMacSystemFont, "Helvetica Neue", "PingFang SC", "Hiragino Sans GB", "Microsoft YaHei UI", "Microsoft YaHei", Arial, sans-serif;white-space: normal;background-color: rgb(255, 255, 255);letter-spacing: 1px;line-height: 1.75em;box-sizing: border-box !important;overflow-wrap: break-word !important;"> 首先看源码给出的解释，这里翻译了一下： </section> <section style="margin-right: 8px;margin-bottom: 15px;margin-left: 8px;max-width: 100%;font-family: -apple-system-font, BlinkMacSystemFont, "Helvetica Neue", "PingFang SC", "Hiragino Sans GB", "Microsoft YaHei UI", "Microsoft YaHei", Arial, sans-serif;white-space: normal;background-color: rgb(255, 255, 255);letter-spacing: 1px;line-height: 1.75em;box-sizing: border-box !important;overflow-wrap: break-word !important;"> Notify： 唤醒一个正在等待这个对象的线程监控。如果有任何线程正在等待这个对象，那么它们中的一个被选择被唤醒。选择是任意的，发生在执行的酌情权。一个线程等待一个对象通过调用一个{@code wait}方法进行监视。 </section> <section style="margin-right: 8px;margin-bottom: 15px;margin-left: 8px;max-width: 100%;font-family: -apple-system-font, BlinkMacSystemFont, "Helvetica Neue", "PingFang SC", "Hiragino Sans GB", "Microsoft YaHei UI", "Microsoft YaHei", Arial, sans-serif;white-space: normal;background-color: rgb(255, 255, 255);letter-spacing: 1px;line-height: 1.75em;box-sizing: border-box !important;overflow-wrap: break-word !important;"> <span style=

为什么我们要从MySQL迁移到TiDB？

作者：微信小助手

<section style="box-sizing: border-box;font-size: 16px;"> <section style="margin-top: 10px;margin-bottom: 10px;box-sizing: border-box;" powered-by="xiumi.us"> <section style="padding-top: 10px;padding-right: 10px;padding-left: 10px;background-color: rgb(239, 239, 239);box-sizing: border-box;"> <section style="box-sizing: border-box;"> “ </section> <section style="display: inline-block;vertical-align: top;float: right;width: 90%;line-height: 1.5;font-size: 15px;color: rgb(89, 89, 89);box-sizing: border-box;"> 当一张百亿数据量的表放在你面前，你将面临着什么？加列？哭吧，怎么也得等个几天甚至几周。加索引？哭吧，不论你用 pt-online-schema，还是 gh-ost，你都面临着拷贝一张临时表用以存储临时数据，磁盘已经 80% 了，这一张表就占几百 G，你咋弄？ </section> <section style="clear: both;box-sizing: border-box;line-height: 0;"> <section style="line-height: 0;width: 0px;"> <svg viewbox="0 0 1 1" style="vertical-align:top;"></svg> </section> </section> </section> </section> </section> <section style="line-height: 1.75em;"> </section> <section style="text-align: center;margin-left: 8px;margin-right: 8px;"> <img class="rich_pages" data-ratio="0.66640625" data-s="300,640" src="/upload/c84691e34f4c4bc260d9d99ecb17de8b.jpg" data-type="jpeg" data-w="1280" style=""> </section> 图片来自 Pexels <section style="line-height: normal;"> </section> 我先说几个最让你兴奋和开心的点吧： <ul class="list-paddingleft-2" style="list-style-type: disc;"> <li>在 TiDB 里，你完全不用担心磁盘容量的问题。</li> <li>在 TiDB 里，原生支持 Online DDL，你完全不用担心第三方改表工具改表出现各种 Bug 的问题，相信用开源工具改过上 T 级别表的同学都遇到过或多或少的各类 error。</li> <li>在 TiDB 里，加列，主键扩容字段都是秒级的，比如我刚刚就刚对一张 19 亿的表加完了字段，1 秒完事，这在 MySQL 里要 8.0 才可以，而且还要求列在最后才行。</li> <li>在 TiDB 里，你会发现 count(*) 惊人的快，一张近 20 亿的表 coun(*) 大概在 1 分钟完事儿，当然，这取决于你的 KV 数量和磁盘性能。</li> <li>在 TiDB 里，从 MySQL 迁移将变得简单，图形化一键迁移，爽不爽？</li> <li>在 TiDB 里，绝大多数情况你会发现比单机 MySQL 有更好的性能，当然也不排除一些例外，例如 enum 这样奇葩的字段类型。 </li> <li>在 TiDB 里，......，您且往下看，我慢慢和您说。</li> </ul> <section style="box-sizing: border-box;font-size: 16px;"> <section style="border-bottom: 1px solid black;margin-top: 0.5em;margin-bottom: 0.5em;line-height: 1.2;box-sizing: border-box;" powered-by="xiumi.us"> <section style="display: inline-block;border-bottom: 6px solid rgb(89, 89, 89);margin-bottom: -1px;border-top-color: rgb(89, 89, 89);border-right-color: rgb(89, 89, 89);border-left-color: rgb(89, 89, 89);font-size: 20px;color: rgb(89, 89, 89);box-sizing: border-box;"> 使用背景 </section> </section> </section> <section style="text-align: justify;margin-left: 8px;margin-right: 8px;line-height: 1.75em;"> 360 云平台对 360 集团各大业务线均有提供服务支持，涉及的数据库支持方案有：MySQL、Redis、MongoDB、ES、GP、PiKA。 </section> <section style="text-align: justify;margin-left: 8px;margin-right: 8px;line-height: 1.75em;"> 其中 MySQL 至今已有过万的实例，目前，对于一些写入量大的业务，已经出现瓶颈。 </section> <section style="text-align: justify;margin-left: 8px;margin-right: 8px;line-height: 1.75em;"> 例如磁盘空间，虽然我们可以通过分库分表的方式，拆分出来，但这对业务和 DBA 来说都是不小的工作量，最痛苦的无异于这些大表的改表。 </section> <section style="text-align: justify;margin-left: 8px;margin-right: 8px;line-height: 1.75em;"> 无论是单表上 T，还是分库分表，对一张大表执行 DDL 的代价是非常大的。 </section> <section style="text-align: justify;margin-left: 8px;margin-right: 8px;line-height: 1.75em;"> 针对业务爆发式增长的数据量，我们开始调研选型是否有其他数据库可以代替传统的 MySQL。 </section> <section style="text-align: justify;margin-left: 8px;margin-right: 8px;line-height: 1.75em;"> 通过调研我们了解到 TiDB，迁移平滑，基本上无需业务变动代码，完全的事务 ACID 支持，分布式的架构，自带高可用、Online DDL。 </section> <section style="text-align: justify;margin-left: 8px;margin-right: 8px;line-height: 1.75em;"> 截止目前，360 云平台这边有三套 TiDB 集群，总节点数在 50+。有 9 个业务线接入使用，有过百亿级表 Online 加索引的案例，总数据量目前在 80T。 </section> <section style="text-align: justify;margin-left: 8px;margin-right: 8px;line-height: 1.75em;"> 版本上，我们从 3.0.1 一路跟随到 3.0.5，DM 版本从内测到 1.0.2 GA 版本，共计提出 9 个 Bug 或改进项反馈。 </section> <section style="text-align: justify;margin-left: 8px;margin-right: 8px;line-height: 1.75em;"> 后续我们计划将 TiDB 上到 360 HULK 云平台上，并定制化开发一些功能为业务提供可视化的服务，以便让 360 集团更多的业务线接触到 TiDB、使用 TiDB。 </section> <section style="text-align: justify;margin-left: 8px;margin-right: 8px;line-height: 1.75em;"> 版本的选择我们之所以从大版本 3 开始，也是看到了一些 2.X 版本的社区反馈，尤其是索引执行计划这块，3.X 版本较之前的版本会好很多。DM 版本我们是直接选取的最新版，后一路跟随新版本升级。 </section> <section style="box-sizing: border-box;font-size: 16px;"> <section style="border-bottom: 1px solid black;margin-top: 0.5em;margin-bottom: 0.5em;line-height: 1.2;box-sizing: border-box;" powered-by="xiumi.us"> <section style="display: inline-block;border-bottom: 6px solid rgb(89, 89, 89);margin-bottom: -1px;border-top-color: rgb(89, 89, 89);border-right-color: rgb(89, 89, 89);border-left-color: rgb(89, 89, 89);font-size: 20px;color: rgb(89, 89, 89);box-sizing: border-box;"> 集群架构 </section> </section> </section> <img data-ratio="0.6405797101449275" src="/upload/29f50a6d55a3d60893e959bad1987b43.jpg" data-type="jpeg" data-w="690" style="-webkit-font-smoothing: antialiased;margin-bottom: 15px;border-width: initial;border-style: none;border-color: initial;word-break: break-all;cursor: pointer;" title="1583751206916700.jpg"> <section style="text-align: justify;margin-left: 8px;margin-right: 8px;line-height: 1.75em;"> 整体架构上，我们除了 TiDB 集群外，还用到了 DM 和 Pump、Drainer 套件。 </section> <section style="text-align: justify;margin-left: 8px;margin-right: 8px;line-height: 1.75em;"> 这块主要是由于我们使用 TiDB 的业务有两种： </section> <section style="text-align: justify;margin-left: 8px;margin-right: 8px;line-height: 1.75em;"> ①老的 MySQL 业务， 因单机磁盘受限，导致单实例磁盘无法支撑爆炸式增长的数据量，数据比较重要，需要备份和支持 7*24 小时的恢复。 </section> <section style="text-align: justify;margin-left: 8px;margin-right: 8px;line-height: 1.75em;"> 这类业务我们用到 DM 套件来实现无障碍迁移，1TB 的导入时间在 16 小时，这里如果是比较大的数据源，且 TiDB 是全新集群，可以使用 TiDB-Lightning，速度可以更快。 </section> <section style="text-align: justify;margin-left: 8px;margin-right: 8px;line-height: 1.75em;"> Lightning 的实测导入速度，37 分钟，导完 2 张大表共计 54G 的数据，符合 100G/H 预估， 是 loader 的 3 倍速度，loader 用时 2 小时 4 分钟。 </section> 说起 DM 使用这块文章后面会单独分享下这块需要注意的问题，如下图所示： <img data-ratio="0.4876632801161103" src="/upload/4567d274b589665d561cf1bdbc9de622.jpg" data-type="jpeg" data-w="689" style="font-family: -apple-system, "Helvetica Neue", Helvetica, Arial, "PingFang SC", "Hiragino Sans GB", "WenQuanYi Micro Hei", "Microsoft Yahei", sans-serif;-webkit-font-smoothing: antialiased;margin-bottom: 15px;border-width: initial;border-style: none;border-color: initial;font-size: 16px;word-break: break-all;cursor: pointer;" title="1583751471331017.jpg"> <section style="text-align: justify;margin-left: 8px;margin-right: 8px;line-height: 1.75em;"> ②全新的业务， 或由业务自行导入到 TiDB 集群中，这种业务数据量一般都会比较大，也是看中了 TiDB 支持 ACID 和分布式的特点。 </section> <section style="text-align: justify;margin-left: 8px;margin-right: 8px;line-height: 1.75em;"> 目前网盾业务有多张表都过 10 亿级别，其中有张表到达了 100 亿+，建索引花了近 10 天（这块其实我们应当注意，不是分布式就一张表就完事儿了，因为表量级过大，清理老旧数据都是个问题）。 </section> <section style="text-align: justify;margin-left: 8px;margin-right: 8px;line-height: 1.75em;"> TiDB 现在支持分区表，但我们在使用过程中发现性能上和普通表有差距，期待后续版本能够让分区表功能和性能更加的完善。 </section> <section style="line-height: normal;"> </section> <section style="box-sizing: border-box;font-size: 16px;"> <section style="border-bottom: 1px solid black;margin-top: 0.5em;margin-bottom: 0.5em;line-height: 1.2;box-sizing: border-box;" powered-by="xiumi.us"> <section style="display: inline-block;border-bottom: 6px solid rgb(89, 89, 89);margin-bottom: -1px;border-top-color: rgb(89, 89, 89);border-right-color: rgb(89, 89, 89);border-left-color: rgb(89, 89, 89);font-size: 20px;color: rgb(89, 89, 89);box-sizing: border-box;"> TiDB 在 360 云平台的使用情况 </section> </section> </section> <section style="line-height: normal;"> </section> <section style="text-align: justify;margin-left: 8px;margin-right: 8px;line-height: 1.75em;"> 对于这一全新的数据库，我们本着大胆用，不拘泥于传统的态度进行使用。 </section> <section style="line-height: normal;"> </section> <section style="text-align: justify;margin-left: 8px;margin-right: 8px;line-height: 1.75em;"> 我们的 MySQL 现在也正在适配 8.0 版本，MongoDB、ES 也都是时刻关注着新版本情况来评估是否适合云平台。 </section> <section style="text-align: justify;margin-left: 8px;margin-right: 8px;line-height: 1.75em;"> 因此 TiDB 的上线也是从离线业务→边缘业务→核心业务来过渡的。 </section> <section style="text-align: justify;margin-left: 8px;margin-right: 8px;line-height: 1.75em;"> 经过大量的测试、也参考了其他公司的使用情况，我们计划将 TiDB 纳入 360 HULK 云平台，并计划后期对其不断完善在云平台中的功能，对全公司业务线开放使用。 </section> <section style="text-align: justify;margin-left: 8px;margin-right: 8px;line-height: 1.75em;"> 定制化开发一些 MySQL 已经具备的，例如 SQL 审核、慢查统计、冗余索引检测、自增索引阈值等各项基础功能等等。 </section> <section style="text-align: justify;margin-left: 8px;margin-right: 8px;line-height: 1.75em;"> 虽然在使用过程中遇到了一些小问题，例如索引的选取、参数的调优，因为一些配置导致的性能抖动，但都得到了 PingCAP 同学快速的响应和回复，这对我们推进 TiDB 有重大的帮助。 </section> <section style="box-sizing: border-box;font-size: 16px;"> <section style="margin-top: 10px;margin-bottom: 10px;box-sizing: border-box;" powered-by="xiumi.us"> <section style="width: 0.6em;display: inline-block;vertical-align: middle;box-sizing: border-box;"> </section> <section style="display: inline-block;vertical-align: middle;font-size: 18px;padding-left: 5px;color: rgb(89, 89, 89);box-sizing: border-box;"> 一键迁移工具 DM 干货分享 </section> </section> </section> <h3 style="text-align: justify;margin-left: 8px;margin-right: 8px;line-height: 1.75em;">DM 使用经验如下：</h3> <h4 style="text-align: justify;margin-left: 8px;margin-right: 8px;line-height: 1.75em;">①权限</h4> <section style="text-align: justify;margin-left: 8px;margin-right: 8px;line-height: 1.75em;"> 官网手册上只说明需要如下权限： </section> <section style="line-height: normal;"> </section> <section style="text-align: justify;margin-left: 8px;margin-right: 8px;line-height: 1.75em;"> TiDB Lightning 需要以下权限： </section> <ul class="list-paddingleft-2" style="list-style-type: disc;"> <li style="font-weight: bold;">SELECT</li> <li style="font-weight: bold;">UPDATE</li> <li style="font-weight: bold;">ALTER</li> <li style="font-weight: bold;">CREATE</li> <li style="font-weight: bold;">DROP</li> </ul> <section style="text-align: justify;margin-left: 8px;margin-right: 8px;line-height: 1.75em;"> 存储断点的数据库额外需要以下权限： </section> <ul class="list-paddingleft-2" style="list-style-type: disc;"> <li style="font-weight: bold;">INSERT</li> <li style="font-weight: bold;">DELETE</li> </ul> <section style="text-align: justify;margin-left: 8px;margin-right: 8px;line-height: 1.75em;"> 但实测过程中发现还需要如下权限： </section> <ul class="list-paddingleft-2" style="list-style-type: disc;"> <li>上游（REPLICATION SLAVE 权限必须具备，要不增量同步会 access deny）。</li> <li>下游（不加 super 会导致 checksum table 无法执行）。</li> </ul> <h4 style="text-align: justify;margin-left: 8px;margin-right: 8px;line-height: 1.75em;">②TiKV Region Score</h4> <section style="line-height: normal;"> </section> <section style="text-align: justify;margin-left: 8px;margin-right: 8px;line-height: 1.75em;"> PD 监控中 -Statistics-balance 中，有 Store-region-score 监控项，这里记录的是各个节点的 Score 得分，正常情况下，我们期望的结果是得分接近的，这说明无需进行 Region 大规模迁移。 </section> <section style="line-height: normal;"> </section> <section style="text-align: justify;margin-left: 8px;margin-right: 8px;line-height: 1.75em;"> ③PD 调度原理 </section> <section style="text-align: justify;margin-left: 8px;margin-right: 8px;line-height: 1.75em;"> Region 负载均衡调度主要依赖 balance-leader 和 balance-region 两个调度器。 </section> <section style="text-align: justify;margin-left: 8px;margin-right: 8px;line-height: 1.75em;"> 二者的调度目标是将 Region 均匀地分散在集群中的所有 Store 上，但它们各有侧重： </section> <ul class="list-paddingleft-2" style="list-style-type: disc;"> <li>balance-leader 关注 Region 的 Leader，目的是分散处理客户端请求的压力。</li> <li>balance-region 关注 Region 的各个 Peer，目的是分散存储的压力，同时避免出现爆盘等状况。</li> </ul> 我们这里重点关注的是 balance-region，当它出现不均衡的时候，我们可以直接在监控中看到类似下图所示： <img data-ratio="0.6347826086956522" src="/upload/e4a6cc8f40e090ca2a084d115adda7e5.jpg" data-type="jpeg" data-w="690" style="-webkit-font-smoothing: antialiased;margin-bottom: 15px;border-width: initial;border-style: none;border-color: initial;word-break: break-all;cursor: pointer;" title="1583752308283682.jpg"> <section style="text-align: justify;margin-left: 8px;margin-right: 8px;line-height: 1.75em;"> 调度期间，不可避免的会出现 IO 争用、磁盘的 lantency，都会出现不同程度的上涨，从业务上的反馈看，就会出现积压，响应不及时等等。而当 Region Balance 完成后， Duration 等都会恢复正常水平。 </section> <section style="line-height: normal;"> </section> <section style="text-align: justify;margin-left: 8px;margin-right: 8px;line-height: 1.75em;"> 因此，我们要关注的地方有两点： </section> <ul class="list-paddingleft-2" style="list-style-type: disc;"> <li>如何控制或减小 Region Balance 大规模迁移时对业务的影响；</li> <li>如何提前规避因磁盘导致的大规模 Region Balance。</li> </ul> <section style="text-align: justify;margin-left: 8px;margin-right: 8px;line-height: 1.75em;"> 对于第一点，我们迁移的时候是有参数可以控制的。 这里无论是磁盘空间阈值，还是 Region Balance 调度速度，或者 Drop 大量表后调整空 Region Merge 的速度，其实都是可以通过 pd-ctl 的 config set 命令来实时调节。 </section> <section style="line-height: normal;"> </section> <section style="text-align: justify;margin-left: 8px;margin-right: 8px;line-height: 1.75em;"> 例如： </section> <ul class="list-paddingleft-2" style="list-style-type: disc;"> <li>high-space-ratio 0.7 #设置空间充裕阈值为 0.7。当节点的空间占用比例小于指定值时，PD 调度时会忽略剩余空间这个指标，主要针对实际数据量进行均衡。</li> <li>region-schedule-limit 8 #最多同时进行 8 个 Region 调度。这个值主要影响 Region Balance 的速度，值越大调度得越快，设置为 0 则关闭调度。Region 调度的开销较大，所以这个值不宜调得太大。也可以通过减小该值来限制调度region对集群产生的影响。</li> <li>merge-schedule-limit 12 #最多同时进行 12 个 merge 调度。设置为 0 则关闭 Region Merge。Merge 调度的开销较大，所以这个值不宜调得过大。</li> <li>leader-schedule-limit 8 #最多同时进行 8 个 leader 调度。这个值主要影响 Leader Balance 的速度，值越大调度得越快，设置为 0 则关闭调度。Leader 调度的开销较小，需要的时候可以适当调大。</li> <li>max-merge-region-keys 50000 #设置 Region Merge 的 keyCount 上限为 50k。当 Region KeyCount 大于指定值时 PD 不会将其与相邻的 Region 合并。</li> <li>max-merge-region-size 16 #设置 Region Merge 的 size 上限为 16M。当 Region Size 大于指定值时 PD 不会将其与相邻的 Region 合并。设置为 0 表示不开启 Region Merge 功能。</li> </ul> <section style="text-align: justify;margin-left: 8px;margin-right: 8px;line-height: 1.75em;"> TIPS： 理解了作用和原理，上述参数都可以根据需求自行控制。 </section> <section style="line-height: normal;"> </section> 例如当我们在 Drop 大量的表后，会产生很多的空 Region。在 Region 数量较多的情况下，Raftstore 需要花费一些时间去处理大量 Region 的心跳，从而带来一些延迟，导致某些读写请求得不到及时处理。 <section style="line-height: normal;"> </section> 如果读写压力较大，Raftstore 线程的 CPU 使用率容易达到瓶颈，导致延迟进一步增加，进而影响性能表现。 <section style="line-height: normal;"> </section> 因此我们会希望尽快的进行 Region Merge，来避免过多的 Region 对集群造成性能损耗时，我们可以同时调小 max-merge-region-keys、max-merge-region-size，来让其更快的触发 Merge 操作，同时调大 merge-schedule-limit 提高并发度。 <section style="line-height: normal;"> </section> 例如当我们发现某台 KV 磁盘空间剩余 40% 开始大量调度时，我们可以将 high-space-ratio 调整到 0.7，以临时避免调度对业务产生的影响。 <section style="line-height: normal;"> </section> 我们也可以控制调度的并发度，来减少对业务产生的影响，实测这都是立竿见影的参数，大家如果遇到这些问题可供参考。 <section style="line-height: normal;"> </section> <section style="text-align: justify;line-height: 1.75em;margin-left: 8px;margin-right: 8px;margin-bottom: 5px;"> 对于第二点，尤其是使用 DM 期间，将 DM-worker 和 TiKV 混合部署的情况下，要注意清理全量备份留下的文件和 Relaylog。 </section> <img data-ratio="0.3443396226415094" src="/upload/a125517977c863298272eee6ae1cf04d.jpg" data-type="jpeg" data-w="424" title="1583752378344947.jpg" style="text-align: center;-webkit-font-smoothing: antialiased;margin-bottom: 15px;border-width: initial;border-style: none;border-color: initial;word-break: break-all;cursor: pointer;"> <section style="text-align: justify;margin-left: 8px;margin-right: 8px;line-height: 1.75em;"> 默认调度策略是当磁盘剩余的有效空间不足 40%，处于中间态时则同时考虑数据量和剩余空间两个因素做加权和当作得分，当得分出现比较大的差异时，就会开始调度。 </section> <section style="text-align: justify;margin-left: 8px;margin-right: 8px;line-height: 1.75em;"> 所以 DM 导入完成后，要记得删除全量备份。就是 dumped_data.task_xxx 文件夹，这个全量备份一般都会比较大，如果 dm-worker 和 TiKV 混部，就会出现某个 TiKV 节点磁盘已使用率高于其他。 </section> 这时 PD 的 store region score 就会相比其他节点出现异常，引起性能抖动和 Duration 升高。 <img data-ratio="0.9328358208955224" src="/upload/a81320e211c050f3e4bdf87384c645f8.jpg" data-type="jpeg" data-w="536" title="1583752409124728.jpg" style="text-align: center;font-family: -apple-system, "Helvetica Neue", Helvetica, Arial, "PingFang SC", "Hiragino Sans GB", "WenQuanYi Micro Hei", "Microsoft Yahei", sans-serif;-webkit-font-smoothing: antialiased;margin-bottom: 15px;border-width: initial;border-style: none;border-color: initial;font-size: 16px;word-break: break-all;cursor: pointer;"> <section style="text-align: justify;margin-left: 8px;margin-right: 8px;line-height: 1.75em;margin-bottom: 5px;"> 一直等待其追上后，才会像下图这样： </section> <img data-ratio="0.48985507246376814" src="/upload/8a694e5d10c659da02f361b83d3ecdcc.jpg" data-type="jpeg" data-w="690" style="-webkit-font-smoothing: antialiased;margin-bottom: 15px;border-width: initial;border-style: none;border-color: initial;word-break: break-all;cursor: pointer;" title="1583752489483514.jpg"> <section style="text-align: justify;margin-left: 8px;margin-right: 8px;line-height: 1.75em;margin-bottom: 5px;"> 此时，balancer 已达平衡状态： </section> <img data-ratio="0.75642965204236" src="/upload/8b18f4c2582214c43526f785d4894c36.jpg" data-type="jpeg" data-w="661" style="-webkit-font-smoothing: antialiased;margin-bottom: 15px;border-width: initial;border-style: none;border-color: initial;word-break: break-all;cursor: pointer;" title="1583752538954594.jpg"> Duration 恢复正常水平，如下图 16:54 分时的情况： <img data-ratio="0.26666666666666666" src="/upload/f4b7975445834a7c22679937df00726b.jpg" data-type="jpeg" data-w="690" style="-webkit-font-smoothing: antialiased;margin-bottom: 15px;border-width: initial;border-style: none;border-color: initial;word-break: break-all;cursor: pointer;" title="1583752577292967.jpg"> QPS 也不再积压，恢复正常水准： <img data-ratio="0.2594202898550725" src="/upload/8e09172ae170b2acc7d013d2f2f5ad71.jpg" data-type="jpeg" data-w="690" style="-webkit-font-smoothing: antialiased;margin-bottom: 15px;border-width: initial;border-style: none;border-color: initial;word-break: break-all;cursor: pointer;" title="1583752610787076.jpg"> <section style="text-align: justify;margin-left: 8px;margin-right: 8px;line-height: 1.75em;"> 关于 relay-log，默认是不清理的，就和 MySQL 的 expire_logs_days 一样，这块可以通过 dm-worker 的配置文件来进行配置。 </section> 例如将 Expires 配置为 7，代表 7 天后删除： <section style="font-size: 16px;color: rgb(62, 62, 62);line-height: 1.6;letter-spacing: 0px;font-family: "Helvetica Neue", Helvetica, "Hiragino Sans GB", "Microsoft YaHei", Arial, sans-serif;"> <pre style="font-size: inherit;color: inherit;line-height: inherit;"><code style="margin-right: 2px;margin-left: 2px;line-height: 18px;font-size: 14px;letter-spacing: 0px;font-family: Consolas, Inconsolata, Courier, monospace;border-radius: 0px;color: rgb(169, 183, 198);background: rgb(40, 43, 46);padding: 0.5em;word-wrap: normal !important;word-break: normal !important;overflow: auto !important;display: -webkit-box !important;">[purge] interval = 3600 expires = 7 remain-space = 15 </code></pre> </section> <section style="text-align: justify;margin-left: 8px;margin-right: 8px;line-height: 1.75em;"> Expires 来控制保留天数。 默认 expires=0，即没有过期时间，而 remain-space=15 意思是当磁盘只剩于 15G 的时候开始尝试清理，这种情况我们极少会碰到，因此这个清理方式其实基本上是用不到的。 </section> <section style="text-align: justify;margin-left: 8px;margin-right: 8px;line-height: 1.75em;"> 所以建议有需要删除过期 relay-log 的小伙伴，直接配置 Expires 保留天数就可以了。 </section> <section style="text-align: justify;margin-left: 8px;margin-right: 8px;line-height: 1.75em;"> DM 导入完成后，应该提供是否在完成后自动删除全备文件的选项，可以默认不删，由使用者决定是否删除。 </section> <section style="text-align: justify;margin-left: 8px;margin-right: 8px;line-height: 1.75em;"> 从使用者角度来说，全量备份目录无论是全量一次性导入还是 all 增量同步，后续都不会再使用到。 </section> <section style="text-align: justify;margin-left: 8px;margin-right: 8px;line-height: 1.75em;"> 如果 dm-worker 和 TiKV 混部，会导致全备文件占据大量磁盘空间，引起 TiKV Region 评分出现异常，导致性能下降，已转化为 PRM 产品需求。 </section> ④关于 DM 使用期间出现数据丢失的问题 <section style="line-height: normal;"> </section> 在早期还没有 dm-portal 自动化生成 task 时，我们都是自行编写 DM 的 task 同步文件。后来有了 dm-portal 自动化生成工具，只要图形页面点点点就可以了。 <img data-ratio="0.6459016393442623" src="/upload/42f1f92f664db0f0ee4ae36a005bfd22.jpg" data-type="jpeg" data-w="610" style="-webkit-font-smoothing: antialiased;margin-bottom: 15px;border-width: initial;border-style: none;border-color: initial;word-break: break-all;cursor: pointer;" title="1583752679901898.jpg"> <section style="text-align: justify;margin-left: 8px;margin-right: 8px;line-height: 1.75em;"> 但该工具目前有一个问题是，没有全库正则匹配，即便你只勾选一个库，他底层是默认把每张表都给你配置一遍。 </section> <section style="text-align: justify;margin-left: 8px;margin-right: 8px;line-height: 1.75em;"> 这就会出现当上层 MySQL 新创建某张表的时候，下游会被忽略掉，例如当你使用改表工具 gh-ost 或者 pt-online-schema-change，你的临时表都会被当做为不在白名单内而被忽略，这个问题使用者需要注意。 </section> <section style="line-height: normal;"> </section> 我们也已经反馈给了官方。未来不久的版本估计就可以修复。 <section style="font-size: 16px;color: rgb(62, 62, 62);line-height: 1.6;letter-spacing: 0px;font-family: "Helvetica Neue", Helvetica, "Hiragino Sans GB", "Microsoft YaHei", Arial, sans-serif;"> <pre style="font-size: inherit;color: inherit;line-height: inherit;"><code style="margin-right: 2px;margin-left: 2px;line-height: 18px;font-size: 14px;letter-spacing: 0px;font-family: Consolas, Inconsolata, Courier, monospace;border-radius: 0px;color: rgb(169, 183, 198);background: rgb(40, 43, 46);padding: 0.5em;word-wrap: normal !important;word-break: normal !important;overflow: auto !important;display: -webkit-box !important;">["skip event"] [task=task_20357] [unit="binlog replication"] [event=query] [statement="ALTER TABLE `360`.`_data_201910_gho` ADD COLUMN `raw_url_md5` CHAR(32) NOT NULL DEFAULT '' COMMENT 'raw_url md5'"] ["skip event"] [task=task_20357] [unit="binlog replication"] [event=query] [statement="ALTER TABLE `360`.`_data_201910_gho` ADD INDEX `idx_rawurl_md5`(`raw_url_md5`)"] [schema=flow] ["skip event"] [task=task_20357] [unit="binlog replication"] [event=query] [statement="ALTER TABLE `360`.`_data_201910_gho` DROP INDEX `idx_raw_url`"] [schema=flow] </code></pre> </section> <section style="text-align: justify;margin-left: 8px;margin-right: 8px;line-height: 1.75em;"> 这里日志可以看到 event 被 skip 了。 </section> <section style="line-height: normal;"> </section> ⑤关于 DM 使用期间偶发性 1062 主键冲突的问题 query-error task 能看到具体的报错信息，下游看都没有该值： <section style="font-size: 16px;color: rgb(62, 62, 62);line-height: 1.6;letter-spacing: 0px;font-family: "Helvetica Neue", Helvetica, "Hiragino Sans GB", "Microsoft YaHei", Arial, sans-serif;"> <pre style="font-size: inherit;color: inherit;line-height: inherit;"><code style="margin-right: 2px;margin-left: 2px;line-height: 18px;font-size: 14px;letter-spacing: 0px;font-family: Consolas, Inconsolata, Courier, monospace;border-radius: 0px;color: rgb(169, 183, 198);background: rgb(40, 43, 46);padding: 0.5em;word-wrap: normal !important;word-break: normal !important;overflow: auto !important;display: -webkit-box !important;">mysql> select * from client where clientid='82b51e6f6eb64955487f978dd94a2c81e492f6170xxxxxxxxxxxxxxxxxxxxxxxxx'; Empty set (0.00 sec) </code></pre> </section> 再去上游看，结果发现也没有值，业务逻辑应该是后来 delete 了： <section style="font-size: 16px;color: rgb(62, 62, 62);line-height: 1.6;letter-spacing: 0px;font-family: "Helvetica Neue", Helvetica, "Hiragino Sans GB", "Microsoft YaHei", Arial, sans-serif;"> <pre style="font-size: inherit;color: inherit;line-height: inherit;"><code style="margin-right: 2px;margin-left: 2px;line-height: 18px;font-size: 14px;letter-spacing: 0px;font-family: Consolas, Inconsolata, Courier, monospace;border-radius: 0px;color: rgb(169, 183, 198);background: rgb(40, 43, 46);padding: 0.5em;word-wrap: normal !important;word-break: normal !important;overflow: auto !important;display: -webkit-box !important;">mysql> select * from client where clientid='82b51e6f6eb64955487f978dd94a2c81e492f6170xxxxxxxxxxxxxxxxxxxxxxxxx'; Empty set (0.00 sec) </code></pre> </section> <section style="text-align: justify;margin-left: 8px;margin-right: 8px;line-height: 1.75em;"> 因为上游也没有值，去上�

超轻量级中文OCR，支持竖排文字识别、ncnn推理，总模型仅17M

作者：微信小助手

<img class="rich_pages" data-ratio="0.11875" src="/upload/debb99064b4679d97f20510588e78bf1.gif" data-type="gif" data-w="640" style=""> <p style="margin: 15px 8px;max-width: 100%;box-sizing: border-box;min-height: 1em;font-family: -apple-system-font, BlinkMacSystemFont, "Helvetica Neue", "PingFang SC", "Hiragino Sans GB", "Microsoft YaHei UI", "Microsoft YaHei", Arial, sans-serif;letter-spacing: 0.544px;text-size-adjust: auto;orphans: 4;color: rgb(93, 94, 93);white-space: pre-wrap;line-height: 1.75em;text-align: right;overflow-wrap: break-word !important;">来源 | AI科技大本营 <section style="margin-right: 8px;margin-left: 8px;max-width: 100%;font-family: -apple-system-font, system-ui, "Helvetica Neue", "PingFang SC", "Hiragino Sans GB", "Microsoft YaHei UI", "Microsoft YaHei", Arial, sans-serif;letter-spacing: 0.544px;white-space: normal;background-color: rgb(255, 255, 255);line-height: 2em;box-sizing: border-box !important;overflow-wrap: break-word !important;"> 光学字符识别（OCR）技术已经得到了广泛应用。比如发票上用来识别关键字样，搜题App用来识别书本上的试题。 </section> <section style="margin-right: 8px;margin-left: 8px;max-width: 100%;font-family: -apple-system-font, system-ui, "Helvetica Neue", "PingFang SC", "Hiragino Sans GB", "Microsoft YaHei UI", "Microsoft YaHei", Arial, sans-serif;letter-spacing: 0.544px;white-space: normal;background-color: rgb(255, 255, 255);line-height: 2em;box-sizing: border-box !important;overflow-wrap: break-word !important;"> <span style="font-size: 15px;letter-spacing: 1px;color: rgb(93, 94, 93);font-family: -apple-system-font, BlinkMacSystemFont, "Helvetica Neue", "PingFang SC", "Hiragino Sans GB", "Microsoft YaHei UI", "Microsoft YaHei", Arial, sans-serif;orphans: 4;white-space: pre-wrap;"> </section> <p style="margin-right: 8px;margin-left: 8px;max-width: 100%;font-family: -apple-system-font, system-ui, "Helvetica Neue", "PingFang SC", "Hiragino Sans GB", "Microsoft YaHei UI", "Microsoft YaHei", Arial, sans-serif;letter-spacing: 0.544px;white-space: normal;background-color: rgb(255, 255, 255);line-height: 2em;box-sizing: border-box !important;overflow-wrap: break-word !important;">近期，这个叫做chineseocr_lite的OCR项目开源了，这是一个超轻量级中文ocr，支持竖排文字识别，支持 ncnn 推理，psenet (8.5M) + crnn (6.3M) + anglenet (1.5M) 总模型仅17M。 <p style="margin-right: 8px;margin-left: 8px;max-width: 100%;font-family: -apple-system-font, system-ui, "Helvetica Neue", "PingFang SC", "Hiragino Sans GB", "Microsoft YaHei UI", "Microsoft YaHei", Arial, sans-serif;letter-spacing: 0.544px;white-space: normal;background-color: rgb(255, 255, 255);line-height: 2em;box-sizing: border-box !important;overflow-wrap: break-word !important;"> <p style="margin-right: 8px;margin-left: 8px;max-width: 100%;font-family: -apple-system-font, system-ui, "Helvetica Neue", "PingFang SC", "Hiragino Sans GB", "Microsoft YaHei UI", "Microsoft YaHei", Arial, sans-serif;letter-spacing: 0.544px;white-space: normal;background-color: rgb(255, 255, 255);line-height: 2em;box-sizing: border-box !important;overflow-wrap: break-word !important;">目前，这个开源项目已在GitHub上标星2400+。 <p style="margin-right: 8px;margin-left: 8px;max-width: 100%;font-family: -apple-system-font, system-ui, "Helvetica Neue", "PingFang SC", "Hiragino Sans GB", "Microsoft YaHei UI", "Microsoft YaHei", Arial, sans-serif;letter-spacing: 0.544px;white-space: normal;background-color: rgb(255, 255, 255);line-height: 2em;box-sizing: border-box !important;overflow-wrap: break-word !important;"> <p style="margin-right: 8px;margin-left: 8px;max-width: 100%;font-family: -apple-system-font, system-ui, "Helvetica Neue", "PingFang SC", "Hiragino Sans GB", "Microsoft YaHei UI", "Microsoft YaHei", Arial, sans-serif;letter-spacing: 0.544px;white-space: normal;background-color: rgb(255, 255, 255);line-height: 2em;box-sizing: border-box !important;overflow-wrap: break-word !important;">这个项目基于chineseocr与psenet实现中文自然场景文字检测及识别，环境是linux/macos。 <p style="margin-right: 8px;margin-left: 8px;max-width: 100%;font-family: -apple-system-font, system-ui, "Helvetica Neue", "PingFang SC", "Hiragino Sans GB", "Microsoft YaHei UI", "Microsoft YaHei", Arial, sans-serif;letter-spacing: 0.544px;white-space: normal;background-color: rgb(255, 255, 255);line-height: 2em;box-sizing: border-box !important;overflow-wrap: break-word !important;"> <p style="margin-right: 8px;margin-left: 8px;max-width: 100%;font-family: -apple-system-font, system-ui, "Helvetica Neue", "PingFang SC", "Hiragino Sans GB", "Microsoft YaHei UI", "Microsoft YaHei", Arial, sans-serif;letter-spacing: 0.544px;white-space: normal;background-color: rgb(255, 255, 255);line-height: 2em;box-sizing: border-box !important;overflow-wrap: break-word !important;">以下为可实现的功能： <p style="margin-right: 8px;margin-left: 8px;max-width: 100%;font-family: -apple-system-font, system-ui, "Helvetica Neue", "PingFang SC", "Hiragino Sans GB", "Microsoft YaHei UI", "Microsoft YaHei", Arial, sans-serif;letter-spacing: 0.544px;white-space: normal;background-color: rgb(255, 255, 255);line-height: 2em;box-sizing: border-box !important;overflow-wrap: break-word !important;"> <ul class="list-paddingleft-2" cid="n20" mdtype="list" data-mark="-" style="margin-right: 8px;margin-left: 8px;max-width: 100%;font-family: -apple-system-font, system-ui, "Helvetica Neue", "PingFang SC", "Hiragino Sans GB", "Microsoft YaHei UI", "Microsoft YaHei", Arial, sans-serif;letter-spacing: 0.544px;white-space: normal;background-color: rgb(255, 255, 255);line-height: 2em;box-sizing: border-box !important;overflow-wrap: break-word !important;"> <li style="margin-right: 8px;margin-left: 8px;max-width: 100%;font-family: -apple-system-font, system-ui, "Helvetica Neue", "PingFang SC", "Hiragino Sans GB", "Microsoft YaHei UI", "Microsoft YaHei", Arial, sans-serif;letter-spacing: 0.544px;white-space: normal;background-color: rgb(255, 255, 255);line-height: 2em;box-sizing: border-box !important;overflow-wrap: break-word !important;"><p style="margin-right: 8px;margin-left: 8px;max-width: 100%;font-family: -apple-system-font, system-ui, "Helvetica Neue", "PingFang SC", "Hiragino Sans GB", "Microsoft YaHei UI", "Microsoft YaHei", Arial, sans-serif;letter-spacing: 0.544px;white-space: normal;background-color: rgb(255, 255, 255);line-height: 2em;box-sizing: border-box !important;overflow-wrap: break-word !important;"> 提供轻量的 backone 检测模型 psenet（8.5M）,crnn_lstm_lite (9.5M) 和行文本方向分类网络（1.5M）</li> <li style="margin-right: 8px;margin-left: 8px;max-width: 100%;font-family: -apple-system-font, system-ui, "Helvetica Neue", "PingFang SC", "Hiragino Sans GB", "Microsoft YaHei UI", "Microsoft YaHei", Arial, sans-serif;letter-spacing: 0.544px;white-space: normal;background-color: rgb(255, 255, 255);line-height: 2em;box-sizing: border-box !important;overflow-wrap: break-word !important;"><p style="margin-right: 8px;margin-left: 8px;max-width: 100%;font-family: -apple-system-font, system-ui, "Helvetica Neue", "PingFang SC", "Hiragino Sans GB", "Microsoft YaHei UI", "Microsoft YaHei", Arial, sans-serif;letter-spacing: 0.544px;white-space: normal;background-color: rgb(255, 255, 255);line-height: 2em;box-sizing: border-box !important;overflow-wrap: break-word !important;"> 任意方向文字检测，识别时判断行文本方向</li> <li style="margin-right: 8px;margin-left: 8px;max-width: 100%;font-family: -apple-system-font, system-ui, "Helvetica Neue", "PingFang SC", "Hiragino Sans GB", "Microsoft YaHei UI", "Microsoft YaHei", Arial, sans-serif;letter-spacing: 0.544px;white-space: normal;background-color: rgb(255, 255, 255);line-height: 2em;box-sizing: border-box !important;overflow-wrap: break-word !important;"><p style="margin-right: 8px;margin-left: 8px;max-width: 100%;font-family: -apple-system-font, system-ui, "Helvetica Neue", "PingFang SC", "Hiragino Sans GB", "Microsoft YaHei UI", "Microsoft YaHei", Arial, sans-serif;letter-spacing: 0.544px;white-space: normal;background-color: rgb(255, 255, 255);line-height: 2em;box-sizing: border-box !important;overflow-wrap: break-word !important;"> crnn\crnn_lite lstm\dense识别（ocr-dense和ocr-lstm是搬运chineseocr 的）</li> <li style="margin-right: 8px;margin-left: 8px;max-width: 100%;font-family: -apple-system-font, system-ui, "Helvetica Neue", "PingFang SC", "Hiragino Sans GB", "Microsoft YaHei UI", "Microsoft YaHei", Arial, sans-serif;letter-spacing: 0.544px;white-space: normal;background-color: rgb(255, 255, 255);line-height: 2em;box-sizing: border-box !important;overflow-wrap: break-word !important;"><p style="margin-right: 8px;margin-left: 8px;max-width: 100%;font-family: -apple-system-font, system-ui, "Helvetica Neue", "PingFang SC", "Hiragino Sans GB", "Microsoft YaHei UI", "Microsoft YaHei", Arial, sans-serif;letter-spacing: 0.544px;white-space: normal;background-color: rgb(255, 255, 255);line-height: 2em;box-sizing: border-box !important;overflow-wrap: break-word !important;"> 支持竖排文本识别</li> <li style="margin-right: 8px;margin-left: 8px;max-width: 100%;font-family: -apple-system-font, system-ui, "Helvetica Neue", "PingFang SC", "Hiragino Sans GB", "Microsoft YaHei UI", "Microsoft YaHei", Arial, sans-serif;letter-spacing: 0.544px;white-space: normal;background-color: rgb(255, 255, 255);line-height: 2em;box-sizing: border-box !important;overflow-wrap: break-word !important;"><p style="margin-right: 8px;margin-left: 8px;max-width: 100%;font-family: -apple-system-font, system-ui, "Helvetica Neue", "PingFang SC", "Hiragino Sans GB", "Microsoft YaHei UI", "Microsoft YaHei", Arial, sans-serif;letter-spacing: 0.544px;white-space: normal;background-color: rgb(255, 255, 255);line-height: 2em;box-sizing: border-box !important;overflow-wrap: break-word !important;">ncnn实现psenet（未实现核扩展）</li> <li style="margin-right: 8px;margin-left: 8px;max-width: 100%;font-family: -apple-system-font, system-ui, "Helvetica Neue", "PingFang SC", "Hiragino Sans GB", "Microsoft YaHei UI", "Microsoft YaHei", Arial, sans-serif;letter-spacing: 0.544px;white-space: normal;background-color: rgb(255, 255, 255);line-height: 2em;box-sizing: border-box !important;overflow-wrap: break-word !important;"><p style="margin-right: 8px;margin-left: 8px;max-width: 100%;font-family: -apple-system-font, system-ui, "Helvetica Neue", "PingFang SC", "Hiragino Sans GB", "Microsoft YaHei UI", "Microsoft YaHei", Arial, sans-serif;letter-spacing: 0.544px;white-space: normal;background-color: rgb(255, 255, 255);line-height: 2em;box-sizing: border-box !important;overflow-wrap: break-word !important;">ncnn实现crnn_dense(改变了全连接为conv1x1)</li> <li style="margin-right: 8px;margin-left: 8px;max-width: 100%;font-family: -apple-system-font, system-ui, "Helvetica Neue", "PingFang SC", "Hiragino Sans GB", "Microsoft YaHei UI", "Microsoft YaHei", Arial, sans-serif;letter-spacing: 0.544px;white-space: normal;background-color: rgb(255, 255, 255);line-height: 2em;box-sizing: border-box !important;overflow-wrap: break-word !important;"><p style="margin-right: 8px;margin-left: 8px;max-width: 100%;font-family: -apple-system-font, system-ui, "Helvetica Neue", "PingFang SC", "Hiragino Sans GB", "Microsoft YaHei UI", "Microsoft YaHei", Arial, sans-serif;letter-spacing: 0.544px;white-space: normal;background-color: rgb(255, 255, 255);line-height: 2em;box-sizing: border-box !important;overflow-wrap: break-word !important;">ncnn实现shuuflenev2角度分类网络</li> <li style="margin-right: 8px;margin-left: 8px;max-width: 100%;font-family: -apple-system-font, system-ui, "Helvetica Neue", "PingFang SC", "Hiragino Sans GB", "Microsoft YaHei UI", "Microsoft YaHei", Arial, sans-serif;letter-spacing: 0.544px;white-space: normal;background-color: rgb(255, 255, 255);line-height: 2em;box-sizing: border-box !important;overflow-wrap: break-word !important;"><p style="margin-right: 8px;margin-left: 8px;max-width: 100%;font-family: -apple-system-font, system-ui, "Helvetica Neue", "PingFang SC", "Hiragino Sans GB", "Microsoft YaHei UI", "Microsoft YaHei", Arial, sans-serif;letter-spacing: 0.544px;white-space: normal;background-color: rgb(255, 255, 255);line-height: 2em;box-sizing: border-box !important;overflow-wrap: break-word !important;">ncnn实现ocr整个流程</li> </ul> <p style="margin-right: 8px;margin-left: 8px;max-width: 100%;font-family: -apple-system-font, system-ui, "Helvetica Neue", "PingFang SC", "Hiragino Sans GB", "Microsoft YaHei UI", "Microsoft YaHei", Arial, sans-serif;letter-spacing: 0.544px;white-space: normal;background-color: rgb(255, 255, 255);line-height: 2em;box-sizing: border-box !important;overflow-wrap: break-word !important;"> <p style="margin-right: 8px;margin-left: 8px;max-width: 100%;font-family: -apple-system-font, system-ui, "Helvetica Neue", "PingFang SC", "Hiragino Sans GB", "Microsoft YaHei UI", "Microsoft YaHei", Arial, sans-serif;letter-spacing: 0.544px;white-space: normal;background-color: rgb(255, 255, 255);line-height: 2em;box-sizing: border-box !important;overflow-wrap: break-word !important;">最近，项目作者对更新了可实现的功能。 <p style="margin-right: 8px;margin-left: 8px;max-width: 100%;font-family: -apple-system-font, system-ui, "Helvetica Neue", "PingFang SC", "Hiragino Sans GB", "Microsoft YaHei UI", "Microsoft YaHei", Arial, sans-serif;letter-spacing: 0.544px;white-space: normal;background-color: rgb(255, 255, 255);line-height: 2em;box-sizing: border-box !important;overflow-wrap: break-word !important;"> <ul class="list-paddingleft-2" style="margin-right: 8px;margin-left: 8px;max-width: 100%;font-family: -apple-system-font, system-ui, "Helvetica Neue", "PingFang SC", "Hiragino Sans GB", "Microsoft YaHei UI", "Microsoft YaHei", Arial, sans-serif;letter-spacing: 0.544px;white-space: normal;background-color: rgb(255, 255, 255);line-height: 2em;box-sizing: border-box !important;overflow-wrap: break-word !important;"> <li style="margin-right: 8px;margin-left: 8px;max-width: 100%;font-family: -apple-system-font, system-ui, "Helvetica Neue", "PingFang SC", "Hiragino Sans GB", "Microsoft YaHei UI", "Microsoft YaHei", Arial, sans-serif;letter-spacing: 0.544px;white-space: normal;background-color: rgb(255, 255, 255);line-height: 2em;box-sizing: border-box !important;overflow-wrap: break-word !important;"><p style="margin-right: 8px;margin-left: 8px;max-width: 100%;font-family: -apple-system-font, system-ui, "Helvetica Neue", "PingFang SC", "Hiragino Sans GB", "Microsoft YaHei UI", "Microsoft YaHei", Arial, sans-serif;letter-spacing: 0.544px;white-space: normal;background-color: rgb(255, 255, 255);line-height: 2em;box-sizing: border-box !important;overflow-wrap: break-word !important;">nihui大佬实现的crnn_lstm推理</li> <li style="margin-right: 8px;margin-left: 8px;max-width: 100%;font-family: -apple-system-font, system-ui, "Helvetica Neue", "PingFang SC", "Hiragino Sans GB", "Microsoft YaHei UI", "Microsoft YaHei", Arial, sans-serif;letter-spacing: 0.544px;white-space: normal;background-color: rgb(255, 255, 255);line-height: 2em;box-sizing: border-box !important;overflow-wrap: break-word !important;"><p style="margin-right: 8px;margin-left: 8px;max-width: 100%;font-family: -apple-system-font, system-ui, "Helvetica Neue", "PingFang SC", "Hiragino Sans GB", "Microsoft YaHei UI", "Microsoft YaHei", Arial, sans-serif;letter-spacing: 0.544px;white-space: normal;background-color: rgb(255, 255, 255);line-height: 2em;box-sizing: border-box !important;overflow-wrap: break-word !important;">升级crnn_lite_lstm_dw.pth模型crnn_lite_lstm_dw_v2.pth , 精度更高</li> <li style="margin-right: 8px;margin-left: 8px;max-width: 100%;font-family: -apple-system-font, system-ui, "Helvetica Neue", "PingFang SC", "Hiragino Sans GB", "Microsoft YaHei UI", "Microsoft YaHei", Arial, sans-serif;letter-spacing: 0.544px;white-space: normal;background-color: rgb(255, 255, 255);line-height: 2em;box-sizing: border-box !important;overflow-wrap: break-word !important;"><p style="margin-right: 8px;margin-left: 8px;max-width: 100%;font-family: -apple-system-font, system-ui, "Helvetica Neue", "PingFang SC", "Hiragino Sans GB", "Microsoft YaHei UI", "Microsoft YaHei", Arial, sans-serif;letter-spacing: 0.544px;white-space: normal;background-color: rgb(255, 255, 255);line-height: 2em;box-sizing: border-box !important;overflow-wrap: break-word !important;">提供竖排文字样例以及字体库（旋转90度的字体）</li> </ul> <p style="margin: 15px 8px;max-width: 100%;box-sizing: border-box;min-height: 1em;font-family: -apple-system-font, BlinkMacSystemFont, "Helvetica Neue", "PingFang SC", "Hiragino Sans GB", "Microsoft YaHei UI", "Microsoft YaHei", Arial, sans-serif;letter-spacing: 0.544px;text-size-adjust: auto;orphans: 4;color: rgb(93, 94, 93);font-size: 1rem;white-space: pre-wrap;line-height: 1.75em;overflow-wrap: break-word !important;">字体样式 <section style="margin-right: 8px;margin-left: 8px;max-width: 100%;caret-color: rgb(51, 51, 51);font-family: -apple-system-font, BlinkMacSystemFont, "Helvetica Neue", "PingFang SC", "Hiragino Sans GB", "Microsoft YaHei UI", "Microsoft YaHei", Arial, sans-serif;letter-spacing: 0.544px;white-space: normal;text-size-adjust: auto;text-align: center;box-sizing: border-box !important;overflow-wrap: break-word !important;"> <img class="rich_pages" data-ratio="0.6372549019607843" data-s="300,640" data-type="png" data-w="612" src="/upload/c0bc56e74532b66670e8cafe7aaec4fa.png" style="box-sizing: border-box !important;overflow-wrap: break-word !important;width: 612px !important;visibility: visible !important;"> </section> <h2 style="margin: 15px 8px;font-size: 1rem;max-width: 100%;box-sizing: border-box;font-family: -apple-system-font, BlinkMacSystemFont, "Helvetica Neue", "PingFang SC", "Hiragino Sans GB", "Microsoft YaHei UI", "Microsoft YaHei", Arial, sans-serif;letter-spacing: 0.544px;text-size-adjust: auto;min-height: 1em;orphans: 4;color: rgb(93, 94, 93);white-space: pre-wrap;line-height: 1.75em;overflow-wrap: break-word !important;">生成的竖排文本样例</h2> <section style="margin-right: 8px;margin-left: 8px;max-width: 100%;caret-color: rgb(51, 51, 51);font-family: -apple-system-font, BlinkMacSystemFont, "Helvetica Neue", "PingFang SC", "Hiragino Sans GB", "Microsoft YaHei UI", "Microsoft YaHei", Arial, sans-serif;letter-spacing: 0.544px;white-space: normal;text-size-adjust: auto;text-align: center;box-sizing: border-box !important;overflow-wrap: break-word !important;"> <img class="rich_pages" data-ratio="0.7848605577689243" data-s="300,640" data-type="png" data-w="502" src="/upload/1554d59f9827766b11e1907cda4f779b.png" style="box-sizing: border-box !important;overflow-wrap: break-word !important;width: 502px !important;visibility: visible !important;"> </section> <section style="margin: 15px 8px;max-width: 100%;box-sizing: border-box;font-family: -apple-system-font, BlinkMacSystemFont, "Helvetica Neue", "PingFang SC", "Hiragino Sans GB", "Microsoft YaHei UI", "Microsoft YaHei", Arial, sans-serif;letter-spacing: 0.544px;text-size-adjust: auto;min-height: 1em;orphans: 4;color: rgb(93, 94, 93);font-size: 1rem;white-space: pre-wrap;line-height: 1.75em;overflow-wrap: break-word !important;"> 识别效果展示： </section> <section style="margin-right: 8px;margin-left: 8px;max-width: 100%;caret-color: rgb(51, 51, 51);font-family: -apple-system-font, BlinkMacSystemFont, "Helvetica Neue", "PingFang SC", "Hiragino Sans GB", "Microsoft YaHei UI", "Microsoft YaHei", Arial, sans-serif;letter-spacing: 0.544px;white-space: normal;text-size-adjust: auto;text-align: center;box-sizing: border-box !important;overflow-wrap: break-word !important;"> <img class="rich_pages" data-ratio="1.4921875" data-s="300,640" data-type="png" data-w="768" src="/upload/28f870ecd1b7fd456c158a470a3023f7.png" style="box-sizing: border-box !important;overflow-wrap: break-word !important;width: 677px !important;visibility: visible !important;"> </section> <h2 style="margin: 15px 8px;font-size: 1rem;max-width: 100%;box-sizing: border-box;font-family: -apple-system-font, BlinkMacSystemFont, "Helvetica Neue", "PingFang SC", "Hiragino Sans GB", "Microsoft YaHei UI", "Microsoft YaHei", Arial, sans-serif;letter-spacing: 0.544px;text-size-adjust: auto;min-height: 1em;orphans: 4;color: rgb(93, 94, 93);white-space: pre-wrap;line-height: 1.75em;overflow-wrap: break-word !important;">ncnn检测识别展示(x86 cpu 单进程)</h2> <section style="margin-right: 8px;margin-left: 8px;max-width: 100%;caret-color: rgb(51, 51, 51);font-family: -apple-system-font, BlinkMacSystemFont, "Helvetica Neue", "PingFang SC", "Hiragino Sans GB", "Microsoft YaHei UI", "Microsoft YaHei", Arial, sans-serif;letter-spacing: 0.544px;white-space: normal;text-size-adjust: auto;text-align: center;box-sizing: border-box !important;overflow-wrap: break-word !important;"> <img class="rich_pages" data-ratio="1.2545045045045045" data-s="300,640" data-type="png" data-w="888" src="/upload/981610066b68a2a547b14c6c713f4d98.png" style="box-sizing: border-box !important;overflow-wrap: break-word !important;width: 459px;visibility: visible !important;height: 576px;"> </section> <section style="margin: 15px 8px;max-width: 100%;box-sizing: border-box;min-height: 1em;font-family: -apple-system-font, BlinkMacSystemFont, "Helvetica Neue", "PingFang SC", "Hiragino Sans GB", "Microsoft YaHei UI", "Microsoft YaHei", Arial, sans-serif;letter-spacing: 0.544px;text-size-adjust: auto;orphans: 4;color: rgb(93, 94, 93);font-size: 1rem;white-space: pre-wrap;line-height: 1.75em;overflow-wrap: break-word !important;"> 项目地址：https://github.com/ouyanghuiyu/chineseocr_lite </section> <section style="margin-right: 8px;margin-left: 8px;max-width: 100%;caret-color: rgb(51, 51, 51);font-family: -apple-system-font, BlinkMacSystemFont, "Helvetica Neue", "PingFang SC", "Hiragino Sans GB", "Microsoft YaHei UI", "Microsoft YaHei", Arial, sans-serif;white-space: normal;text-size-adjust: auto;font-size: 15px;text-align: center;line-height: 1.75em;min-height: 1em;letter-spacing: 1.5px;visibility: visible;box-sizing: border-box !important;overflow-wrap: break-word !important;"> </section> <section style="margin-right: 8px;margin-left: 8px;max-width: 100%;caret-color: rgb(51, 51, 51);font-family: -apple-system-font, BlinkMacSystemFont, "Helvetica Neue", "PingFang SC", "Hiragino Sans GB", "Microsoft YaHei UI", "Microsoft YaHei", Arial, sans-serif;white-space: normal;text-size-adjust: auto;font-size: 15px;text-align: center;line-height: 1.75em;min-height: 1em;letter-spacing: 1.5px;visibility: visible;box-sizing: border-box !important;overflow-wrap: break-word !important;"> —end— </section> <span style="letter-spacing: 0.544px;text-align: right;caret-color: rgb(51, 51, 51);color: rgb(56, 56, 56);font-family: "Helvetica Neue", Helvetica, "Hiragino Sans GB", "Microsoft YaHei", "Apple Color Emoji", "Emoji Symbols Font", "Segoe UI Symbol", Arial, sans-serif;font-size: 14px;font-weight: 700;background-color: rgb(255, 255, 255);">你<span style="margin-right: 3px;margin-left: 3px;padding: 1px 3px;letter-spacing: 0.544px;text-align: right;caret-color: rgb(51, 51, 51);max-width: 100%;box-sizing: border-box;color: rgb(56, 56, 56);font-family: "Helvetica Neue", Helvetica, "Hiragino Sans GB", "Microsoft YaHei", "Apple Color Emoji", "Emoji Symbols Font", "Segoe UI Symbol", Arial, sans-serif;font-size: 14px;font-weight: 700;user-select: text;-webkit-user-drag: none;-webkit-tap-highlight-color: transparent;background-color: rgb(171, 224, 225);border-radius: 2px;overflow-wrap: break-word !important;">“在看”<span style="letter-spacing: 0.544px;text-align: right;caret-color: rgb(51, 51, 51);color: rgb(56, 56, 56);font-family: "Helvetica Neue", Helvetica, "Hiragino Sans GB", "Microsoft YaHei", "Apple Color Emoji", "Emoji Symbols Font", "Segoe UI Symbol", Arial, sans-serif;font-size: 14px;font-weight: 700;background-color: rgb(255, 255, 255);">我吗？♡

Mybatis Plus 多租户架构(Multi-tenancy)实现

作者：微信小助手

<section style="font-size: 14px;color: rgb(62, 62, 62);line-height: 1.6;letter-spacing: 0px;font-family: "Helvetica Neue", Helvetica, "Hiragino Sans GB", "Microsoft YaHei", Arial, sans-serif;" data-mpa-powered-by="yiban.io"> <p style="font-family: -apple-system-font, BlinkMacSystemFont, "Helvetica Neue", "PingFang SC", "Hiragino Sans GB", "Microsoft YaHei UI", "Microsoft YaHei", Arial, sans-serif;letter-spacing: 0.544px;white-space: normal;color: rgb(62, 62, 62);background-color: rgb(255, 255, 255);text-size-adjust: auto;font-size: 15px;word-spacing: 2px;text-align: center;">点击蓝色“架构文摘”关注我哟 <p style="margin-bottom: 10px;font-family: -apple-system-font, BlinkMacSystemFont, "Helvetica Neue", "PingFang SC", "Hiragino Sans GB", "Microsoft YaHei UI", "Microsoft YaHei", Arial, sans-serif;letter-spacing: 0.544px;white-space: normal;color: rgb(62, 62, 62);background-color: rgb(255, 255, 255);text-size-adjust: auto;font-size: 15px;word-spacing: 2px;text-align: center;">加个“星标”，每天上午 09:25，干货推送！ <p style="letter-spacing: 0.544px;white-space: normal;font-family: -apple-system-font, system-ui, "Helvetica Neue", "PingFang SC", "Hiragino Sans GB", "Microsoft YaHei UI", "Microsoft YaHei", Arial, sans-serif;color: rgb(62, 62, 62);font-size: 14px;background-color: rgb(255, 255, 255);text-align: center;"><img data-backh="36" data-backw="578" data-ratio="0.0625" data-s="300,640" data-type="jpeg" data-w="640" width="100%" src="/upload/8c292e55ba5a23cb6ebc11f2a2c4fece.null" style="font-family: -apple-system-font, BlinkMacSystemFont, "Helvetica Neue", "PingFang SC", "Hiragino Sans GB", "Microsoft YaHei UI", "Microsoft YaHei", Arial, sans-serif;letter-spacing: 0.544px;widows: 1;word-spacing: 2px;color: rgb(136, 136, 136);visibility: visible !important;width: 677px !important;"> 作者：吴汶泽    来源：https://segmentfault.com/a/1190000017197768 在进行多租户架构(Multi-tenancy)实现之前，先了解一下相关的定义吧： <h3 style="line-height: inherit;margin-top: 1.5em;margin-bottom: 1.5em;font-weight: bold;color: rgb(0, 172, 193);font-size: 1.3em;">什么是多租户</h3> 多租户技术或称多重租赁技术，简称<code style="font-size: inherit;line-height: inherit;padding: 2px 4px;border-radius: 4px;margin-right: 2px;margin-left: 2px;color: rgb(233, 105, 0);background: rgb(248, 248, 248);">SaaS</code>，是一种软件架构技术，是实现如何在多用户环境下（此处的多用户一般是面向企业用户）共用相同的系统或程序组件，并且可确保各用户间数据的隔离性。 简单讲：在一台服务器上运行单个应用实例，它为多个租户（客户）提供服务。从定义中我们可以理解：多租户是一种架构，目的是为了让多用户环境下使用同一套程序，且保证用户间数据隔离。那么重点就很浅显易懂了，多租户的重点就是同一套程序下实现多用户数据的隔离。 <h3 style="line-height: inherit;margin-top: 1.5em;margin-bottom: 1.5em;font-weight: bold;color: rgb(0, 172, 193);font-size: 1.3em;">数据隔离方案</h3> 多租户在数据存储上存在三种主要的方案，分别是： <h4 style="line-height: inherit;margin-top: 1.5em;margin-bottom: 1.5em;font-weight: bold;color: rgb(0, 172, 193);font-size: 1.2em;">独立数据库</h4> 即一个租户一个数据库，这种方案的用户数据隔离级别最高，安全性最好，但成本较高。 <ul style="font-size: inherit;color: inherit;line-height: inherit;padding-left: 32px;" class="list-paddingleft-2"> <li style="font-size: inherit;color: inherit;line-height: inherit;margin-bottom: 0.5em;">优点：为不同的租户提供独立的数据库，有助于简化数据模型的扩展设计，满足不同租户的独特需求；如果出现故障，恢复数据比较简单。</li> <li style="font-size: inherit;color: inherit;line-height: inherit;margin-bottom: 0.5em;">缺点：增多了数据库的安装数量，随之带来维护成本和购置成本的增加。</li> </ul> <h4 style="line-height: inherit;margin-top: 1.5em;margin-bottom: 1.5em;font-weight: bold;color: rgb(0, 172, 193);font-size: 1.2em;">共享数据库，独立 Schema</h4> 多个或所有租户共享Database，但是每个租户一个Schema（也可叫做一个user）。底层库比如是：DB2、ORACLE等，一个数据库下可以有多个SCHEMA。 <ul style="font-size: inherit;color: inherit;line-height: inherit;padding-left: 32px;" class="list-paddingleft-2"> <li style="font-size: inherit;color: inherit;line-height: inherit;margin-bottom: 0.5em;">优点：为安全性要求较高的租户提供了一定程度的逻辑数据隔离，并不是完全隔离；每个数据库可支持更多的租户数量。</li> <li style="font-size: inherit;color: inherit;line-height: inherit;margin-bottom: 0.5em;">缺点：如果出现故障，数据恢复比较困难，因为恢复数据库将牵涉到其他租户的数据；</li> </ul> <h4 style="line-height: inherit;margin-top: 1.5em;margin-bottom: 1.5em;font-weight: bold;color: rgb(0, 172, 193);font-size: 1.2em;">共享数据库，共享 Schema，共享数据表</h4> 即租户共享同一个Database、同一个Schema，但在表中增加TenantID多租户的数据字段。这是共享程度最高、隔离级别最低的模式。 简单来讲，即每插入一条数据时都需要有一个客户的标识。这样才能在同一张表中区分出不同客户的数据，这也是我们系统目前用到的(provider_id) <ul style="font-size: inherit;color: inherit;line-height: inherit;padding-left: 32px;" class="list-paddingleft-2"> <li style="font-size: inherit;color: inherit;line-height: inherit;margin-bottom: 0.5em;">优点：三种方案比较，第三种方案的维护和购置成本最低，允许每个数据库支持的租户数量最多。</li> <li style="font-size: inherit;color: inherit;line-height: inherit;margin-bottom: 0.5em;">缺点：隔离级别最低，安全性最低，需要在设计开发时加大对安全的开发量；数据备份和恢复最困难，需要逐表逐条备份和还原。</li> </ul> <h3 style="line-height: inherit;margin-top: 1.5em;margin-bottom: 1.5em;font-weight: bold;color: rgb(0, 172, 193);font-size: 1.3em;">利用MybatisPlus实现</h3> 这里我们选用了第三种方案（<code style="font-size: inherit;line-height: inherit;padding: 2px 4px;border-radius: 4px;margin-right: 2px;margin-left: 2px;color: rgb(233, 105, 0);background: rgb(248, 248, 248);">共享数据库，共享 Schema，共享数据表</code>）来实现，也就意味着，每个数据表都需要有一个租户标识(<code style="font-size: inherit;line-height: inherit;padding: 2px 4px;border-radius: 4px;margin-right: 2px;margin-left: 2px;color: rgb(233, 105, 0);background: rgb(248, 248, 248);">provider_id</code>) 现在有数据库表(<code style="font-size: inherit;line-height: inherit;padding: 2px 4px;border-radius: 4px;margin-right: 2px;margin-left: 2px;color: rgb(233, 105, 0);background: rgb(248, 248, 248);">user</code>)如下： <figure style="font-size: inherit;color: inherit;line-height: inherit;"> <img data-ratio="0.31537450722733246" src="/upload/19facc068f5565dc2cc3a756f610cea.png" data-type="png" data-w="761" style="font-size: inherit;color: inherit;line-height: inherit;display: block;margin-right: auto;margin-left: auto;" title=""> </figure> 将<code style="font-size: inherit;line-height: inherit;padding: 2px 4px;border-radius: 4px;margin-right: 2px;margin-left: 2px;color: rgb(233, 105, 0);background: rgb(248, 248, 248);">provider_id</code>视为租户ID，用来隔离租户与租户之间的数据，如果要查询当前服务商的用户，SQL大致如下： <pre style="font-size: inherit;color: inherit;line-height: inherit;"><code style="margin-right: 2px;margin-left: 2px;line-height: 18px;letter-spacing: 0px;font-family: Consolas, Inconsolata, Courier, monospace;border-radius: 0px;color: rgb(169, 183, 198);background: rgb(40, 43, 46);padding: 0.5em;overflow-wrap: normal !important;word-break: normal !important;overflow: auto !important;display: -webkit-box !important;">SELECT * FROM user t WHERE t.name LIKE '%Tom%' AND t.provider_id = 1; </code></pre> 试想一下，除了一些系统共用的表以外，其他租户相关的表，我们都需要不厌其烦的加上AND t.provider_id = ?查询条件，稍不注意就会导致数据越界，数据安全问题让人担忧。 好在有了MybatisPlus这个神器，可以极为方便的实现多租户SQL解析器，官方文档如下： <blockquote style="line-height: inherit;padding: 15px 15px 15px 1rem;font-size: 0.9em;color: rgb(129, 145, 152);border-left-width: 6px;border-left-color: rgb(220, 230, 240);background: rgb(242, 247, 251);overflow: auto;overflow-wrap: normal;word-break: normal;"> http://mp.baomidou.com/guide/tenant.html </blockquote> 这里终于进入了正题，开始搭建一个极为简单的开发环境吧! <h3 style="line-height: inherit;margin-top: 1.5em;margin-bottom: 1.5em;font-weight: bold;color: rgb(0, 172, 193);font-size: 1.3em;">新建SpringBoot环境</h3> POM文件如下，主要集成了MybatisPlus以及H2数据库(方便测试) <pre style="font-size: inherit;color: inherit;line-height: inherit;"><code style="margin-right: 2px;margin-left: 2px;line-height: 18px;letter-spacing: 0px;font-family: Consolas, Inconsolata, Courier, monospace;border-radius: 0px;color: rgb(169, 183, 198);background: rgb(40, 43, 46);padding: 0.5em;overflow-wrap: normal !important;word-break: normal !important;overflow: auto !important;display: -webkit-box !important;"><?xml version="1.0" encoding="UTF-8"?> <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"          xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">     <modelVersion>4.0.0</modelVersion>     <groupId>com.wuwenze</groupId>     <artifactId>mybatis-plus-multi-tenancy</artifactId>     <version>0.0.1-SNAPSHOT</version>     <packaging>jar</packaging>     <name>mybatis-plus-multi-tenancy</name>     <description>Demo project for Spring Boot</description>     <parent>         <groupId>org.springframework.boot</groupId>         <artifactId>spring-boot-starter-parent</artifactId>         <version>2.1.0.RELEASE</version>         <relativePath/>      </parent>     <properties>         <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>         <project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>         <java.version>1.8</java.version>     </properties>     <dependencies>         <dependency>             <groupId>org.springframework.boot</groupId>             <artifactId>spring-boot-starter</artifactId>         </dependency>         <dependency>             <groupId>org.springframework.boot</groupId>             <artifactId>spring-boot-starter-test</artifactId>             <scope>test</scope>         </dependency>         <dependency>             <groupId>org.projectlombok</groupId>             <artifactId>lombok</artifactId>             <scope>provided</scope>         </dependency>         <dependency>             <groupId>com.google.guava</groupId>             <artifactId>guava</artifactId>             <version>19.0</version>         </dependency>         <dependency>             <groupId>com.baomidou</groupId>             <artifactId>mybatis-plus-boot-starter</artifactId>             <version>3.0.5</version>         </dependency>         <dependency>             <groupId>com.baomidou</groupId>             <artifactId>mybatis-plus</artifactId>             <version>3.0.5</version>         </dependency>         <dependency>             <groupId>com.baomidou</groupId>             <artifactId>mybatis-plus-generator</artifactId>             <version>3.0.5</version>         </dependency>         <dependency>             <groupId>com.h2database</groupId>             <artifactId>h2</artifactId>         </dependency>     </dependencies>     <build>         <plugins>             <plugin>                 <groupId>org.springframework.boot</groupId>                 <artifactId>spring-boot-maven-plugin</artifactId>             </plugin>         </plugins>     </build> </project> </code></pre> 数据源配置(application.yml) <pre style="font-size: inherit;color: inherit;line-height: inherit;"><code style="margin-right: 2px;margin-left: 2px;line-height: 18px;letter-spacing: 0px;font-family: Consolas, Inconsolata, Courier, monospace;border-radius: 0px;color: rgb(169, 183, 198);background: rgb(40, 43, 46);padding: 0.5em;overflow-wrap: normal !important;word-break: normal !important;overflow: auto !important;display: -webkit-box !important;">spring:   datasource:     driver-class-name: org.h2.Driver     schema: classpath:db/schema.sql     data: classpath:db/data.sql     url: jdbc:h2:mem:test     username: root     password: test logging:   level:     com.wuwenze.mybatisplusmultitenancy: debug </code></pre> 对应的H2数据库初始化schema文件 <pre style="font-size: inherit;color: inherit;line-height: inherit;"><code style="margin-right: 2px;margin-left: 2px;line-height: 18px;letter-spacing: 0px;font-family: Consolas, Inconsolata, Courier, monospace;border-radius: 0px;color: rgb(169, 183, 198);background: rgb(40, 43, 46);padding: 0.5em;overflow-wrap: normal !important;word-break: normal !important;overflow: auto !important;display: -webkit-box !important;">#schema.sql DROP TABLE IF EXISTS user; CREATE TABLE user (     id BIGINT(20) NOT NULL COMMENT '主键',     provider_id BIGINT(20) NOT NULL COMMENT '服务商ID',     name VARCHAR(30) NULL DEFAULT NULL COMMENT '姓名',     PRIMARY KEY (id) ); #data.sql INSERT INTO user (id, provider_id, name) VALUES (1, 1, 'Tony老师'); INSERT INTO user (id, provider_id, name) VALUES (2, 1, 'William老师'); INSERT <span style="font-size: inherit;line-h

【生活现场】从诗词大会飞花令到elasticsearch原理解析

作者：微信小助手

<p style="max-width: 100%;min-height: 1em;letter-spacing: 0.544px;white-space: normal;background-color: rgb(255, 255, 255);text-indent: 2em;box-sizing: border-box !important;word-wrap: break-word !important;">郑重声明：漫画人物和故事情节均是原创，未经同意禁止任何形式的转载，侵权将进入法律程序。大佬除外。 <p style="max-width: 100%;min-height: 1em;letter-spacing: 0.544px;white-space: normal;background-color: rgb(255, 255, 255);text-indent: 2em;box-sizing: border-box !important;word-wrap: break-word !important;">欢迎大家转发文章，谢谢！ <p style="max-width: 100%;min-height: 1em;letter-spacing: 0.544px;white-space: normal;background-color: rgb(255, 255, 255);text-align: center;box-sizing: border-box !important;word-wrap: break-word !important;"><img class="" data-copyright="0" data-ratio="0.44802867383512546" data-s="300,640" data-type="jpeg" data-w="558" src="/upload/2ff1805dda7d70892bb3f61234ae0ac2.jpg" style="box-sizing: border-box !important;word-wrap: break-word !important;visibility: visible !important;width: 558px !important;"> <p style="max-width: 100%;min-height: 1em;letter-spacing: 0.544px;white-space: normal;background-color: rgb(255, 255, 255);text-indent: 2em;box-sizing: border-box !important;word-wrap: break-word !important;">小史是一个非科班的程序员，虽然学的是电子专业，但是通过自己的努力成功通过了面试，现在要开始迎接新生活了。 <p style="max-width: 100%;min-height: 1em;letter-spacing: 0.544px;white-space: normal;background-color: rgb(255, 255, 255);text-indent: 2em;box-sizing: border-box !important;word-wrap: break-word !important;">对小史面试情况感兴趣的同学可以观看<a href="http://mp.weixin.qq.com/s?__biz=MzIzMTE1ODkyNQ==&mid=2649410956&idx=1&sn=50942b435c41f60c1fc02b0d10c6b553&chksm=f0b6086ec7c181787c5b4a2936026941db14a4ac177e78f42d2374860999a0e8c3911858056f&scene=21#wechat_redirect" target="_blank" data-linktype="2" style="color: rgb(87, 107, 149);-webkit-tap-highlight-color: rgba(0, 0, 0, 0);max-width: 100%;box-sizing: border-box !important;word-wrap: break-word !important;">面试现场系列</a>。 <img class="" data-copyright="0" data-ratio="0.5628517823639775" data-s="300,640" src="/upload/ce39fdee7e3828ca1c32d809383acf72.jpg" data-type="jpeg" data-w="533" style=""> 随着央视诗词大会的热播，小史开始对诗词感兴趣，最喜欢的就是飞花令的环节。 但是由于小史很久没有背过诗词了，飞一个字很难说出一句，很多之前很熟悉的诗句也想不起来。 <img class="" data-copyright="0" data-ratio="1.0223214285714286" data-s="300,640" src="/upload/260d2de164bb35b28f3d709e7fe45428.jpg" data-type="jpeg" data-w="448" style=""> <img class="" data-copyright="0" data-ratio="1.0223214285714286" data-s="300,640" src="/upload/f6fe4ce748da747e0da89147532db04b.jpg" data-type="jpeg" data-w="448" style=""> <img class="" data-copyright="0" data-ratio="1.0223214285714286" data-s="300,640" src="/upload/8c04395163b0f7110b5dd71a8303d772.jpg" data-type="jpeg" data-w="448" style=""> <img class="" data-copyright="0" data-ratio="1.0223214285714286" data-s="300,640" src="/upload/a983e56a4c39ec78fca6281a711e9c51.jpg" data-type="jpeg" data-w="448" style=""> <img class="" data-copyright="0" data-ratio="1.0223214285714286" data-s="300,640" src="/upload/53c9fb769ca4efbf6df01f0cbb74560b.jpg" data-type="jpeg" data-w="448" style=""> <img class="" data-copyright="0" data-ratio="1.0223214285714286" data-s="300,640" src="/upload/9743c03d6181f134383e4a54c3c17357.jpg" data-type="jpeg" data-w="448" style=""> <img class="" data-copyright="0" data-ratio="1.0223214285714286" data-s="300,640" src="/upload/ac1d224e4eab473e2b0b72f5b9b8c645.jpg" data-type="jpeg" data-w="448" style=""> <img class="" data-copyright="0" data-ratio="1.0223214285714286" data-s="300,640" src="/upload/42e78004a1756f9dbd4673efdf6afd4b.jpg" data-type="jpeg" data-w="448" style=""> <img class="" data-copyright="0" data-ratio="1.0223214285714286" data-s="300,640" src="/upload/c6d53a863620f4cdd8a7d9ba89c24f38.jpg" data-type="jpeg" data-w="448" style=""> <img class="" data-copyright="0" data-ratio="1.0223214285714286" data-s="300,640" src="/upload/7f861516127189bf10322b64fa25469f.jpg" data-type="jpeg" data-w="448" style=""> 【倒排索引】 <img class="" data-copyright="0" data-ratio="1.0223214285714286" data-s="300,640" src="/upload/329267e6c1e657776d5a03035e4d62de.jpg" data-type="jpeg" data-w="448" style=""> <img class="" data-copyright="0" data-ratio="1.0223214285714286" data-s="300,640" src="/upload/b826ec2f4f5354916eb9b16bcd403f40.jpg" data-type="jpeg" data-w="448" style=""> <img class="" data-copyright="0" data-ratio="1.0223214285714286" data-s="300,640" src="/upload/c66232c2ba41b4edb7bc2c25eb237405.jpg" data-type="jpeg" data-w="448" style=""> <img class="" data-copyright="0" data-ratio="1.1840796019900497" data-s="300,640" src="/upload/31a9b633e67644b15861b1bb90146f18.jpg" data-type="jpeg" data-w="402" style=""> <img class="" data-copyright="0" data-ratio="1.0223214285714286" data-s="300,640" src="/upload/88f11db3984bf512cb66edda518fc50c.jpg" data-type="jpeg" data-w="448" style=""> <img class="" data-copyright="0" data-ratio="1.0223214285714286" data-s="300,640" src="/upload/6d58f3fd3fbdf8cf064f45f3554629e6.jpg" data-type="jpeg" data-w="448" style=""> 吕老师：但是我让你说出带“前”字的诗句，由于没有索引，你只能遍历脑海中所有诗词，当你的脑海中诗词量大的时候，就很难在短时间内得到结果了。 <img class="" data-copyright="0" data-ratio="1.0223214285714286" data-s="300,640" src="/upload/3a7054af4769d828e15173ebade48d57.jpg" data-type="jpeg" data-w="448" style=""> <img class="" data-copyright="0" data-ratio="1.0223214285714286" data-s="300,640" src="/upload/3361290f99cd2494296c61ba802124ec.jpg" data-type="jpeg" data-w="448" style=""> <img class="" data-copyright="0" data-ratio="0.9601990049751243" data-s="300,640" src="/upload/230f0fe69c57c1e5fa151b9d49841673.jpg" data-type="jpeg" data-w="402" style=""> <img class="" data-copyright="0" data-ratio="1.0223214285714286" data-s="300,640" src="/upload/54b6b18851b52642a5325fc2319d77ec.jpg" data-type="jpeg" data-w="448" style=""> <img class="" data-copyright="0" data-ratio="1.0223214285714286" data-s="300,640" src="/upload/3153e55325cf2c723656d5964d84b03a.jpg" data-type="jpeg" data-w="448" style=""> <img class="" data-copyright="0" data-ratio="1.0223214285714286" data-s="300,640" src="/upload/9f63ff12b13c980d72c10a7c3582e35f.jpg" data-type="jpeg" data-w="448" style=""> <img class="" data-copyright="0" data-ratio="1.0223214285714286" data-s="300,640" src="/upload/f320234118bc37a35ff4527e77321ad9.jpg" data-type="jpeg" data-w="448" style=""> 【索引量爆炸】 <img class="" data-copyright="0" data-ratio="1.0223214285714286" data-s="300,640" src="/upload/2a663bda1a15a7a31be3bece6145d34e.jpg" data-type="jpeg" data-w="448" style=""> <img class="" data-copyright="0" data-ratio="0.8349056603773585" data-s="300,640" src="/upload/7b858c26704572fa4e4b8ee0f3c31a18.jpg" data-type="jpeg" data-w="424" style=""> <img class="" data-copyright="0" data-ratio="1.0223214285714286" data-s="300,640" src="/upload/c4e722a6395648a7f5ce3f37b269444c.jpg" data-type="jpeg" data-w="448" style=""> <img class="" data-copyright="0" data-ratio="2.644278606965174" data-s="300,640" src="/upload/443b95185971c36edc7ac0bd026845f.jpg" data-type="jpeg" data-w="402" style=""> <img class="" data-copyright="0" data-ratio="1.0223214285714286" data-s="300,640" src="/upload/a8cc59fd0f42df7d0d3ab0a6114ceb0f.jpg" data-type="jpeg" data-w="448" style=""> <img class="" data-copyright="0" data-ratio="1.0223214285714286" data-s="300,640" src="/upload/55ec85f40c6cdaf2c165d1cc00dffc34.jpg" data-type="jpeg" data-w="448" style=""> <img class="" data-copyright="0" data-ratio="1.0223214285714286" data-s="300,640" src="/upload/a14c0010578bd81fe73816249862cb03.jpg" data-type="jpeg" data-w="448" style=""> <img class="" data-copyright="0" data-ratio="1.0223214285714286" data-s="300,640" src="/upload/7c2816691a1ca61fd2dc96fd5688d341.jpg" data-type="jpeg" data-w="448" style=""> <img class="" data-copyright="0" data-ratio="1.0223214285714286" data-s="300,640" src="/upload/4fbdf101bc5340119c5305bf509d4082.jpg" data-type="jpeg" data-w="448" style=""> <img class="" data-copyright="0" data-ratio="1.0223214285714286" data-s="300,640" src="/upload/6221c12fa56574635863190706939826.jpg" data-type="jpeg" data-w="448" style=""> <img class="" data-copyright="0" data-ratio="1.8781094527363185" data-s="300,640" src="/upload/edcde07c2f14c297e9ad0c4a5d8f7f61.jpg" data-type="jpeg" data-w="402" style=""> <img class="" data-copyright="0" data-ratio="1.0223214285714286" data-s="300,640" src="/upload/130d1b44ffa3bac78a8f896bc256e783.jpg" data-type="jpeg" data-w="448" style=""> <img class="" data-copyright="0" data-ratio="1.0223214285714286" data-s="300,640" src="/upload/cad373d81f99f336e70f3a0ed6ae7ded.jpg" data-type="jpeg" data-w="448" style=""> <img class="" data-copyright="0" data-ratio="1.8781094527363185" data-s="300,640" src="/upload/98c94a49661e6a2415edcda806408d1d.jpg" data-type="jpeg" data-w="402" style=""> <img class="" data-copyright="0" data-ratio="1.0223214285714286" data-s="300,640" src="/upload/a249e1cffab63464dd692d7ea7e5ff00.jpg" data-type="jpeg" data-w="448" style=""> <img class="" data-copyright="0" data-ratio="1.1741293532338308" data-s="300,640" src="/upload/6ea2b2419e62f647a9701bd7c9d7613c.jpg" data-type="jpeg" data-w="402" style=""> <img class="" data-copyright="0" data-ratio="1.0223214285714286" data-s="300,640" src="/upload/8d350699ae03a29b2fc02e5e63cb48db.jpg" data-type="jpeg" data-w="448" style=""> <img class="" data-copyright="0" data-ratio="1.1766169154228856" data-s="300,640" src="/upload/272ae5bcc9213f2d2aa3499df2b371e6.jpg" data-type="jpeg" data-w="402" style=""> <img class="" data-copyright="0" data-ratio="1.0223214285714286" data-s="300,640" src="/upload/f9a8ee774799a34412d819fc955e5a35.jpg" data-type="jpeg" data-w="448" style=""> 【搜索引擎原理】 <img class="" data-copyright="0" data-ratio="1.0223214285714286" data-s="300,640" src="/upload/1676dbaaa86c31af76468e46757c96f6.jpg" data-type="jpeg" data-w="448" style=""> <img class="" data-copyright="0" data-ratio="1.0223214285714286" data-s="300,640" src="/upload/24f85a11904447a8ba03cb04c6d92d49.jpg" data-type="jpeg" data-w="448" style=""> <img class="" data-copyright="0" data-ratio="0.9245283018867925" data-s="300,640" src="/upload/f4cb5834b7a9c6b2ea90f454236f3de9.jpg" data-type="jpeg" data-w="424" style=""> <img class="" data-copyright="0" data-ratio="1.0223214285714286" data-s="300,640" src="/upload/215c759b9d93ff10e4573d9929dfc2da.jpg" data-type="jpeg" data-w="448" style=""> <img class="" data-copyright="0" data-ratio="1.0223214285714286" data-s="300,640" src="/upload/43ab76b69cdcc2fe52af29f3681e8c0d.jpg" data-type="jpeg" data-w="448" style=""> <img class="" data-copyright="0" data-ratio="1.0223214285714286" data-s="300,640" src="/upload/a7b2f38af125fec4401d11a50c4d5a99.jpg" data-type="jpeg" data-w="448" style=""> <img class="" data-copyright="0" data-ratio="1.0223214285714286" data-s="300,640" src="/upload/d908105e34386ad7c22cadab71767f4e.jpg" data-type="jpeg" data-w="448" style=""> <img class="" data-copyright="0" data-ratio="1.0223214285714286" data-s="300,640" src="/upload/4f725555c54f940a4955909c80374c21.jpg" data-type="jpeg" data-w="448" style=""> <img class="" data-copyright="0" data-ratio="1.1766169154228856" data-s="300,640" src="/upload/541b371c0c6a32c4c010989552c8f1bf.jpg" data-type="jpeg" data-w="402" style=""> <img class="" data-copyright="0" data-ratio="1.0223214285714286" data-s="300,640" src="/upload/9377097e29edb9d5e3610611b1b034e4.jpg" data-type="jpeg" data-w="448" style=""> <img class="" data-copyright="0" data-ratio="1.0223214285714286" data-s="300,640" src="/upload/6a9bdc27330cdb31e7a1b7c8ae415c7f.jpg" data-type="jpeg" data-w="448" style=""> <img class="" data-copyright="0" data-ratio="1.0223214285714286" data-s="300,640" src="/upload/ca9023e9b135db8578bf0f1feb94bc9e.jpg" data-type="jpeg" data-w="448" style=""> <img class="" data-copyright="0" data-ratio="1.0223214285714286" data-s="300,640" src="/upload/439cdbae4dc622098a7bdab6d16dca50.jpg" data-type="jpeg" data-w="448" style=""> <img class="" data-copyright="0" data-ratio="1.3962264150943395" data-s="300,640" src="/upload/60ee5be19cdfdbebf1db5fa2465ec495.jpg" data-type="jpeg" data-w="424" style=""> <img class="" data-copyright="0" data-ratio="1.0223214285714286" data-s="300,640" src="/upload/6268f88732fe19ac2379526649f62b52.jpg" data-type="jpeg" data-w="448" style=""> 【elasticsearch简介】 <img class="" data-copyright="0" data-ratio="1.0223214285714286" data-s="300,640" src="/upload/2f3d854dd00f7188c9f6c595babc1324.jpg" data-type="jpeg" data-w="448" style=""> <img class="" data-copyright="0" data-ratio="1.0223214285714286" data-s="300,640" src="/upload/df17ee822bed9a88c8efe8fdefd06e79.jpg" data-type="jpeg" data-w="448" style=""> <img class="" data-copyright="0" data-ratio="1.0223214285714286" data-s="300,640" src="/upload/ad6295340ad5a30098e7a8f29f4b7b86.jpg" data-type="jpeg" data-w="448" style=""> <img class="" data-copyright="0" data-ratio="1.0223214285714286" data-s="300,640" src="/upload/57560c7fa4c1b89ba885bc795e6134f2.jpg" data-type="jpeg" data-w="448" style=""> <img class="" data-copyright="0" data-ratio="1.0223214285714286" data-s="300,640" src="/upload/2fe7c9ef3d43f7bb4424c31f2e85a235.jpg" data-type="jpeg" data-w="448" style=""> 吕老师：但是lucene还是一个库，必须要懂一点搜索引擎原理的人才能用的好，所以后来又有人基于lucene进行封装，写出了elasticsearch。 <img class="" data-copyright="0" data-ratio="1.0223214285714286" data-s="300,640" src="/upload/dadfa1d74e626bcebda61a4208e79192.jpg" data-type="jpeg" data-w="448" style=""> <img class="" data-copyright="0" data-ratio="1.0223214285714286" data-s="300,640" src="/upload/ad2bc40aa13c47b3c4abcb5cdde3a4bd.jpg" data-type="jpeg" data-w="448" style=""> <img class="" data-copyright="0" data-ratio="1.0223214285714286" data-s="300,640" src="/upload/be0b5c58a61d68be79226346aa1ff0ad.jpg" data-type="jpeg" data-w="448" style=""> <img class="" data-copyright="0" data-ratio="1.0223214285714286" data-s="300,640" src="/upload/50242586788c8234edfb4758055a1ff6.jpg" data-type="jpeg" data-w="448" style=""> <img class="" data-copyright="0" data-ratio="1.0223214285714286" data-s="300,640" src="/upload/157bc5846ba1ea5529dd286182f3bb82.jpg" data-type="jpeg" data-w="448" style=""> <img class="" data-copyright="0" data-ratio="1.0223214285714286" data-s="300,640" src="/upload/2c73db7be9b5216acd91e2e540d73ec8.jpg" data-type="jpeg" data-w="448" style=""> 【elasticsearch基本概念】 <img class="rich_pages" data-copyright="0" data-ratio="1.0223214285714286" data-s="300,640" src="/upload/3e5f4e5e1fd129b444d064fafae81418.jpg" data-type="jpeg" data-w="448" style=""> <img class="rich_pages" data-copyright="0" data-ratio="1.0223214285714286" data-s="300,640" src="/upload/623299d66a1cb0f55b855a076b8e39ee.jpg" data-type="jpeg" data-w="448" style=""> <img class="rich_pages" data-copyright="0" data-ratio="1.0223214285714286" data-s="300,640" src="/upload/96eb01f4f01e12837e59862bd6c280fe.jpg" data-type="jpeg" data-w="448" style=""> <img class="rich_pages" data-copyright="0" data-ratio="1.0223214285714286" data-s="300,640" src="/upload/2a27f0cb64eb50a90c21af1b6547099a.jpg" data-type="jpeg" data-w="448" style=""> <img class="rich_pages" data-copyright="0" data-ratio="1.0223214285714286" data-s="300,640" src="/upload/afec305c8c6ff56171ec5978b5865216.jpg" data-type="jpeg" data-w="448" style=""> 吕老师：类型是用来定义数据结构的，你可以认为是mysql中的一张表。文档就是最终的数据了，你可以认为一个文档就是一条记录。 <img class="rich_pages" data-copyright="0" data-ratio="1.0522388059701493" data-s="300,640" src="/upload/18c27d8905b6836454ff95673a015c4e.jpg" data-type="jpeg" data-w="402" style=""> <img class="rich_pages" data-copyright="0" data-ratio="1.0223214285714286" data-s="300,640" src="/upload/98db9c20871bbb99dffcc06653bbd484.jpg" data-type="jpeg" data-w="448" style=""> <img class="rich_pages" data-copyright="0" data-ratio="1.0223214285714286" data-s="300,640" src="/upload/5a4fc25c2fbe31926b53c6de4a5ba7f.jpg" data-type="jpeg" data-w="448" style=""> 吕老师：比如一首诗，有诗题、作者、朝代、字数、诗内容等字段，那么首先，我们可以建立一个名叫poems的索引，然后创建一个名叫poem的类型，类型是通过mapping来定义每个字段的类型，比如诗题、作者、朝代都是keyword类型，诗内容是text类型，而字数是integer类型，最后就是把数据组织成json格式存放进去了。 <img class="rich_pages" data-copyright="0" data-ratio="3.2661691542288556" data-s="300,640" src="/upload/c8a86008eb022b2915bedb51b2126d09.jpg" data-type="jpeg" data-w="402" style=""> <img class="rich_pages" data-copyright="0" data-ratio="1.0223214285714286" data-s="300,640" src="/upload/1bc794f7ba8871fade8f4f771f476be.jpg" data-type="jpeg" data-w="448" style=""> <img class="rich_pages" data-copyright="0" data-ratio="1.0223214285714286" data-s="300,640" src="/upload/930cc64cc8a65b95e7911be318e2a84f.jpg" data-type="jpeg" data-w="448" style=""> 吕老师：这个问题问得好，这涉及到分词的问题，keyword类型是不会分词的，直接根据字符串内容建立反向索引，text类型在存入elasticsearch的时候，会先分词，然后根据分词后的内容建立反向索引。 <img class="rich_pages" data-copyright="0" data-ratio="0.9353233830845771" data-s="300,640" src="/upload/917c9e0a6a9fbe05c5fab273ab58726e.jpg" data-type="jpeg" data-w="402" style=""> <img class="rich_pages" data-copyright="0" data-ratio="1.0223214285714286" data-s="300,640" src="/upload/be138e0ede7e939b2fbae5c822af2a1d.jpg" data-type="jpeg" data-w="448" style=""> <img class="rich_pages" data-copyright="0" data-ratio="1.0223214285714286" data-s="300,640" src="/upload/51100a504a15e240c2e48d331ff1b970.jpg" data-type="jpeg" data-w="448" style=""> 吕老师：之前我们说过，elasticsearch把操作都封装成了http的api，我们只要给elasticsearch发送http请求就行。比如使用curl -XPUT 'http://ip:port/poems'，就能建立一个名为poems的索引，其他操作也是类似的。 <img class="rich_pages" data-copyright="0" data-ratio="1.0223214285714286" data-s="300,640" src="/upload/1d4c5988901e447cff1d90ac4deff4d.jpg" data-type="jpeg" data-w="448" style=""> 【elasticsearch分布式原理】 <img class="rich_pages" data-copyright="0" data-ratio="1.0223214285714286" data-s="300,640" src="/upload/6d6a8af1b8cff5478f0cbd89017f4179.jpg" data-type="jpeg" data-w="448" style=""> <img class="rich_pages" data-copyright="0" data-ratio="1.0223214285714286" data-s="300,640" src="/upload/f553a8929ac8f0cd1b42ba6c22b7d3d9.jpg" data-type="jpeg" data-w="448" style=""> 吕老师：没错，elasticsearch也是会对数据进行切分，同时每一个分片会保存多个副本，其原因和hdfs是一样的，都是为了保证分布式环境下的高可用。 <img class="rich_pages" data-copyright="0" data-ratio="1.492537313432836" data-s="300,640" src="/upload/7acad273dc4f6bf89abbad27df6f6132.jpg" data-type="jpeg" data-w="402" style=""> <img class="rich_pages" data-copyright="0" data-ratio="1.0223214285714286" data-s="300,640" src="/upload/3f42f2666018d6a1ebd37cc7f7de6162.jpg" data-type="jpeg" data-w="448" style=""> <img class="rich_pages" data-copyright="0" data-ratio="1.0223214285714286" data-s="300,640" src="/upload/f9953748e9b9bc950bc5baf1025adbf7.jpg" data-type="jpeg" data-w="448" style=""> 吕老师：没错，在elasticsearch中，节点是对等的，节点间会通过自己的一些规则选取集群的master，master会负责集群状态信息的改变，并同步给其他节点。 <img class="rich_pages" data-copyright="0" data-ratio="1.0223214285714286" data-s="300,640" src="/upload/ceb5d93b3a9bcdd47e1c02c9b28b3545.jpg" data-type="jpeg" data-w="448" style=""> <img class="rich_pages" data-copyright="0" data-ratio="1.1691542288557213" data-s="300,640" src="/upload/4038d8292d2ad80ead1abc11ecfc1d0b.jpg" data-type="jpeg" data-w="402" style=""> <img class="rich_pages" data-copyright="0" data-ratio="1.0223214285714286" data-s="300,640" src="/upload/61054315875780a2d4dbf817d3d5a398.jpg" data-type="jpeg" data-w="448" style=""> <img class="rich_pages" data-copyright="0" data-ratio="1.0223214285714286" data-s="300,640" src="/upload/dbdcf2f3d1870a5d0f140994319ebc2b.jpg" data-type="jpeg" data-w="448" style=""> <img class="rich_pages" data-copyright="0" data-ratio="1.0223214285714286" data-s="300,640" src="/upload/ee7fdfb8ad499676b8942ecf72d5b1c7.jpg" data-type="jpeg" data-w="448" style=""> 吕老师：注意，只有建立索引和类型需要经过master，数据的写入有一个简单的routing规则，可以route到集群中的任意节点，所以数据写入压力是分散在整个集群的。 <img class="rich_pages" data-copyright="0" data-ratio="1.0223214285714286" data-s="300,640" src="/upload/40701c5ad9fdbccc9359b9bebf1e3d46.jpg" data-type="jpeg" data-w="448" style=""> 【elk系统】 <img class="rich_pages" data-copyright="0" data-ratio="1.0223214285714286" data-s="300,640" src="/upload/b73b537e9cf173ebad68fb7bd4b6cf03.jpg" data-type="jpeg" data-w="448" style=""> <img class="rich_pages" data-copyright="0" data-ratio="1.0223214285714286" data-s="300,640" src="/upload/5fcb32f2903422f408e4cb807578e577.jpg" data-type="jpeg" data-w="448" style=""> 吕老师：其实很多公司都用elasticsearch搭建elk系统，也就是日志分析系统。其中e就是elasticsearch，l是logstash，是一个日志收集系统，k是kibana，是一个数据可视化平台。 <img class="rich_pages" data-copyright="0" data-ratio="1.6019900497512438" data-s="300,640" src="/upload/9ca834e25bc1566b41248a8624a3d6db.jpg" data-type="jpeg" data-w="402" style=""> <img class="rich_pages" data-copyright="0" data-ratio="1.0223214285714286" data-s="300,640" src="/upload/e053c6c7bd24c4afaa0f7d65263d6e59.jpg" data-type="jpeg" data-w="448" style=""> <img class="rich_pages" data-copyright="0" data-ratio="1.0223214285714286" data-s="300,640" src="/upload/39428dae1a51ab8803880a01e59baa43.jpg" data-type="jpeg" data-w="448" style=""> 吕老师：分析日志的用处可大了，你想，假如一个分布式系统有1000台机器，系统出现故障时，我要看下日志，还得一台一台登录上去查看，是不是非常麻烦？ <img class="rich_pages" data-copyright="0" data-ratio="1.0223214285714286" data-s="300,640" src="/upload/c19cc1255d0d49a36cddfe1e44e241ad.jpg" data-type="jpeg" data-w="448" style=""> <img class="rich_pages" data-copyright="0" data-ratio="1.0223214285714286" data-s="300,640" src="/upload/9fefd674ac87a9051242210c88f8db2f.jpg" data-type="jpeg" data-w="448" style=""> 吕老师：但是如果日志接入了elk系统就不一样。比如系统运行过程中，突然出现了异常，在日志中就能及时反馈，日志进入elk系统中，我们直接在kibana就能看到日志情况。如果再接入一些实时计算模块，还能做实时报警功能。 <img class="rich_pages" data-copyright="0" data-ratio="1.0223214285714286" data-s="300,640" src="/upload/f0d1dcf88c93810bb86f19e43a6ec685.jpg" data-type="jpeg" data-w="448" style=""> <img class="rich_pages" data-copyright="0" data-ratio="1.0223214285714286" data-s="300,640" src="/upload/7b026905ca08eaf77c423042b2616559.jpg" data-type="jpeg" data-w="448" style=""> <img class="rich_pages" data-copyright="0" data-ratio="1.0223214285714286" data-s="300,640" src="/upload/43ce189fffd2556945603495b6d81fe4.jpg" data-type="jpeg" data-w="448" style=""> 【笔记】 小史学完了elasticsearch，在笔记本上写下了如下记录： 1、反向索引又叫倒排索引，是根据文章内容中的关键字建立索引 2、搜索引擎原理就是建立反向索引 3、elasticsearch在lucene的基础上进行封装，实现了分布式搜索引擎 4、elasticsearch中的索引、类型和文档的概念比较重要，类似于mysql中的数据库、表和行 5、elasticsearch也是master-slave架构，也实现了数据的分片和备份 6、elasticsearch一个典型应用就是elk日志分析系统 写完，又高高兴兴背诗去了。 观书有感 朱熹 半亩方塘一鉴开，天光云影共徘徊。 问渠那得清如许？为有源头活水来。 <hr style="max-width: 100%;letter-spacing: 0.544px;white-space: normal;background-color: rgb(255, 255, 255);box-sizing: border-box !important;word-wrap: break-word !important;"> <p style="max-width: 100%;min-height: 1em;letter-spacing: 0.544px;white-space: normal;background-color: rgb(255, 255, 255);text-indent: 2em;box-sizing: border-box !important;word-wrap: break-word !important;"> <p style="max-width: 100%;min-height: 1em;letter-spacing: 0.544px;white-space: normal;background-color: rgb(255, 255, 255);text-indent: 2em;box-sizing: border-box !important;word-wrap: break-word !important;">生活现场是互联网侦察推出的现场系列中的另一个板块，旨在通过生活中的场景，来解释大数据微服务技术中的基本原理，希望对大家学习技术原理有所帮助。 <p style="max-width: 100%;min-height: 1em;letter-spacing: 0.544px;white-space: normal;background-color: rgb(255, 255, 255);text-indent: 2em;box-sizing: border-box !important;word-wrap: break-word !important;"> <p style="max-width: 100%;min-height: 1em;letter-spacing: 0.544px;white-space: normal;background-color: rgb(255, 255, 255);text-indent: 2em;box-sizing: border-box !important;word-wrap: break-word !important;">往期回顾 <p style="max-width: 100%;min-height: 1em;letter-spacing: 0.544px;white-space: normal;background-color: rgb(255, 255, 255);text-indent: 2em;box-sizing: border-box !important;word-wrap: break-word !important;"><a href="http://mp.weixin.qq.com/s?__biz=MzIzMTE1ODkyNQ==&mid=2649411489&idx=1&sn=cc830718ac8af0287a0f2c4a880a87c9&chksm=f0b60a43c7c18355a89616d96dd7fec3529c0c307f282a14c9efba46f25cc504d71f61b93cc3&scene=21#wechat_redirect" target="_blank" data-itemshowtype="0" data-linktype="2">【生活现场】从洗袜子到hbase存储原理解析</a> <p style="max-width: 100%;min-height: 1em;letter-spacing: 0.544px;white-space: normal;background-color: rgb(255, 255, 255);text-indent: 2em;box-sizing: border-box !important;word-wrap: break-word !important;"><a href="http://mp.weixin.qq.com/s?__biz=MzIzMTE1ODkyNQ==&mid=2649411210&idx=1&sn=749f6a034d91ed3292a9f7167dee9c41&chksm=f0b60b68c7c1827e82d4d3ba39cd1acd4f564ec556ef7bbc023d6517fe563a34f3d385244b7a&scene=21#wechat_redirect" target="_blank" data-itemshowtype="0" data-linktype="2" style="color: rgb(87, 107, 149);-webkit-tap-highlight-color: rgba(0, 0, 0, 0);max-width: 100%;box-sizing: border-box !important;word-wrap: break-word !important;">【生活现场】从生日请客到hdfs工作原理解析</a> <p style="max-width: 100%;min-height: 1em;letter-spacing: 0.544px;white-space: normal;background-color: rgb(255, 255, 255);text-indent: 2em;box-sizing: border-box !important;word-wrap: break-word !important;"><a href="http://mp.weixin.qq.com/s?__biz=MzIzMTE1ODkyNQ==&mid=2649411054&idx=1&sn=f5fc02ccfb32cbe0421f13255027ae3f&chksm=f0b6080cc7c1811a7a333994bd77f2b57b6d6962e90d52ef173b06345f309ce6ecfa62519b7e&scene=21#wechat_redirect" target="_blank" data-linktype="2" style="color: rgb(87, 107, 149);-webkit-tap-highlight-color: rgba(0, 0, 0, 0);max-width: 100%;box-sizing: border-box !important;word-wrap: break-word !important;">【生活现场】从打牌到map-reduce工作原理解析</a> <p style="max-width: 100%;min-height: 1em;letter-spacing: 0.544px;white-space: normal;background-color: rgb(255, 255, 255);text-indent: 2em;box-sizing: border-box !important;word-wrap: break-word !important;"><a href="http://mp.weixin.qq.com/s?__biz=MzIzMTE1ODkyNQ==&mid=2649410963&idx=1&sn=dab123af8a39a0d7eb3d06c22b5d7ad9&chksm=f0b60871c7c18167801f71b952f3201835f9deb30aaec5fbfce2fe4769124e29cdfe52f20bed&scene=21#wechat_redirect" target="_blank" data-linktype="2" style="color: rgb(87, 107, 149);-webkit-tap-highlight-color: rgba(0, 0, 0, 0);max-width: 100%;box-sizing: border-box !important;word-wrap: break-word !important;">【生活现场】从搬家到容器技术docker应用场景解析</a> <p style="max-width: 100%;min-height: 1em;letter-spacing: 0.544px;white-space: normal;background-color: rgb(255, 255, 255);text-align: center;box-sizing: border-box !important;word-wrap: break-word !important;"><img class="" data-copyright="0" data-ratio="0.5" data-s="300,640" data-type="jpeg" data-w="400" src="/upload/27a0fd22d324a1af95f534078afa17c1.jpg" style="box-sizing: border-box !important;word-wrap: break-word !important;width: 400px !important;visibility: visible !important;">

Hbase表设计策略

作者：微信小助手

<section class="xmteditor" style="display:none;" data-tools="新媒体管家" data-label="powered by xmt.cn"></section> <img class="rich_pages" data-ratio="0.10025062656641603" data-s="300,640" src="/upload/7c40ff0276ccfdbc1921f28067646510.png" data-type="png" data-w="399" style=""> <img class="rich_pages" data-ratio="0.23416666666666666" src="/upload/dcb64950a27454fb05ff8a2f9b7a1173.gif" data-type="gif" data-w="1200" style=""> <section style="font-size: 14px;line-height: 1.8;letter-spacing: 1.2px;padding-right: 8px;padding-left: 8px;box-sizing: border-box;"> <section class="horizontal-tb" style="box-sizing: border-box;" powered-by="xiumi.us"> </section> <section style="margin-right: 0%;margin-left: 0%;text-align: center;box-sizing: border-box;" powered-by="xiumi.us"> <section style="display: inline-block;box-sizing: border-box;"> <section style="border-radius: 0.5em;border-color: rgb(207, 207, 207);border-style: solid;border-width: 1px;padding: 10px;display: inline-block;box-sizing: border-box;"> <section class="horizontal-tb" style="text-align: justify;line-height: 1.4em;box-sizing: border-box;" powered-by="xiumi.us"> 本文字数：4849字 预计阅读时间：20分钟 </section> </section> <section style="width: 0px;border-top: 0.8em solid rgb(207, 207, 207);margin: auto;border-left: 0.8em solid transparent !important;border-right: 0.8em solid transparent !important;box-sizing: border-box;"></section> </section> </section> <section class="horizontal-tb" style="box-sizing: border-box;" powered-by="xiumi.us"> </section> <section style="margin-right: 0%;margin-left: 0%;text-align: center;box-sizing: border-box;" powered-by="xiumi.us"> <section style="display: inline-block;vertical-align: top;box-sizing: border-box;"> <section style="width: 45px;height: 5px;line-height: 5px;background-color: rgb(255, 203, 58);box-sizing: border-box;"></section> <section class="horizontal-tb" style="margin-top: -3px;margin-bottom: -3px;transform: rotate(0deg);border-width: 1px;border-style: solid;border-color: rgb(0, 0, 0);padding-left: 8px;padding-right: 8px;font-size: 16px;box-sizing: border-box;"> 导读 </section> <section style="width: 45px;height: 5px;margin-left: auto;line-height: 5px;background-color: rgb(255, 203, 58);box-sizing: border-box;"></section> </section> </section> <section class="horizontal-tb" style="letter-spacing: 1.2px;line-height: 1.75;padding-right: 8px;padding-left: 8px;box-sizing: border-box;" powered-by="xiumi.us"> Hbase是一个高可靠性，高性能、列存储、可伸缩、多版本的Nosql的分布式数据存储系统，而Hbase的表设计是发挥出Hbase优异性能的关键所在。一个好的Hbase的表设计在满足业务需求下遵循哪些规则呢？如何设计出既满足业务需求性能又好的的Hbase表呢？今天我们就来谈谈Hbase的表设计策略。 <section> Hbase非常适用于数据量大、写入＞读取、写入并发量高的业务场景之下。在介绍表设计策略之前，先来介绍一下有关Hbase表设计需要知道的相关基本概念。 </section> </section> <section class="horizontal-tb" style="box-sizing: border-box;" powered-by="xiumi.us"> </section> <section style="margin: 10px 0% 20px;box-sizing: border-box;" powered-by="xiumi.us"> <section style="display: inline-block;width: 100%;vertical-align: top;border-width: 1px;border-radius: 0px;border-style: solid none;border-color: rgb(62, 62, 62);box-sizing: border-box;"> <section style="box-sizing: border-box;" powered-by="xiumi.us"> <section style="align-items: center;display: flex;margin: -3px 0%;box-sizing: border-box;"> <section style="display: inline-block;vertical-align: bottom;width: auto;flex: 0 0 auto;border-width: 0px 0px 0px 2px;border-radius: 1px;border-style: none none none solid;border-color: rgb(62, 62, 62);overflow: hidden;box-sizing: border-box;"> <section style="text-align: center;margin-right: 0%;margin-left: 0%;transform: translate3d(-10px, 0px, 0px);box-sizing: border-box;" powered-by="xiumi.us"> <section style="display: inline-block;min-width: 10%;max-width: 100%;vertical-align: top;transform: matrix(1, 0, -0.2, 1, 0, 0);-webkit-transform: matrix(1, 0, -0.2, 1, 0, 0);-moz-transform: matrix(1, 0, -0.2, 1, 0, 0);-o-transform: matrix(1, 0, -0.2, 1, 0, 0);border-style: solid;border-width: 2px;border-radius: 0px;border-color: rgb(62, 62, 62);padding: 6px 10px 6px 20px;background-color: rgb(255, 222, 23);box-sizing: border-box;"> <section class="horizontal-tb" style="text-align: justify;line-height: 1;letter-spacing: 0px;box-sizing: border-box;" powered-by="xiumi.us"> </section> </section> </section> </section> <section style="display: inline-block;vertical-align: bottom;width: auto;flex: 1 1 auto;border-width: 0px;box-sizing: border-box;"> <section class="horizontal-tb" style="font-size: 18px;line-height: 1;letter-spacing: 1px;box-sizing: border-box;" powered-by="xiumi.us"> HBase数据模型 </section> </section> <section style="display: inline-block;vertical-align: bottom;width: auto;flex: 0 0 auto;border-width: 0px;border-radius: 1px;border-style: none;border-color: rgb(62, 62, 62);overflow: hidden;box-sizing: border-box;"> <section style="text-align: center;margin: -2px 0%;transform: translate3d(10px, 0px, 0px);-webkit-transform: translate3d(10px, 0px, 0px);-moz-transform: translate3d(10px, 0px, 0px);-o-transform: translate3d(10px, 0px, 0px);box-sizing: border-box;" powered-by="xiumi.us"> <section style="display: inline-block;min-width: 10%;max-width: 100%;vertical-align: top;transform: matrix(1, 0, -0.2, 1, 0, 0);-webkit-transform: matrix(1, 0, -0.2, 1, 0, 0);-moz-transform: matrix(1, 0, -0.2, 1, 0, 0);-o-transform: matrix(1, 0, -0.2, 1, 0, 0);border-style: solid;border-width: 2px;border-radius: 0px;border-color: rgb(62, 62, 62);padding: 6px 10px;background-color: rgb(255, 222, 23);box-sizing: border-box;"> <section class="horizontal-tb" style="text-align: justify;line-height: 1;letter-spacing: 0px;box-sizing: border-box;" powered-by="xiumi.us"> </section> </section> </section> </section> </section> </section> </section> </section> <section class="horizontal-tb" style="letter-spacing: 1.2px;line-height: 1.75;padding-right: 8px;padding-left: 8px;box-sizing: border-box;" powered-by="xiumi.us"> •表<span style="background-color: rgb(248, 248, 248);color: rgb(233, 105, 0);font-size: inherit;font-family: "Helvetica Neue", Helvetica, "Hiragino Sans GB", "Microsoft YaHei", Arial, sans-serif;letter-spacing: 0px;">Table 对应于关系型数据库中的一张张表，HBase以“表”为单位组织数据，表由多行组成； •行 <span style="background-color: rgb(248, 248, 248);color: rgb(233, 105, 0);font-size: inherit;font-family: "Helvetica Neue", Helvetica, "Hiragino Sans GB", "Microsoft YaHei", Arial, sans-serif;letter-spacing: 0px;">Colum 行由一个<span style="background-color: rgb(248, 248, 248);color: rgb(233, 105, 0);font-size: inherit;font-family: "Helvetica Neue", Helvetica, "Hiragino Sans GB", "Microsoft YaHei", Arial, sans-serif;letter-spacing: 0px;">RowKey和多个列族组成，一个行有一个<span style="background-color: rgb(248, 248, 248);color: rgb(233, 105, 0);font-size: inherit;font-family: "Helvetica Neue", Helvetica, "Hiragino Sans GB", "Microsoft YaHei", Arial, sans-serif;letter-spacing: 0px;">RowKey，用来唯一标识； •行键<span style="background-color: rgb(248, 248, 248);color: rgb(233, 105, 0);font-size: inherit;font-family: "Helvetica Neue", Helvetica, "Hiragino Sans GB", "Microsoft YaHei", Arial, sans-serif;letter-spacing: 0px;">RowKey Rowkey类似RDBMS的主键，唯一标识该行，是定位该行的索引； •列族<span style="background-color: rgb(248, 248, 248);color: rgb(233, 105, 0);font-size: inherit;font-family: "Helvetica Neue", Helvetica, "Hiragino Sans GB", "Microsoft YaHei", Arial, sans-serif;letter-spacing: 0px;">Row Family 每一行由若干列族组成，每个列族下可包含多个列，列族是列共性的一些体现。注意：物理上，同一列族的数据存储在一起的； •列限定符<span style="background-color: rgb(248, 248, 248);color: rgb(233, 105, 0);font-size: inherit;font-family: "Helvetica Neue", Helvetica, "Hiragino Sans GB", "Microsoft YaHei", Arial, sans-serif;letter-spacing: 0px;">Row Qualifier 列由列族和列限定符唯一指定； •单元格<span style="background-color: rgb(248, 248, 248);color: rgb(233, 105, 0);font-size: inherit;font-family: "Helvetica Neue", Helvetica, "Hiragino Sans GB", "Microsoft YaHei", Arial, sans-serif;letter-spacing: 0px;">Cell 单元格由<span style="background-color: rgb(248, 248, 248);color: rgb(233, 105, 0);font-size: inherit;font-family: "Helvetica Neue", Helvetica, "Hiragino Sans GB", "Microsoft YaHei", Arial, sans-serif;letter-spacing: 0px;">RowKey、列族、列限定符唯一定位，单元格之中存放一个值（Value）和一个版本号； •时间戳<span style="background-color: rgb(248, 248, 248);color: rgb(233, 105, 0);font-size: inherit;font-family: "Helvetica Neue", Helvetica, "Hiragino Sans GB", "Microsoft YaHei", Arial, sans-serif;letter-spacing: 0px;">TimStamp 单元格内不同版本的值按时间倒序排列，最新的数据排在最前面； •区<span style="background-color: rgb(248, 248, 248);color: rgb(233, 105, 0);font-size: inherit;font-family: "Helvetica Neue", Helvetica, "Hiragino Sans GB", "Microsoft YaHei", Arial, sans-serif;letter-spacing: 0px;">Region HBase自动把表水平（按<span style="background-color: rgb(248, 248, 248);color: rgb(233, 105, 0);font-size: inherit;font-family: "Helvetica Neue", Helvetica, "Hiragino Sans GB", "Microsoft YaHei", Arial, sans-serif;letter-spacing: 0px;">Row）划分成多个区 (<span style="background-color: rgb(248, 248, 248);color: rgb(233, 105, 0);font-size: inherit;font-family: "Helvetica Neue", Helvetica, "Hiragino Sans GB", "Microsoft YaHei", Arial, sans-serif;letter-spacing: 0px;">Region)，每个<span style="background-color: rgb(248, 248, 248);color: rgb(233, 105, 0);font-size: inherit;font-family: "Helvetica Neue", Helvetica, "Hiragino Sans GB", "Microsoft YaHei", Arial, sans-serif;letter-spacing: 0px;">Region会保存一个表里面某段连续的数据。 </section> <section class="horizontal-tb" style="box-sizing: border-box;" powered-by="xiumi.us"> </section> <section style="margin: 10px 0% 20px;box-sizing: border-box;" powered-by="xiumi.us"> <section style="display: inline-block;width: 100%;vertical-align: top;border-width: 1px;border-radius: 0px;border-style: solid none;border-color: rgb(62, 62, 62);box-sizing: border-box;"> <section style="box-sizing: border-box;" powered-by="xiumi.us"> <section style="align-items: center;display: flex;margin: -3px 0%;box-sizing: border-box;"> <section style="display: inline-block;vertical-align: bottom;width: auto;flex: 0 0 auto;border-width: 0px 0px 0px 2px;border-radius: 1px;border-style: none none none solid;border-color: rgb(62, 62, 62);overflow: hidden;box-sizing: border-box;"> <section style="text-align: center;margin-right: 0%;margin-left: 0%;transform: translate3d(-10px, 0px, 0px);box-sizing: border-box;" powered-by="xiumi.us"> <section style="display: inline-block;min-width: 10%;max-width: 100%;vertical-align: top;transform: matrix(1, 0, -0.2, 1, 0, 0);-webkit-transform: matrix(1, 0, -0.2, 1, 0, 0);-moz-transform: matrix(1, 0, -0.2, 1, 0, 0);-o-transform: matrix(1, 0, -0.2, 1, 0, 0);border-style: solid;border-width: 2px;border-radius: 0px;border-color: rgb(62, 62, 62);padding: 6px 10px 6px 20px;background-color: rgb(255, 222, 23);box-sizing: border-box;"> <section class="horizontal-tb" style="text-align: justify;line-height: 1;letter-spacing: 0px;box-sizing: border-box;" powered-by="xiumi.us"> </section> </section> </section> </section> <section style="display: inline-block;vertical-align: bottom;width: auto;flex: 1 1 auto;border-width: 0px;box-sizing: border-box;"> <section class="horizontal-tb" style="font-size: 18px;line-height: 1;letter-spacing: 1px;box-sizing: border-box;" powered-by="xiumi.us"> 列描述 </section> </section> <section style="display: inline-block;vertical-align: bottom;width: auto;flex: 0 0 auto;border-width: 0px;border-radius: 1px;border-style: none;border-color: rgb(62, 62, 62);overflow: hidden;box-sizing: border-box;"> <section style="text-align: center;margin: -2px 0%;transform: translate3d(10px, 0px, 0px);-webkit-transform: translate3d(10px, 0px, 0px);-moz-transform: translate3d(10px, 0px, 0px);-o-transform: translate3d(10px, 0px, 0px);box-sizing: border-box;" powered-by="xiumi.us"> <section style="display: inline-block;min-width: 10%;max-width: 100%;vertical-align: top;transform: matrix(1, 0, -0.2, 1, 0, 0);-webkit-transform: matrix(1, 0, -0.2, 1, 0, 0);-moz-transform: matrix(1, 0, -0.2, 1, 0, 0);-o-transform: matrix(1, 0, -0.2, 1, 0, 0);border-style: solid;border-width: 2px;border-radius: 0px;border-color: rgb(62, 62, 62);padding: 6px 10px;background-color: rgb(255, 222, 23);box-sizing: border-box;"> <section class="horizontal-tb" style="text-align: justify;line-height: 1;letter-spacing: 0px;box-sizing: border-box;" powered-by="xiumi.us"> </section> </section> </section> </section> </section> </section> </section> </section> <section class="horizontal-tb" style="letter-spacing: 1.2px;line-height: 1.75;padding-right: 8px;padding-left: 8px;box-sizing: border-box;" powered-by="xiumi.us"> 常用的列描述有： •<span style="background-color: rgb(248, 248, 248);color: rgb(233, 105, 0);font-size: inherit;font-family: "Helvetica Neue", Helvetica, "Hiragino Sans GB", "Microsoft YaHei", Arial, sans-serif;letter-spacing: 0px;">BLOOMFILTER 表示是否开启<span style="background-color: rgb(248, 248, 248);color: rgb(233, 105, 0);font-size: inherit;font-family: "Helvetica Neue", Helvetica, "Hiragino Sans GB", "Microsoft YaHei", Arial, sans-serif;letter-spacing: 0px;">Bloomfilter，以及<span style="background-color: rgb(248, 248, 248);color: rgb(233, 105, 0);font-size: inherit;font-family: "Helvetica Neue", Helvetica, "Hiragino Sans GB", "Microsoft YaHei", Arial, sans-serif;letter-spacing: 0px;">Bloomfilte<span style="background-color: rgb(248, 248, 248);color: rgb(233, 105, 0);font-size: inherit;font-family: "Helvetica Neue", Helvetica, "Hiragino Sans GB", "Microsoft YaHei", Arial, sans-serif;letter-spacing: 0px;">r的细粒度； •<span style="background-color: rgb(248, 248, 248);color: rgb(233, 105, 0);font-size: inherit;font-family: "Helvetica Neue", Helvetica, "Hiragino Sans GB", "Microsoft YaHei", Arial, sans-serif;letter-spacing: 0px;">COMPRESSION 表示是否开启压缩，以及使用压缩的方式； •<span style="background-color: rgb(248, 248, 248);color: rgb(233, 105, 0);font-size: inherit;font-family: "Helvetica Neue", Helvetica, "Hiragino Sans GB", "Microsoft YaHei", Arial, sans-serif;letter-spacing: 0px;">VERSIONS 表示允许存储的数据最大版本数； •<span style="background-color: rgb(248, 248, 248);color: rgb(233, 105, 0);font-size: inherit;font-family: "Helvetica Neue", Helvetica, "Hiragino Sans GB", "Microsoft YaHei", Arial, sans-serif;letter-spacing: 0px;">TTL 表示允许数据存活的最大时间； •<span style="background-color: rgb(248, 248, 248);color: rgb(233, 105, 0);font-size: inherit;font-family: "Helvetica Neue", Helvetica, "Hiragino Sans GB", "Microsoft YaHei", Arial, sans-serif;letter-spacing: 0px;">MINVERSION 表示数据保存的最少版本数。 </section> <section class="horizontal-tb" style="box-sizing: border-box;" powered-by="xiumi.us"> </section> <section style="margin: 10px 0% 20px;box-sizing: border-box;" powered-by="xiumi.us"> <section style="display: inline-block;width: 100%;vertical-align: top;border-width: 1px;border-radius: 0px;border-style: solid none;border-color: rgb(62, 62, 62);box-sizing: border-box;"> <section style="box-sizing: border-box;" powered-by="xiumi.us"> <section style="align-items: center;display: flex;margin: -3px 0%;box-sizing: border-box;"> <section style="display: inline-block;vertical-align: bottom;width: auto;flex: 0 0 auto;border-width: 0px 0px 0px 2px;border-radius: 1px;border-style: none none none solid;border-color: rgb(62, 62, 62);overflow: hidden;box-sizing: border-box;"> <section style="text-align: center;margin-right: 0%;margin-left: 0%;transform: translate3d(-10px, 0px, 0px);box-sizing: border-box;" powered-by="xiumi.us"> <section style="display: inline-block;min-width: 10%;max-width: 100%;vertical-align: top;transform: matrix(1, 0, -0.2, 1, 0, 0);-webkit-transform: matrix(1, 0, -0.2, 1, 0, 0);-moz-transform: matrix(1, 0, -0.2, 1, 0, 0);-o-transform: matrix(1, 0, -0.2, 1, 0, 0);border-style: solid;border-width: 2px;border-radius: 0px;border-color: rgb(62, 62, 62);padding: 6px 10px 6px 20px;background-color: rgb(255, 222, 23);box-sizing: border-box;"> <section class="horizontal-tb" style="text-align: justify;line-height: 1;letter-spacing: 0px;box-sizing: border-box;" powered-by="xiumi.us"> </section> </section> </section> </section> <section style="display: inline-block;vertical-align: bottom;width: auto;flex: 1 1 auto;border-width: 0px;box-sizing: border-box;"> <section class="horizontal-tb" style="font-size: 16px;line-height: 1;letter-spacing: 1px;box-sizing: border-box;" powered-by="xiumi.us"> 列描述角度的设计策略 </section> </section> <section style="display: inline-block;vertical-align: bottom;width: auto;flex: 0 0 auto;border-width: 0px;border-radius: 1px;border-style: none;border-color: rgb(62, 62, 62);overflow: hidden;box-sizing: border-box;"> <section style="text-align: center;margin: -2px 0%;transform: translate3d(10px, 0px, 0px);-webkit-transform: translate3d(10px, 0px, 0px);-moz-transform: translate3d(10px, 0px, 0px);-o-transform: translate3d(10px, 0px, 0px);box-sizing: border-box;" powered-by="xiumi.us"> <section style="display: inline-block;min-width: 10%;max-width: 100%;vertical-align: top;transform: matrix(1, 0, -0.2, 1, 0, 0);-webkit-transform: matrix(1, 0, -0.2, 1, 0, 0);-moz-transform: matrix(1, 0, -0.2, 1, 0, 0);-o-transform: matrix(1, 0, -0.2, 1, 0, 0);border-style: solid;border-width: 2px;border-radius: 0px;border-color: rgb(62, 62, 62);padding: 6px 10px;background-color: rgb(255, 222, 23);box-sizing: border-box;"> <section class="horizontal-tb" style="text-align: justify;line-height: 1;letter-spacing: 0px;box-sizing: border-box;" powered-by="xiumi.us"> </section> </section> </section> </section> </section> </section> </section> </section> <section style="margin: 10px 0% 15px;box-sizing: border-box;" powered-by="xiumi.us"> <section style="display: inline-block;width: 100%;border-width: 0px 0px 0px 5px;border-style: solid;border-left-color: rgb(253, 209, 0);border-right-color: rgb(253, 209, 0);padding-left: 5px;border-bottom-left-radius: 0px;box-sizing: border-box;"> <section class="horizontal-tb" style="box-sizing: border-box;" powered-by="xiumi.us"> BLOOMFILTER </section> </section> </section> <section class="horizontal-tb" style="line-height: 1.75;padding-right: 8px;padding-left: 8px;box-sizing: border-box;" powered-by="xiumi.us"> <span style="background-color: rgb(248, 248, 248);color: rgb(233, 105, 0);font-size: inherit;font-family: "Helvetica Neue", Helvetica, "Hiragino Sans GB", "Microsoft YaHei", Arial, sans-serif;letter-spacing: 0px;">BloomFilter的原理这里不过多描述。是否采用<span style="background-color: rgb(248, 248, 248);color: rgb(233, 105, 0);font-size: inherit;font-family: "Helvetica Neue", Helvetica, "Hiragino Sans GB", "Microsoft YaHei", Arial, sans-serif;letter-spacing: 0px;">BloomFilter需要根据实际业务考虑。由于<span style="background-color: rgb(248, 248, 248);color: rgb(233, 105, 0);font-size: inherit;font-family: "Helvetica Neue", Helvetica, "Hiragino Sans GB", "Microsoft YaHei", Arial, sans-serif;letter-spacing: 0px;">BloomFilter的原理，BloomFilter可以快速确认某个<span style="background-color: rgb(248, 248, 248);color: rgb(233, 105, 0);font-size: inherit;font-family: "Helvetica Neue", Helvetica, "Hiragino Sans GB", "Microsoft YaHei", Arial, sans-serif;letter-spacing: 0px;">Rowkey或<span style="background-color: rgb(248, 248, 248);color: rgb(233, 105, 0);font-size: inherit;font-family: "Helvetica Neue", Helvetica, "Hiragino Sans GB", "Microsoft YaHei", Arial, sans-serif;letter-spacing: 0px;">Rowkey<span style="background-color: rgb(248, 248, 248);color: rgb(233, 105, 0);font-size: inherit;font-family: "Helvetica Neue", Helvetica, "Hiragino Sans GB", "Microsoft YaHei", Arial, sans-serif;letter-spacing: 0px;"> + <span style="background-color

30条书写高质量SQL的建议，太有用了！

作者：微信小助手

文章列表