百度技术沙龙第71期 百度万亿量级数据库Tera架构应用、设计与实践全攻略

在 5 月 20 日百度开发者中心主办、极客邦科技承办的 71 期百度技术沙龙上,百度网页搜索基础架构技术经理齐志宏和资深工程师郑然,为大家免费放送了大型分布式表格系统 Tera 在百度搜索引擎中的应用、以及 Tera 架构设计与实践的全攻略。

Tera 在百度搜索引擎中的应用

在讲解 Tera 的应用之前,百度网页搜索基础架构技术经理齐志宏首先介绍了百度搜索架构,百度搜索引擎的作用是连接人与信息、连接人与服务,信息抓取、索引构建、检索系统构成了搜索引擎最经典的三大板块。

互联网上的信息是如何通过搜索引擎最终展示给用户的?首先,网页被搜索引擎发现,通过抓取进入搜索引擎;然后,有价值的网页经过筛选,进行正排计算和倒排计算,完成索引构建;最后,通过检索系统将最终的结果呈现给用户。

伴随互联网信息爆发式的增长,百度搜索架构也在逐渐向实时化方向演进,在介绍完搜索架构之后,齐志宏从链接存储、索引筛选、用户行为分析三个场景切入,详细讲解了 Tera 在实时搜索架构中的应用。

齐志宏先为大家解释什么是 Tera:一种大型分布式表格存储系统,具有高性能、可伸缩等存储特点,最初的设计是为了管理万亿量级的超链和网页信息。Tera 在架构演进中到底扮演了怎样的核心角色呢?

首先来看存储链接。百度推出的 Spider 3.0 系统是基于 Tera 的实时架构,以 Tera 为核心,承载了链接库、网页库的存储,将原有基于 MapReduce 的批量计算转变为基于 Tera 的实时计算,实现每秒亿级的数据随机读写、每天处理万亿量级的链接操作,信息抓取模块(即 Spider)进入了实时处理时代。

第二个是索引筛选。 索引筛选的核心作用是让有价值的信息进入索引。Tera 架构作为数据存储中心,存储了包含网页库、去重库、结果库在内的所有中间数据和最终结果,通过流式计算的方式完成页面特征拼接、页面价值计算、网页去重以及索引排序等核心操作的实时化改造。网页从抓取到筛选完成的整个过程,实现了从天级变到分钟级甚至秒级的飞跃。

最后一个是用户行为分析。 用户行为分析在搜索效果改进和搜索引擎的评价等方面,都具有重要价值。基于 Tera 的实时用户行为数据流,将用户数据的时效性推向新高度。实时数据产出的延迟可降至秒级,突发时效性识别、用户意图分析、产品迭代评估等多个维度均可实时获取用户数据,进行实时分析,对时效性和用户体验有很大的提升。

总体上,Tera 支撑了着搜索引擎大规模的实时数据读写,将批量、全量计算转变为增量、实时的数据计算,极大的提升了搜索引擎的实时数据处理能力,Tera 是百度搜索引擎从批量处理迈向实时计算的架构基础。

Tera 大型分布式表格系统的设计与实践

Tera 完成了百度搜索向万亿级数据实时搜索的跃进,成为炙手可热的数据库系统,那么,如何做好 Tera 架构的设计与实践成为开发者最为关心的问题。百度网页搜索基础架构资深工程师郑然在演讲过程中,围绕背景、数据模型、架构与设计、高可用实践以及性能优化等方面,详细讲解了 Tera 设计和实践过程。

郑然表示,百度搜索引擎面临三大业务特点:

  1. 数据量大,PB 到百 PB 这样的量级;

  2. 离线处理过程中,以站点等前缀方式访问数据是普遍的需求;

  3. 数据类型不固定。

这样的业务特点决定着 Tera 设计和实践的过程。

Tera 设计的数据模型

Tera 的数据模型有以下几个特点,首先它是 Key-Value 模型,再深入一层,它是典型的 BigTable 模型,同时,一个非常重要的特点就是全局有序。这几个特点结合在一起,就是 Tera 数据模型的设计目标。

Tera 设计的系统架构

Tera 系统主要由 Tabletserver、Master 和 ClientSDK 三部分构成, 数据持久化到底层的分布式文件系统中。其中 Tabletserver 是核心服务器,承载着所有的数据管理与访问;Master 是系统的仲裁者,负责表格的创建、Schema 更新与负载均衡;ClientSDK 包含供管理员使用的命令行工具 Teracli 和给用户使用的 SDK。

表格被按 RowKey 全局排序,并横向切分成多个 Tablet,每个 Tablet 负责服务 RowKey 的一个区间,表格又被纵向且分为多个 LocalityGroup,一个 Tablet 的多个 LocalityGroup 在物理上单独存储,可以选择不同的存储介质,用以优化访问效率。

Tera 的高可用实践

Tera 的高可用性比较关键,直接影响整个系统的服务质量,其实现方式包括两个方面:Tablet Server 可用性以及负载均衡。

  • Tablet Server 的可用性:1)Tablet Server 向 ZooKeeper 注册,利用 ZooKeeper 检测 Tablet Server 的存活;2)Tablet Server 挂掉之后,Master 收到 ZooKeeper 通知,进行 Tablet 迁移。具体迁移过程,会把挂掉的 Tablet Server 节点迁移到 Kick 节点上,当 Tablet Server 发现自己出现在 Kick 节点下面,自行退出。

  • 负载均衡:负载均衡会直接影响整个集群的可用性,所以负载均衡更本质上来说是实现高可用的技术手段。影响 Tera 负载均衡的因素相对较少,主要在 SSD 容量、随机读和随机写这三个方面。针对上述影响因素, Tera 从两个层面来进行负载均衡策略的设计。首先平衡各个 Tablet Server 读请求 Pending 的数据量, 同时利用历史值来平滑负载短时间内抖动的影响 ; 其次根据 SSD 容量平衡各个 Tablet 的数据大小。

Tera 设计的性能优化

郑然表示,Tera 设计的性能优化,是百度在做设计过程中总结出来的,实用性较强。

第一个经验是需要考虑对分布式文件系统友好。Tera 的数据持久化在分布式文件系统上,必须考虑对其友好的使用。根据 LevelDb 的特点,数据首先要持久化在 WAL 上,确保异常情况下不丢数据,所以写 WAL 的延迟和吞吐直接决定了用户写请求的延迟和吞吐。然而分布式文件系统需要写多个数据副本,在某些副本异常情况下,如果依赖分布式文件系统层面去自动恢复的话,可能大幅增加延迟。Tera 针对写 WAL 异常情况,采用关闭旧文件创建新文件的方法,规避分布式文件系统的短板。同时 WAL 持久化成功才能保证用户数据不丢,所以 WAL 写完之后必须 sync 强制数据落盘,而 sst 文件不强制要求每次写请求落盘,从而减少对分布式文件系统的压力。

第二个经验是关于 SSD 的运用。SATA 的随机读能力很差,虽然 LevelDb 做了很多优化,但是仍然无法突破硬件瓶颈,SSD 的价格现在是越来越便宜,但成本依然比 SATA 高。Tera 的数据全部持久化在 SATA 上,仅把 SSD 作为 Cache 使用,这是平衡性能和成本的一种途径。

第三个经验是异步逻辑设计。Tera 里面所有可能阻塞的逻辑都是异步的,异步逻辑可以很好提高性能,另外客户端缓存 Tablet 位置信息,因为 tablet 位置信息通常情况下变化的也不频繁,同时扩展了 LevelDb 的 BloomFilter 机制,可以提升 20% 左右的读性能。

收藏 评论(3)
分享到:
共3条回复 最后由代开深圳票 回复于2019-08-04 22:51
#2 筱Myselfkv 回复于2019-07-29

https://www.viki.com/collections/2643625l
https://www.viki.com/collections/2643633l
https://www.viki.com/collections/2643644l
https://www.viki.com/collections/2643651l
https://www.viki.com/collections/2643656l
https://www.viki.com/collections/2643666l
https://www.viki.com/collections/2643674l
https://www.viki.com/collections/2643676l
https://www.viki.com/collections/2643680l
https://www.viki.com/collections/2643683l
https://www.viki.com/collections/2643688l
https://www.viki.com/collections/2643713l
https://www.viki.com/collections/2643716l
https://www.viki.com/collections/2643718l
https://www.viki.com/collections/2643719l
https://www.viki.com/collections/2643726l
https://www.viki.com/collections/2643728l
https://www.viki.com/collections/2643732l
https://www.viki.com/collections/2643733l
https://www.viki.com/collections/2643735l
https://www.viki.com/collections/2643737l
https://www.viki.com/collections/2643739l
https://www.viki.com/collections/2643743l
https://www.viki.com/collections/2643746l
https://www.viki.com/collections/2643749l
https://www.viki.com/collections/2643750l
https://www.viki.com/collections/2643754l
https://www.viki.com/collections/2643758l
https://www.viki.com/collections/2643760l
https://www.viki.com/collections/2643762l
https://www.viki.com/collections/2643763l
https://www.viki.com/collections/2643765l
https://www.viki.com/collections/2643767l
https://www.viki.com/collections/2643771l
https://www.viki.com/collections/2643773l
https://www.viki.com/collections/2643776l
https://www.viki.com/collections/2643778l
https://www.viki.com/collections/2643779l
https://www.viki.com/collections/2643790l
https://www.viki.com/collections/2643791l
https://www.viki.com/collections/2643793l
https://www.viki.com/collections/2643794l
https://www.viki.com/collections/2643795l
https://www.viki.com/collections/2643809l
https://www.viki.com/collections/2643811l
https://www.viki.com/collections/2643815l
https://www.viki.com/collections/2643821l
https://www.viki.com/collections/2643817l
https://www.viki.com/collections/2643825l
https://www.viki.com/collections/2643829l
https://www.viki.com/collections/2643833l
https://www.viki.com/collections/2643837l
https://www.viki.com/collections/2643838l
https://www.viki.com/collections/2643860l
https://www.viki.com/collections/2643861l
https://www.viki.com/collections/2643862l
https://www.viki.com/collections/2643864l
https://www.viki.com/collections/2643867l
https://www.viki.com/collections/2643868l
https://www.viki.com/collections/2643869l
https://www.viki.com/collections/2643870l
https://www.viki.com/collections/2643872l
https://www.viki.com/collections/2643873l
https://www.viki.com/collections/2643876l
https://www.viki.com/collections/2643877l
https://www.viki.com/collections/2643879l
https://www.viki.com/collections/2643882l
https://www.viki.com/collections/2643885l
https://www.viki.com/collections/2643887l
https://www.viki.com/collections/2643888l
https://www.viki.com/collections/2643894l
https://www.viki.com/collections/2643896l
https://www.viki.com/collections/2643898l
https://www.viki.com/collections/2643899l
https://www.viki.com/collections/2643902l
https://www.viki.com/collections/2643905l
https://www.viki.com/collections/2643907l
https://www.viki.com/collections/2643908l
https://www.viki.com/collections/2643911l
https://www.viki.com/collections/2643915l
https://www.viki.com/collections/2643917l
https://www.viki.com/collections/2643919l
https://www.viki.com/collections/2643968l
https://www.viki.com/collections/2643970l
https://www.viki.com/collections/2643972l
https://www.viki.com/collections/2643974l
https://www.viki.com/collections/2643978l
https://www.viki.com/collections/2643979l
https://www.viki.com/collections/2643983l
https://www.viki.com/collections/2643984l
https://www.viki.com/collections/2643985l
https://www.viki.com/collections/2643986l
https://www.viki.com/collections/2643989l
https://www.viki.com/collections/2643992l
https://www.viki.com/collections/2643993l
https://www.viki.com/collections/2643994l
https://www.viki.com/collections/2643997l
https://www.viki.com/collections/2643999l
https://www.viki.com/collections/2644000l
https://www.viki.com/collections/2644001l
https://www.viki.com/collections/2644005l
https://www.viki.com/collections/2644009l
https://www.viki.com/collections/2644010l
https://www.viki.com/collections/2644012l
https://www.viki.com/collections/2644014l
https://www.viki.com/collections/2644018l
https://www.viki.com/collections/2644020l
https://www.viki.com/collections/2644021l
https://www.viki.com/collections/2644023l
https://www.viki.com/collections/2644025l
https://www.viki.com/collections/2644026l
https://www.viki.com/collections/2644028l
https://www.viki.com/collections/2644032l
https://www.viki.com/collections/2644035l
https://www.viki.com/collections/2644036l
https://www.viki.com/collections/2644038l
https://www.viki.com/collections/2644040l
https://www.viki.com/collections/2644042l
https://www.viki.com/collections/2644045l
https://www.viki.com/collections/2644046l
https://www.viki.com/collections/2644050l
https://www.viki.com/collections/2644051l
https://www.viki.com/collections/2644052l
https://www.viki.com/collections/2644054l
https://www.viki.com/collections/2644057l
https://www.viki.com/collections/2644059l
https://www.viki.com/collections/2644060l
https://www.viki.com/collections/2644061l
https://www.viki.com/collections/2644064l
https://www.viki.com/collections/2644067l
https://www.viki.com/collections/2644069l
https://www.viki.com/collections/2644071l
https://www.viki.com/collections/2644072l
https://www.viki.com/collections/2644074l
https://www.viki.com/collections/2644076l
https://www.viki.com/collections/2644079l
https://www.viki.com/collections/2644081l
https://www.viki.com/collections/2644084l
https://www.viki.com/collections/2644087l
https://www.viki.com/collections/2644094l
https://www.viki.com/collections/2644096l
https://www.viki.com/collections/2644098l
https://www.viki.com/collections/2644099l
https://www.viki.com/collections/2644100l
https://www.viki.com/collections/2644102l
https://www.viki.com/collections/2644103l
https://www.viki.com/collections/2644105l
https://www.viki.com/collections/2644107l
https://www.viki.com/collections/2644109l
https://www.viki.com/collections/2644111l
https://www.viki.com/collections/2644113l
https://www.viki.com/collections/2644114l
https://www.viki.com/collections/2644115l
https://www.viki.com/collections/2644118l
https://www.viki.com/collections/2644120l
https://www.viki.com/collections/2644122l
https://www.viki.com/collections/2644124l
https://www.viki.com/collections/2644126l
https://www.viki.com/collections/2644129l
https://www.viki.com/collections/2644130l
https://www.viki.com/collections/2644134l
https://www.viki.com/collections/2644135l
https://www.viki.com/collections/2644138l
https://www.viki.com/collections/2644140l
https://www.viki.com/collections/2644144l
https://www.viki.com/collections/2644147l
https://www.viki.com/collections/2644148l
https://www.viki.com/collections/2644150l
https://www.viki.com/collections/2644152l
https://www.viki.com/collections/2644158l
https://www.viki.com/collections/2644162l
https://www.viki.com/collections/2644165l
https://www.viki.com/collections/2644166l
https://www.viki.com/collections/2644167l
https://www.viki.com/collections/2644168l
https://www.viki.com/collections/2644169l
https://www.viki.com/collections/2644172l
https://www.viki.com/collections/2644174l
https://www.viki.com/collections/2644175l
https://www.viki.com/collections/2644176l
https://www.viki.com/collections/2644178l
https://www.viki.com/collections/2644179l
https://www.viki.com/collections/2644180l
https://www.viki.com/collections/2644182l
https://www.viki.com/collections/2644184l
https://www.viki.com/collections/2644186l
https://www.viki.com/collections/2644193l
https://www.viki.com/collections/2644194l
https://www.viki.com/collections/2644197l
https://www.viki.com/collections/2644199l
https://www.viki.com/collections/2644202l
https://www.viki.com/collections/2644204l
https://www.viki.com/collections/2644210l
https://www.viki.com/collections/2644475l
https://www.viki.com/collections/2644478l
https://www.viki.com/collections/2644485l
https://www.viki.com/collections/2644493l
https://www.viki.com/collections/2644495l
https://www.viki.com/collections/2644497l
https://www.viki.com/collections/2644621l
https://www.viki.com/collections/2644623l
https://www.viki.com/collections/2644624l
https://www.viki.com/collections/2645949l
https://www.viki.com/collections/2645951l
https://www.viki.com/collections/2645956l
https://www.viki.com/collections/2645957l
https://www.viki.com/collections/2645959l
https://www.viki.com/collections/2645960l
https://www.viki.com/collections/2645962l
https://www.viki.com/collections/2645965l
https://www.viki.com/collections/2645966l
https://www.viki.com/collections/2645968l
https://www.viki.com/collections/2645970l
https://www.viki.com/collections/2645975l
https://www.viki.com/collections/2645976l
https://www.viki.com/collections/2645977l
https://www.viki.com/collections/2645981l
https://www.viki.com/collections/2645983l
https://www.viki.com/collections/2645984l
https://www.viki.com/collections/2645990l
https://www.viki.com/collections/2645991l
https://www.viki.com/collections/2645993l
https://www.viki.com/collections/2645995l
https://www.viki.com/collections/2645997l
https://www.viki.com/collections/2645998l
https://www.viki.com/collections/2646001l
https://www.viki.com/collections/2646005l
https://www.viki.com/collections/2646007l
https://www.viki.com/collections/2646022l
https://www.viki.com/collections/2646026l
https://www.viki.com/collections/2646029l
https://www.viki.com/collections/2646031l
https://www.viki.com/collections/2646033l
https://www.viki.com/collections/2646036l
https://www.viki.com/collections/2646037l
https://www.viki.com/collections/2646040l
https://www.viki.com/collections/2646044l
https://www.viki.com/collections/2646047l
https://www.viki.com/collections/2646049l
https://www.viki.com/collections/2646054l
https://www.viki.com/collections/2646057l
https://www.viki.com/collections/2646061l
https://www.viki.com/collections/2646063l
https://www.viki.com/collections/2646068l
https://www.viki.com/collections/2646072l
https://www.viki.com/collections/2646077l
https://www.viki.com/collections/2646080l
https://www.viki.com/collections/2646081l
https://www.viki.com/collections/2646082l
https://www.viki.com/collections/2646084l
https://www.viki.com/collections/2646086l
https://www.viki.com/collections/2646088l
https://www.viki.com/collections/2646094l
https://www.viki.com/collections/2646095l
https://www.viki.com/collections/2646097l
https://www.viki.com/collections/2646098l
https://www.viki.com/collections/2646102l
https://www.viki.com/collections/2646103l
https://www.viki.com/collections/2646106l
https://www.viki.com/collections/2646108l
https://www.viki.com/collections/2646111l
https://www.viki.com/collections/2646112l
https://www.viki.com/collections/2646114l
https://www.viki.com/collections/2646116l
https://www.viki.com/collections/2646119l
https://www.viki.com/collections/2646120l
https://www.viki.com/collections/2646123l
https://www.viki.com/collections/2646125l
https://www.viki.com/collections/2646129l
https://www.viki.com/collections/2646130l
https://www.viki.com/collections/2646132l
https://www.viki.com/collections/2646133l
https://www.viki.com/collections/2646134l
https://www.viki.com/collections/2646136l
https://www.viki.com/collections/2646137l
https://www.viki.com/collections/2646139l
https://www.viki.com/collections/2646141l
https://www.viki.com/collections/2646142l
https://www.viki.com/collections/2646143l
https://www.viki.com/collections/2646145l
https://www.viki.com/collections/2646148l
https://www.viki.com/collections/2646150l
https://www.viki.com/collections/2646151l
https://www.viki.com/collections/2646153l
https://www.viki.com/collections/2646157l
https://www.viki.com/collections/2646158l
https://www.viki.com/collections/2646161l
https://www.viki.com/collections/2646163l
https://www.viki.com/collections/2646165l

0
#3 Q1058204131 回复于2019-08-04

http://gbsgjd.wikidot.com/,
http://mfghdg.wikidot.com/,
http://jsdfghd.wikidot.com/,
http://hfsgf.wikidot.com/,
http://fcnfgsg.wikidot.com/,
http://jfjdsgf.wikidot.com/,
http://hkdtrj.wikidot.com/,
http://fsdgshk.wikidot.com/,
http://fhsdgj.wikidot.com/,
http://fgsdfgs.wikidot.com/,
http://mczfjkm.wikidot.com/,
http://nzgsgj.wikidot.com/,
http://fhsahh.wikidot.com/,
http://ghsdhd.wikidot.com/,
http://nfdgsdd.wikidot.com/,
http://mfgxjch.wikidot.com/,
http://gfhfsa.wikidot.com/,
http://goyfhglk.wikidot.com/,
http://fkgkjjk.wikidot.com/,
http://hgdhds.wikidot.com/,
http://mhdfhs.wikidot.com/,
http://yursdff.wikidot.com/,
http://hdfgdgs.wikidot.com/,
http://shanghhfg.wikidot.com/,
http://fhsshbx.wikidot.com/,
http://mngbsd.wikidot.com/,
http://jsrebx.wikidot.com/,
http://kdgszg.wikidot.com/,
http://hdfhsa.wikidot.com/,
http://jfdsfhd.wikidot.com/,
http://nfgssnt.wikidot.com/,
http://fdgsdas.wikidot.com/,
http://hjrszd.wikidot.com/,
http://hgjskk.wikidot.com/,
http://hedjszm.wikidot.com/,
http://hjuskf.wikidot.com/,
http://kdsrjsg.wikidot.com/,
http://nfdgsar.wikidot.com/,
http://ghkddsj.wikidot.com/,
http://fgjcbxg.wikidot.com/,
http://tgsahd.wikidot.com/,
http://zgsgsrk.wikidot.com/,
http://qrysxk.wikidot.com/,
http://gstrho.wikidot.com/,
http://hckdsk.wikidot.com/,
http://shafhf.wikidot.com/,
http://zsfxmnh.wikidot.com/,
http://mzjtia.wikidot.com/,
http://nfdhssg.wikidot.com/,
http://hkffbd.wikidot.com/,
http://zdfwjt.wikidot.com/,
http://xcrghs.wikidot.com/,
http://nsgzfg.wikidot.com/,
http://hgsdsa.wikidot.com/,
http://aezfgh.wikidot.com/,
http://bxbned.wikidot.com/,
http://gabzjjg.wikidot.com/,
http://hsgyfj.wikidot.com/,
http://hsfdhf.wikidot.com/,
http://zvshetg.wikidot.com/,
http://zzyurej.wikidot.com/,
http://lchdhjw.wikidot.com/,
http://hzjstgrs.wikidot.com/,
http://dzjhkfy.wikidot.com/,
http://jndhas.wikidot.com/,
http://zbhfher.wikidot.com/,
http://ytdghs.wikidot.com/,
http://whhsvs.wikidot.com/,
http://wfkfdsz.wikidot.com/,
http://qdjsgt.wikidot.com/,
http://mgssgtg.wikidot.com/,
http://jdgsjah.wikidot.com/,
http://fhkfak.wikidot.com/,
http://hfhfdds.wikidot.com/,
http://hskefx.wikidot.com/,
http://xykjdg.wikidot.com/,
http://qankfhd.wikidot.com/,
http://xakfhd.wikidot.com/,
http://bjsgsw.wikidot.com/,
http://hzgdstg.wikidot.com/,
http://hfgfhn.wikidot.com/,
http://ashdfyh.wikidot.com/,
http://dldhds.wikidot.com/,
http://wsjfh.wikidot.com/,
http://bxkdhkf.wikidot.com/,
http://ykkfpgh.wikidot.com/,
http://jzkfphj.wikidot.com/,
http://pjkfpof.wikidot.com/,
http://hdkdjf.wikidot.com/,
http://ddkfpgj.wikidot.com/,
http://cykfpjf.wikidot.com/,
http://hldkfpt.wikidot.com/,
http://jygkfphj.wikidot.com/,
http://lzkfphj.wikidot.com/,
http://tlkfphk.wikidot.com/,
http://jqkfpjh.wikidot.com/,
http://tskfpjg.wikidot.com/,
http://xzkfojh.wikidot.com/,
http://wxkfpjh.wikidot.com/,
http://zykfpjh.wikidot.com/,
http://yckfpkjn.wikidot.com/,
http://ntkfpjh.wikidot.com/,
http://szkfpjh.wikidot.com/,
http://lygkfphj.wikidot.com/,
http://chgeesd.wikidot.com/,
http://cckfplk.wikidot.com/,
http://jlkfplk.wikidot.com/,
http://thkfplk.wikidot.com/,
http://spkfplk.wikidot.com/,
http://yzkfplk.wikidot.com/,
http://bckfplk.wikidot.com/,
http://ddjkfplk.wikidot.com/,
http://jjhekfplk.wikidot.com/,
http://henkfplk.wikidot.com/,
http://jnskfplk.wikidot.com/,
http://dqkfplk.wikidot.com/,
http://gykfplk.wikidot.com/,
http://qthkfplk.wikidot.com/,
http://lpskfplk.wikidot.com/,
http://zykfplk.wikidot.com/,
http://kmkfplk.wikidot.com/,
http://ljkfplk.wikidot.com/,
http://qjkfplk.wikidot.com/,
http://yxkfplk.wikidot.com/,
http://zygkfplk.wikidot.com/,
http://zzkfplk.wikidot.com/,
http://qzkfplk.wikidot.com/,
http://xmkfplk.wikidot.com/,
http://ptkfplk.wikidot.com/,
http://fzkfplk.wikidot.com/,
http://tjkfplk.wikidot.com/,
http://cskfplk.wikidot.com/,
http://sjzkfplk.wikidot.com/,
http://cqkfplk.wikidot.com/,
http://bjkfplk.wikidot.com/,
http://dfsgfda.wikidot.com/,
http://dsfgdsf.wikidot.com/,
http://dsfgdsg.wikidot.com/,
http://sdgdsfyh.wikidot.com/,
http://dsgdhg.wikidot.com/,
http://ljmdsgdfg.wikidot.com/,
http://fdhggfkj.wikidot.com/,
http://fhfdsf.wikidot.com/,
http://dsfgsdhy.wikidot.com/,
http://dsfgdfftg.wikidot.com/,
http://sdtgdfd.wikidot.com/,
http://dsgdszfg.wikidot.com/,
http://jhtgjh.wikidot.com/,
http://zdfgds.wikidot.com/,
http://dsgdfdcx.wikidot.com/,
http://vdsghg.wikidot.com/,
http://aghfghj.wikidot.com/,
http://dsgdsdfg.wikidot.com/,
http://dfgsfg.wikidot.com/,
http://dfggd.wikidot.com/,
http://dfjhklfg.wikidot.com/,
http://gfvjhfg.wikidot.com/,
http://dsfgfdf.wikidot.com/,
http://fdstdy.wikidot.com/,
http://dsftdrcg.wikidot.com/,
http://bgfcgh.wikidot.com/,
http://ghfhj.wikidot.com/,
http://hgfghv.wikidot.com/,
http://sdgf.wikidot.com/,
http://gjmhfgh.wikidot.com/,
http://vhgvh.wikidot.com/,
http://hfghg.wikidot.com/,
http://fdgddf.wikidot.com/,
http://sdgd.wikidot.com/,
http://dsfdf.wikidot.com/,
http://sdgfd.wikidot.com/,
http://dsgdf.wikidot.com/,
http://fgdfsh.wikidot.com/,
http://fgsdf.wikidot.com/,
http://dsfgdf.wikidot.com/,
http://sdgfd.wikidot.com/main:layout,
http://sdgfd.wikidot.com/system:join,
http://sdgfd.wikidot.com/help:_home,
http://sdgfd.wikidot.com/main:contact,
http://sdgfd.wikidot.com/help:first-time-user,
http://sdgfd.wikidot.com/help:creating-pages,
http://sdgfd.wikidot.com/help:editing-pages,
http://sdgfd.wikidot.com/help:navigation-bars,
http://sdgfd.wikidot.com/help:using-modules,
http://sdgfd.wikidot.com/help:templates,
http://dsgdf.wikidot.com/main:layout,
http://dsgdf.wikidot.com/system:join,
http://dsgdf.wikidot.com/help:_home,
http://dsgdf.wikidot.com/main:contact,
http://dsgdf.wikidot.com/help:first-time-user,
http://dsgdf.wikidot.com/help:quick-reference,
http://dsgdf.wikidot.com/help:creating-pages,
http://dsgdf.wikidot.com/help:editing-pages,
http://dsgdf.wikidot.com/help:navigation-bars,
http://dsgdf.wikidot.com/help:templates,
http://dsgdf.wikidot.com/help:css-themes,
http://fgdfsh.wikidot.com/main:layout,
http://fgdfsh.wikidot.com/system:join,
http://fgdfsh.wikidot.com/main:about,
http://dsgdf.wikidot.com/main:about,
http://sdgfd.wikidot.com/main:about,
http://fgsdf.wikidot.com/main:about,
http://fgsdf.wikidot.com/main:layout,
http://fgsdf.wikidot.com/system:join,
http://fgsdf.wikidot.com/help:_home,
http://fgsdf.wikidot.com/main:contact,
http://fgsdf.wikidot.com/help:first-time-user,
http://fgsdf.wikidot.com/help:quick-reference,
http://fgsdf.wikidot.com/help:creating-pages,
http://fgsdf.wikidot.com/help:editing-pages,
http://fgsdf.wikidot.com/help:navigation-bars,
http://fgsdf.wikidot.com/help:templates,
http://fgsdf.wikidot.com/help:css-themes,
http://dsfgdf.wikidot.com/main:about,
http://dsfgdf.wikidot.com/main:layout,
http://dsfgdf.wikidot.com/system:join,
http://dsfgdf.wikidot.com/help:_home,
http://dsfgdf.wikidot.com/main:contact,
http://dsfgdf.wikidot.com/help:first-time-user,
http://dsfgdf.wikidot.com/help:quick-reference,
http://dsfgdf.wikidot.com/help:creating-pages,
http://dsfgdf.wikidot.com/help:editing-pages,
http://dsfgdf.wikidot.com/help:navigation-bars,
http://dsfgdf.wikidot.com/help:templates,
http://dsfgdf.wikidot.com/help:css-themes,
http://fgsdf.wikidot.com/main:home,
http://bcvfdf.wikidot.com/,
http://fdsgfxd.wikidot.com/,
http://lmhjvb.wikidot.com/,
http://dsfsdfg.wikidot.com/,
http://mfcgdsfg.wikidot.com/,
http://mhhdf.wikidot.com/,
http://ljkhjc.wikidot.com/,
http://wefdslo.wikidot.com/,
http://iuydfh.wikidot.com/,
http://pfyfdf.wikidot.com/,
http://ertryrt.wikidot.com/,
http://werdsfsx.wikidot.com/,
http://puhjigh.wikidot.com/,
http://iyufh.wikidot.com/,
http://iytrc.wikidot.com/,
http://uysedhb.wikidot.com/,
http://opirfgh.wikidot.com/,
http://uyrthf.wikidot.com/,
http://ytydfgc.wikidot.com/,
http://retgvcx.wikidot.com/,
http://yuifdg.wikidot.com/,
http://qwreersd.wikidot.com/,
http://tyutgf.wikidot.com/,
http://resfdsf.wikidot.com/,
http://rteddf.wikidot.com/,
http://aqddcz.wikidot.com/,
http://ohjvf.wikidot.com/,
http://yursdf.wikidot.com/,
http://wzsdasd.wikidot.com/,
http://phgfdc.wikidot.com/,
http://sdfwew.wikidot.com/,
http://dtfgrerf.wikidot.com/,
http://reweas.wikidot.com/,
http://qwerad.wikidot.com/,
http://qasdfd.wikidot.com/,
http://dfsgdf.wikidot.com/,
http://fddsfgdf.wikidot.com/,
http://jhiujyu.wikidot.com/,
http://sgtyrt.wikidot.com/,
http://asdsz.wikidot.com/,
http://srfsdf.wikidot.com/,
http://desrs.wikidot.com/,
http://sdfser.wikidot.com/,
http://sddfgs.wikidot.com/,
http://sdfsdg.wikidot.com/,
http://dsfsgfe.wikidot.com/,
http://gwserg.wikidot.com/,
http://xcfdg.wikidot.com/,
http://klfghgf.wikidot.com/,
http://gfdaa.wikidot.com/,
http://yedrcg.wikidot.com/,
http://swefzf.wikidot.com/,
http://dsfsza.wikidot.com/,
http://swezd.wikidot.com/,
http://hghgshg.wikidot.com/,
http://rtsdf.wikidot.com/,
http://dsaew.wikidot.com/,
http://dsfrwlo.wikidot.com/,
http://srdtg.wikidot.com/,
http://dfgwews.wikidot.com/,
http://dsggrw.wikidot.com/,
http://aserdtg.wikidot.com/,
http://dsgadtg.wikidot.com/,
http://eytder.wikidot.com/,
http://dsghsfh.wikidot.com/,
http://dsrww.wikidot.com/,
http://dsfzgds.wikidot.com/,
http://sdtgdf.wikidot.com/,
http://dszfas.wikidot.com/,
http://dfgfxd.wikidot.com/,
http://dsfdse.wikidot.com/,
http://sdrse.wikidot.com/,
http://eawfxf.wikidot.com/,
http://dssdfg.wikidot.com/,
http://dsfwef.wikidot.com/,
http://yyhfc.wikidot.com/,
http://fdsgd.wikidot.com/,
http://scxdfgdf.wikidot.com/,
http://dfsdraa.wikidot.com/,
http://fghdftgsd.wikidot.com/,
http://fsfplk.wikidot.com/,
http://gzfolk.wikidot.com/,
http://stfplk.wikidot.com/,
http://szfplk.wikidot.com/,
http://zhfplk.wikidot.com/,
http://zqfplk.wikidot.com/,
http://zjfplk.wikidot.com/,
http://sgfplk.wikidot.com/,
http://mmfplk.wikidot.com/,
http://jmfplk.wikidot.com/,
http://hyfplk.wikidot.com/,
http://hzfplk.wikidot.com/,
http://swfplk.wikidot.com/,
http://mzfplk.wikidot.com/,
http://yjfplk.wikidot.com/,
http://zsfplk.wikidot.com/,
http://nnfplk.wikidot.com/,
http://qyfplk.wikidot.com/,
http://glfplk.wikidot.com/,
http://dgfplk.wikidot.com/,
http://bhfplk.wikidot.com/,
http://lzfplk.wikidot.com/,
http://qzfplk.wikidot.com/,
http://rlfplk.wikidot.com/,
http://wzfplk.wikidot.com/,
http://ncfplk.wikidot.com/,
http://hezfplk.wikidot.com/,
http://bsfplk.wikidot.com/,
http://fcgfplk.wikidot.com/,
http://ggfplk.wikidot.com/,
http://ganzfolk.wikidot.com/,
http://ycfplk.wikidot.com/,
http://xxfplk.wikidot.com/,
http://pxfplk.wikidot.com/,
http://jafplk.wikidot.com/,
http://srfplk.wikidot.com/,
http://jjfplk.wikidot.com/,
http://fzfplk.wikidot.com/,
http://fuzfplk.wikidot.com/,
http://jdzfplk.wikidot.com/,
http://zzfplk.wikidot.com/,
http://xmfplk.wikidot.com/,
http://quzfplk.wikidot.com/,
http://ptfplk.wikidot.com/,
http://lyfplk.wikidot.com/,
http://czfplk.wikidot.com/,
http://tyfplk.wikidot.com/,
http://shzfplk.wikidot.com/,
http://yqfplk.wikidot.com/,
http://dtfplk.wikidot.com/,
http://csfplk.wikidot.com/,
http://zjjfplk.wikidot.com/,
http://huyfplk.wikidot.com/,
http://xtfplk.wikidot.com/,
http://zzhfplk.wikidot.com/,
http://xyfplk.wikidot.com/,
http://ximfplk.wikidot.com/,
http://whfplk.wikidot.com/,
http://syfplk.wikidot.com/,
http://hsfplk.wikidot.com/,
http://hgfplk.wikidot.com/,
http://shzhfplk.wikidot.com/,
http://xgfplk.wikidot.com/,
http://xnfplk.wikidot.com/,
http://xzfplk.wikidot.com/,
http://zgfplk.wikidot.com/,
http://guyfplk.wikidot.com/,
http://luzfplk.wikidot.com/,
http://deyfplk.wikidot.com/,
http://cdfplk.wikidot.com/,
http://suzfplk.wikidot.com/,
http://qrfplk.wikidot.com/,
http://hffplk.wikidot.com/,
http://fgfplk.wikidot.com/,
http://fdgfplk.wikidot.com/,
http://aqfplk.wikidot.com/,
http://fhfplk.wikidot.com/,
http://masfplk.wikidot.com/,
http://xbfplk.wikidot.com/,
http://xcfplk.wikidot.com/,
http://yuntfplk.wikidot.com/,
http://weihfplk.wikidot.com/,
http://qdfplk.wikidot.com/,
http://jnfplk.wikidot.com/,
http://dyfplk.wikidot.com/,
http://lwfplk.wikidot.com/,
http://hezefplk.wikidot.com/,
http://wffplk.wikidot.com/,
http://zbfplk.wikidot.com/,
http://zgxfplk.wikidot.com/,
http://dezfplk.wikidot.com/,
http://lcfplk.wikidot.com/,
http://lffplk.wikidot.com/,
http://tafplk.wikidot.com/,
http://zhzhfplk.wikidot.com/,
http://zhengzfplk.wikidot.com/,
http://pdsfplk.wikidot.com/,
http://xinxfplk.wikidot.com/,
http://luoyfplk.wikidot.com/,
http://kffplk.wikidot.com/,
http://yyfplk.wikidot.com/,
http://xucfplk.wikidot.com/,
http://jzuofplk.wikidot.com/,
http://hbfplk.wikidot.com/,
http://anyfplk.wikidot.com/,
http://zkfplk.wikidot.com/,
http://xinyfplk.wikidot.com/,
http://sxmfplk.wikidot.com/,
http://sqfplk.wikidot.com/,
http://lhfplk.wikidot.com/,
http://thfplk.wikidot.com/,
http://spfplk.wikidot.com/,
http://liaoyfplk.wikidot.com/,
http://jlfplk.wikidot.com/,
http://ccfplk.wikidot.com/,
http://qqhefplk.wikidot.com/,
http://hegfplk.wikidot.com/,
http://jmsfplk.wikidot.com/,
http://hebfplk.wikidot.com/,
http://dqfplk.wikidot.com/,
http://yucfplk.wikidot.com/,
http://qthfplk.wikidot.com/,
http://mdfplk.wikidot.com/,
http://jxfplk.wikidot.com/,
http://hehfplk.wikidot.com/,
http://wxfplk.wikidot.com/,
http://taizfplk.wikidot.com/,
http://suzhfplk.wikidot.com/,
http://ntfplk.wikidot.com/,
http://njfplk.wikidot.com/,
http://hzhfplk.wikidot.com/,
http://lygfplk.wikidot.com/,
http://xuzfplk.wikidot.com/,
http://yancfplk.wikidot.com/,
http://yzhfplk.wikidot.com/,
http://wzhfplk.wikidot.com/,
http://jiaxfplk.wikidot.com/,
http://zxfplk.wikidot.com/,
http://nbfplk.wikidot.com/,
http://huzfplk.wikidot.com/,
http://hjfplk.wikidot.com/,
http://trfplk.wikidot.com/,
http://tzfplk.wikidot.com/,
http://weizfplk.wikidot.com/,
http://zhsfplk.wikidot.com/,
http://lpsfplk.wikidot.com/,
http://guizfplk.wikidot.com/,
http://bjfplk.wikidot.com/,
http://ansfplk.wikidot.com/,
http://zyfplk.wikidot.com/,
http://ztfplk.wikidot.com/,
http://rxfplk.wikidot.com/,
http://qjfplk.wikidot.com/,
http://ljfplk.wikidot.com/,
http://kmfplk.wikidot.com/,
http://xiyfplk.wikidot.com/,
http://wnfplk.wikidot.com/,
http://xafplk.wikidot.com/,
http://bjifplk.wikidot.com/,
http://tcfplk.wikidot.com/,
http://tsfplk.wikidot.com/,
http://lzhfplk.wikidot.com/,
http://jygfplk.wikidot.com/,
http://jcfplk.wikidot.com/,
http://byfplk.wikidot.com/,
http://zhyfplk.wikidot.com/,
http://plfplk.wikidot.com/,
http://wwfplk.wikidot.com/,
http://jqfplk.wikidot.com/,
http://hmfplk.wikidot.com/,
http://shyfplk.wikidot.com/,
http://fusfplk.wikidot.com/,
http://dlfplk.wikidot.com/,
http://bxfplk.wikidot.com/,
http://asfplk.wikidot.com/,
http://liayfplk.wikidot.com/,
http://fxfplk.wikidot.com/,
http://ykfplk.wikidot.com/,
http://jzfplk.wikidot.com/,
http://ddfplk.wikidot.com/,
http://cyfplk.wikidot.com/,
http://hhhtfplk.wikidot.com/,
http://pjfplk.wikidot.com/,
http://hldfplk.wikidot.com/,
http://tlfplk.wikidot.com/,
http://wuhfplk.wikidot.com/,
http://tonglfplk.wikidot.com/,
http://tjfplk.wikidot.com/,
http://btfplk.wikidot.com/,
http://cffplk.wikidot.com/,
http://wlmqfplk.wikidot.com/,
http://sjzfplk.wikidot.com/,
http://lsfplk.wikidot.com/,
http://hdfplk.wikidot.com/,
http://chdfplk.wikidot.com/,
http://mxcsa.wikidot.com/,
http://wxzaz.wikidot.com/,
http://zxsa.wikidot.com/,
http://zsdc.wikidot.com/,
http://zxcvh.wikidot.com/,
http://sddgh.wikidot.com/,
http://zdsw.wikidot.com/,
http://dfsw.wikidot.com/,
http://dsgf1.wikidot.com/,
http://s44as.wikidot.com/,
http://gzxd.wikidot.com/,
http://fgse.wikidot.com/,
http://rdcs.wikidot.com/,
http://fsec.wikidot.com/,
http://mkhy.wikidot.com/,
http://nbvc.wikidot.com/,
http://cvfd.wikidot.com/,
http://bcvf.wikidot.com/,
http://fdty.wikidot.com/,
http://ufgh.wikidot.com/,
http://hifv.wikidot.com/,
http://ytre.wikidot.com/,
http://tyug.wikidot.com/,
http://oytf.wikidot.com/,
http://sdeo.wikidot.com/,
http://hjsa.wikidot.com/,
http://fdgd.wikidot.com/,
http://fgrw.wikidot.com/,
http://ghfo.wikidot.com/,
http://ngiw.wikidot.com/,
http://jybf.wikidot.com/,
http://fvup.wikidot.com/,
http://hjfy.wikidot.com/,
http://pdun.wikidot.com/,
http://cvbs.wikidot.com/,
http://fdgdfsgdf.wikidot.com/,
http://xzczs.wikidot.com/,
http://dcvxc.wikidot.com/,
http://sdd4ds.wikidot.com/,
http://jkyv.wikidot.com/,
http://fadsgwsea.wikidot.com/,
http://sadas.wikidot.com/,
http://ffdg1df.wikidot.com/,
http://migr.wikidot.com/,
http://ffdgdf.wikidot.com/,
http://czsa.wikidot.com/,
http://hqyv.wikidot.com/,
http://lfdsds.wikidot.com/,
http://fghfh.wikidot.com/,
http://hghfg.wikidot.com/,
http://fdstgf.wikidot.com/,
http://jgsw.wikidot.com/,
http://piyr.wikidot.com/,
http://ghfs.wikidot.com/,
http://fjtv.wikidot.com/,
http://taobao.wikidot.com/,
http://chengbang.ccedpw.com/,
http://nanzhuo3.shop.bxgtd.com/,
http://qiaojiao778.shop.bxgtd.com/,
http://doushi4566.shop.bxgtd.com/,
http://taodao.shop.bxgtd.com/,
http://jidun84.shop.bxgtd.com/,
http://tianquang67.shop.bxgtd.com/,
http://bojuenu1639.shop.bxgtd.com/,
http://cengtied15.shop.bxgtd.com/,
http://yeniangzh49.shop.bxgtd.com/,
http://www.andu729.yhtx.tv/,
http://www.tangyan83.yhtx.tv/,
http://www.ditang2.yhtx.tv/,
http://www.kaonai43043.yhtx.tv/,
http://www.duishuo.yhtx.tv/,
http://www.pabenye4186.yhtx.tv/,
http://www.waxuanqi61.yhtx.tv/,
http://www.haoganson86.yhtx.tv/,
http://www.piaoqinc13.yhtx.tv/,
http://www.qianghuai78.yhtx.tv/,
http://www.shenxiao34.yhtx.tv/,
http://www.fengniuge59.yhtx.tv/,
http://www.daojiongr71.yhtx.tv/,
http://www.sunkaizu34.yhtx.tv/,
http://www.cengshes84.yhtx.tv/,
http://www.51xsq.com/qy/yunqiang208/,
http://www.51xsq.com/qy/youshi66/,
http://www.51xsq.com/qy/kongwei55447/,
http://www.51xsq.com/qy/wendou/,
http://www.51xsq.com/qy/bibei8414689/,
http://www.51xsq.com/qy/hudiecu7642/,
http://www.51xsq.com/qy/tuncangt95/,
http://www.51xsq.com/qy/jianniuda12/,
http://www.51xsq.com/qy/yanpaijio77/,
http://www.51xsq.com/qy/feixiong68/,
http://www.51xsq.com/qy/tuofengj14/,
http://www.51xsq.com/qy/mengsong30/,

0
#4 代开深圳票 回复于2019-08-04

不错的转发下,有啥东东,抱歉
http://cphoto.wikidot.com
http://fengniao.wikidot.com
http://fsjiaju.wikidot.com
http://110199.wikidot.com
http://txtnovel.wikidot.com
http://fayifa.wikidot.com
http://cssmoban.wikidot.com
http://evergrande.wikidot.com
http://10086shenzhen.wikidot.com
http://ccedip.wikidot.com
http://lollipop.wikidot.com
http://wpumpkin.wikidot.com
http://moment.wikidot.com
http://wbanana.wikidot.com
http://wbubble.wikidot.com
http://cherish.wikidot.com
http://wsunflower.wikidot.com
http://gorgeous.wikidot.com
http://wrainbow.wikidot.com
http://wsunshine.wikidot.com
http://hilarious.wikidot.com
http://wlullaby.wikidot.com
http://wgalaxy.wikidot.com
http://wbutterfly.wikidot.com
http://wtwinkle.wikidot.com
http://wsmile.wikidot.com
http://lovetu.wikidot.com
http://wmother.wikidot.com
http://wpassion.wikidot.com
http://fantastic.wikidot.com
http://opsteel.wikidot.com

0