百度技术沙龙第67期 百度开源专场

具体的产品案例,分享百度开源技术最新实践经验。目前这些项目都已经在 github/baidu 上开源。

什么是 PaddlePaddle 深度学习平台?

首先做个简单的介绍,PaddlePaddle 是百度自主研发的性能优先、灵活易用的深度学习平台,是一个已经解决和将要解决一些实际问题的平台。目前百度有超过30个主要产品都在使用 PaddlePaddle。关于机器学习、深度学习和浅层学习的内容就不详细介绍了,接下来重点讲述一下 PaddlePaddle 的整体架构。

关于 PaddlePaddle 整体架构

说到 PaddlePaddle 的整体架构,主要从这几个方面入手:多机并行架构、多 GPU 并行架构、Sequence 序列模型和大规模稀疏训练。多机的并行架构和序列模型的实现都是实现神经网络最复杂的东西,那么具体怎么实现全连接?

PaddlePaddle 是2013年启动时比较流行的架构是 Pserver 和 Trainer 的架构。在多机并行架构中数据分配到不同节点,下图里灰色部分表示机器,方框里表示一个进程,Pserver 和 Trainer 是分布在两个进程里,中间的部分是网络通讯连接。

下面来介绍一下什么是大规模稀疏模型训练。稀疏模型训练是说输入数据是稀疏的,由于稀疏输入,那么灰色的神经元和连接在训练中都没有作用,灰色神经元的输出是0,灰色连接的梯度是0,梯度是0的话,简单的 SGD 不更新权重。所以只有蓝色的连接有价值,需要从 PServer 服务器获得最新参数,需要计算梯度,并将梯度传送回参数服务器。(如下图)

除了上面所提到的,还有两外两种情况下的稀疏模型:

  • 大规模稀疏模型(多机器)——每个 Trainer Prefetch 出自身需要的参数和服务器通信。

  • 大规模稀疏模型(正则化)——简单的 SGD 确实在梯度为0的时候,不去更新参数,但是加上正则化就不一定了;比如L2正则化,就要求参数的2范数持续减小。

PaddlePaddle 实现时的一些思考

基于 OP(操作)还是基于 Layer(层)?

  • 基于 OP——从矩阵乘法配起,一步一步对应一个一个数学运算。

  • 基于层——直接写一个全连接层,LSTM 层。

  • 基于 OP 的优势 Tensorflow——更灵活,更可以让研究人员构造新的东西

  • 基于 Layer 的优势 Caffe——更易用,让细节暴露的更少;更容易优化。

基于 OP 还是基于 Layer?——支持大部分 Layer,但是也支持从 OP 开始配网络(矩阵乘发,加法,激活等等);对于成型的 Layer(LSTM)使用 C++重新优化。原因在于,PaddlePaddle 是企业解决现有问题的框架,不是纯粹的科研框架;企业需要性能,也需要灵活性。

多机通信基于 MPI 还是 Spark 还是 K8s + Docker?PaddlePaddle 底层通信不依赖于任何网络框架,PaddlePaddle 的网络任务需求相对简单,根源在于任务周期短(连续运行几周);任务可以失败(多存checkpoint)。同时,PaddlePaddle 的网络需要高性能,从头手写网络库更方便性能调优,RDMA 可以更好的支持。同理,PaddlePaddle 底层不依赖任何 GPU 通信框架。

百度搜索开源基础架构

颜世光是百度搜索基础架构负责人,在这次沙龙上介绍了百度当前的这套搜索引擎,以及搜索引擎背后的事件。重点部分是百度这套开源的基础架构软件站,它包括分布式数据库、文件系统、管理系统、分布式协调服务、网络通信框架。下面来一一介绍。

当前,用户通过互联网搜索引擎的期望在不断的变化,整个搜索引擎的期望从之前的几周变成现在的几分钟,之前几周之内可以处理几百亿的数据,现在要在几分钟之内处理几万亿的数据,这是个鲜明的矛盾。其实解决方案就是构建一个大数据处理平台,也称之为“基础架构系统”。这个基础架构系统目标首先是海量的目标数据。其次就是在于集群利用率的保证,这个利用率可能是 CPU 利用率,它会为你节省成本。

这里可以简单介绍一下百度内从事开发的平台——baidu stack(如上图)。这个平台分三层,最的底层是网络通讯,是一个高性能的 RPC 框架,它会把所有的网络问题屏蔽掉,让上层的系统在开发中不需要考虑网络拓扑。中间一层是基础服务,包括分布式文件系统,它解决了数据处理。

第二就是集群管理系统,它管理的数据可以让程序部署变得代价很小。第三是分布式的协调服务,一方面用做服务发现,另外就是分布式。最上层是核心数据库和数据处理系统。

在理解上可以将这套系统和 Hadoop 相关的系统类比。从中间这层说起,Hadoop 有 HDPS,Hadoop 在分布式服务这块使用 Cukaber,比如也有 Storm、Spark 这些。整个基础架构系统的设计思想有两个,第一是分层。无论是 Hadoop 系还是谷歌,他们都使用类似的思想,这个思想主要是分工和借用,让不同分工解决不同问题。

另外一个思想就是高效,解决用户对处理速度的期望。百度主要使用 SSD、万兆网卡,这套分布式基础架构完全是用 C++实现的。首先是核心的数据库 Tera,这里列了 Tera 数据库的核心功能,包括全局有序、自动分裂、合并、支持快照。

Tera 的架构可以看(如上简图),从图上我们可以看到它有核心就是绿色两部分,是 Master 和 TableServer,提供整个数据节点都是 TableServer,所有的访问经过 Master,让它扩展到几千台服务器中。

灰色的底层数据都是在分布式文件系统上,自身没有任何数据,被设计成无状态,当一个 TableServer 宕机后,会从另外一个机器拉取数据,不会有任何损失。同样底层的分布式文件系统可以提供很大的帮助,它是通过 Nexus 做的。

这里简单介绍一下 Tera 的核心技术。⽔水平扩展方面能做到无单点;在线分裂、合并;⾃自动负载均衡;通过 LSM-Tree 做到实时同步,并且 Tera 还具有多语言支持的特点。Tera 在百度内部有非常广泛的应用,例如百度的网页库, 百度将万亿量级的网页存储在 Tera 中。

这里再介绍一下如上图的百度文件处理系统(BFS),不仅在百度,BFS 在外部的使用价值也是很高的。那么 BFS 具有哪些特点呢?首先是持续可⽤:分布式 Master,多机房数据冗余。其次就是实时高吞吐:慢节点处理,元数据管理。

收藏 评论(2)
分享到:
共2条回复 最后由Q1058204131 回复于2019-08-05 15:40
#2 代开深圳票 回复于2019-08-04

不错的转发下,有啥东东,抱歉
http://cphoto.wikidot.com
http://fengniao.wikidot.com
http://fsjiaju.wikidot.com
http://110199.wikidot.com
http://txtnovel.wikidot.com
http://fayifa.wikidot.com
http://cssmoban.wikidot.com
http://evergrande.wikidot.com
http://10086shenzhen.wikidot.com
http://ccedip.wikidot.com
http://lollipop.wikidot.com
http://wpumpkin.wikidot.com
http://moment.wikidot.com
http://wbanana.wikidot.com
http://wbubble.wikidot.com
http://cherish.wikidot.com
http://wsunflower.wikidot.com
http://gorgeous.wikidot.com
http://wrainbow.wikidot.com
http://wsunshine.wikidot.com
http://hilarious.wikidot.com
http://wlullaby.wikidot.com
http://wgalaxy.wikidot.com
http://wbutterfly.wikidot.com
http://wtwinkle.wikidot.com
http://wsmile.wikidot.com
http://lovetu.wikidot.com
http://wmother.wikidot.com
http://wpassion.wikidot.com
http://fantastic.wikidot.com
http://opsteel.wikidot.com

0
#3 Q1058204131 回复于2019-08-05

不错的转发下,有啥东东,抱歉
http://gbsgjd.wikidot.com/,
http://mfghdg.wikidot.com/,
http://jsdfghd.wikidot.com/,
http://hfsgf.wikidot.com/,
http://fcnfgsg.wikidot.com/,
http://jfjdsgf.wikidot.com/,
http://hkdtrj.wikidot.com/,
http://fsdgshk.wikidot.com/,
http://fhsdgj.wikidot.com/,
http://fgsdfgs.wikidot.com/,
http://mczfjkm.wikidot.com/,
http://nzgsgj.wikidot.com/,
http://fhsahh.wikidot.com/,
http://ghsdhd.wikidot.com/,
http://nfdgsdd.wikidot.com/,
http://mfgxjch.wikidot.com/,
http://gfhfsa.wikidot.com/,
http://goyfhglk.wikidot.com/,
http://fkgkjjk.wikidot.com/,
http://hgdhds.wikidot.com/,
http://mhdfhs.wikidot.com/,
http://yursdff.wikidot.com/,
http://hdfgdgs.wikidot.com/,
http://shanghhfg.wikidot.com/,
http://fhsshbx.wikidot.com/,
http://mngbsd.wikidot.com/,
http://jsrebx.wikidot.com/,
http://kdgszg.wikidot.com/,
http://hdfhsa.wikidot.com/,
http://jfdsfhd.wikidot.com/,
http://nfgssnt.wikidot.com/,
http://fdgsdas.wikidot.com/,
http://hjrszd.wikidot.com/,
http://hgjskk.wikidot.com/,
http://hedjszm.wikidot.com/,
http://hjuskf.wikidot.com/,
http://kdsrjsg.wikidot.com/,
http://nfdgsar.wikidot.com/,
http://ghkddsj.wikidot.com/,
http://fgjcbxg.wikidot.com/,
http://tgsahd.wikidot.com/,
http://zgsgsrk.wikidot.com/,
http://qrysxk.wikidot.com/,
http://gstrho.wikidot.com/,
http://hckdsk.wikidot.com/,
http://shafhf.wikidot.com/,
http://zsfxmnh.wikidot.com/,
http://mzjtia.wikidot.com/,
http://nfdhssg.wikidot.com/,
http://hkffbd.wikidot.com/,
http://zdfwjt.wikidot.com/,
http://xcrghs.wikidot.com/,
http://nsgzfg.wikidot.com/,
http://hgsdsa.wikidot.com/,
http://aezfgh.wikidot.com/,
http://bxbned.wikidot.com/,
http://gabzjjg.wikidot.com/,
http://hsgyfj.wikidot.com/,
http://hsfdhf.wikidot.com/,
http://zvshetg.wikidot.com/,
http://zzyurej.wikidot.com/,
http://lchdhjw.wikidot.com/,
http://hzjstgrs.wikidot.com/,
http://dzjhkfy.wikidot.com/,
http://jndhas.wikidot.com/,
http://zbhfher.wikidot.com/,
http://ytdghs.wikidot.com/,
http://whhsvs.wikidot.com/,
http://wfkfdsz.wikidot.com/,
http://qdjsgt.wikidot.com/,
http://mgssgtg.wikidot.com/,
http://jdgsjah.wikidot.com/,
http://fhkfak.wikidot.com/,
http://hfhfdds.wikidot.com/,
http://hskefx.wikidot.com/,
http://xykjdg.wikidot.com/,
http://qankfhd.wikidot.com/,
http://xakfhd.wikidot.com/,
http://bjsgsw.wikidot.com/,
http://hzgdstg.wikidot.com/,
http://hfgfhn.wikidot.com/,
http://ashdfyh.wikidot.com/,
http://dldhds.wikidot.com/,
http://wsjfh.wikidot.com/,
http://bxkdhkf.wikidot.com/,
http://ykkfpgh.wikidot.com/,
http://jzkfphj.wikidot.com/,
http://pjkfpof.wikidot.com/,
http://hdkdjf.wikidot.com/,
http://ddkfpgj.wikidot.com/,
http://cykfpjf.wikidot.com/,
http://hldkfpt.wikidot.com/,
http://jygkfphj.wikidot.com/,
http://lzkfphj.wikidot.com/,
http://tlkfphk.wikidot.com/,
http://jqkfpjh.wikidot.com/,
http://tskfpjg.wikidot.com/,
http://xzkfojh.wikidot.com/,
http://wxkfpjh.wikidot.com/,
http://zykfpjh.wikidot.com/,
http://yckfpkjn.wikidot.com/,
http://ntkfpjh.wikidot.com/,
http://szkfpjh.wikidot.com/,
http://lygkfphj.wikidot.com/,
http://chgeesd.wikidot.com/,
http://cckfplk.wikidot.com/,
http://jlkfplk.wikidot.com/,
http://thkfplk.wikidot.com/,
http://spkfplk.wikidot.com/,
http://yzkfplk.wikidot.com/,
http://bckfplk.wikidot.com/,
http://ddjkfplk.wikidot.com/,
http://jjhekfplk.wikidot.com/,
http://henkfplk.wikidot.com/,
http://jnskfplk.wikidot.com/,
http://dqkfplk.wikidot.com/,
http://gykfplk.wikidot.com/,
http://qthkfplk.wikidot.com/,
http://lpskfplk.wikidot.com/,
http://zykfplk.wikidot.com/,
http://kmkfplk.wikidot.com/,
http://ljkfplk.wikidot.com/,
http://qjkfplk.wikidot.com/,
http://yxkfplk.wikidot.com/,
http://zygkfplk.wikidot.com/,
http://zzkfplk.wikidot.com/,
http://qzkfplk.wikidot.com/,
http://xmkfplk.wikidot.com/,
http://ptkfplk.wikidot.com/,
http://fzkfplk.wikidot.com/,
http://tjkfplk.wikidot.com/,
http://cskfplk.wikidot.com/,
http://sjzkfplk.wikidot.com/,
http://cqkfplk.wikidot.com/,
http://bjkfplk.wikidot.com/,
http://dfsgfda.wikidot.com/,
http://dsfgdsf.wikidot.com/,
http://dsfgdsg.wikidot.com/,
http://sdgdsfyh.wikidot.com/,
http://dsgdhg.wikidot.com/,
http://ljmdsgdfg.wikidot.com/,
http://fdhggfkj.wikidot.com/,
http://fhfdsf.wikidot.com/,
http://dsfgsdhy.wikidot.com/,
http://dsfgdfftg.wikidot.com/,
http://sdtgdfd.wikidot.com/,
http://dsgdszfg.wikidot.com/,
http://jhtgjh.wikidot.com/,
http://zdfgds.wikidot.com/,
http://dsgdfdcx.wikidot.com/,
http://vdsghg.wikidot.com/,
http://aghfghj.wikidot.com/,
http://dsgdsdfg.wikidot.com/,
http://dfgsfg.wikidot.com/,
http://dfggd.wikidot.com/,
http://dfjhklfg.wikidot.com/,
http://gfvjhfg.wikidot.com/,
http://dsfgfdf.wikidot.com/,
http://fdstdy.wikidot.com/,
http://dsftdrcg.wikidot.com/,
http://bgfcgh.wikidot.com/,
http://ghfhj.wikidot.com/,
http://hgfghv.wikidot.com/,
http://sdgf.wikidot.com/,
http://gjmhfgh.wikidot.com/,
http://vhgvh.wikidot.com/,
http://hfghg.wikidot.com/,
http://fdgddf.wikidot.com/,
http://sdgd.wikidot.com/,
http://dsfdf.wikidot.com/,
http://sdgfd.wikidot.com/,
http://dsgdf.wikidot.com/,
http://fgdfsh.wikidot.com/,
http://fgsdf.wikidot.com/,
http://dsfgdf.wikidot.com/,
http://sdgfd.wikidot.com/main:layout,
http://sdgfd.wikidot.com/system:join,
http://sdgfd.wikidot.com/help:_home,
http://sdgfd.wikidot.com/main:contact,
http://sdgfd.wikidot.com/help:first-time-user,
http://sdgfd.wikidot.com/help:creating-pages,
http://sdgfd.wikidot.com/help:editing-pages,
http://sdgfd.wikidot.com/help:navigation-bars,
http://sdgfd.wikidot.com/help:using-modules,
http://sdgfd.wikidot.com/help:templates,
http://dsgdf.wikidot.com/main:layout,
http://dsgdf.wikidot.com/system:join,
http://dsgdf.wikidot.com/help:_home,
http://dsgdf.wikidot.com/main:contact,
http://dsgdf.wikidot.com/help:first-time-user,
http://dsgdf.wikidot.com/help:quick-reference,
http://dsgdf.wikidot.com/help:creating-pages,
http://dsgdf.wikidot.com/help:editing-pages,
http://dsgdf.wikidot.com/help:navigation-bars,
http://dsgdf.wikidot.com/help:templates,
http://dsgdf.wikidot.com/help:css-themes,
http://fgdfsh.wikidot.com/main:layout,
http://fgdfsh.wikidot.com/system:join,
http://fgdfsh.wikidot.com/main:about,
http://dsgdf.wikidot.com/main:about,
http://sdgfd.wikidot.com/main:about,
http://fgsdf.wikidot.com/main:about,
http://fgsdf.wikidot.com/main:layout,
http://fgsdf.wikidot.com/system:join,
http://fgsdf.wikidot.com/help:_home,
http://fgsdf.wikidot.com/main:contact,
http://fgsdf.wikidot.com/help:first-time-user,
http://fgsdf.wikidot.com/help:quick-reference,
http://fgsdf.wikidot.com/help:creating-pages,
http://fgsdf.wikidot.com/help:editing-pages,
http://fgsdf.wikidot.com/help:navigation-bars,
http://fgsdf.wikidot.com/help:templates,
http://fgsdf.wikidot.com/help:css-themes,
http://dsfgdf.wikidot.com/main:about,
http://dsfgdf.wikidot.com/main:layout,
http://dsfgdf.wikidot.com/system:join,
http://dsfgdf.wikidot.com/help:_home,
http://dsfgdf.wikidot.com/main:contact,
http://dsfgdf.wikidot.com/help:first-time-user,
http://dsfgdf.wikidot.com/help:quick-reference,
http://dsfgdf.wikidot.com/help:creating-pages,
http://dsfgdf.wikidot.com/help:editing-pages,
http://dsfgdf.wikidot.com/help:navigation-bars,
http://dsfgdf.wikidot.com/help:templates,
http://dsfgdf.wikidot.com/help:css-themes,
http://fgsdf.wikidot.com/main:home,
http://bcvfdf.wikidot.com/,
http://fdsgfxd.wikidot.com/,
http://lmhjvb.wikidot.com/,
http://dsfsdfg.wikidot.com/,
http://mfcgdsfg.wikidot.com/,
http://mhhdf.wikidot.com/,
http://ljkhjc.wikidot.com/,
http://wefdslo.wikidot.com/,
http://iuydfh.wikidot.com/,
http://pfyfdf.wikidot.com/,
http://ertryrt.wikidot.com/,
http://werdsfsx.wikidot.com/,
http://puhjigh.wikidot.com/,
http://iyufh.wikidot.com/,
http://iytrc.wikidot.com/,
http://uysedhb.wikidot.com/,
http://opirfgh.wikidot.com/,
http://uyrthf.wikidot.com/,
http://ytydfgc.wikidot.com/,
http://retgvcx.wikidot.com/,
http://yuifdg.wikidot.com/,
http://qwreersd.wikidot.com/,
http://tyutgf.wikidot.com/,
http://resfdsf.wikidot.com/,
http://rteddf.wikidot.com/,
http://aqddcz.wikidot.com/,
http://ohjvf.wikidot.com/,
http://yursdf.wikidot.com/,
http://wzsdasd.wikidot.com/,
http://phgfdc.wikidot.com/,
http://sdfwew.wikidot.com/,
http://dtfgrerf.wikidot.com/,
http://reweas.wikidot.com/,
http://qwerad.wikidot.com/,
http://qasdfd.wikidot.com/,
http://dfsgdf.wikidot.com/,
http://fddsfgdf.wikidot.com/,
http://jhiujyu.wikidot.com/,
http://sgtyrt.wikidot.com/,
http://asdsz.wikidot.com/,
http://srfsdf.wikidot.com/,
http://desrs.wikidot.com/,
http://sdfser.wikidot.com/,
http://sddfgs.wikidot.com/,
http://sdfsdg.wikidot.com/,
http://dsfsgfe.wikidot.com/,
http://gwserg.wikidot.com/,
http://xcfdg.wikidot.com/,
http://klfghgf.wikidot.com/,
http://gfdaa.wikidot.com/,
http://yedrcg.wikidot.com/,
http://swefzf.wikidot.com/,
http://dsfsza.wikidot.com/,
http://swezd.wikidot.com/,
http://hghgshg.wikidot.com/,
http://rtsdf.wikidot.com/,
http://dsaew.wikidot.com/,
http://dsfrwlo.wikidot.com/,
http://srdtg.wikidot.com/,
http://dfgwews.wikidot.com/,
http://dsggrw.wikidot.com/,
http://aserdtg.wikidot.com/,
http://dsgadtg.wikidot.com/,
http://eytder.wikidot.com/,
http://dsghsfh.wikidot.com/,
http://dsrww.wikidot.com/,
http://dsfzgds.wikidot.com/,
http://sdtgdf.wikidot.com/,
http://dszfas.wikidot.com/,
http://dfgfxd.wikidot.com/,
http://dsfdse.wikidot.com/,
http://sdrse.wikidot.com/,
http://eawfxf.wikidot.com/,
http://dssdfg.wikidot.com/,
http://dsfwef.wikidot.com/,
http://yyhfc.wikidot.com/,
http://fdsgd.wikidot.com/,
http://scxdfgdf.wikidot.com/,
http://dfsdraa.wikidot.com/,
http://fghdftgsd.wikidot.com/,
http://fsfplk.wikidot.com/,
http://gzfolk.wikidot.com/,
http://stfplk.wikidot.com/,
http://szfplk.wikidot.com/,
http://zhfplk.wikidot.com/,
http://zqfplk.wikidot.com/,
http://zjfplk.wikidot.com/,
http://sgfplk.wikidot.com/,
http://mmfplk.wikidot.com/,
http://jmfplk.wikidot.com/,
http://hyfplk.wikidot.com/,
http://hzfplk.wikidot.com/,
http://swfplk.wikidot.com/,
http://mzfplk.wikidot.com/,
http://yjfplk.wikidot.com/,
http://zsfplk.wikidot.com/,
http://nnfplk.wikidot.com/,
http://qyfplk.wikidot.com/,
http://glfplk.wikidot.com/,
http://dgfplk.wikidot.com/,
http://bhfplk.wikidot.com/,
http://lzfplk.wikidot.com/,
http://qzfplk.wikidot.com/,
http://rlfplk.wikidot.com/,
http://wzfplk.wikidot.com/,
http://ncfplk.wikidot.com/,
http://hezfplk.wikidot.com/,
http://bsfplk.wikidot.com/,
http://fcgfplk.wikidot.com/,
http://ggfplk.wikidot.com/,
http://ganzfolk.wikidot.com/,
http://ycfplk.wikidot.com/,
http://xxfplk.wikidot.com/,
http://pxfplk.wikidot.com/,
http://jafplk.wikidot.com/,
http://srfplk.wikidot.com/,
http://jjfplk.wikidot.com/,
http://fzfplk.wikidot.com/,
http://fuzfplk.wikidot.com/,
http://jdzfplk.wikidot.com/,
http://zzfplk.wikidot.com/,
http://xmfplk.wikidot.com/,
http://quzfplk.wikidot.com/,
http://ptfplk.wikidot.com/,
http://lyfplk.wikidot.com/,
http://czfplk.wikidot.com/,
http://tyfplk.wikidot.com/,
http://shzfplk.wikidot.com/,
http://yqfplk.wikidot.com/,
http://dtfplk.wikidot.com/,
http://csfplk.wikidot.com/,
http://zjjfplk.wikidot.com/,
http://huyfplk.wikidot.com/,
http://xtfplk.wikidot.com/,
http://zzhfplk.wikidot.com/,
http://xyfplk.wikidot.com/,
http://ximfplk.wikidot.com/,
http://whfplk.wikidot.com/,
http://syfplk.wikidot.com/,
http://hsfplk.wikidot.com/,
http://hgfplk.wikidot.com/,
http://shzhfplk.wikidot.com/,
http://xgfplk.wikidot.com/,
http://xnfplk.wikidot.com/,
http://xzfplk.wikidot.com/,
http://zgfplk.wikidot.com/,
http://guyfplk.wikidot.com/,
http://luzfplk.wikidot.com/,
http://deyfplk.wikidot.com/,
http://cdfplk.wikidot.com/,
http://suzfplk.wikidot.com/,
http://qrfplk.wikidot.com/,
http://hffplk.wikidot.com/,
http://fgfplk.wikidot.com/,
http://fdgfplk.wikidot.com/,
http://aqfplk.wikidot.com/,
http://fhfplk.wikidot.com/,
http://masfplk.wikidot.com/,
http://xbfplk.wikidot.com/,
http://xcfplk.wikidot.com/,
http://yuntfplk.wikidot.com/,
http://weihfplk.wikidot.com/,
http://qdfplk.wikidot.com/,
http://jnfplk.wikidot.com/,
http://dyfplk.wikidot.com/,
http://lwfplk.wikidot.com/,
http://hezefplk.wikidot.com/,
http://wffplk.wikidot.com/,
http://zbfplk.wikidot.com/,
http://zgxfplk.wikidot.com/,
http://dezfplk.wikidot.com/,
http://lcfplk.wikidot.com/,
http://lffplk.wikidot.com/,
http://tafplk.wikidot.com/,
http://zhzhfplk.wikidot.com/,
http://zhengzfplk.wikidot.com/,
http://pdsfplk.wikidot.com/,
http://xinxfplk.wikidot.com/,
http://luoyfplk.wikidot.com/,
http://kffplk.wikidot.com/,
http://yyfplk.wikidot.com/,
http://xucfplk.wikidot.com/,
http://jzuofplk.wikidot.com/,
http://hbfplk.wikidot.com/,
http://anyfplk.wikidot.com/,
http://zkfplk.wikidot.com/,
http://xinyfplk.wikidot.com/,
http://sxmfplk.wikidot.com/,
http://sqfplk.wikidot.com/,
http://lhfplk.wikidot.com/,
http://thfplk.wikidot.com/,
http://spfplk.wikidot.com/,
http://liaoyfplk.wikidot.com/,
http://jlfplk.wikidot.com/,
http://ccfplk.wikidot.com/,
http://qqhefplk.wikidot.com/,
http://hegfplk.wikidot.com/,
http://jmsfplk.wikidot.com/,
http://hebfplk.wikidot.com/,
http://dqfplk.wikidot.com/,
http://yucfplk.wikidot.com/,
http://qthfplk.wikidot.com/,
http://mdfplk.wikidot.com/,
http://jxfplk.wikidot.com/,
http://hehfplk.wikidot.com/,
http://wxfplk.wikidot.com/,
http://taizfplk.wikidot.com/,
http://suzhfplk.wikidot.com/,
http://ntfplk.wikidot.com/,
http://njfplk.wikidot.com/,
http://hzhfplk.wikidot.com/,
http://lygfplk.wikidot.com/,
http://xuzfplk.wikidot.com/,
http://yancfplk.wikidot.com/,
http://yzhfplk.wikidot.com/,
http://wzhfplk.wikidot.com/,
http://jiaxfplk.wikidot.com/,
http://zxfplk.wikidot.com/,
http://nbfplk.wikidot.com/,
http://huzfplk.wikidot.com/,
http://hjfplk.wikidot.com/,
http://trfplk.wikidot.com/,
http://tzfplk.wikidot.com/,
http://weizfplk.wikidot.com/,
http://zhsfplk.wikidot.com/,
http://lpsfplk.wikidot.com/,
http://guizfplk.wikidot.com/,
http://bjfplk.wikidot.com/,
http://ansfplk.wikidot.com/,
http://zyfplk.wikidot.com/,
http://ztfplk.wikidot.com/,
http://rxfplk.wikidot.com/,
http://qjfplk.wikidot.com/,
http://ljfplk.wikidot.com/,
http://kmfplk.wikidot.com/,
http://xiyfplk.wikidot.com/,
http://wnfplk.wikidot.com/,
http://xafplk.wikidot.com/,
http://bjifplk.wikidot.com/,
http://tcfplk.wikidot.com/,
http://tsfplk.wikidot.com/,
http://lzhfplk.wikidot.com/,
http://jygfplk.wikidot.com/,
http://jcfplk.wikidot.com/,
http://byfplk.wikidot.com/,
http://zhyfplk.wikidot.com/,
http://plfplk.wikidot.com/,
http://wwfplk.wikidot.com/,
http://jqfplk.wikidot.com/,
http://hmfplk.wikidot.com/,
http://shyfplk.wikidot.com/,
http://fusfplk.wikidot.com/,
http://dlfplk.wikidot.com/,
http://bxfplk.wikidot.com/,
http://asfplk.wikidot.com/,
http://liayfplk.wikidot.com/,
http://fxfplk.wikidot.com/,
http://ykfplk.wikidot.com/,
http://jzfplk.wikidot.com/,
http://ddfplk.wikidot.com/,
http://cyfplk.wikidot.com/,
http://hhhtfplk.wikidot.com/,
http://pjfplk.wikidot.com/,
http://hldfplk.wikidot.com/,
http://tlfplk.wikidot.com/,
http://wuhfplk.wikidot.com/,
http://tonglfplk.wikidot.com/,
http://tjfplk.wikidot.com/,
http://btfplk.wikidot.com/,
http://cffplk.wikidot.com/,
http://wlmqfplk.wikidot.com/,
http://sjzfplk.wikidot.com/,
http://lsfplk.wikidot.com/,
http://hdfplk.wikidot.com/,
http://chdfplk.wikidot.com/,
http://mxcsa.wikidot.com/,
http://wxzaz.wikidot.com/,
http://zxsa.wikidot.com/,
http://zsdc.wikidot.com/,
http://zxcvh.wikidot.com/,
http://sddgh.wikidot.com/,
http://zdsw.wikidot.com/,
http://dfsw.wikidot.com/,
http://dsgf1.wikidot.com/,
http://s44as.wikidot.com/,
http://gzxd.wikidot.com/,
http://fgse.wikidot.com/,
http://rdcs.wikidot.com/,
http://fsec.wikidot.com/,
http://mkhy.wikidot.com/,
http://nbvc.wikidot.com/,
http://cvfd.wikidot.com/,
http://bcvf.wikidot.com/,
http://fdty.wikidot.com/,
http://ufgh.wikidot.com/,
http://hifv.wikidot.com/,
http://ytre.wikidot.com/,
http://tyug.wikidot.com/,
http://oytf.wikidot.com/,
http://sdeo.wikidot.com/,
http://hjsa.wikidot.com/,
http://fdgd.wikidot.com/,
http://fgrw.wikidot.com/,
http://ghfo.wikidot.com/,
http://ngiw.wikidot.com/,
http://jybf.wikidot.com/,
http://fvup.wikidot.com/,
http://hjfy.wikidot.com/,
http://pdun.wikidot.com/,
http://cvbs.wikidot.com/,
http://fdgdfsgdf.wikidot.com/,
http://xzczs.wikidot.com/,
http://dcvxc.wikidot.com/,
http://sdd4ds.wikidot.com/,
http://jkyv.wikidot.com/,
http://fadsgwsea.wikidot.com/,
http://sadas.wikidot.com/,
http://ffdg1df.wikidot.com/,
http://migr.wikidot.com/,
http://ffdgdf.wikidot.com/,
http://czsa.wikidot.com/,
http://hqyv.wikidot.com/,
http://lfdsds.wikidot.com/,
http://fghfh.wikidot.com/,
http://hghfg.wikidot.com/,
http://fdstgf.wikidot.com/,
http://jgsw.wikidot.com/,
http://piyr.wikidot.com/,
http://ghfs.wikidot.com/,
http://fjtv.wikidot.com/,
http://taobao.wikidot.com/,
http://chengbang.ccedpw.com/,
http://nanzhuo3.shop.bxgtd.com/,
http://qiaojiao778.shop.bxgtd.com/,
http://doushi4566.shop.bxgtd.com/,
http://taodao.shop.bxgtd.com/,
http://jidun84.shop.bxgtd.com/,
http://tianquang67.shop.bxgtd.com/,
http://bojuenu1639.shop.bxgtd.com/,
http://cengtied15.shop.bxgtd.com/,
http://yeniangzh49.shop.bxgtd.com/,
http://www.andu729.yhtx.tv/,
http://www.tangyan83.yhtx.tv/,
http://www.ditang2.yhtx.tv/,
http://www.kaonai43043.yhtx.tv/,
http://www.duishuo.yhtx.tv/,
http://www.pabenye4186.yhtx.tv/,
http://www.waxuanqi61.yhtx.tv/,
http://www.haoganson86.yhtx.tv/,
http://www.piaoqinc13.yhtx.tv/,
http://www.qianghuai78.yhtx.tv/,
http://www.shenxiao34.yhtx.tv/,
http://www.fengniuge59.yhtx.tv/,
http://www.daojiongr71.yhtx.tv/,
http://www.sunkaizu34.yhtx.tv/,
http://www.cengshes84.yhtx.tv/,
http://www.51xsq.com/qy/yunqiang208/,
http://www.51xsq.com/qy/youshi66/,
http://www.51xsq.com/qy/kongwei55447/,
http://www.51xsq.com/qy/wendou/,
http://www.51xsq.com/qy/bibei8414689/,
http://www.51xsq.com/qy/hudiecu7642/,
http://www.51xsq.com/qy/tuncangt95/,
http://www.51xsq.com/qy/jianniuda12/,
http://www.51xsq.com/qy/yanpaijio77/,
http://www.51xsq.com/qy/feixiong68/,
http://www.51xsq.com/qy/tuofengj14/,
http://www.51xsq.com/qy/mengsong30/,

0